Leadership team

TranscEngram Research

Research by Yi Ma, Shenghua Gao, Yanchao Yang and their collaborators, spanning 3D vision, machine learning and embodied AI.

Research by our academic co-founders

NeurIPS 2026Accepted at a conference

Generative Structure from Motion with Native 3D Diffusion

Generative Structure from Motion with Native 3D Diffusion

Haobin Duan, Binbin Huang, Mochu Xiang, Yiqun Zhao, Zibo Zhao, Yi Ma, Shenghua Gao

GenSfM uses a few uncalibrated images to estimate camera poses and object shape together. A 3D diffusion model infers surfaces outside the observed views.

GenSfM 从少量未标定图像中联合估计相机位姿和物体形状,并用三维扩散模型推断未观测到的表面。

NeurIPS 2026Accepted at a conference

Scaling Optimization-Oriented Hypernetworks for Implicit Neural Representations

Scaling Optimization-Oriented Hypernetworks for Implicit Neural Representations

Lulu Cai, Hanxiang Ren, Siyan Dong, Li Sun, Jiefeng Wu, Youyi Zheng, Yi Ma, Yanchao Yang

An attention-based hypernetwork generates the weights of an implicit neural representation in one forward pass, avoiding per-instance weight fitting at inference.

研究如何扩大优化导向超网络的规模,使网络能够直接生成隐式神经表示模型的参数。

CoRL 2026Conference paper

The Imitator Game: Benchmarking Robot Imitative Ability Beyond Action Prediction

The Imitator Game: Benchmarking Robot Imitative Ability Beyond Action Prediction

Xunzhe Zhou, Yiyang Cai, Fengyi Wang, Ran Ju, Hanxiang Ren, Ruizhe Liu, Yu Zhang, Qian Luo, Feng Chen, Pei Zhou, Yi Ma, Yanchao Yang

用人类示范视频测试机器人能否理解示范意图,并在工具、物体或布局发生变化时完成任务。

arXivPreprint

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

Yunchao Yao, Zhuxiu Xu, Tianqi Zhang, Zixian Liu, Sikai Li, Zhenyu Wei, Feng Chen, Dihong Huang, Kechang Wan, Chenyang Ma, Shuqi Zhao, Shenghua Gao, Masayoshi Tomizuka, Yi Ma, Mingyu Ding

建立模块化灵巧操作基准,比较策略在多种任务、传感条件和机器人硬件之间的泛化能力。

RSSConference paper

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation

Hanxiang Ren, Pei Zhou, Xunzhe Zhou, Yanchao Yang

A hypernetwork turns a language instruction into a task-specific control policy. The resulting policy uses observations to choose the robot’s actions.

用语言指令生成任务专用策略,使机器人在执行时根据当前状态控制动作。

NeurIPS 2026 Workshop on On-Device Intelligence (ODI)Accepted at a workshop

DexHoldem: An Agentic Robotics Benchmark for Dexterous Manipulation in Texas Hold'em

DexHoldem: An Agentic Robotics Benchmark for Dexterous Manipulation in Texas Hold'em

Feng Chen, Tianzhe Chu, Li Sun, Pei Zhou, Zhuxiu Xu, Shenghua Gao, Yuexiang Zhai, Yanchao Yang, Yi Ma

以真实灵巧手执行德州扑克任务,评估机器人在多轮游戏中的视觉感知、策略判断和精细操作。

CoRL 2026Conference paper

Flow Policy Gradients for Robot Control

Flow Policy Gradients for Robot Control

Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa

研究如何利用策略梯度训练基于流模型的连续控制策略,并在机器人控制任务上评估。

CVPRConference paper

CUPID: Generative 3D Reconstruction via Joint Object and Pose Modeling

CUPID: Generative 3D Reconstruction via Joint Object and Pose Modeling

Binbin Huang, Haobin Duan, Yiqun Zhao, Zibo Zhao, Yi Ma, Shenghua Gao

From a single image, CUPID jointly estimates an object’s geometry, texture and camera pose, keeping the reconstructed object aligned with the input view.

联合建模物体三维形状与相机姿态,再利用与像素对齐的图像特征细化重建结果。

RSS 2026Conference paper

Viser: Imperative, Web-based 3D Visualization for Python

Viser: Imperative, Web-based 3D Visualization for Python

Brent Yi, Chung Min Kim, Justin Kerr, Gina Wu, Rebecca Feng, Anthony Zhang, Jonas Kulhanek, Hongsuk Choi, Yi Ma, Matthew Tancik, Angjoo Kanazawa

面向机器人和计算机视觉的 Python 三维可视化工具,支持在浏览器中交互展示三维场景。

Journal of Machine Learning ResearchJournal article

Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions

Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions

Peng Wang, Huijie Zhang, Zekai Zhang, Siyi Chen, Yi Ma, Qing Qu

在低秩高斯混合模型下,将扩散模型训练与子空间聚类联系起来,分析学习低维分布所需的样本量。

Proceedings of the National Academy of SciencesJournal article

Flexible protein-ligand docking with diffusion-based side-chain packing

Runze Zhang, Xinyu Jiang, Duanhua Cao, Zhaokun Wang, Jie Yu, Mingan Chen, Zhehuan Fan, Xiangtai Kong, Jiacheng Xiong, Zimei Zhang, Wei Zhang, Shengkun Ni, Yitian Wang, Minda Liao, Shenghua Gao, Sulin Zhang, Mingyue Zheng

通过扩散模型预测氨基酸侧链构象,处理蛋白质与配体对接时的柔性。

NeurIPS 2025Conference paper

Cameras as Relative Positional Encoding

Cameras as Relative Positional Encoding

Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, Angjoo Kanazawa

将相机内外参与相对位姿引入多视角 Transformer,并比较不同几何位置编码方案。

ICMLConference paper

Simplifying DINO via Coding Rate Regularization

Simplifying DINO via Coding Rate Regularization

Ziyang Wu, Jingyuan Zhang, Druv Pai, Xudong Wang, Chandan Singh, Jianwei Yang, Jianfeng Gao, Yi Ma

A coding-rate regularizer helps prevent collapse when training vision models with self-distillation, simplifying the DINO training procedure.

将编码率正则化引入 DINO 自监督视觉训练,研究如何控制表征坍缩并简化训练流程。

IEEE Transactions on Pattern Analysis and Machine IntelligenceJournal article

Surfel-Based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction From Monocular Videos

Surfel-Based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction From Monocular Videos

Yiqun Zhao, Chenming Wu, Binbin Huang, Yihao Zhi, Chen Zhao, Jingdong Wang, Shenghua Gao

从单目视频重建动态人体,并支持通过高斯逆渲染调整照明。

ICLR 2025Conference paper

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction

Token Statistics Transformer: Linear-Time Attention via Variational Rate Reduction

Ziyang Wu, Tianjiao Ding, Yifu Lu, Druv Pai, Jingyuan Zhang, Weida Wang, Yaodong Yu, Yi Ma, Benjamin Haeffele

The attention mechanism uses token statistics to keep attention cost linear as the sequence grows.

从变分编码率约减推导基于词元统计量的注意力机制,将计算量降至随序列长度线性增长。

NeurIPSConference paper

Scaling White-Box Transformers for Vision

Scaling White-Box Transformers for Vision

Jinrui Yang, Xianhang Li, Druv Pai, Yuyin Zhou, Yi Ma, Yaodong Yu, Cihang Xie

The authors scale up CRATE on visual recognition tasks and study how changes to its architecture and training affect accuracy.

研究如何扩展用于视觉任务的白盒 Transformer。

CPALConference paper

Unsupervised Learning of Structured Representation via Closed-Loop Transcription

Unsupervised Learning of Structured Representation via Closed-Loop Transcription

Shengbang Tong, Xili Dai, Yubei Chen, Mingyang Li, Zengyi Li, Brent Yi, Yann LeCun, Yi Ma

A closed-loop training objective learns features from unlabeled images that can support both classification and image generation.

通过闭环转录,在无监督条件下学习具有结构的表征。

J. Mach. Learn. ResJournal article

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Hao Bai, Yuexiang Zhai, Benjamin D. Haeffele, Yi Ma

The expanded journal study develops the theory behind CRATE and tests the architecture in both encoding and decoding tasks across images and text.

从稀疏编码率约减出发推导可解释的白盒 Transformer。

NeurIPSConference paper

White-Box Transformers via Sparse Rate Reduction

White-Box Transformers via Sparse Rate Reduction

Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin D. Haeffele, Yi Ma

Alternating compression and sparsification steps yield a transformer-like block and a mathematical interpretation of its attention and feed-forward operations.

依据稀疏编码率约减原理构建白盒 Transformer。

EntropyJournal article

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

Xili Dai, Shengbang Tong, Mingyang Li, Ziyang Wu, Michael Psenka, Kwan Ho Ryan Chan, Pengyuan Zhai, Yaodong Yu, Xiaojun Yuan, Heung-Yeung Shum, Yi Ma

An encoder and a decoder form a loop. The model compares features before and after reconstruction to learn a separate linear subspace for each class.

通过编码率约减的极小极大优化,将数据闭环转录为线性判别表征。

J. Mach. Learn. ResJournal article

ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction

ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction

Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, Yi Ma

ReduNet builds each network layer from an optimization step for coding rate reduction, giving the layer’s operation an explicit mathematical interpretation.

从最大编码率约减原理推导可解释的白盒深度网络 ReduNet。

CVPRConference paper

Prior Based Human Completion

Zibo Zhao, Wen Liu, Yanyu Xu, Xianing Chen, Weixin Luo, Lei Jin, Bohui Zhu, Tong Liu, Binqiang Zhao, Shenghua Gao

利用人体先验补全图像中缺失的人体部分。

NeurIPSConference paper

Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction

Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction

Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, Yi Ma

The objective measures how much more compact the data become when each class is encoded separately, encouraging features that distinguish classes and retain variation within them.

通过最大编码率约减学习兼具多样性与判别性的表征。

Proceedings of the ACM on Computer Graphics and Interactive TechniquesJournal article

Deep Surface Light Fields

Anpei Chen, Minye Wu, Yingliang Zhang, Nianyi Li, Jie Lu, Shenghua Gao, Jingyi Yu

用深度模型表示表面光场。

The International Journal of Medical Robotics and Computer Assisted SurgeryJournal article

Intraocular robotic interventional surgical system (IRISS): Semi‐automated OCT‐guided cataract removal

Cheng‐Wei Chen, Yu‐Hsiu Lee, Matthew J. Gerber, Harrison Cheng, Yan‐Chao Yang, Andrea Govetto, Anibal Andrés Francone, Stefano Soatto, Warren S. Grundfest, Jean‐Pierre Hubschman, Tsu‐Chin Tsao

将光学相干断层成像与眼内手术机器人结合,研究半自动白内障去除。

CVPRConference paper

Seeing through the blur

Hossein Mobahi, C. Lawrence Zitnick, Yi Ma

为仿射及单应变换下的图像配准推导平滑目标函数的模糊核。

1st IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis (HLK)Workshop paper

Symmetry-Based Photo Editing

Kun Huang, Wei Hong, Yi Ma

研究如何借助几何对称性进行照片编辑。

Further reading