Kuan Fang

Assistant Professor
Department of Computer Science
Cornell University
Email: kuanfang [at] cornell [dot] edu
Office: Room 425, Gates Hall

| |


I am an Assistant Professor of Computer Science at Cornell University. Previously, I was a postdoc at UC Berkeley, working with Sergey Levine, and a researcher at Boston Dynamics AI Institute. I received a Ph.D. and M.S. from Stanford University, advised by Fei-Fei Li and Silvio Savarese, and a bachelor's degree from Tsinghua University. I have also spent time at Google Brain, Google X Robotics, and Microsoft Research Asia.

Prospective students: I am actively seeking motivated students and postdocs of relevant backgrounds to join our lab. If you are interested in working with me, please review the information on this page before reaching out.

Research

I conduct research at the intersection of robotics, machine learning, and computer vision. My research aims to enable robots to perform diverse and complex tasks in unstructured environments using deep learning. To achieve this, my lab and I develop scalable algorithms and systems for robot perception and control with the following focuses:

News

Teaching

Publications

(*: equal contribution, : equal advising)

Blox-Net: Generative Design-for-Robot-Assembly using VLM Supervision, Physics Simulation, and A Robot with Reset
Andrew Goldberg, Kavish Kondap, Tianshuang Qiu, Zehan Ma, Letian Fu, Justin Kerr, Huang Huang, Kaiyuan Chen, Kuan Fang, Ken Goldberg
arXiv Preprint

KALIE: Fine-Tuning Vision-Language Models for Open-World Manipulation without Robot Data
Grace Tang*, Swetha Rajkumar*, Yifei Zhou, Homer Rich Walke, Sergey Levine, Kuan Fang
arXiv Preprint

Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation
Xuanlin Li, Tong Zhao, Xinghao Zhu, Jiuguang Wang, Tao Pang, Kuan Fang
arXiv Preprint

Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
Vivek Myers*, Bill Chunyuan Zheng*, Oier Mees, Sergey Levine, Kuan Fang
Conference on Robot Learning (CoRL) 2024

Jacta: A Versatile Planner for Learning Dexterous and Whole-Body Manipulation
Jan Bruedigam, Ali Adeeb Abbas, Maks Sorokin, Kuan Fang, Brandon Hung, Maya Guru, Stefan Georg Sosnowski, Jiuguang Wang, Sandra Hirche, Simon Le Cleac'h
Conference on Robot Learning (CoRL) 2024

Affordance-Guided Reinforcement Learning via Visual Prompting
Olivia Y. Lee, Annie Xie, Kuan Fang, Karl Pertsch, Chelsea Finn
RSS Lifelong Robot Learning Workshop 2024

MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
Kuan Fang*, Fangchen Liu*, Pieter Abbeel, Sergey Levine
Robotics: Science and Systems (RSS) 2024

Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Open X-Embodiment Collaboration
International Conference on Robotics and Automation (ICRA) 2024

Stabilizing Contrastive RL: Techniques for Offline Goal Reaching
Chongyi Zheng, Benjamin Eysenbach, Homer Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, Sergey Levine
International Conference on Learning Representations(ICLR) 2024

Multi-Stage Cable Routing Through Hierarchical Imitation Learning
Jianlan Luo*, Charles Xu*, Xinyang Geng*, Gilbert Feng, Kuan Fang, Liam Tan, Stefan Schaal, Sergey Levine
IEEE Transactions on Robotics (T-RO) 2024

Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control
Vivek Myers*, Andre He*, Kuan Fang, Homer Walke, Philippe Hansen-Estruch, Ching-An Cheng, Mihai Jalobeanu, Andrey Kolobov, Anca Dragan, Sergey Levine
Conference on Robot Learning (CoRL) 2023

BridgeData V2: A Dataset for Robot Learning at Scale
Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, Sergey Levine
Conference on Robot Learning (CoRL) 2023

Active Task Randomization: Learning Robust Skills Via Unsupervised Generation of Diverse and Feasible Tasks
Kuan Fang*, Toki Migimatsu*, Ajay Mandlekar, Li Fei-Fei, Jeannette Bohg
International Conference on Intelligent Robots and Systems (IROS) 2023

Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Tasks
Kuan Fang, Patrick Yin, Ashvin Nair, Homer Walke, Gengchen Yan, Sergey Levine

Conference on Robot Learning (CoRL) 2022 (Oral Presentation)

Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space
Kuan Fang*, Patrick Yin*, Ashvin Nair, Sergey Levine
International Conference on Intelligent Robots and Systems (IROS) 2022

Discovering Generalizable Skills via Automated Generation of Diverse Tasks
Kuan Fang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
Robotics: Science and Systems (RSS) 2021

Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations
Zhenyu Jiang, Yifeng Zhu, Maxwell Svetlik, Kuan Fang, Yuke Zhu,
Robotics: Science and Systems (RSS) 2021

Adaptive Procedural Task Generation for Hard-Exploration Problems
Kuan Fang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
International Conference on Learning Representations(ICLR) 2021

KETO: Learning Keypoint Representations for Tool Manipulation
Zengyi Qin, Kuan Fang, Yuke Zhu, Li Fei-Fei, Silvio Savarese
International Conference on Robotics and Automation (ICRA) 2020

Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation
Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei

Conference on Robot Learning (CoRL) 2019 (Oral Presentation)

Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Kuan Fang, Alexander Toshev, Li Fei-Fei, Silvio Savarese
Conference on Computer Vision and Pattern Recognition (CVPR) 2019

Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, Silvio Savarese
Robotics: Science and Systems (RSS) 2018
International Journal of Robotics Research (IJRR) 2019

Multi-Task Domain Adaptation for Deep Learning of Instance Grasping from Simulation
Kuan Fang, Yunfei Bai, Stefan Hinterstoisser, Silvio Savarese, Mrinal Kalakrishnan
International Conference on Robotics and Automation (ICRA) 2018

Demo2Vec: Reasoning Object Affordances from Online Videos
Kuan Fang*, Te-Lin Wu*, Daniel Yang, Silvio Savarese, Joseph J. Lim
Conference on Computer Vision and Pattern Recognition (CVPR) 2018

Recurrent Autoregressive Networks for Online Multi-Object Tracking
Kuan Fang, Yu Xiang, Xiaocheng Li, Silvio Savarese
Winter Conference on Applications of Computer Vision (WACV) 2018

DeLay: Robust Spatial Layout Estimation for Cluttered Indoor Scenes
Saumitro Dasgupta, Kuan Fang*, Kevin Chen*, Silvio Savarese
Conference on Computer Vision and Pattern Recognition (CVPR) 2016