Jingxi Xu 许靖西

jxu at cs dot columbia dot edu

PhD Student, Department of Computer Science
Member, Robotic Manipulation and Mobility (ROAM) Lab
Columbia University

Member, Robotics and Embodied Artificial Intelligence (REAL) Lab
Stanford University

Google Scholar / GitHub / X (Twitter) / YouTube

I am a fourth-year PhD student in Computer Science at Columbia University, co-advised by Professor Matei Ciocarlie and Professor Shuran Song. I received a Bachelor's degree from Edinburgh. Prior to joining Columbia, I also spent some time at University of Pennsylvania, with Dinesh Jayaraman and Nikolai Matni; MIT, with Leslie Pack Kaelbling and Tomás Lozano-Pérez; and Columbia with Peter Allen. My research interest is at the intersection of robotics, machine learning and computer vision.

Mentoring

I am always interested in mentoring highly-motivated undergrads/masters. If you are at Columbia (preferred) and want to discuss project ideas/research opportunities, please feel free to shoot me an email.

I currently have the luck to work with: Luke Hsu, Pedro Leandro La Rotta, Yolanda Zhu, Yuan Qing.

Students that I have mentored in the past: Han Lin (now CS PhD at UNC), Yinsen Jia (now EE PhD at Duke), Wenxi Chen (now ME PhD at Purdue), Patrick Meng (now CS PhD at Rutgers), Zihan Guo (now ME PhD at Northwestern), Runsheng Wang (now ME PhD at Columbia), Dongixao Yang (now robotics engineer at Xiaomi), Siqi Shang (now ECE PhD at UT Autstin).

Highlights

Retrieve burried objects with tactile sensors.

Generative AI for rehabilitation robots.

How to grasp moving objects?

Learning a tactile exploration policy.

Visual navigation.

Learning-based motion planning.

News

[2024-06] I will give a talk at the RSS workshop: Noosphere: Tactile Sensing for General Purpose Robot Learning.
[2024-05] I received the Computer Science Service Award from Columbia University.
[2024-03] Excited to join the Boston Dynamics AI Institute as a summer research intern, working with the Foundation Models team!
[2024-03] I gave a guest lecture on The Practice of Robot Learning at Matei Ciocarlie's Robot Learning class. The video is now on YouTube.
[2024-02] GEOTACT is in the IEEE Spectrum Video Friday roundup.
[2023-09] I will give a talk at the Global Perspectives on Medicine, Rehabilitation and Robotics Webinar.
[2023-02] I will give a guest lecture on Tactile Sensing and Its Applications at Shuran Song's Computational Aspects of Robotics course.
[2022-08] Our paper on thumb stabilization is selected as Best Paper Finalist at BioRob 2022.
[2022-03] Our work on developing a hand orthosis to help stroke patients reuse their hands is now featured on the latest issue of Columbia Engineering magazine: Better Living with Robots.
[2021-12] We released our dynamic grasping code. Check it out if you want some dynamic manipulation tasks to benchmark your algorithms.
[2019-04] I am deeply honored to receive the Andrew P. Kosoresow Memorial Award for Excellence in Teaching and Service from the Computer Science Department.
[2018-08] I received the CA Fellowship from Columbia University.
[2017-06] I started an internship at arm, Cambridge, UK, working on CPU pipeline visualization.
[2017-05] I graduated from the University of Edinburgh with First Class Honours and Class Medal (overall grade ranks first in class).

Publications

*: indicating equal contribution or alphabetic ordering. You can also check my Google Scholar profile.

2024

ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

Jingxi Xu*, Runsheng Wang*, Siqi Shang*, Ava Chen, Lauren Winterbottom, To-Liang Hsu, Wenxi Chen, Khondoker Ahmed, Pedro Leandro La Rotta, Xinyue Zhu, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Preprint

[arXiv] [Webpage] [Video]

Tactile-based Object Retrieval From Granular Media

Jingxi Xu*, Yinsen Jia*, Dongxiao Yang*, Patrick Meng, Xinyue Zhu, Zihan Guo, Shuran Song, Matei Ciocarlie

Preprint

[arXiv] [Webpage] [Video]

Media Coverage: [IEEE Spectrum Video Friday]

Meta-Learning for Fast Adaptation in Intent Inferral on a Robotic Hand Orthosis for Stroke

Pedro Leandro La Rotta*, Jingxi Xu*, Ava Chen, Lauren Winterbottom, Wenxi Chen, Dawn Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Intelligent Robots and Systems (IROS) 2024

[arXiv]

Dynamic Grasping with a Learned Meta-controller

Yinsen Jia*, Jingxi Xu*, Dinesh Jayaraman, Shuran Song

🏆 NeurIPS Registration Award

5th Robot Learning Workshop at Neural Information Processing Systems (NeurIPS) 2022

International Conference on Automation Science and Engineering (CASE) 2024

[arXiv] [Webpage]

Volitional Control of the Paretic Hand Post-Stroke Increases Finger Stiffness and Resistance to Robot-Assisted Movement

Ava Chen*, Katelyn Lee*, Lauren Winterbottom, Jingxi Xu, Connor Lee, Grace Munger, Alexandra Deli-Ivanov, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2024

[arXiv]

An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

Eric T. Chang*, Runsheng Wang*, Peter Ballentine, Jingxi Xu, Trey Smith, Brian Coltin, Ioannis Kymissis, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2024

[arXiv]

Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation

Shutaro Aoyama, Jen-Shuo Liu, Portia Wang, Shreeya Jain, Xuezhen Wang, Jingxi Xu, Shuran Song, Barbara Tversky, Steven Feiner

Conference on Virtual Reality and 3D User Interfaces (VR) 2024

[IEEEXplore] [Video]

Collaboration between Occupational Therapists, Engineers, and People with Neurological Conditions in the Development of Wearable Robotic Devices

Lauren Winterbottom, Dawn M. Nilsen, Rochelle Mendonca, Ava Chen, Sara Lin, Kevin Carroll, Jingxi Xu, Matei Ciocarlie, and Joel Stein

American Occupational Therapy Association (AOTA) INSPIRE Conference 2024

[Poster]

2023

TANDEM3D: Active Tactile Exploration for 3D Object Recognition

Jingxi Xu*, Han Lin*, Shuran Song, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2023

[arXiv] [Webpage]

2022

TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors

Jingxi Xu, Shuran Song, Matei Ciocarlie

Robotics and Automation Letters (RA-L)

International Conference on Intelligent Robots and Systems (IROS) 2022

[arXiv] [Webpage]

Thumb Stabilization and Assistance in a Robotic Hand Orthosis for Post-Stroke Hemiparesis

Ava Chen, Lauren Winterbottom, Sangwoo Park, Jingxi Xu, Dawn Nilsen, Joel Stein, Matei Ciocarlie

🏆 Best Paper Finalist

Robotics and Automation Letters (RA-L)

International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2022

[arXiv]

Adaptive Semi-Supervised Intent Inferral to Control a Powered Hand Orthosis for Stroke

Jingxi Xu, Cassie Meeker, Ava Chen, Lauren Winterbottom, Michaela Fraser, Sangwoo Park, Lynne M. Weber, Mitchell Miya, Dawn Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2022

[arXiv] [Webpage] [Poster]

2021

Dynamic Grasping with Reachability and Motion Awareness

Iretiayo Akinola*, Jingxi Xu*, Shuran Song, Peter Allen

International Conference on Intelligent Robots and Systems (IROS) 2021

[arXiv] [Webpage]
How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?

Jingxi Xu, Bruce Lee, Nikolai Matni, Dinesh Jayaraman

Learning for Dynamics & Control Conference (L4DC) 2021

[arXiv] [Webpage] [Poster]

2020

Learning a Decentralized Multi-Arm Motion Planner

Huy Ha, Jingxi Xu, Shuran Song

Conference on Robot Learning (CoRL) 2020

[arXiv] [Webpage]
Learning Your Way Without Map or Compass: Panoramic Target Driven Visual Navigation

David Watkins*, Jingxi Xu*, Nicholas Waytowich, Peter Allen

International Conference on Intelligent Robots and Systems (IROS) 2020

[arXiv] [Webpage]
Accelerated Robot Learning via Human Brain Signals

Iretiayo Akinola*, Zizhao Wang*, Junyao Shi, Xiaomin He, Pawan Lapborisuth, Jingxi Xu, David Watkins, Paul Sajda, Peter Allen

International Conference on Robotics and Automation (ICRA) 2020

[arXiv] [Webpage]

2019

Active Multitask Learning with Committees

Jingxi Xu, Tang Da, Tony Jebara

🏆 Travel Award

Adaptive and Multi-Task Learning Workshop at the International Conference on Machine Learning (ICML) 2019

[arXiv] [Poster]

Teaching

Teaching assistant at Columbia University for:

Robot Learning (MECE6616) - Spring 2023
Robot Learning (MECE6616) - Spring 2021
Computational Aspects of Robotics (COMS4733) - Fall 2019
Humanoid Robots (COMS6731) - Spring 2019
Computational Aspects of Robotics (COMS4733) - Fall 2018
Machine Learning (COMS4771) - Spring 2018

Talks

The Practice of Robot Learning I (video)
- Guest lecture at course Robot Learning (MECE6616), Columbia University, 02/20/2024
- Guest lecture at course Robot Learning (MECE6616), Columbia University, 01/31/2023
- Guest lecture at course Robot Learning (MECE6616), Columbia University, 02/08/2022
Robotic Hand Orthoses for Assistance and Rehabilitation After Stroke
- Global Perspectives on Medicine, Rehabilitation and Robotics Webinar Series, 09/20/2023
Tactile Sensing and Its Applications
- Guest lecture at course Computational Aspects of Robotics (COMS4733), Columbia University, 03/24/2023
- Guest lecture at course Computational Aspects of Robotics (COMS4733), Columbia University, 02/25/2022
The Practice of Robot Learning II
- Guest lecture at course Robot Learning (MECE6616), Columbia University, 04/28/2022
MyHand: a Wearable Hand Orthosis for Stroke
- Invited talk at Challenges and Opportunities of Human-robot Symbiosis: from Wearable Robots to Neurorobotics Workshop, IROS 2021, 10/01/2021 (Co-give with Ava Chen, Matei Ciocarlie, video)
Dynamic Grasping with Reachability and Motion Awareness

Invited talk at AIRLab, Lehigh University, 09/27/2021 (video)
Invited talk at Professor Dinesh Jayaraman's group, University of Pennsylvania, 07/14/2021
Invited talk at Professor Yi Ma's group, University of California Berkeley, 04/09/2021

ROS Tutorial

An introductory tutorial on ROS and use of robotics in the Columbia Robotics Lab, 02/08/2019

Introduction to Robotics

An invited lecture on robotics to summer school students at MIT, 08/21/2018

Projects

Deep Temporal Multi-scale Net for Apparent Personality Analysis - advised by Shih-Fu Chang
Integrate Eye-tracking with BCI System for Assistive Robots - advised by Peter Allen
Hybrid HW/SW CPU Simulation using Zync - advised by Nigel Topham and Boris Grot

Misc

I am an academic descendent of Issac Newton, according to this website that tracks the academic genealogy. Here is the specific branch.
During my non-existent free time, I enjoy swimming, snowboarding, piano, billiards, basketball, hiking, travelling, and photography.
I used to work on computer architecture during my undergraduate years at Edinburgh. I got accepted to DPhil in Computer Science at Oxford University with full funding (Jason Hu Scholarship from Balliol College) when I graduated from Edinburgh. However, in order to explore AI/robotics a bit more than staying in computer architecture for my PhD, I decided to do a master first at Columbia to gain more research experience in AI/robotics.
I am a supporter of and allergic to 500-pound websites. I am consistently making efforts to make my website as simple as possible while still being responsive on different devices. This website is built with a single css and html file.
I am a supporter of Slow Science.
I am originally from Zhuhai, Guangdong, China. I understand Cantonese.