Jingxi Xu 许靖西

jxu at cs dot columbia dot edu

PhD Student, Department of Computer Science
Member, Robotic Manipulation and Mobility (ROAM) Lab
Columbia University

Member, Robotics and Embodied Artificial Intelligence (REAL) Lab
Stanford University

Google Scholar / GitHub / X (Twitter) / YouTube

I am a fifth-year Ph.D. in Computer Science at Columbia University, co-advised by Professor Matei Ciocarlie and Shuran Song. I received my bachelor’s degree from Edinburgh, with First Class Honours and Class Medal (overall grade ranks first in class), and I received an MS from Columbia, where I worked closely with Professor Peter Allen. I also spent some time as a research intern at Boston Dynamics AI Institute and as a visiting student at MIT (with Leslie Pack Kaelbling and Tomás Lozano-Pérez) and UPenn (with Dinesh Jayaraman and Nikolai Matni).

My research focuses are robotics, machine learning and healthcare. My most recent line of work [RA-L'24, IROS'24, ICRA'22] builds wearable assistive and rehabilitation robots that use machine learning to predict user intent from multimodal biosignals for people with disabilities. I also develop efficient exploration and manipulation policies with tactile sensing [arXiv'24, ICRA'23, RA-L'22]. At the same time, I am a multifaceted roboticist with research experience in a variety of applications such as dynamic grasping [CASE'24, IROS'21], visual navigation [IROS'20], motion planning [CoRL'20], optimal control [L4DC'21], and brain-computer interface [ICRA'20].

Highlights

Retrieve burried objects with tactile sensors.

Generative AI for rehabilitation robots.

How to grasp moving objects?

Learning a tactile exploration policy.

Visual navigation.

Learning-based motion planning.

News

[2024-11] ChatEMG is accepted at RA-L.
[2024-10] I gave a talk at the Stanford Vision and Learning Lab (SVL), hosted by Fei-Fei Li.
[2024-06] Excited to join the Boston Dynamics AI Institute as a summer research intern.
[2024-05] I received the Computer Science Service Award from Columbia University.
[2024-02] GEOTACT is in the IEEE Spectrum Video Friday roundup.
[2022-08] Our paper on thumb stabilization is selected as Best Paper Finalist at BioRob 2022.
[2022-03] Our work on developing a hand orthosis to help stroke patients reuse their hands is now featured on the latest issue of Columbia Engineering magazine: Better Living with Robots.
[2019-04] I am deeply honored to receive the Andrew P. Kosoresow Memorial Award for Excellence in Teaching and Service from the Computer Science Department.
[2018-08] I received the CA Fellowship from Columbia University.
[2017-06] I started an internship at arm, Cambridge, UK, working on CPU pipeline visualization.
[2017-05] I graduated from the University of Edinburgh with First Class Honours and Class Medal (overall grade ranks first in class).

Publications

*: indicating equal contribution or alphabetic ordering. You can also check my Google Scholar profile.
(Co-)First-author papers are highlighted.

2024

Tactile-based Object Retrieval From Granular Media

Jingxi Xu*, Yinsen Jia*, Dongxiao Yang*, Patrick Meng, Xinyue Zhu, Zihan Guo, Shuran Song, Matei Ciocarlie

Preprint, under review by Science Robotics

[arXiv] [Webpage] [Video]

Media Coverage: [IEEE Spectrum Video Friday]
ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

Jingxi Xu*, Runsheng Wang*, Siqi Shang*, Ava Chen, Lauren Winterbottom, To-Liang Hsu, Wenxi Chen, Khondoker Ahmed, Pedro Leandro La Rotta, Xinyue Zhu, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Robotics and Automation Letters (RA-L)

[arXiv] [Webpage] [Video]
Meta-Learning for Fast Adaptation in Intent Inferral on a Robotic Hand Orthosis for Stroke

Pedro Leandro La Rotta*, Jingxi Xu*, Ava Chen, Lauren Winterbottom, Wenxi Chen, Dawn Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Intelligent Robots and Systems (IROS) 2024

[arXiv]
Dynamic Grasping with a Learned Meta-controller

Yinsen Jia*, Jingxi Xu*, Dinesh Jayaraman, Shuran Song

🏆 NeurIPS Registration Award

5th Robot Learning Workshop at Neural Information Processing Systems (NeurIPS) 2022

International Conference on Automation Science and Engineering (CASE) 2024

[arXiv] [Webpage]
Volitional Control of the Paretic Hand Post-Stroke Increases Finger Stiffness and Resistance to Robot-Assisted Movement

Ava Chen*, Katelyn Lee*, Lauren Winterbottom, Jingxi Xu, Connor Lee, Grace Munger, Alexandra Deli-Ivanov, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2024

[arXiv]
An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

Eric T. Chang*, Runsheng Wang*, Peter Ballentine, Jingxi Xu, Trey Smith, Brian Coltin, Ioannis Kymissis, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2024

[arXiv]
Asynchronously Assigning, Monitoring, and Managing Assembly Goals in Virtual Reality for High-Level Robot Teleoperation

Shutaro Aoyama, Jen-Shuo Liu, Portia Wang, Shreeya Jain, Xuezhen Wang, Jingxi Xu, Shuran Song, Barbara Tversky, Steven Feiner

Conference on Virtual Reality and 3D User Interfaces (VR) 2024

[IEEEXplore] [Video]
Collaboration between Occupational Therapists, Engineers, and People with Neurological Conditions in the Development of Wearable Robotic Devices

Lauren Winterbottom, Dawn M. Nilsen, Rochelle Mendonca, Ava Chen, Sara Lin, Kevin Carroll, Jingxi Xu, Matei Ciocarlie, and Joel Stein

American Occupational Therapy Association (AOTA) INSPIRE Conference 2024

[Poster]

2023

TANDEM3D: Active Tactile Exploration for 3D Object Recognition

Jingxi Xu*, Han Lin*, Shuran Song, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2023

[arXiv] [Webpage]

2022

TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors

Jingxi Xu, Shuran Song, Matei Ciocarlie

Robotics and Automation Letters (RA-L)

International Conference on Intelligent Robots and Systems (IROS) 2022

[arXiv] [Webpage]
Thumb Stabilization and Assistance in a Robotic Hand Orthosis for Post-Stroke Hemiparesis

Ava Chen, Lauren Winterbottom, Sangwoo Park, Jingxi Xu, Dawn Nilsen, Joel Stein, Matei Ciocarlie

🏆 Best Paper Finalist

Robotics and Automation Letters (RA-L)

International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2022

[arXiv]
Adaptive Semi-Supervised Intent Inferral to Control a Powered Hand Orthosis for Stroke

Jingxi Xu, Cassie Meeker, Ava Chen, Lauren Winterbottom, Michaela Fraser, Sangwoo Park, Lynne M. Weber, Mitchell Miya, Dawn Nilsen, Joel Stein, Matei Ciocarlie

International Conference on Robotics and Automation (ICRA) 2022

[arXiv] [Webpage] [Poster]

2021

Dynamic Grasping with Reachability and Motion Awareness

Iretiayo Akinola*, Jingxi Xu*, Shuran Song, Peter Allen

International Conference on Intelligent Robots and Systems (IROS) 2021

[arXiv] [Webpage]
How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?

Jingxi Xu, Bruce Lee, Nikolai Matni, Dinesh Jayaraman

Learning for Dynamics & Control Conference (L4DC) 2021

[arXiv] [Webpage] [Poster]

2020

Learning a Decentralized Multi-Arm Motion Planner

Huy Ha, Jingxi Xu, Shuran Song

Conference on Robot Learning (CoRL) 2020

[arXiv] [Webpage]
Learning Your Way Without Map or Compass: Panoramic Target Driven Visual Navigation

David Watkins*, Jingxi Xu*, Nicholas Waytowich, Peter Allen

International Conference on Intelligent Robots and Systems (IROS) 2020

[arXiv] [Webpage]
Accelerated Robot Learning via Human Brain Signals

Iretiayo Akinola*, Zizhao Wang*, Junyao Shi, Xiaomin He, Pawan Lapborisuth, Jingxi Xu, David Watkins, Paul Sajda, Peter Allen

International Conference on Robotics and Automation (ICRA) 2020

[arXiv] [Webpage]

2019

Active Multitask Learning with Committees

Jingxi Xu, Tang Da, Tony Jebara

🏆 Travel Award

Adaptive and Multi-Task Learning Workshop at the International Conference on Machine Learning (ICML) 2019

[arXiv] [Poster]

Talks

[2024-10-07] "Robot Learning with Sparsity and Scarcity" @ Stanford, hosted by Fei-Fei Li.
[2024-07-19] "Learning Tactile-only Exploration and Manipulation Policies" @ RSS 2024 Workshop: Noosphere: Tactile Sensing for General Purpose Robot Learning.
[2024-02-20] "The Practice of Robot Learning I" (video) @ Columbia, guest lecture hosted by Matei Ciocarlie.
[2023-09-20] "Robotic Hand Orthoses for Assistance and Rehabilitation After Stroke" @ Global Perspectives on Medicine, Rehabilitation and Robotics Webinar Series.
[2023-03-24] "Tactile Sensing and Its Applications" @ Columbia, guest lecture hosted by Shuran Song.
[2023-01-31] "The Practice of Robot Learning I" @ Columbia, guest lecture hosted by Matei Ciocarlie.
[2022-04-28] "The Practice of Robot Learning II" @ Columbia, guest lecture hosted by Matei Ciocarlie.
[2022-02-25] "Tactile Sensing and Its Applications" @ Columbia, guest lecture hosted by Shuran Song.
[2022-02-08] "The Practice of Robot Learning I" @ Columbia, guest lecture hosted by Matei Ciocarlie.
[2021-10-01] "MyHand: a Wearable Hand Orthosis for Stroke" @ IROS 2021 Workshop: Challenges and Opportunities of Human-robot Symbiosis: from Wearable Robots to Neurorobotics Workshop.
[2021-09-27] "Dynamic Grasping with Reachability and Motion Awareness" @ Lehigh, hosted by Mooi Choo Chuah.
[2021-07-14] "Dynamic Grasping with Reachability and Motion Awareness" @ UPenn, hosted by Dinesh Jayaraman.
[2021-04-09] "Dynamic Grasping with Reachability and Motion Awareness" @ Berkeley, hosted by Yi Ma.

Mentoring

I am always interested in mentoring highly-motivated undergrads/masters. If you are at Columbia (preferred) and want to discuss project ideas/research opportunities, please feel free to shoot me an email.

David Choi (CS MS)
Shreenithi Navaneethan (BME MS)
Yuan Qing (MS EE)
Bingyao Du (CS MS)
Yolanda Zhu (Undergrad CS)
Luke Hsu (Undergrad CS)
Pedro Leandro La Rotta (MS ME → Machine Learning Engineer at Accrete AI)
Runsheng Wang (MA Stats → ME PhD at Columbia)
Dongixao Yang (MS ME → Robotics Engineer at Xiaomi)
Siqi Shang (MS CS → ECE PhD at UT Austin)
Zihan Guo (MS ME → ME PhD at Northwestern)
Wenxi Chen (MS ME → ME PhD at Purdue)
Patrick Meng (MS CS → CS PhD at Rutgers)
Yinsen Jia (MS EE → EE PhD at Duke)
Han Lin (MS CS → CS PhD at UNC)

Teaching

Teaching assistant at Columbia University for:

Robot Learning (MECE6616) - Spring 2023
Robot Learning (MECE6616) - Spring 2021
Computational Aspects of Robotics (COMS4733) - Fall 2019
Humanoid Robots (COMS6731) - Spring 2019
Computational Aspects of Robotics (COMS4733) - Fall 2018
Machine Learning (COMS4771) - Spring 2018

Projects

Deep Temporal Multi-scale Net for Apparent Personality Analysis - advised by Shih-Fu Chang
Integrate Eye-tracking with BCI System for Assistive Robots - advised by Peter Allen
Hybrid HW/SW CPU Simulation using Zync - advised by Nigel Topham and Boris Grot

Misc

I am an academic descendent of Issac Newton, according to this website that tracks the academic genealogy. Here is the specific branch.
During my non-existent free time, I enjoy swimming, snowboarding, piano, billiards, basketball, hiking, travelling, and photography.
I used to work on computer architecture during my undergraduate years at Edinburgh. I got accepted to DPhil in Computer Science at Oxford University with full funding (Jason Hu Scholarship from Balliol College) when I graduated from Edinburgh. However, in order to explore AI/robotics a bit more than staying in computer architecture for my PhD, I decided to do a master first at Columbia to gain more research experience in AI/robotics.
I am a supporter of and allergic to 500-pound websites. I am consistently making efforts to make my website as simple as possible while still being responsive on different devices. This website is built with a single css and html file.
I am a supporter of Slow Science.
I am originally from Zhuhai, Guangdong, China. I understand Cantonese.