The first workshop on

4D Hand Object Interaction:

Geometric Understanding and Applications in Dexterous Manipulation

June 18, CVPR 2023 Partially Virtual


The whole workshop video is here.


Hand-object interactions (HOI) feature regularly in our daily activities in which we use one hand or two to manipulate objects directly or to use various tools. Through HOI, humans excel in manipulation tasks, master new skills, and adapt to complex and continuously changing environments. Recently, there is a surge of interest in understanding and generating hand-object interaction, to support applications in robotics, augmented reality, and other important fields. Such applications usually require a detailed understanding of hands dynamically interacting with a wide range of objects in 4D (3D space + 1D time), and struggle to synthesize or execute dexterous manipulations in 4D at the same level of skill as humans.

The key goal of this workshop is to assemble experts in 4D vision, hand-object interaction, dexterous manipulation, and animation synthesis to synchronize and coordinate the efforts. First of all, the workshop would help researchers working on 4D HOI understanding to know about dexterous manipulation and tailor their research topics toward interaction-oriented 4D understanding. Secondly, embodied AI researchers could improve their understanding of the limitation of current 4D HOI perception and design their methods properly. Finally, in the workshop we will also host a competition designed for benchmarking the progress of 4D HOI geometric understanding, to invite more researchers to the field.

We have invited a few leading researchers with extensive related experience to make keynote presentations at our workshop. We will also organize a competition to benchmark the current progress of 4D HOI geometric understanding and to also encourage more researchers to join the field. We will organize a panel discussion at the conclusion of our full-day workshop. We envision that such discussions will inspire people and stimulate novel research for 4D hand-object interaction.


In this workshop, we will encourage participating researchers to focus on topics below:

HOI4D Challenge Timeline (11:59 PM Pacific Standard Time)

We host an online leaderboard( through which participants could indicate the track they are working on and submit their results. We will evaluate the submissions on our server and update the leaderboard accordingly

01-20Announce HOI4D Challenge and call for participants
02-01Challenge Begin
06-05Challenge Deadline
06-10Notification to Authors

Workshop Schedule (PDT)

Start End Event Speaker
1:00pm1:10pmOpening Remark
1:10pm1:20pmHOI4D Challenge overview
1:20pm1:40pmHOI4D Challenge winner talks
1:40pm2:10pmInvited TalkJitendra Malik
2:10pm2:40pmHandling Complex Hand-Object ManipulationsVincent Lepetit
2:40pm3:10pmToward a 3D Understanding of Complex Hand InteractionsZicong Fan
3:10pm4:00pmStudent talk
4:00pm4:30pmUnderstanding Hand-Object Interaction in Mixed-RealityMarc Pollefeys
4:30pm5:00pmThree Lessons from Teaching DexterityLerrel Pinto
5:00pm5:30pm4D Reconstruction of Human Interaction MotionFeng Xu


Li Yi
Tsinghua University
He Wang
Peking University
Xiaolong Wang
UC San Diego
Yu-Wei Chao
David Fouhey
University of Michigan
Sifei Liu
Yunze Liu
Tsinghua University
Hao Su
UC San Diego
Shubham Tulsiani


© 2023 4D Hand Object Interaction: Geometric Understanding and Applications in Dexterous Manipulation