Deep reinforcement learning (DRL) has shown immense potential in robotic learning, such as locomotion [1] and detexterous manipulation [1]. Nevertheless, the DRL policies are mainly trained in simuation, and transferring policies from simulation to the real world (sim-to-real) remains a significant challenge due to discrepancies between simulated and physical environments. Digital twins offer a promising solution by creating highly accurate virtual models that facilitate robust DRL training and adaptation for real-world deployment. In this project, we aim to build a high-fidelity digital twin using Nvidia Issac Sim [3] that provides diverse and realistic scenarios for training DRL policy, enabing zero-shot sim-to-real generalization.

Main Objectives:

• Literature research on digital twin for locomotion
• Setup quadruped robot in Nvidia Issac Sim
• Interface the quadruped robot in simulation and real world.
• Interface Vicon system and mapping real-world environment into simulation.
• Demonstrate the realism of the digital twin in a path following task: the path is generated by a path planner for a virtual scenario in simulation; compare the path following performance of both simulated and real robot to understand the sim-to-real gap.
• Using real world data to improve the simulation and refine pre-trained policy (Optional)

 

Deliverables:

• Clean and optimized workflow for mapping real world robot into Simulation
• Reusable and documented code base
• Master thesis

 

[1] Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

[2] Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

[3] developer.nvidia.com/isaac/sim

 

Mandatory:

• Programming skills in Python
• Proficient background in robotics, control systems
• Proficient background in Linux system
• Background in robot simulation, such as Issac, Gazebo, Pybullet

 

Optional:

• Familiar with ROS2
• Familiar with Deep learning frameworks, such as Tensorflow, Pytorch