Bi-Directional Sim-to-Real Transfer for GelSight Tactile Sensors with CycleGAN
Published in IEEE Robotics and Automation Letters (RA-L), under review, 2021
GelSight optical tactile sensors have high-resolution and low-cost advantages and have witnessed growing adoption in various contact-rich robotic applications. Sim2Real for GelSight sensors can reduce the time cost and sensor damage during data collection and is crucial for learning-based tactile perception and control. However, it remains difficult for existing simulation methods to resemble the complex and non-ideal light transmission of real sensors. In this paper, we propose to narrow the gap between simulation and real world by using CycleGAN. Due to the bi-directional generators of CycleGAN, the proposed metho can not only generate more realistic simulated tactile images, but also improve the deformation measurement accuracy of real sensors by transferring them to simulation domain. Experiments on a public dataset and our own GelSight sensors have validated the effectiveness of our method. Our code will be released upon acceptance.