Simulated virtual environments help to develop and evaluate intelligent agent algorithms. However, most of the existing ones focus on rigid-body dynamics, although soft-body environments can be widely applied. For instance, soft bodies can be used to simulate virtual surgery or develop biomimetic actuators in robotics. Therefore, a recent paper suggests a framework for running and evaluating ten soft-body manipulation tasks.
The tasks require complex operations, including rolling, chopping, or molding. For instance, in one of the tasks, the agent has to wind a rope, modeled as a long plasticine piece, around a rigid pillar with two spherical manipulators. The framework uses differentiable physics and provides analytical gradient information, which can be used in supervised learning with gradient-based optimization. The study allows comparing reinforcement learning and gradient-based planning algorithms.
Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only simulate rigid body physics. Additionally, the simulation process usually does not provide gradients that might be useful for planning and control optimizations. We introduce a new differentiable physics benchmark called PasticineLab, which includes a diverse collection of soft body manipulation tasks. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. The underlying physics engine supports differentiable elastic and plastic deformation using the DiffTaichi system, posing many under-explored challenges to robotic agents. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark. Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning. We expect that PlasticineLab will encourage the development of novel algorithms that combine differentiable physics and RL for more complex physics-based skill learning tasks.
Link to the research article: Huang, Z. et al, “PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics”, 2021.
Link to the project page: https://plasticinelab.csail.mit.edu/