[Bug]: SO101 unable to pick up elastic object

Author: johnsutorCreated Oct 14, 2025Updated Aug 22, 2026
LabelsbugP0

Bug Description

I am working on a project involving teleoperation of a SO101 robot. I am using the Mujoco file as-is from the original repository. I can pick up a rigid cube just fine. However, I am unable to pick up an elastic cube as the grippers pass right through the object. I have tried a few different fixes to no avail, including

  • Making the cube denser
  • Using a larger target decimation face number
  • Using all other solvers besides MPM
  • Using MPM with ElastoPlastic material

Any guidance or help would be greatly appreciated! Please view the video below for more context.

Steps to Reproduce

python
import numpy as np
import genesis as gs
from lerobot.teleoperators.so101_leader import SO101LeaderConfig, SO101Leader

gs.init(backend=gs.gpu)

scene = gs.Scene(
    sim_options=gs.options.SimOptions(
        dt=1e-2,
        substeps=33,
    ),
    mpm_options=gs.options.MPMOptions(
        lower_bound=(-1.5, -1.5, -0.5),
        upper_bound=(1.5, 1.5, 1.5),
    ),
    rigid_options=gs.options.RigidOptions(
        box_box_detection=True,
    ),
    vis_options=gs.options.VisOptions(
        visualize_mpm_boundary=True,
    ),
    viewer_options=gs.options.ViewerOptions(
        camera_pos=(1, 1, 1),
        camera_lookat=(0.0, 0.0, 0.0),
        camera_fov=30,
        max_FPS=30,
    ),
    show_viewer=True,
)

cam = scene.add_camera(
    res=(1280, 960), pos=(1, 1, 1), lookat=(0.0, 0.0, 0.0), fov=30, GUI=False
)


teleop_config = SO101LeaderConfig(
    port="/dev/ttyACM0",
    id="sutie_leader",
    use_degrees=True,
)
teleop_device = SO101Leader(teleop_config)

plane = scene.add_entity(
    gs.morphs.Plane(),
)

so101 = scene.add_entity(
    gs.morphs.MJCF(
        file="assets/so101_new_calib.xml",
        pos=(0.0, 0.0, 0.0),
        euler=(0, 0, 0),
    ),
)

sponge = scene.add_entity(
    material=gs.materials.MPM.Elastic(),
    morph=gs.morphs.Box(
        size=(0.04, 0.04, 0.04),
        pos=(0.15, 0.15, 0.1),
        euler=(0, 0, 0),
    ),
    surface=gs.surfaces.Default(
        color=(1.0, 0.4, 0.4),
        vis_mode="visual",
    ),
)

joints = [
    "shoulder_pan",
    "shoulder_lift",
    "elbow_flex",
    "wrist_flex",
    "wrist_roll",
    "gripper",
]

dof_indices = [so101.get_joint(name).dof_idx for name in joints]

scene.build()
teleop_device.connect()


def convert_action_to_genesis(action):
    return np.array(
        [
            np.deg2rad(action["shoulder_pan.pos"]),
            np.deg2rad(action["shoulder_lift.pos"]),
            np.deg2rad(action["elbow_flex.pos"]),
            np.deg2rad(action["wrist_flex.pos"]),
            np.deg2rad(action["wrist_roll.pos"]),
            np.deg2rad(action["gripper.pos"]),
        ]
    )


cam.start_recording()

try:
    while True:
        action = teleop_device.get_action()
        dof_positions = convert_action_to_genesis(action)
        so101.control_dofs_position(dof_positions, dof_indices)
        cam.render()
        scene.step()
except Exception as e:
    print(f"Error: {e}")
finally:
    cam.stop_recording("./so101_teleop.mp4")

Expected Behavior

It should be able to grip and pick up the cube, with the cube deforming.

Screenshots/Videos

https://github.com/user-attachments/assets/c7f02a4c-1726-43bb-82f7-637b39ced1dc

Relevant log output

bash

Environment

OS: Ubuntu 25.04 GPU: RTX 5090 CUDA Version: 12.8, CUDA Driver Version: 570.172.08 CPU: AMD 9950X Package Manager: uv Installed Packages: "genesis-world>=0.3.4", "lerobot[feetech]>=0.3.2", "ruff>=0.14.0", "torch>=2.8.0",

Release version or Commit ID

genesis-world>=0.3.4

Additional Context

No response

Source: Genesis-Embodied-AI/genesis-world