[Bug Report] Reacher observation mixes post-step qpos with stale body positions

Author: HadushHailuCreated Sep 9, 2026Updated Sep 12, 2026

Labelsbug

Describe the bug

ReacherEnv._get_obs() appears to construct a single observation from quantities corresponding to different points of the MuJoCo simulation step.

After mj_step, the joint angles are read from the updated data.qpos, while the fingertip position is obtained from MuJoCo's derived body-position data. Since _step_mujoco_simulation() does not call mj_forward after mj_step, these quantities can be slightly inconsistent.

In our tests, this produces a forward-kinematics residual of approximately 6.2e-9. Calling mj_forward after mj_step reduces the residual to approximately 2.8e-17, near machine precision.

The result is that the Reacher observation does not exactly satisfy its expected forward-kinematics relationship when the joints are moving.

Code example

bash

import gymnasium as gym
import mujoco
import numpy as np

L1, L2 = 0.1, 0.11
JOINT_LIMIT = 3.0


def residuals(n=2000, seed=7, do_step=True, zero_qvel=False):
    env = gym.make("Reacher-v5", disable_env_checker=True)
    u = env.unwrapped
    env.reset(seed=0)

    rng = np.random.default_rng(seed)
    errors = []

    # Stay 0.1 rad inside joint1's [-3, 3] limit.
    limit = JOINT_LIMIT - 0.1

    for _ in range(n):
        theta1 = rng.uniform(-np.pi, np.pi)
        theta2 = rng.uniform(-limit, limit)
        target = rng.uniform(-0.15, 0.15, 2)

        if zero_qvel:
            qvel = np.zeros(4)
        else:
            qvel = np.array([
                rng.uniform(-1, 1),
                rng.uniform(-1, 1),
                0.0,
                0.0,
            ])

        u.set_state(
            np.array([theta1, theta2, target[0], target[1]]),
            qvel,
        )

        if do_step:
            env.step(np.zeros(2, dtype=np.float32))
        else:
            mujoco.mj_forward(u.model, u.data)

        t1, t2 = u.data.qpos[:2]
        fingertip = u.data.body("fingertip").xpos

        error = max(
            abs(
                fingertip[0]
                - (L1 * np.cos(t1) + L2 * np.cos(t1 + t2))
            ),
            abs(
                fingertip[1]
                - (L1 * np.sin(t1) + L2 * np.sin(t1 + t2))
            ),
        )

        errors.append(error)

    env.close()
    return np.asarray(errors)


conditions = [
    ("mj_forward only", dict(do_step=False)),
    ("after mj_step", dict(do_step=True)),
    ("step, qvel = 0", dict(do_step=True, zero_qvel=True)),
]

for label, kwargs in conditions:
    errors = residuals(**kwargs)
    print(
        f"{label:>16}  "
        f"median={np.median(errors):.3e}  "
        f"max={errors.max():.3e}"
    )

System info

Gymnasium 1.3.0 MuJoCo 3.10.0 Python 3.10 Ubuntu 24.04

Additional context

Running the minimal reproduction above gives:

mj_forward only median=2.082e-17 max=8.327e-17 after mj_step median=6.199e-09 max=2.274e-08 step, qvel = 0 median=2.776e-17 max=1.110e-16

The complete reproduction, cross-version audit, and results are available here:

https://github.com/TesfayZ/algebraicRLtest

Relevant files:

audit/reacher_fk.py
audit/Results/reacher_fk.csv
audit/UPSTREAM_ISSUE.md

After following the repository installation instructions, the audit can be reproduced with:

bash

./venv/bin/python audit/reacher_fk.py

This reproduces the behavior on both Reacher-v4 and Reacher-v5. In the tested environment, the residual is approximately at machine precision for mj_forward only and step, qvel = 0, but increases to approximately 6.2e-9 after mj_step.

These results support the interpretation that the discrepancy is associated with synchronization of MuJoCo's derived kinematic quantities after stepping.

Checklist

I have checked that there is no similar issue in the repo

Source: Farama-Foundation/Gymnasium

View original on GitHub View discussion on GitHub