[DATA] VLABench default eef Box rejects all 3,114,872 unified actions

Ticket Type

Bug Report (Something isn't working)

Environment & System Info

• LeRobot: 22bd7a2f489b367d8df42de803b1e8c4ca63a3f9
• Dataset: lerobot/vlabench_unified@f61ad898199e746c20af3112f278cc86eb3ca3c4
• VLABench semantic reference: cf588fe60c0c7282174fe979f5913170cfe69017
• Audit: DuckDB 1.5.5; Python 3.10 / 3.12 / 3.13

Description

The default VLABenchEnv eef mode declares a seven-dimensional Gym Box with low [-1,-1,-1,-1,-1,-1,0] and high [1,1,1,1,1,1,1]. The wrapper documents and consumes the first six values as absolute robot-frame XYZ and xyz Euler targets, converts Euler to quaternion, and sends the pose through inverse kinematics.

A checksum-verified full scan found that 3,114,872 / 3,114,872 rows (100%) in the official unified mirror lie outside that declared Box. All 10,977 episodes and all 295 task strings contain at least one rejected action; 5,545,679 / 21,804,104 scalar coordinates lie outside it.

This does not show that the trajectories are corrupt or that Gym clips them. The pinned adapter consumes the absolute targets directly. The defect is the mismatch between the executable interface metadata and the action representation.

Context & Reproduction

Exact evidence, pinned source manifest, independent validation, and CI gate:

• https://github.com/sawhney17/vlabench-contract-audit
• https://github.com/sawhney17/vlabench-contract-audit/releases/tag/v0.1.0
• https://sawhney17.github.io/vlabench-contract-audit/demo/

git clone https://github.com/sawhney17/vlabench-contract-audit
cd vlabench-contract-audit
uv sync --frozen --group dev
uv run vlabench-contract-audit download --data-root .audit-data

set +e
uv run vlabench-contract-audit scan --data-root .audit-data \
  --output reproduced-audit.json
status=$?
set -e
test "$status" -eq 1  # verified finding
cmp artifacts/audit-evidence.json reproduced-audit.json

Relevant output

A broad representation-level control—unbounded XYZ, float32 [-pi, pi] Euler, and [0,1] binary gripper—contains every row. That zero-violation envelope is a negative control, not a proposed calibrated robot workspace.

Suggested repair

1. Introduce one named, versioned absolute-pose action specification shared by the environment, dataset adapter, policy configuration, and tests.
2. Document order, frame, and units explicitly: robot-frame XYZ metres, xyz Euler radians, and binary 1 = open.
3. Replace the normalized-looking Box with maintainer-approved calibrated workspace limits, or a clearly documented representational Box if calibrated limits are unavailable.
4. If normalized/delta control is desired, expose it as a separate mode with an explicit transform.
5. Add a pinned fixture whose real actions must satisfy action_space.contains().

I am not proposing bounds fitted to dataset extrema. The correct finite workspace should be chosen by VLABench/LeRobot maintainers with simulator knowledge.

Related semantic hazard

The published state also preserves a legacy closed-bit while actions use an open-bit. Because changing polarity can break existing checkpoints, I filed that compatibility issue separately: https://github.com/OpenMOSS/VLABench/issues/88

Claim boundary

No rollout failure, policy-score degradation, image/pose corruption, calibrated workspace, or exact dataset-generator checkout is claimed. The VLABench source pin is corroborating semantic evidence because the dataset card omits exact generator provenance.

Checklist

• I searched existing issues and pull requests for this exact action-space mismatch.
• I scanned the complete pinned Parquet population rather than a sample.
• I separated the interface defect from unmeasured runtime and policy impact.
• I published reproducible, checksum-anchored evidence and an independently reviewed release.