[RDA Audit] Data Quality Report for lerobot/svla_so101_pickplace
Hi LeRobot team
We built an open-source tool called RDA (Robot Data Audit) (github.com/liesliy/rda) for automated quality auditing on LeRobot-format datasets. We ran an audit on lerobot/svla_so101_pickplace (50 episodes, 11,939 frames) and wanted to share findings that may be useful for dataset users.
✅ What Looks Good
- Data integrity: All 50 episodes pass missing-frame, invalid-value, and schema consistency checks.
- Action continuity: 260 spikes across 50 episodes (median 5/episode) — relatively moderate compared to other datasets we have audited.
- State space occupancy: median 4.5%, reasonable for a pick-and-place task.
⚠️ Findings Worth Noting
1. High Idle Ratio
Median effective motion is only 13.3% — meaning 86.7% of frames show minimal state change. This is notably higher idle than most datasets we have audited. For downstream policy training, this may reduce sample efficiency.
Possible explanations:
- Long pause periods during teleoperation
- Conservative recording strategy (start early, stop late)
- Task-specific: pick-and-place may naturally have more static phases
2. Action Discontinuity
100% of episodes contain at least one action discontinuity spike (sudden jumps between consecutive frames). While the spike count (median 5/episode) is moderate, the universal presence suggests a systematic pattern rather than random noise.
Tool Info
- Tool: RDA v0.9.7 (pip install robot-data-audit)
- Audit scope: Standard audit (video quality not enabled, as video files were not downloaded)
What Would Help
- Whether the high idle ratio is intentional (e.g., conservative recording boundaries)
- Whether action discontinuities are expected artifacts from the data collection pipeline
Happy to share the full audit report if helpful. Thanks for maintaining such a valuable ecosystem for the robotics community!
Source: huggingface/lerobot