pi0-FAST reaches ~85% on LIBERO-10 with a config-only fine-tuning change, up from the widely cited 60.2% baseline
The LIBERO README at commit 7cf5f60 reported 60.2% success for pi0-FAST on LIBERO-10. This result has been widely cited as a baseline, including in:
- MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation
- Unified Vision-Language-Action Model
- VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models
- VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model
- Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
Our experiments (@iqr-lab, @ziyaosg) show that setting extra_delta_transform=False during fine-tuning raises LIBERO-10 success from ~60% to ~85%, without modifying the model or adding new training code.
LIBERO actions are already deltas (goal_pose - current_pose), but pi0_fast_libero applies DeltaActions, subtracting the current pose again. The model is therefore trained to predict goal_pose - 2 * current_pose. At inference, AbsoluteActions adds the current pose back. Although this reverses the extra subtraction, it changes the distribution of what the model learns to predict. Issue #750 raises similar concern.
We fine-tuned from pi0_fast_base with the additional delta transform enabled or disabled under two normalization settings, using matching training and inference statistics in each:
- Quantile: the default at commit
215abfb, which we used as the base for our experiments. - Mean/std: applying PR #971 to restore the normalization used to train the released checkpoint.
| Action normalization | extra_delta_transform=True |
extra_delta_transform=False |
|---|---|---|
| Mean/std, restored using #971 | 60.5 ± 1.0% | 84.7 ± 1.7% |
| Quantile, as configured at 215abfb | 75.1 ± 1.5% | 85.9 ± 0.6% |
Setup: 30k training steps, seeds 42/43/44, and evaluation through examples/libero/main.py with 50 trials per task. Results are mean ± std across three seeds. Conditions differ only in the additional delta transform and action normalization.
Other suites show smaller changes: Goal gains 5.9–7.5 percentage points; Spatial and Object change by ≤3.5 points. Please refer to full results and codes.
I plan to submit a PR adding pi0_fast_libero_no_delta_transform as an optional configuration, while preserving the existing configuration for compatibility with the released checkpoint.
Source: Physical-Intelligence/openpi