DeepSpeed - Q4 2026 Roadmap

This is the roadmap for DeepSpeed Q4 2026. Feedback is welcome — please leave comments on this issue or join the #2026q4-roadmap channel on the DeepSpeed Slack.

Some of the items are the extension of the Q3 roadmap: https://github.com/deepspeedai/DeepSpeed/issues/8104

Tagging all the PoCs (code maintainers) for each workstream, the PoCs can coordinate developer meetings, sort out implementation plan, and are responsible for updating the status.

New feature and enhancement

Mixture-of-Expert (MoE) Training Scaling and Kernel Development

Mixture-of-Expert (MoE) is the mainstream model architecture for frontier AI models. Scaling MoE training requires further improvement on the MoE backend of DeepSpeed, enabling better efficiency, scalability and performance.

• Pipelined Communication & Compute (More Efficient All-to-All Communications and Reduce HBM activation spikes) (PoC: @PKUWZP )
• Block-Sparse Kernels for efficient token dropping (PoC: @PKUWZP )
• Hardware-Aware Memory Optimization & Model Sharding (PoC: @PKUWZP )
• Expert Loading Balance & Hotspot Mitigations (PoC: @PKUWZP )
• Resiliency & Silence Data Corruption Detection (PoC: @PKUWZP )
• AutoEP Performance Optimization & Parallel Folding (AutoEP + AutoTP) (PoC: @tohtana @pengdurice )

DeepCompile

• Automatic optimization pass orchestration (@tohtana @sfc-gh-truwase @pengdurice )
• Expert parallel support (@tohtana @pengdurice )
• AutoSP with ZeRO (@tohtana )
• Model coverage improvements including VL/VLA models (@tohtana @sfc-gh-truwase , optionally @PKUWZP )
• Save/load optimized IR graphs (@tohtana )
• Robustness improvements (@tohtana )

New Accelerators Support (Q4)

• DeepSpeed TPU accelerators support and performance optimization (@PKUWZP @tohtana @sfc-gh-truwase )
• Apple Silicon Accelerators support for DeepSpeed (@PKUWZP )
• Performance Optimization on Emerging AI Accelerators (@PKUWZP @delock )

DeepSpeed HybridEngine Optimization (Q4)

• Achieving on-par performance with small batch size compared to mainstream LLM Inference Engines (@delock @PKUWZP )
• Continuous Batching Rollout support (@delock )
• Kernel Injection for co-located rollout support (@delock )

On-Policy Distillation (OPD) Trainer

• Systems Optimization for Large model OPD across multi-accelerators. (@delock @minjiazhang @PKUWZP )
• Coupled with HybridEngine Optimization for end-to-end benchmarking (@delock @minjiazhang @PKUWZP )
• Documentation for OPD trainer onboarding (@delock @PKUWZP )

AI Agent-driven Distributed Training Optimization (Optional)

• DeepSpeed-native Agentic training optimization (TBD, tentatively @PKUWZP )

Agent Sandbox enablement for Agentic post-training capabilities (Optional)

• Implement Agent Sandbox Environment for DeepSpeed to enable Agentic Post-training (@PKUWZP )

Universal Checkpointing (UCP) feature extension (Optional)

• Affine mapping as Semantic free UCP info (Phase 1) (@delock )
• UCP info support Yuan model TP sharding. (@delock )