DeepSpeed - Q3 Roadmap

This is the roadmap for DeepSpeed Q3 2026. Feedback is welcome — please leave comments on this issue or join the #2026q3-roadmap channel on the DeepSpeed Slack.

Some of the items are the extension of the Q2 roadmap: https://github.com/deepspeedai/DeepSpeed/issues/7861

New feature and enhancement

Enhance Expert-Parallelism (EP) support

• AutoEP Support: AutoEP enables Expert Parallelism (EP) for major Mixture-of-Experts (MoE) models out of the box, eliminating the need for users to write model-specific parallelization code. By automatically distributing expert layers across devices, AutoEP allows users to scale MoE training with minimal configuration changes.
• Combining AutoEP with AutoTP
  • #8064
  • #8670
• Extend AutoTP capabilities by integrating Liger Kernel (sharding LM heads and adopting Online Softmax)
  • #8309
  • #8540
  • #8582
• EP performance optimization and benchmarking
  • #8190
  • #8209
  • #8213
  • #8423
  • #8359
  • #8180
  • #8244
  • #8326
  • #8331
  • #8451
  • #8380
  • #8356
  • #8424
  • #8581
  • #8621
  • #8557
  • #8636
  • #8644
  • #8652
  • #8671
  • #8678
  • https://github.com/deepspeedai/DeepSpeedExamples/pull/1007

New Accelerators Support (Q3)

• DeepSpeed TPU accelerators support
• Performance Optimization on Emerging AI Accelerators (e.g. Biren AI Accelerators)

On-Policy Distillation Trainer Support (Q3)

• Systems Design, prototyping and benchmarking (reference: https://arxiv.org/abs/2604.14084)

DeepCompile efficiency and robustness improvements

• Formal pass contracts and validation of optimization passes: Add lightweight optimization pass contracts for automatic compatibility validation and ordering.

  • #8139
  • #8251

• Composable AutoTP / AutoEP / SP optimization passes: Implement AutoTP and AutoEP at a compiler level and integrate them with AutoSP.

  • #8204
  • #8355
  • #8457
  • #8667

• Enhance optimization passes: Add some optimization passes including activation offloading.

  • #8186
  • #8233
  • #8255
  • #8258
  • #8388
  • #8418
  • #8508

• DeepCompile robustness improvements:

  • #8106
  • #8149
  • #8157
  • #8159
  • #8214
  • #8254
  • #8328

Pipeline parallelism with Ray

• Ray-backed pipeline stage placement and execution: Prototype pipeline stage placement and execution using Ray actor groups for a representative transformer workload, enabling heterogeneous resource allocation across pipeline stages. (Will be a generalized implementation of this).
  • rdsp: pipeline parallelism on Ray + DeepSpeed with per-stage layouts
• Create some examples: Provide example implementations of Ray-backed pipeline parallelism for representative transformer workloads, demonstrating the benefits of heterogeneous resource allocation.

Tuning guide with benchmarking results

• Recommended configurations for representative models: Provide practical configuration guidance for representative dense and MoE models across several common GPU setups.
• Benchmark-backed guidance: Run targeted benchmarks to support the recommended configurations and document the observed throughput, memory usage, and limitations.

Stability (Q3)

• Performance regression test
• Enable nightly full test
  • CUDA
  • AMD
  • Intel XPU
  • Intel Gaudi
  • NPU