#6453·ColossalAI

CI: replace explicit example skips with hermetic smoke tests

Author: richardoo-707Created Sep 25, 2026Updated Sep 26, 2026

Context

Several example directories had empty test_ci.sh files. Bash exits successfully for an empty script, so PR and scheduled example checks reported green without running a test.

The empty-script fix adds real smoke tests where the examples can run deterministically. The following examples remain explicitly skipped until they can satisfy the same contract:

  • examples/language/deepseek: even the tiny configuration eagerly calls AutoConfig.from_pretrained(..., trust_remote_code=True), making CI depend on network access, the Hugging Face cache, and remote code.
  • examples/community/roberta: the example requires external preprocessed data, tokenizer/config assets, multi-host setup, and legacy ColossalAI APIs.
  • examples/inference/benchmark_ops: the entry points are full performance grids with warmups/repetitions and do not expose a bounded correctness-oriented smoke mode.

Acceptance criteria

  • Each skipped directory has a deterministic smoke command that performs meaningful work and fails on regressions.
  • The smoke test does not download model weights, datasets, tokenizers, or executable remote code.
  • It runs in the example CI image and completes within five minutes on at most two GPUs.
  • It uses synthetic/local fixtures and returns non-zero when the example is broken.
  • Replace the corresponding explicit SKIP script with the smoke command.

Until these conditions are met, CI must print an explicit warning linked to this issue instead of silently succeeding.