[RFC] AI-guided fault injection + cross-region latency & Redis cluster failure NetworkChaos actions
Feature request
Background
Chaos engineering today requires practitioners to translate informal resilience goals ("our checkout service should survive a database latency spike") into precise Chaos Mesh YAML. This translation is manual, error-prone, and requires deep knowledge of both the application topology and the Chaos Mesh API surface.
Large language models (LLMs) have demonstrated strong capability in understanding natural-language system descriptions and generating structured configuration from them. Combining Chaos Mesh's rich fault vocabulary with an LLM-backed planning layer could dramatically lower the barrier to running principled chaos experiments.
Feature Request
Add an AI-guided fault injection workflow to Chaos Mesh that allows an operator to describe a resilience hypothesis in natural language and receive a complete, ready-to-apply Workflow YAML.
Proposed user experience
$ chaosctl ai suggest \
--describe "Simulate a 200ms latency spike between the order service and the payments database, \
then verify that the circuit breaker opens within 5 seconds"The AI layer is additive and offline-capable:
- Intent parsing — the LLM extracts: target services, fault type, magnitude, duration, and verification steps
- Topology enrichment —
chaosctlqueries the Kubernetes API to resolve service labels and pod selectors - YAML generation — the LLM produces a
WorkflowYAML with the CRD JSON Schema injected as a system prompt to constrain output - Dry-run validation — generated YAML is validated client-side against the CRD schema
- Apply with confirmation — user reviews then applies
The LLM is called only at step 3. Steps 1, 2, 4, and 5 are deterministic. Architecture supports any OpenAI-compatible API endpoint (including self-hosted via Ollama/vLLM).
New fault patterns (shipped with this RFC as a companion PR)
To demonstrate AI suggestion capability and address long-requested community features, this RFC accompanies a PR adding two new NetworkChaosAction values:
cross-region-latency
Simulates variable WAN latency toward external CIDR ranges representing cloud-provider regional endpoints. Each region gets independent latency, jitter, and bandwidth cap applied simultaneously via separate filtered tc qdiscs.
action: cross-region-latency
crossRegionLatency:
profiles:
- regionName: us-east-1
cidrs: ["52.0.0.0/8", "54.80.0.0/13"]
latency: 5ms
jitter: 1ms
- regionName: eu-west-1
cidrs: ["54.72.0.0/13", "176.34.0.0/16"]
latency: 80ms
jitter: 8ms
bandwidthRate: "50mbit"redis-cluster-failure
Injects targeted network faults scoped to Redis protocol ports (6379 + cluster bus 16379) without disturbing unrelated pod traffic. Three modes: partition (drops 100% of Redis traffic), latency (netem delay on Redis port), bandwidth (TBF throttle on Redis port).
action: redis-cluster-failure
redisClusterFailure:
mode: partition
redisPort: 6379
clusterBusPort: 16379Implementation plan
Phase 1 (companion PR): New fault types
api/v1alpha1/networkchaos_types_additions.go— type definitionscontrollers/chaosimpl/networkchaos/trafficcontrol/impl_additions.go— ApplyTc case handlers
Phase 2 (follow-on PR): chaosctl ai suggest
- CLI command, OpenAI-compatible HTTP client, CRD-schema-based prompt builder, dry-run validator
- Config:
CHAOS_MESH_AI_ENDPOINT+CHAOS_MESH_AI_API_KEYenv vars
Phase 3 (optional): UI panel in Chaos Dashboard
Related issues
- #2201 — NetworkChaos: support for multi-region latency simulation
- #1893 — Redis-aware chaos actions
Proposed solution
See implementation plan in the feature request body above. The companion PR (to be linked once opened) contains:
api/v1alpha1/networkchaos_types_additions.go— CrossRegionLatencySpec, RegionLatencyProfile, RedisClusterFailureSpec, RedisFailureMode typescontrollers/chaosimpl/networkchaos/trafficcontrol/impl_additions.go— applyCrossRegionLatency() and applyRedisClusterFailure() helpers that extend the ApplyTc switch
The AI suggestion layer (Phase 2) will live in a new pkg/ai/ package and chaosctl ai suggest CLI subcommand, supporting any OpenAI-compatible endpoint.
Additional context
No response
Source: chaos-mesh/chaos-mesh