API loops and judge examples lack cost / rate-limit safeguards

Author: MozzamShahidCreated Jul 14, 2026Updated Jul 14, 2026

Labels: security, safety, cost, examples, orchestration

Multiple example scripts and the researcher orchestration loop invoke paid LLM APIs repeatedly with no budget caps, token limits, or rate-limit handling:

  • examples/interleaved-thinking/reasoning_trace_optimizer/loop.py
  • examples/llm-as-judge-skills/src/config/index.ts
  • researcher/scripts/loop_step.py

Impact: A user running the examples could unexpectedly exhaust API credits. A misconfigured orchestration loop could repeat expensive calls indefinitely.

Suggested fix:

  1. Add max_cost_usd, max_total_tokens, max_requests_per_minute, and max_retries controls.
  2. Implement a lightweight budget tracker that aborts the loop when limits are reached.
  3. Add exponential backoff and HTTP 429 handling.
  4. Display a clear cost warning in every affected README and default new users to dry-run or stub mode.

Source: muratcankoylan/Agent-Skills-for-Context-Engineering