API loops and judge examples lack cost / rate-limit safeguards
Author: MozzamShahidCreated Jul 14, 2026Updated Jul 14, 2026
Labels: security, safety, cost, examples, orchestration
Multiple example scripts and the researcher orchestration loop invoke paid LLM APIs repeatedly with no budget caps, token limits, or rate-limit handling:
- examples/interleaved-thinking/reasoning_trace_optimizer/loop.py
- examples/llm-as-judge-skills/src/config/index.ts
- researcher/scripts/loop_step.py
Impact: A user running the examples could unexpectedly exhaust API credits. A misconfigured orchestration loop could repeat expensive calls indefinitely.
Suggested fix:
- Add max_cost_usd, max_total_tokens, max_requests_per_minute, and max_retries controls.
- Implement a lightweight budget tracker that aborts the loop when limits are reached.
- Add exponential backoff and HTTP 429 handling.
- Display a clear cost warning in every affected README and default new users to dry-run or stub mode.
Source: muratcankoylan/Agent-Skills-for-Context-Engineering