RFC: Generator-Evaluator Dual-Agent Architecture (Harness Mode)
Motivation
The current Ralph is a highly effective single-agent loop. After extensive real-world use and battle-testing (especially on Windows/MSYS2), I've developed a significant enhancement: a Generator-Evaluator dual-agent architecture that adds contract negotiation, phase discipline, and 4-dimension quality scoring.
This RFC proposes adding "Harness Mode" as an optional architecture alongside the existing simple mode.
Key Concept
In Harness Mode, two specialized AI agents collaborate per story — Generator (builds, never judges) and Evaluator (judges, never builds):
- Contract Negotiation — Generator proposes scope → Evaluator reviews → back-and-forth until locked
- Build — Generator reads locked contract → implements → typecheck/lint/test → commits
- Evaluate — Evaluator tests in browser (Playwright MCP) → scores on 4 dimensions → writes evaluation.json
- Retry or Advance — Pass all thresholds → next story. Fail → feedback to Generator → retry
Proposed Changes
New (4 files): generator-prompt.md, evaluator-prompt.md, contract.json.example, AGENTS.md
Modified (1 file): ralph.sh — purely additive, run_simple_mode() untouched
Default mode remains simple — fully backward compatible:
./ralph.sh # simple mode (unchanged)
./ralph.sh --mode harness --tool claude # new harness modeQuestions
- Is the dual-agent architecture direction aligned with your vision?
- Any concerns about naming or structure before I submit a PR?
- Would you prefer harness code in
ralph.shor a separate script?
Happy to iterate based on your feedback.
Source: snarktank/ralph