RFC: Ralph Series.

RFC-001: Ralph Series — Tiered Model Routing & Privacy-Preserving Sub-Agent Orchestration

Status: Submitted Authors: Angelo Reale and Perplexity Repository: https://github.com/snarktank/ralph
Created: 2026-05-30
Last Updated: 2026-05-30

Abstract

This RFC proposes the Ralph Series — a set of extensions to the Ralph autonomous agent loop that introduce (1) a cost-aware, tiered model routing system ("Model Series"), (2) a privacy-preserving proxy layer for EU/regulated-environment deployments ("EU Proxy"), and (3) an output validation and compliance gate ("Compliance Guard"). Together, these components make Ralph the most cost-efficient and reliable prompt ingress layer for autonomous coding workflows, while enabling organizations to delegate to cheap external models without exposing sensitive codebase context.

1. Motivation

Ralph today operates as a single-model loop: one PRD story → one AI coding tool invocation (Amp or Claude Code) → one commit. This design works well but leaves two large inefficiencies unaddressed.

Cost inefficiency. The majority of tasks in a typical PRD (renaming a variable, adding a field to a schema, writing a unit test) do not require frontier-grade models. Routing every story through the most capable — and most expensive — model inflates costs without improving outcomes. A nano-tier model resolving 70% of tasks at 1/20th the cost would produce identical results for those stories while dramatically reducing the total bill.

Privacy and compliance risk. Organizations operating under GDPR or handling proprietary intellectual property cannot freely delegate codebase context to untrusted third-party LLMs (OpenAI, Anthropic, Deepseek, Grok, etc.). There is no current mechanism to sanitize or redact context before it leaves the trust boundary. The EU Proxy addresses this by placing a sovereign, self-hosted LLM as the sole interface to external models, ensuring that only redacted, non-attributable task chunks cross the boundary.

2. Terminology

3. Design Goals

1. Pay-as-you-consist — spend the minimum required to produce a passing result, scaling up only when cheaper tiers fail.
2. Parallelism-first — sub-tasks that are independent should be dispatched concurrently rather than sequentially.
3. Compliance by default — no raw codebase context crosses a trust boundary without explicit allowlisting.
4. Drop-in compatibility — all Series extensions are backward-compatible with the existing ralph.sh interface; single-model usage remains fully supported.
5. Deterministic stop condition — the loop terminates only when all stories are validated, not merely returned.

4. System Overview

┌─────────────────────────────────────────────────────────┐
│                      ralph.sh (loop)                    │
│                                                         │
│  ┌────────────┐    ┌──────────────────────────────────┐ │
│  │  prd.json  │───▶│        Ingress Layer             │ │
│  └────────────┘    │  (task decomposition + routing)  │ │
│                    └───────────────┬──────────────────┘ │
│                                    │                     │
│              ┌─────────────────────▼──────────────────┐ │
│              │           Model Series Router           │ │
│              │  nano ──▶ mini ──▶ sonnet ──▶ opus     │ │
│              └─────────────────────┬──────────────────┘ │
│                                    │                     │
│              ┌─────────────────────▼──────────────────┐ │
│              │         Compliance Guard               │ │
│              │  (validate output → pass / escalate)   │ │
│              └─────────────────────┬──────────────────┘ │
│                                    │                     │
│                    ┌───────────────▼──────────────────┐ │
│                    │     commit + prd.json update      │ │
│                    └──────────────────────────────────┘ │
│                                                         │
│  ┌──────────────────────────────────────────────────┐   │
│  │  EU Proxy (optional, activated by env flag)      │   │
│  │  trusted LLM ──▶ redact ──▶ external model       │   │
│  └──────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘

5. Component Specifications

5.1 Model Series Router

The Series Router replaces the single --tool flag with a declarative tier configuration. Each story dispatched by ralph.sh is first attempted at the cheapest eligible tier; on failure or escalation, the next tier is tried.

5.1.1 Tier Configuration

Tiers are declared in a new top-level key in prd.json (or a companion ralph.config.json):

{
  "series": {
    "tiers": [
      { "id": "nano",   "model": "o4-mini",           "costCeiling": 0.10 },
      { "id": "mini",   "model": "claude-3-5-haiku",  "costCeiling": 0.50 },
      { "id": "sonnet", "model": "claude-sonnet-4",   "costCeiling": 1.00 },
      { "id": "opus",   "model": "claude-opus-4",     "costCeiling": 5.00 }
    ],
    "startTier": "nano",
    "maxCostPerStory": 5.00
  }
}

• costCeiling is the maximum spend allowed before the tier is considered failed and the next tier is tried.
• maxCostPerStory is a hard cap; if the final tier would exceed it, the story is marked blocked and added to a human review queue.
• startTier can be overridden per-story via a "minTier" field on the user story object.

5.1.2 Escalation Criteria

A tier result is escalated to the next tier if any of the following are true:

• Quality checks (typecheck, lint, test) fail after the model's output is applied.
• The Compliance Guard rejects the output (see §5.3).
• The model returns an explicit uncertainty signal (<uncertain> tag or equivalent).
• The model's cost for this invocation exceeds costCeiling.

The escalation is logged in progress.txt with tier, cost, and failure reason.

5.1.3 Parallelism

When a story is decomposed into multiple independent sub-tasks (see §5.2), each sub-task is dispatched concurrently up to a configurable parallelism limit:

{ "series": { "parallelism": 4 } }

Sub-task results are merged and validated as a unit before committing. Any sub-task failure triggers re-dispatch of only that sub-task (not the entire story).

5.1.4 Consistency Voting (Optional)

When voting is enabled, N concurrent calls at the same tier produce N candidate outputs. The output closest to the median (or passing the most quality checks) is selected. This is particularly useful for non-deterministic or ambiguous tasks:

{ "series": { "voting": { "enabled": true, "n": 3, "strategy": "majority" } } }

This is the "compare 1/N LLM outputs to pick the best result" capability described in the motivation.

5.2 Ingress Layer & Task Decomposition

The Ingress Layer sits between prd.json and the Series Router. Its responsibilities:

1. Codebase context assembly — reads relevant files, git diff, and AGENTS.md to build a context bundle for the story.
2. Task decomposition — uses a fast, cheap model (or deterministic heuristics) to split large stories into independent sub-tasks, each completable in a single model call.
3. Dependency resolution — ensures sub-tasks with data dependencies are ordered correctly; independent sub-tasks are flagged for parallel dispatch.
4. Context scoping — passes only the files and context genuinely relevant to each sub-task, minimizing token usage at cheap tiers.

The decomposition output is an ordered/annotated array of Chunk objects:

[
  {
    "chunkId": "US-4.1",
    "parentStoryId": "US-4",
    "description": "Add `verified` boolean column to users table migration",
    "files": ["db/migrations/", "db/schema.ts"],
    "dependsOn": [],
    "sensitive": false
  },
  {
    "chunkId": "US-4.2",
    "parentStoryId": "US-4",
    "description": "Update UserService.create() to set verified=false by default",
    "files": ["src/services/user.ts"],
    "dependsOn": ["US-4.1"],
    "sensitive": false
  }
]

The sensitive flag controls whether the chunk is eligible for routing through the EU Proxy (see §5.4).

5.3 Compliance Guard

The Compliance Guard validates model output before it is committed to the repository. Validation runs as a pluggable pipeline; each validator is a small script or configuration block.

5.3.1 Built-in Validators

5.3.2 Custom Validators

Project-specific validators are declared in ralph.config.json:

{
  "complianceGuard": {
    "validators": [
      { "id": "no-console-log", "command": "grep -rn 'console.log' src/", "exitCodeOnMatch": 1 },
      { "id": "api-contract", "command": "npm run validate:openapi" }
    ]
  }
}

5.3.3 Guard Disposition

After all validators run, the Guard emits one of three dispositions:

• PASS — output accepted, proceed to commit.
• ESCALATE — output rejected due to fixable quality issues; try next tier.
• BLOCK — output rejected due to security, licensing, or irreversible concerns; halt loop, write to human review queue.

5.4 EU Proxy

The EU Proxy is an optional, self-hosted component activated by setting EU_PROXY=true in the environment. It provides a sovereign LLM intermediary that ensures sensitive codebase context never reaches untrusted external models in raw form.

5.4.1 Architecture

ralph.sh
   │
   ▼
EU Proxy (self-hosted, e.g., Mistral on-prem / llama.cpp / Ollama)
   │  1. Receives full task + context
   │  2. Runs full codebase analysis
   │  3. Produces redacted chunks (strips proprietary identifiers,
   │     internal URLs, credentials, business logic comments)
   │  4. Dispatches redacted chunks to external model
   │  5. Receives responses
   │  6. Re-contextualizes responses back to original codebase
   ▼
External Model (Deepseek / OpenAI / Anthropic / Grok / etc.)

The proxy operates as a local HTTP server compatible with the OpenAI Chat Completions API, making it a drop-in replacement for any model endpoint.

5.4.2 Redaction Rules

Redaction is configured per project in ralph.config.json:

{
  "euProxy": {
    "redact": [
      { "type": "regex",  "pattern": "COMPANY_[A-Z_]+",     "replacement": "CONST_VAR" },
      { "type": "regex",  "pattern": "https://internal\\..*", "replacement": "https://internal.example.com" },
      { "type": "entity", "kind": "credential" },
      { "type": "entity", "kind": "pii" }
    ],
    "allowExternalModels": ["deepseek-coder", "gpt-4o"],
    "trustedModel": "mistral-7b-instruct"  // runs locally
  }
}

5.4.3 Re-contextualization

After receiving output from the external model, the proxy maps redacted tokens back to their originals before returning the result to the Compliance Guard. The mapping is held in memory only and never serialized to disk or logs.

5.4.4 Audit Log

The EU Proxy writes an append-only audit log (eu-proxy-audit.jsonl) recording:

• Timestamp, story ID, chunk ID
• Which external model was called
• Redaction summary (count of tokens redacted by category, never the tokens themselves)
• Response disposition (accepted / rejected by Compliance Guard)

This log is suitable for GDPR compliance documentation.

6. Updated ralph.sh Interface

The existing CLI flags are preserved. New flags:

./scripts/ralph/ralph.sh [options] [max_iterations]

Options:
  --tool <amp|claude>       AI coding tool (default: amp)
  --series                  Enable Model Series Router (default: off)
  --start-tier <tier>       Override starting tier for this run (default: from config)
  --eu-proxy                Enable EU Proxy for all external model calls
  --parallel <n>            Max concurrent sub-task dispatches (default: 1)
  --vote <n>                Enable consistency voting with N candidates (default: 1)
  --dry-run                 Decompose and plan without executing any model calls

7. prd.json Schema Extensions

New optional fields on each user story:

{
  "id": "US-7",
  "title": "Add email verification flow",
  "passes": false,
  "minTier": "mini",
  "maxTier": "sonnet",
  "sensitive": true,
  "parallelizable": true,
  "acceptanceCriteria": ["..."],
  "qualityChecks": ["npm run typecheck", "npm test -- --filter=auth"]
}

8. Cost Model

For a PRD with N stories, let $$c_k$$ be the average cost of a tier-$$k$$ call and $$p_k$$ the probability that tier $$k$$ resolves the story without escalation. The expected cost per story under Series routing is:

$$ E[\text{cost}] = \sum_{k=1}^{K} c_k \prod_{j=1}^{k-1}(1 - p_j) \cdot p_k $$

Empirically, if nano resolves 60% of stories and mini resolves a further 25%, the blended cost per story is dominated by the cheap tiers. At representative 2026 pricing (nano ≈ €0.05, mini ≈ €0.20, sonnet ≈ €0.80, opus ≈ €4.00), a 100-story PRD costs roughly €18–25 under Series routing versus €400 if every story hits opus — a ~20× reduction at no quality loss for routine tasks.

9. Failure Modes & Mitigations

10. Security Considerations

• Secret scanning is a hard-block validator — no tier escalation, no EU Proxy routing can override a detected credential in output.
• The EU Proxy mapping table (redacted token → original token) exists only in process memory. It is never written to disk, never included in audit logs, and is destroyed at process exit.
• External model API keys used by the EU Proxy are scoped to the proxy process only; ralph.sh never holds or logs them.
• Chunk boundaries are designed so that no single chunk contains enough context to reconstruct proprietary business logic.

11. Open Questions

1. Decomposition model — Should the Ingress Layer decompose tasks using a deterministic parser, a dedicated cheap model call, or a hybrid? Deterministic is faster and free; model-based is more accurate for complex stories.
2. Voting strategy — majority works for code with tests; for open-ended tasks, what is the right selection criterion?
3. EU Proxy trust model — Should the proxy be a generic OpenAI-compatible proxy or Ralph-specific? A generic proxy enables broader reuse but increases attack surface.
4. Tier failure accounting — Should escalation cost count against the story's maxCostPerStory, or only the final accepted tier?
5. Progress.txt format — Should Series metadata (tier used, cost, escalation path) be appended to progress.txt or kept in a separate series-log.jsonl?

12. Implementation Plan

Phase 1 — Series Router (no EU Proxy)

• Add series config parsing to ralph.sh
• Implement tier loop with escalation on quality check failure
• Extend prd.json schema with minTier, maxTier, qualityChecks
• Append tier/cost metadata to progress.txt

Phase 2 — Ingress Layer & Parallelism

• Implement task decomposition (deterministic heuristics first)
• Add --parallel flag and concurrent sub-task dispatch
• Implement sub-task merge and conflict detection

Phase 3 — Compliance Guard

• Secret scan validator (integrate truffleHog or detect-secrets)
• Diff-scope validator
• Custom validator plugin interface
• Human review queue output

Phase 4 — EU Proxy

• OpenAI-compatible proxy server (Node.js or Go)
• Redaction engine with regex + NER entity types
• Re-contextualization mapping
• Append-only audit log
• Docker Compose service definition

Phase 5 — Voting & Observability

• --vote N flag and majority/best-of-N selection
• Cost dashboard (series-summary.json per run)
• Integration tests for escalation paths

13. References

• Geoffrey Huntley's Ralph pattern
• snarktank/ralph README
• Amp documentation
• Claude Code documentation
• OpenAI Chat Completions API
• GDPR Article 28 — Processor obligations