paritok-4b-v1

Paritok

A non-destructive compression gateway for coding agents — cut your input-token bill without changing your agent.

Paritok sits between your agent and the LLM as a drop-in proxy. On every request it strips the tool-schema bloat, compresses tool results and file reads, and summarizes stale history — then forwards upstream, billed on the compressed tokens. Nothing is ever permanently discarded: the agent pulls back any exact original on demand. Works with Claude Code, Cursor, Codex, OpenHands, and any agent that honors BASE_URL — you don't change a line of your agent.

Powered by the first open-source 4B compression model trained specifically for coding agents (45K real agent trajectories). Cut input token bills from ~25% on turn one to past 85% in long, context-saturated sessions, and fit ~3× more turns in the same context window.

What it does · The three levers · Compounding savings · Cost · Pricing · Quick Start · The engine (model) · Team

--- ## News - **2026-08-17**   **v1.3.8** — a visual `/stats` dashboard (live token savings + per-request original→compressed before/after, in place of the raw JSON) and the new [Paritok VS Code extension](https://github.com/Paritok-official/paritok-vscode). - **2026-07-31**   **v1.3.0** — stability release: edit-recovery, `read_original` API rename (from `expand_context`). - **2026-07-19**   **v1.2.0** ships the embedding-based tool filter — the biggest single-turn lever (~29K → ~8K on a typical Claude Code turn), unlocking prompt-cache-friendly tool selection with `gateway_search_tools` recall. - **2026-07-15**   **Paritok gateway v1.0.0** open-sourced — the proxy/middleware that turns the 4B model into a drop-in Claude Code / Cursor / Codex compression layer. - **2026-07-14**   **Paritok-4B-v1** released on Hugging Face Hub with full SWE-bench Lite end-to-end evaluation. - **2026-06-25**   Finished training. 45K teacher-distilled samples on the Qwen3-4B backbone. --- ## What Paritok does Paritok runs as a **middle layer between your agent and the LLM API** — your agent points at Paritok instead of Anthropic/OpenAI, and everything else stays the same. ``` Your Agent (Claude Code / Cursor / Codex) → builds request (tool schemas + history + tool results / file reads) ★ Paritok gateway rewrites the request here ★ → forwarded to Anthropic / OpenAI (billed on the compressed tokens) ← response flows back unchanged; compressed refs expand on demand ``` The token bill for a coding agent is dominated by **input you re-send every turn**: dozens of tool schemas, an ever-growing message history, and big file-read / tool-output blocks. Paritok attacks all three — and because it's a **non-destructive** gateway, anything it compresses or filters is still recoverable on demand. It's lossy on the wire, fully recoverable when it counts. --- ## ️ The three levers Paritok saves tokens through three independent mechanisms. They stack, and they hit different parts of the bill: ### 1. Tool-schema filter — the biggest single-turn win Coding agents expose dozens of tools — often **70+** once you add MCP servers — in full JSON schema on **every** request. Most are irrelevant to the task at hand. Paritok filters them semantically (`tool_discovery.strategy: embedding`), keeping only the handful relevant to the user's intent in full schema and stubbing the rest. - **This is the largest single-turn lever.** On a typical Claude Code turn the tool block alone is ~29K tokens; filtered it drops to ~8K — a saving no amount of file compression matches on a single turn. - **Prompt-cache friendly.** The selection is frozen per conversation, so the `tools[]` block stays byte-stable turn-to-turn and never invalidates the LLM's KV cache. - **Never destructive.** Anything filtered is recoverable — the model calls `gateway_search_tools` and gets the full schema back. - Runs a small open embedding model ([BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5), MIT, ~130MB) **entirely locally on CPU** — no API, no per-token fee. - An agent's **core execution tool** (shell / exec / apply_patch) is never stubbed, so agents like Codex that expose only a handful of tools always keep the one they can't work without. ### 2. Content compression — file reads, tool output, history Each `tool_result`, file read, and (once the window fills) stale history turn is compressed by the 4B model down to **~26% of its original size**, tagged `[REF:id]`. This is where the trained model earns its keep: it knows a function signature from a debug line, so it protects identifiers, paths, and error strings while dropping the noise. - Single-turn this is the *smaller* lever (a few % — most of a turn's cost is the fixed prefix). **Its power shows up across a session — see below.** - Every compressed segment is recoverable: the agent calls `read_original` / `expand_context` to pull back the exact untouched bytes, locally and instantly. ### 3. History summarization — keep long sessions under the window Turns beyond the recent window are summarized once the context fills up, so a long session stays inside the model's context window instead of overflowing (or forcing an aggressive client-side compaction that drops detail). --- ## Savings compound over a session This is the part a single-turn benchmark hides. In a real multi-turn session the two levers **grow at different rates**, and together they compound. We ran the same read-only "find the bug" task for **5 consecutive turns in one Claude Code session** (Sonnet, GPU model). **Important:** in this A/B the tool filter was left **on for both sides**, so the delta below isolates **content compression alone** — it does *not* include the tool-schema saving. **Content compression only** (tool filter on both sides): | Turns | Paritok (files compressed) | Baseline (files raw) | Content-only saving | | :---: | :------------------------: | :-------------------: | :-----------------: | | 1 | 72,041 | 75,507 | 4.6% | | 5 | 293,389 (cumulative) | 377,099 (cumulative) | 22.2% | But that's only one of the three levers. The real "do I use Paritok or not?" comparison must add the tool filter back — and **without Paritok the agent sends the entire ~29K tool-schema block every turn, not the filtered ~8K.** Folding that ~21K/turn back onto the no-Paritok side: **Full stack** (filter + compression vs. no Paritok at all): | Turns | Paritok (filter + compression) | No Paritok (full tools + raw files) | End-to-end saving | | :---: | :----------------------------: | :---------------------------------: | :---------------: | | 1 | 72,041 | ~96,500 | **~25%** | | 5 | 293,389 (cumulative) | ~482,000 (cumulative) | **~39%** | So the 4.6% / 22.2% above is the **floor** (content only). Against a real no-Paritok baseline it's **~25% on turn 1, past ~39% by turn 5** — because the tool filter saves a fixed ~21K *every* turn on top of the compounding content compression. **Why it grows:** every file you read stays in history and is re-sent (cache-read) every subsequent turn — so content compression keeps paying off turn after turn, while the tool filter adds a fixed cut on top. - **Content compression → quadratic.** Cumulative saving ≈ `3,350 × N²` — each turn's compressed reads keep paying off on every later turn. - **Tool filter → linear.** Cumulative saving ≈ `21,000 × N` — a fixed block saved every turn. - **Crossover ≈ turn 6:** early on the tool filter dominates; past ~turn 6 content compression overtakes it and the gap widens. Plugging these formulas into a range of N (baseline ~96,500 tokens/turn), **capped at a ~200K context budget** (typical Sonnet-tier configuration; larger-context models like Opus 1M push the flatten point out proportionally): | Turn (N) | Content saved | Tool filter saved | Cumulative saved | Cumulative baseline | **% saved** | |:---:|---:|---:|---:|---:|:---:| | 1 | 3,350 | 21,000 | 24,350 | 96,500 | **25%** | | 5 | 83,750 | 105,000 | 188,750 | 482,500 | **39%** | | 10 | 308,150 | 210,000 | 518,150 | 965,000 | **54%** | | 12 | 404,150 | 252,000 | 656,150 | 1,158,000 | **57%** | | 15 | 548,150 | 315,000 | 863,150 | 1,447,500 | **60%** | | 20 | 788,150 | 420,000 | 1,208,150 | 1,930,000 | **63%** | **Capped at a ~200K context budget.** The quadratic only holds while history is still growing. Once the accumulated context fills the configured budget (around turn ~8–12 here on Sonnet-tier ~200K), client-side compaction holds it flat, per-turn content saving stops growing (freezes at ~48K/turn), and **% saved plateaus toward the ~72% default ceiling instead of diverging**. Turns 1–5 match the measured tables above; later rows are in-window projections. **Ceilings shift with deployment shape:** | Deployment scenario | Baseline / turn | Ceiling % saved | | ------------------------------------------------ | :-------------: | :-------------: | | Default (~40 tools, moderate reads) | 96,500 | **~72%** | | MCP-heavy (70+ tools) | 127,500 | **~78%** | | Context-saturated (no-Paritok forced to compact) | ~200,000 | **~85%+** | The projection above uses the default ceiling. MCP-heavy setups earn a bigger per-turn tool-filter cut; context-saturated sessions get an even larger effective saving because they're now compared against a compacted, information-lossy baseline. > **Honest cap:** the quadratic doesn't run forever — whatever session context budget you configure bounds it. In practice a ~200K budget (typical for Sonnet-tier deployments) makes the curve flatten around turn ~12–20 as the window fills; larger-context models like Opus 1M push the flatten point out proportionally. But that ceiling is itself a feature: **because each turn's prefix is smaller, the agent fits more turns before hitting the budget** — Paritok effectively buys back context length. **Same budget, more room to think.** Because each turn's prefix is smaller, the agent fits far more turns in the same window before compaction kicks in. | Context budget | Turns without Paritok | Turns with Paritok | | :------------------------------------ | :-------------------: | :----------------: | | 128K (typical Claude Code default) | ~10 | ~30 | | 200K (Claude Sonnet standard) | ~15 | ~44 | | 1M (Claude Opus, GPT-5 beta) | ~75 | ~220 | That's roughly **3× the runway on any budget**, giving the agent more turns to think and finish the task before hitting the wall. **One line:** *use more, save more*. Compression frees up the window and lets the agent go deeper and longer in the same session. Strongest on **long, multi-turn, read-heavy** work (auditing, Q&A over a big codebase, long debugging sessions). --- ## Cost Impact The **74%** figure is Paritok's **content compression rate** — file reads, tool output, and hist