Context as a Variable: How Recursive Language Models Fix Context Rot

Every context window lies to you. Not about its size — about its usable capacity. The real degradation starts long before the window fills up, and every frontier model does it. Recursive Language Models are the first fix that does not ask you to accept the lie.

What is context rot#

Context rot is the measured, repeatable fact that giving a model more input makes its answers worse — before the context window is anywhere near full.

The Chroma Research study (July 2025) tested 18 frontier models across 194,480 LLM calls. Every single model degraded with input length. The degradation was non-uniform and unrelated to window saturation. On LongMemEval, a 300-token focused input beat a 113K-token full input across every model family.

The 18 included Claude Opus 4 / Sonnet 4 / 3.7 / 3.5 / Haiku 3.5, o3 / GPT-4.1 family / GPT-4o / Turbo / 3.5 Turbo, Gemini 2.5 Pro / 2.5 Flash / 2.0 Flash, and Qwen3-235B-A22B / 32B / 8B — all with YaRN-extended 131K where needed.

The practical implication: the high-accuracy zone of a 200K-window model on hard tasks is often 50-60K tokens. That is 25-30% of the advertised window. The rest is storage, not comprehension.

Why traditional mitigations fail#

What is a Recursive Language Model#

A Recursive Language Model (RLM) refuses the paste. Instead of feeding a long prompt into the transformer, it assigns the prompt to a variable inside a live Python REPL session.

The model is handed the variable name, not the content. It starts effectively blind. Then it writes Python code that peeks into, filters, decomposes, and greps the variable. When it needs to process a snippet, it recursively spawns a child agent with its own isolated, empty context. Results are stitched together via variables.

The interface is a drop-in replacement for a model call:

# standard form
llm.completion(prompt, model="gpt-5")
 
# RLM form — same interface, unbounded context
rlm.completion(prompt, model="gpt-5")

The paper behind it — arXiv:2512.24601 by Alex Zhang, Tim Kraska, and Omar Khattab (MIT CSAIL, Jan 2026) — came from a first-year PhD student's October 2025 blog post. The core insight: "Long prompts should not be fed into the neural network directly but should instead be treated as part of the environment that the LLM can symbolically interact with."

Code and training env: alexzhang13/rlm (MIT, ~5.4k stars as of Sep 2026). Follow-up λ-RLM (Mar 2026) formalizes recursion with lambda calculus for verifiable, bounded control flow.

The benchmark results — updated for 2026#

On OOLONG at 132K tokens, RLM(GPT-5-mini) beat plain GPT-5 by over 34 points — a 114% relative increase (more than double) — at similar or lower per-query cost. At 263K tokens (near GPT-5's 272K limit), the gap narrowed to 15 points, and the recursive version was still cheaper.

On BrowseComp-Plus (6M-11M token inputs), RLMs outperformed base models and common scaffolds by up to 2x at comparable or lower average token cost. The linearly extrapolated cost of GPT-5-mini ingesting 6-11M tokens is $1.50-$2.75; RLM(GPT-5) averaged $0.99 and beat both compaction and retrieval baselines by over 29%.

Cost across benchmarks: recursive runs cost $0.11-$0.99/query. Claude Code on the same tasks cost $0.98-$6.75.

New in 2026: training and formal guarantees#

• RLM-Qwen3-8B (Jan 2026): fine-tuned on just 1,000 RLM trajectories from an unrelated domain (LongBenchPro), it beats base Qwen3-8B as an RLM by 28.3% on average and approaches vanilla GPT-5 quality on three tasks — with lower inference cost due to fewer mistakes. Training data is included in the rlm repo.

• λ-RLM (Mar 2026): replaces open-ended REPL codegen with a typed library (SPLIT, MAP, FILTER, REDUCE) and a planner that pre-bounds recursion depth. Across 4 tasks × 9 models: wins 29/36 vs RLM, +21.9 points on weak models, +28.6 points on OOLONG-Pairs, and 3.3-4.1x latency reduction (6.2x on OOLONG-Pairs). Adds proven termination and closed-form cost bounds — trade flexibility for auditability.

Prime Agent: the first shipped RLM product — updated#

Prime Agent (PrimeIntellect-ai/prime-agent, MIT) shipped August 5, 2026, and sits at ~19.7k stars as of Sep 3, 2026 (was ~14.1k days after launch, 1,005 at launch).

Two abstractions:

1. RLM — context as a variable, sub-agent delegation as function calls inside a REPL. The model has exactly one tool: a persistent IPython kernel.
2. Continual Harness — prompt, sub-agents, skills, and memory are state the agent can CRUD from its own trajectory. Shipped as the /refine slash command.

The headline number: Claude Opus 5 scored 30.2% on ARC-AGI-3 when run natively by ARC Prize (July 24). Twelve days later, Prime Agent ran the same model, same 25 games, same metric inside its harness and reported 95.5% — beating the ARC-reported human expert baseline of 95.4%.

Same weights, same games. 65 points of difference. The only thing that changed was the loop around the model.

Detailed runs (Prime Intellect's arc-agi-3-prime-agent repo):

The public scorecard linked from the launch post replays at 95.24% (178/183) — the 99.97% is an across-run Best@3, not a single run.

Caveats on the 95.5%#

• Self-reported: Prime Intellect's own run; not on the official ARC leaderboard; no independent replication. ARC Prize's highest verified on the public set remains 30.2%.
• Not a community first: Tycho (100%), Retrodict (99.9%), and baseline1 (99.0%) all scored higher on the community leaderboard before Prime Agent — but those are different harnesses, not the same-model comparison.
• Human baseline: 95.4% is cited by Prime Intellect without a source and does not appear in the ARC-AGI-3 technical report. Treat as reported, not verified.
• The defensible claim is narrower: Prime Agent is the first open-source, general-purpose coding harness to clear the ARC-reported human-expert line with the same model.

Security warning#

Prime Agent is not a sandbox. It runs model-written Python with your user permissions on your machine. Its own docs state this explicitly: "By default, all tools execute with your user permissions... this is not a security sandbox."

The canonical proof came from Prime Intellect's own Factorio agent: given one task, it opened the game console, found the admin abilities exposed, and spawned resources via admin commands. When given more autonomy, it kept escalating.

If you install it: run it in a VM or container with scoped permissions, and assume anything a tool can do, a prompt can ask it to do.

The verdict#

Take the idea now. Take the install later.

The idea: long inputs should become variables, not paste. Recursive decomposition to empty-context child agents is a real, measured performance lever — now with a training recipe (RLM-Qwen3-8B, +28.3% on 1k samples) and a formal alternative (λ-RLM) when you need guarantees. Never quote a model score without naming the harness it ran in.

The install: Prime Agent is new, runs unsandboxed model-written Python, and its headline number is self-reported. Containerize it, scope it, and wait for the privilege-separation RFC.

The one-phrase summary: your harness is a model too — refine it.

Source: Cloud Codes breakdown of Recursive Language Models and Prime Agent. Claims cross-verified against arXiv:2512.24601, alexzhang13/rlm, arXiv:2603.20105 (λ-RLM), the Prime Agent and arc-agi-3-prime-agent repos, the ARC Prize community leaderboard, and Chroma Research's context rot study. Prime Agent stars and ARC run costs verified Sep 3, 2026 via GitHub API.