SoL-Pi Cuts Coding-Agent Tokens by up to 49.0% Without Touching the Model, by Auto-Searching the Harness

NVIDIA researchers released SoL-Pi on September 17, 2026 as arXiv 2609.20519, a coding-agent harness improved through automated search that reduces token traffic by 44.7-49.0%. The work leaves model weights untouched and changes only the execution layer surrounding the agent, holding scores comparable to the Pi baseline harness on the 51-task EdgeBench evaluation while cutting API cost by about one third. The paper puts estimated savings at $8.75-$13.50 per hour relative to the native Codex and Claude Code harnesses. ASAP examines this work through its central move: applying recursive self-improvement to the harness layer rather than the model.

The target of improvement is the harness, not the model

SoL-Pi is a set of changes to the harness rather than to the model, which means the weights of GPT-5.6 Sol and Opus 5 are untouched in every result the paper reports. A harness is the execution layer that mediates tool calls between model and environment, manages context, and returns observations, and the paper applies the idea of recursive self-improvement (RSI) to that layer. The baseline and starting point is Pi, an existing coding-agent harness, and SoL-Pi is Pi with four selected mechanisms layered on top.

The problem the paper frames is tokens, not accuracy. As the abstract puts it, coding agents moving from supervised code completion to unattended, around-the-clock exploration turn their work from isolated predictions into long trajectories of reasoning, tool use, and feedback. The longer the trajectory, the more tokens dominate the cost of producing the same result, which makes token efficiency a precondition for scaling recursive self-improvement.

That framing sits one step away from the recent line of harness research. EvoHarness-RL, a prior work that treated the harness as a learnable object, used reinforcement learning to train a harness policy and raise success rates. SoL-Pi instead targets the same score at lower cost, and discovers mechanisms through automated search rather than training. The two are not competitors; they attack the same layer with different objective functions.

152 directions, more than 3,000 runs, four survivors

The scale of the search is the paper's substantive contribution. The researchers proposed roughly 150 mechanism directions spanning context, tools, delegation, prompt and policy, and improvement and evaluation, and assembled about 500 executable environments from 495 GitHub issue-PR pairs plus 40 synthetic verifier-driven tasks. Across these they logged more than 3,000 runs and more than 60,000 agent-environment interactions.

The structure is a two-stage funnel. A broad outer search sweeps 152 directions, and each direction then runs an independent inner development cycle. That inner cycle follows the Ralph Loop: an implementer refines a candidate until it meets an explicit completion criterion, an independent reviewer checks it before evaluation, and failed reviews trigger revision. The design principle the paper states explicitly is separation, keeping development feedback from ever reaching the final evaluation. Of EdgeBench's 134 tasks the study uses the 51 public ones, opening only 11 to one-way acceptance testing during development and reserving 40 for held-out evaluation.

Four mechanisms survive the funnel. Action Fusion combines a mutation and its follow-up command into a single request. Online Context Compact decides whether to compress context at task-completion boundaries. ObservationPack archives large outputs locally and, after the initial request, sends stable handles instead of full results. The Evidence-Preserving Reducer uses a lower-cost model to extract key evidence from build and test logs, with a verification fallback to the original.

What the four share reveals what the search actually found. None of them asks the model to think better; all four reduce the volume of text placed in front of the model. They cut round trips (Action Fusion), trim accumulated history (Online Context Compact), replace bulk output with references (ObservationPack), or delegate preprocessing to a cheaper model (Evidence-Preserving Reducer). That no prompt-and-policy direction made the final four tells you that at this search scale, the savings came from plumbing rather than from better instructions.

Measured results moved the same way on both models

The EdgeBench numbers are the same shape on both models, with token traffic, API cost, and average score all moving in one direction across GPT-5.6 Sol and Opus 5. On GPT-5.6 Sol, Pi used 2.1538 billion tokens and $1,339 in API cost for an average score of 44.833, while the efficiency configuration of SoL-Pi used 1.0990 billion tokens and $894 for 42.003. Tokens fall 49.0% and cost 33.2%, with the score holding at 93.7% of Pi. Expressed as cost per point, the figure drops from $0.5855 to $0.4174, a 28.7% reduction.

Opus 5 points the same direction. Pi used 2.3697 billion tokens and $1,741 for 44.756, and the SoL-Pi efficiency configuration used 1.3101 billion tokens and $1,158 for 42.224. Tokens fall 44.7% and cost 33.5%, with the score at 94.3% of Pi. The 44.7-49.0% range in the abstract is simply the two endpoints from these two models.

One more configuration deserves attention. On GPT-5.6 Sol, the performance configuration of SoL-Pi recorded 47.208 using 2.0224 billion tokens and $1,271, which is 2.375 points above Pi at $68 less cost. Tightening the same set of mechanisms toward savings gives up 2.8 points to cut cost by a third, while loosening them toward performance raises the score at a modest cost reduction. That is the evidence that token efficiency here is not purely a trade against capability.

Several figures call for caution. First, the $8.75-$13.50 hourly savings is an estimate, as the paper says, and rests on an agent running continuously. That assumption holds for near-100% utilization such as unattended overnight work, and does not reproduce in ordinary development flows where a human sits in the loop. Second, what the gap between 44.833 and 42.003 means in practice depends on how EdgeBench scores, and an average alone does not say which task families absorbed the roughly 3% drop. Third, the comparison against Pi is an improvement within one harness lineage, while the figures against native Codex and Claude Code harnesses are an indirect comparison across configurations.

Where this lands for engineering organizations

The practical implication of SoL-Pi is about where control over cost structure sits, because an organization running coding agents through an API cannot change the model but can change the harness sitting in front of it. Most engineering organizations do not train models; they run coding agents through an API, and under that constraint cost reduction usually narrows to two options, dropping to a cheaper model or capping usage. The third option SoL-Pi demonstrates is keeping the same model and reducing the text sent to it, and that layer is one API consumers can modify directly.

Two of the four mechanisms are comparatively easy to implement in-house. Storing build logs, test output, and large file dumps locally and passing references instead of pushing them into context, and pre-reducing long logs with a cheaper model behind a fallback that restores the original when the reduction is inadequate, both transfer directly into an internal agent pipeline. Action Fusion, by contrast, changes the tool-call contract and requires checking for conflicts with existing workflows first.

One reading of the absolute figures is worth stating. That a single EdgeBench run costs API money in the thousand-dollar range shows that the operating cost of unattended coding agents has already passed what an individual absorbs. Token-efficiency research is active now not because thrift is a virtue, but because running agents around the clock has started to hit a cost wall. An organization evaluating overnight automation or an always-on refactoring agent should calculate token consumption per operating hour before it compares capability benchmarks.

Four limitations the authors state themselves

The paper lists four limitations, and they narrow the scope of the result. The first is harness pre-training: sustaining gains requires scaling the search across environments more diverse than those used here. The second is multi-backend training. The current harness was optimized on GPT-5.6 Sol alone, and lower activation rates for the mechanisms were observed on Opus 5. Transfer to Opus 5 working is a different claim from SoL-Pi being optimal on Opus 5.

The third is that recursive efficiency improvement remains theoretical. Whether efficiency gains actually reduce the cost of future search is unexplored, and the recursive self-improvement framing only closes into a loop once that is demonstrated. The fourth is search cost: high compute expense prevented controlled comparisons of breadth against depth under a fixed budget.

One addition belongs to that fourth point. The paper reports more than 3,000 runs and more than 60,000 interactions but does not disclose what the search itself cost. Without knowing what was spent to find a harness that saves $8.75-$13.50 per hour, no other organization can judge whether reproducing the methodology in its own environment is worth it. Adopting the four discovered mechanisms and running the same automated search are decisions of entirely different magnitude, and the option genuinely open to most organizations is the former.

Sources: arXiv 2609.20519, "SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness" (submitted September 17, 2026, NVIDIA et al.), drawing on the abstract, Tables 1 and 2, and the limitations section. EvoHarness-RL, referenced for comparison, is arXiv 2608.05446.