EvoHarness-RL: How an 8B Model Reached 96.9% on ALFWorld by Learning Its Own Harness

EvoHarness-RL is a training framework that takes Qwen3-8B, an 8-billion-parameter model, to a 96.9% success rate on ALFWorld. Researchers from the University of Illinois Urbana-Champaign and Meta AI released it on August 5, 2026 as arXiv 2608.05446. In the same table, Claude Opus 4.5 with ReAct scores 96.4%, and the ReAct baseline on the same backbone scores 47.9%. The paper's claim is that the gain comes not from a larger model or a larger memory, but from training a policy that decides when to read and write an external workspace.

Moving the Harness From Prompt Convention to Learned Policy

The starting point of EvoHarness-RL is a BPE workspace that organizes agent external state into Belief, Progress, and Experience. Belief holds the current state of the environment, Progress records which subgoals are finished and which remain, and Experience stores knowledge carried over from earlier episodes. The agent touches this workspace only through four meta-actions: track, commit, recall, and note.

What changes here is ownership, not the components. Conventional agents leave the questions of how to build state and when to retrieve it to prompts, heuristics, and domain-specific conventions written by hand. The paper names this as the problem, describing the external workspace and its usage policy as manually engineered. EvoHarness-RL learns that usage policy offline and deploys it at runtime, which moves the harness from something a developer designs into something the model learns.

Training runs in two stages. Supervised harness fine-tuning first teaches the four meta-actions and how to construct useful external state. Cost-aware GRPO then explores when to read, update, and consolidate that state. The design axis of the second stage is the cost term inside the reward.

The Conditions That Belong Next to That 96.9%

The reported 96.9% is the result of training a Qwen3-8B policy model with GRPO for 150 epochs on eight NVIDIA H200 GPUs. The setup lists a group size of 8, a learning rate of 1e-6, a KL coefficient of 0.01, a maximum prompt length of 12,288 tokens, and vLLM with TP=4 as the inference engine. Episodes cap at 70 steps, Progress holds 8 subgoals, and the Experience store holds 80 entries per category.

The most important condition sits outside the results table. Claude Opus serves as the teacher for SFT trajectory collection and as the consolidation model for the experience store. Running the teacher across 500 ALFWorld training games and keeping only successful episodes yielded 87 trajectories and 1,153 next-action conversation pairs, averaging 26.5 turns per episode. The teacher issued 405 harness calls in total, roughly 18% of all turns, split into commit (202), recall (114), note (55), and track (34).

Read plainly, the sentence "an 8B model beat Opus 4.5" summarizes a result in which Opus participated as a teacher inside the training pipeline. The accurate reading is not that a frontier model was replaced, but that a frontier model's harness-use habits were transplanted into an 8B policy and then refined with reinforcement learning. The useful lens is where cost moved rather than where performance moved: Opus calls shifted from inference time to training time.

Why Harness Calls Fall as Training Continues

Harness annealing is the pattern where per-episode harness calls converge toward roughly one call as GRPO training proceeds. The SFT-initialized agent starts by calling the harness often, using it as an explicit scaffold to track state, recall procedures, and narrow the search space. As reinforcement learning progresses, usage drops quickly and stabilizes near one call per episode.

The curve is interesting because it runs against intuition. The agent was trained to use its tools well and ended up using them less. The paper attributes this to recurring scaffolded patterns being internalized into the policy, leaving external access only where the expected benefit outweighs its step cost. The cost term in the reward pays off precisely here.

For practitioners, this observation is a counterexample to treating tool-call frequency as a proxy for agent quality. A chatty agent may still need the scaffold, and a quiet agent may have already internalized the procedure. The same number points at opposite states, and a log that shows only tool usage cannot separate them.

Why the 86.6% on the Unseen Split Carries More Weight

On the ALFWorld unseen split, EvoHarness-RL reaches 86.6% while ReAct on the same backbone reaches 50.0%. The interesting values sit in the middle configurations. EvoHarness-Base, which adds BPE through prompting alone with no training, scores 77.6%, while EvoHarness-SFT, which adds supervised training, drops to 69.4%.

The paper explains that inversion by noting supervised imitation learns the teacher's harness-use patterns without learning when access is actually worthwhile. Habits observed in training environments transfer into new ones at a loss. Only with cost-aware GRPO does the number climb to 86.6%.

The unseen-split figure is closer to practice than the headline 96.9%. The higher number comes from the same distribution the policy trained on; the lower one comes from unfamiliar layouts, and that is what an adoption decision should weigh. The same table also carries no unseen-split score for Opus 4.5, which means the headline comparison holds only where the training distribution overlaps.

Where This Diverges From ExpeL, ReasoningBank, and SkillOS

On the same Qwen3-8B backbone, ExpeL scores 49.3%, ReasoningBank 55.7%, MemP 49.7%, Dynamic Cheatsheet 52.1%, and ACE 51.4%. Among trained approaches, plain GRPO reaches 65.6%, SkillOS reaches 80.2%, and SkillRL on a Qwen2.5-7B base reaches 89.9%.

EvoHarness-RL's position in that lineup is specific. The paper argues that existing self-evolving agents separate cross-episode skill curation from within-episode state tracking into different modules. BPE merges the two into one interface, so a single policy governs not only how long-term experience gets used but how it stays synchronized with active environmental belief and execution progress.

The ablations price that merger. Removing Belief from the prompt-time BPE harness drops it from 56.4% to 50.0%, removing Progress lands at 50.7%, and removing Experience lands at 48.6%. None of the three axes substitutes for another, which is exactly where memory-only approaches part ways with this design.

What a Team Can Take From the Paper Today

The transferable parts of EvoHarness-RL are the BPE interface partition and the cost-aware reward design, not the eight H200 GPUs. Splitting an in-house agent's external state into Belief, Progress, and Experience pays off at the prompt layer alone, which is what EvoHarness-Base's 77.6% on the unseen split demonstrates. That configuration requires no training at all.

The reward design travels even more easily. A structure that prices tool calls and optimizes them jointly with a success reward can be imitated as an evaluation metric in pipelines that never touch reinforcement learning. Multiplying an internal success-rate dashboard by a per-call cost selects prompts that reach the same success rate with fewer calls.

Plans to reproduce the training stage deserve more caution. They carry teacher-model API calls, an eight-GPU budget, and 150 epochs, and this paper offers no evidence that the result holds outside ALFWorld.

The Open Question: Is One Benchmark Enough

The evaluation in EvoHarness-RL is confined to a single environment, ALFWorld, across six task families. Pick, Look, Clean, Heat, Cool, and Pick2 all involve object manipulation in a simulated household, and whether the same BPE partition survives in messier state spaces such as code repositories, web browsers, or internal tool chains remains unanswered. The paper itself notes that the belief tracker is a rule-based action-observation parser, and that component has to be rebuilt for every environment.

The gap between 96.9% and 96.4% also deserves a literal reading. The difference is 0.5 percentage points, and the table publishes neither standard deviations nor confidence intervals, so nothing in the paper establishes that the gap exceeds noise. What the result does establish is not a ranking but a range: an 8B open model has entered a band where it is indistinguishable from a frontier model, and the tool that put it there was a learned harness policy rather than a bigger model.

Source: arXiv 2608.05446, "EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents" (Xuying Ning et al., University of Illinois Urbana-Champaign and Meta AI, released August 5, 2026; BPE workspace with track, commit, recall and note meta-actions, two-stage training of supervised harness fine-tuning and cost-aware GRPO, ALFWorld success rates of 96.9% for EvoHarness-RL, 96.4% for Claude Opus 4.5 with ReAct, 98.5% for Opus 4.5 with EvoHarness-Base, 60.7% to 85.0% for GPT-5, 47.9% to 70.0% for GPT-4.1, 47.9% for the Qwen3-8B ReAct baseline, 89.9% for SkillRL, 80.2% for SkillOS, 49.3% for ExpeL, 55.7% for ReasoningBank, 49.7% for MemP, 52.1% for Dynamic Cheatsheet and 51.4% for ACE; unseen-split scores of 86.6%, 77.6%, 69.4% and 50.0%; ablations at 50.0%, 50.7% and 48.6%; Qwen3-8B as policy model with Claude Opus as teacher and consolidation model; SFT data of 87 trajectories and 1,153 conversation pairs from 500 games averaging 26.5 turns with 405 teacher harness calls; GRPO for 150 epochs on eight H200 GPUs with vLLM TP=4), compiled by ASAP.