Environment-Probing Curation: Checking Agent Memory Before Writing It Raised Pass Rate From 39% to 73%

Microsoft researchers introduced environment-probing curation in arXiv paper 2609.11060, published on September 10, 2026, giving a post-task curator agent read-only world tools to verify a candidate memory before it is written. On the CLBench database-exploration benchmark the method raises pass rate from 39% to 73% and pass-discounted reward from 8.60 to 22.60, while cutting queries from 8.8 to 4.7 per question and task-agent cost from $3.38 to $1.68. The point is that the task agent gained nothing. The model, retriever, memory representation, and production write authority stay unchanged, and only the curator receives new tools, which is why the authors describe the gain as an improvement in write-time evidence quality. ASAP works only from the numbers and design stated in the paper to separate what is demonstrated from what is left open.

Memory written from a finished trajectory fails in five specific ways

The problem the paper names is not memory representation or retrieval but the evidence boundary. A post-task curator normally sees only the completed trajectory and the grade, and a trajectory is a single, partial, often mistake-laden observation of the environment. A record built from it can memorize an instance answer instead of the procedure, inherit an inefficient path, assert an unverifiable scope, leave blind spots in regions the agent never visited, or go stale as the world changes. Deferring verification to task time forces the next agent to spend scarce tool calls rechecking uncertain memories rather than solving the task in front of it.

Environment-probing curation inserts one step. After a task closes, an asynchronous curator receives the raw trajectory, a distilled version of it, and the terminal grade, proposes a candidate record, makes targeted read-only calls to investigate the specific uncertainty, and only then creates, revises, narrows, deletes, or skips the record. The probes are pointed: distinguishing an incidental answer from a reusable relation, comparing the observed procedure against a shorter path, testing a claimed relation on another slice, checking a procedure's preconditions, inspecting states the trajectory omitted, and re-querying the environment when drift is suspected.

Half of the method is the permission design

What the curator receives is a least-privilege, read-only subset of existing connectors or MCP tools. Probes cannot mutate the environment, cannot enter the task trajectory, cannot consume the task agent's budget, and cannot see future tasks or labels. Because they run asynchronously off the user-facing critical path, they add no latency, and they inherit platform authentication and auditing. Where no safe read surface exists, the system falls back to trajectory-only curation.

The task agent, meanwhile, is locked down. It reaches memory only through memory_read and holds no create, update, or delete tool, so it cannot change shared memory while solving a task. Write authority belongs to the curator alone, and the curator finishes before the next task is revealed, so future tasks cannot leak into memory. No retraining is involved. That design is what makes the measured increment attributable to evidence quality rather than to added task-agent capacity.

Four configurations diverge on the 40-question CLBench schedule

The primary experiment runs GPT-5.4 on a 40-question drift schedule where a hidden SQLite schema changes after question 20, testing reuse and stale-memory repair together. Joins are hidden, timestamp and price encodings are mixed, and drift renames fields and adds soft deletes. The comparison covers GHCP with no memory, GHCP + Full ICL which prepends prior trajectories, GHCP + Mem which uses an indexed store, and GHCP + Mem with environment probing. Every configuration was run five times with paired seeds.

All three memory configurations beat the baseline. Pass rate rises from 39% to 61–73% and total reward from 8.60 to 20.00–22.60, which is 2.3–2.6 times the baseline, while queries fall from 8.8 to 3.0–5.6 per question and task-agent cost from $3.38 to $1.68–$2.01. The gain is not brute force: the budget previously spent rediscovering schema, encodings, and tool conventions is what shrinks.

The configurations differ in character. Full ICL uses the fewest queries at 3.0 but consumes 5.42M input tokens because its context grows with the stream. Indexed memory reaches 70% pass at 2.13M input tokens, and the probing variant reaches the highest pass rate and reward at only 1.69M. On the drift curve, probing already leads trajectory-only memory at the migration boundary (0.541 versus 0.486 cumulative reward) and finishes at 0.565 versus 0.500, while the memoryless run ends at 0.215.

A separate 30-question schedule without drift isolates the effect across models. On Opus 4.7, trajectory-only memory improves over its paired baseline by 0.252 and probing by 0.263; on Sonnet 4.6, the figures are 0.351 and 0.421. Probing holds the highest mean reward on both models, at 0.721 and 0.748.

On APEX, the difference shows up in reward per dollar

The second benchmark adapts 90 management-consulting tasks across six document worlds that require discovery and quantitative analysis over PDF, XLSX, DOCX, and PPTX files. All 18 memory-versus-baseline mean reward comparisons, six worlds by three systems, are positive, and task-agent tool calls fall by 16–75%. Measured as reward gain per task-agent dollar, probing wins five of six worlds, with trajectory-only memory marginally ahead in the last one.

The largest reduction lands where baseline discovery costs the most. In world 941eba66, tool calls fall from 71.6 to 17.7–19.3, input consumption from 53.92M tokens to 6.56–7.67M, and cost from $54.30 to $7–$9 per run. Across the suite, baseline tool-call means of 30.0–71.6 drop to 13.9–28.4. Full ICL can win raw reward in an individual world, but its growing context makes those wins expensive, and in one world it costs more than the stateless baseline.

The sentences inside the records explain the result

The qualitative comparison in arXiv 2609.11060 is where the shift becomes concrete, because the difference between the two curators is a difference in what each record instructs a later agent to do. Trajectory-only curation produced an answer-anchored warning: do not answer with AVG(items_g2.prc_usd) over non-null rows, because that yields about 52.96 while the benchmark's correct result is 96.23, so a different price field or row subset is required. The probing curator wrote an executable procedure instead: join items_g2 to taxn_g2 on ref_id, filter cat_lvl=1 and the exact cat_nm, keep items_g2.prc>0, then compare against the filtered AVG(prc). A stale record telling the agent to use attrs_g3 became one pointing at product_attributes_g3 with brand filters and grouping attached.

Matched trajectories tell the same story. On one CLBench task, the memoryless run used seven queries and failed, trajectory-only memory used nine and passed, and the run holding a probe-validated ref_id relation passed in two. On a hard APEX task, the memoryless run failed both criteria after 96 calls, trajectory-only memory transferred a revenue-per-head procedure and passed in 11, and a validated workbook map brought probing down to six. The authors state explicitly that these selected cases do not establish the aggregate effect.

Read the impressive numbers and the cautious ones separately

The impressive part is that accuracy and cost moved the same direction. Raising a pass rate usually means allowing more exploration and longer context, which costs more. Here the pass rate climbs from 39% to 73% while input tokens fall from 3.14M to 1.69M and cost from $3.38 to $1.68, because repeated environment discovery moved out of task time and into curation time, which sits off the user's critical path.

The cautious part is accounting and uncertainty. The reported costs are task-agent costs, with distillation and curation tracked separately and excluded, so a total-cost-of-ownership comparison that includes the probes is a question this paper does not answer. The confidence intervals also overlap: trajectory-only memory reports 70±16 pass rate against 73±5 for probing, so the variance exceeds the gap between the means, and the more defensible contribution of probing is the tighter spread rather than the higher center.

The authors themselves treat the effect as conditional. Probing's additional gain reaches +1.77 in world 2a87e5cb and +1.09 in 2f84c98b but is −0.04 in 941eba66, and in the no-drift setting it adds +0.075 on Sonnet 4.6 against +0.025 on Opus 4.7. Their reading is that probing helps most when a trajectory leaves a join, a file location, or a procedure unresolved, and adds little when the trajectory already supports an actionable record. Because the uncertainty intervals overlap, they present this as a mechanism interpretation rather than a resolved subgroup effect.

Conditions to check before porting this design

The first condition is a safe read surface, since the design assumes a least-privilege subset of connectors or MCP tools already exists. Where an internal database or document store already exposes read-only access with audit logging, the method adds a curator permission on top of the existing stack. Where that access has to be built, the build is the real adoption cost, and the paper says such deployments fall back to trajectory-only curation.

The second condition is the shape of the workload. The experiments assume a stream of related tasks in one environment, no task repeated, and an environment that can change without announcement. Recurring internal analytics requests and repeated document-based work fit that shape; one-off tasks in ever-changing environments never accumulate the discovery that memory is supposed to amortize.

The third condition is the scoring convention. Reward multiplies a binary pass by the share of the tool-call budget left unspent, with the budget fixed at 15 SQL queries for CLBench and 100 tool calls for APEX, and multi-criteria tasks scored as a strict pass where every criterion must clear. Reproducing this evaluation internally means setting that budget for your own workload first and deciding whether strict pass is the right bar.

What the paper leaves open

The first gap is total cost, since the probe phase is excluded from the cost tables, leaving open whether probing is still cheaper than trajectory-only memory once curation is counted. The second is scale: the streams are 40 and 30 CLBench questions and 90 APEX tasks, and how a curator handles duplication and conflict once records number in the hundreds or thousands is not examined. The third is the safety of the read-only boundary itself, which is presented as a design principle rather than tested, with no experiment on whether probes might pull sensitive data into long-lived memory.

The contribution here is not a new memory structure but a change in what evidence is available at write time. Because the task-time interface stays fixed and only the curator gains tools, the source of the improvement is unusually well isolated. The next checkpoints are a comparison that includes curation cost and a replication in an environment where records have accumulated over a long deployment.

Source: Suresh, Mak, Bhatnagar, Methani and Gutierrez Munoz, "Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents" (arXiv:2609.11060v1, September 10, 2026, Microsoft)