Two searches improve the same agent's working method. The one that scores higher during selection transfers less well to other benchmarks. Their retained edits help explain what the scores leave unresolved.
An AI completes a batch of tasks, reads the feedback, revises how it works and attempts the tasks again. Each round helps choose the next revision. As the score rises, those tasks are doing two jobs: measuring the agent and shaping the method it will use afterward.
That creates a problem for judging self-improvement. A procedure selected by repeated grading may help on new work, but the selection score cannot establish how far that help extends. RRSI's workspace experiment makes the distinction visible: Meta-Harness wins on the tasks used to choose edits, while RRSI wins on the average across three other benchmarks. Both improve that average over the starting agent. Understanding the reversal requires looking at what each search kept.
The model's weights remain fixed. The editable part is the harness: the instructions, tools and procedures surrounding the model, including which files it reads, how it checks its work and when it stops. The paper calls the repeatedly used selection tasks the evolve set. Results from those tasks guide successive edits; after the search, the final harness runs unchanged on tasks excluded from selection. Harness and evolution protocol, v3
The comparison begins with Harvey LAB, a legal-work benchmark. Its tasks supply Word, Excel and PDF sources and require deliverable files under specified names. RRSI and Meta-Harness start with the same harness and frozen model, use the same evolve tasks and receive the same candidate budget. That matches the candidate allowance, not token consumption or dollar cost. Workspace setup and evaluation
According to the authors' inspection, Meta-Harness adds a 730-word standing system instruction. For each item, it organizes work around citation, the controlling rule or playbook provision, severity, consequence and a recommended fix. Those dimensions become part of the agent's routine. The instruction names no individual task, but the authors interpret it as restating the evolve suite's grading scheme. This alignment can occur at the level of grading dimensions.
RRSI's final additions act at a different point in the work. The authors report four accepted edits from 32 evaluated candidates over 20 rounds: one coverage and depth recheck and three small programs that read deliverables back. All attach to a check before finalization that the starting harness already had. One flags a source value absent from every written file. It supplies feedback about the output at the moment the agent is about to finish. These are author-reported descriptions of the retained harnesses. Appendix F, Table 9
The diagram below makes that last check concrete. In this teaching scenario, a task requires an amount, represented by x, from source.pdf to appear in report.docx. The report exists, but x is missing. Reading the report back exposes the omission and directs the agent back to that draft before completion.
The filenames and x are editorial placeholders. The diagram illustrates feedback to a draft; it records no experimental repair.
There are two decisions here. Within a task, omission feedback directs further work on the current report. Across rounds, the improvement search decides whether the checking procedure stays in the harness for later tasks. In the illustration, x belongs to the current input; the procedure for reading the output back is the element that could carry forward.
That makes reuse easier to reason about. The usefulness of Harvey's standing checklist depends on whether its grading dimensions fit the next assignment. A check for information missing from a required deliverable can remain meaningful when the subject changes, provided the relevant sources and outputs remain accessible. JobBench exposes a filesystem and tools for producing office and PDF files; APEX-Agents also supplies file and document tools. These interfaces preserve a condition that readback needs. The authors say the retained probes contain neither rubric nor domain vocabulary and still operate on all three transfer benchmarks. File environments and retained probes
This is a plausible explanation for reuse, with a specific boundary. File access makes the check possible; it does not make the deliverable correct. Discovering that x is absent can expose an omission. Finding x somewhere in the report would not establish that it was used correctly or that the reasoning was sound.
Harvey is partitioned once into 120 evolve tasks and 40 same-suite held-out tasks. The separate cross-benchmark tests are JobBench, GDPval and APEX-Agents. The reported results are:
| Harness | Harvey evolve score | Mean on the three other benchmarks |
|---|---|---|
| Starting harness | 89.4 | 39.7 |
| Meta-Harness | 93.0 | 40.6 |
| RRSI | 90.5 | 43.6 |
The Harvey score is the fraction of rubric criteria passed, rather than the fraction of fully solved tasks. The last column averages three benchmark scores; it is neither pooled task accuracy nor a scale of general intelligence. Tables 1, 2 and 9
On the selection tasks, Meta-Harness ranks higher. On the cross-benchmark mean, RRSI ranks higher. The same-Harvey test adds an essential qualification: on the 40 reserved tasks, both score 89.2. That test does not favor RRSI, and Meta-Harness's cross-benchmark mean also exceeds the starting harness's. The comparison concerns stronger transfer, not an absence of transfer from Meta-Harness.
The retained edits help interpret these measurements. Repeated use of a grading scheme gives a search repeated opportunities to align its working method with that scheme. The authors' rubric-shaped instruction is a concrete example of their concern. Readback instead addresses an identifiable step between producing a file and declaring the task finished. That difference suggests why the procedures might travel differently. The score reversal alone cannot prove this explanation.
RRSI puts controls on both proposing a change and allowing it to persist. On the proposal side, it gradually reduces the number of independent edits a candidate may bundle. It also records the component changed, the hypothesis, the actual diff, score and cost changes, and whether the candidate was accepted. Later proposals use that history, including rejected ideas. The authors' rationale for smaller late-stage updates is attribution: fewer simultaneous changes make it easier to identify what produced a measured improvement. Proposal controls
On the selection side, a critic reads the actual diff before full evaluation and rejects explicit benchmark-specific content, including task names, entities, task-specific values and answers. Generic improvements to prompts or tool descriptions remain eligible. Candidates must meet admission criteria before a score can justify replacing the current harness. This constrains how feedback becomes permanent behavior; it does not ban a particular kind of component. Candidate selection
A screen for explicit benchmark content still leaves transfer to be measured. The Meta-Harness example shows why: an instruction can name no task while remaining shaped around a recurring rubric. Passing that narrower screen would not answer whether its work method remains useful elsewhere.
The paper supplies additional evidence by removing the controls in groups. Removing either the proposal-side or acceptance-side regularizers raises the evolve score and lowers the three-benchmark mean. These ablations share the base harness, model, evolve split, round count and candidate budget. They support a contribution from the grouped controls to transfer in this experiment. They do not separately establish the effect of the critic, the smaller-edit rule or any one readback probe. Matched ablations, Table 2
The missing-amount scenario brings the opening question back into focus. Sending an agent back to one report concerns repairing that task. Keeping the checking procedure and testing it on tasks excluded from selection concerns improving its future work. RRSI's reported workspace results support transferable improvements to procedures around a fixed model. They do not establish general or unlimited recursive self-improvement. To understand the claimed improvement, the score must be connected to the edit it selected and the new work on which that edit still helped.
This analysis presents author-reported results and retained-harness descriptions from the frozen arXiv v3 material, alongside editorial explanations. It provides no independent replication results and makes no claim that the findings first appeared in v3.
The primary materials used in this analysis. Source versions stay explicit.
https://arxiv.org/abs/2609.24972v3 | L8; L27-L29
https://arxiv.org/abs/2609.24972v3
https://arxiv.org/html/2609.24972v3 | L78–L120; L159–L163
https://arxiv.org/html/2609.24972v3 | L126-L158
https://arxiv.org/html/2609.24972v3 | L168–L207
https://arxiv.org/html/2609.24972v3 | L307-L338
https://arxiv.org/html/2609.24972v3 | L549–L577
https://arxiv.org/html/2609.24972v3
Recorded attempts let policies test different continuations without repeating discovery. The Lasso results show why fewer search calls and faster solver execution must be read separately.
A fixed coder, four patch candidates, one execution: the code experiment isolates predictor adaptation. The wider framework adds conditional updates and checks on which policy–model pair to retain.