Self-Evolving Agents (2023–2026)

Executive summary

The field has moved through four recognizable stages. In 2023, agents began retaining linguistic reflections, experiences, and executable skills across attempts. In 2024, researchers made more of the agent stack optimizable—parameters, prompt graphs, workflows, tools, and source code. In 2025, archive-based code evolution and proposer–solver co-evolution made the loop more autonomous. By 2026, the frontier had shifted to evolving the learning machinery itself: memory architecture, skill managers, critics, consolidation rules, and meta-improvement procedures.

The headline finding is narrower than the phrase recursive self-improvement suggests. There is strong evidence that bounded agents improve on repeated tasks when they can store experience, train on verified trajectories, or search over agent code. There is not yet strong evidence of open-ended, domain-general, safe, and indefinitely compounding improvement. Nearly every convincing result depends on one or more external anchors: a fixed foundation model, a task generator, a reward function, an executable verifier, a benchmark, a simulator, or a human gate.

Three scientific conclusions are especially robust:

1. The substrate matters. Textual memory is cheap and reversible but context-bounded; parameter updates compress experience but introduce forgetting and drift; prompt/workflow/code evolution expands the design space but makes validation and safety harder.
2. Verification is the rate limiter. The largest gains occur in coding, mathematics, games, web tasks, and tool use—domains with cheap feedback. Intrinsic self-critique alone is unreliable, and self-judging loops can reward-hack.
3. Evaluation is behind capability. Most papers report endpoint success on a fixed benchmark. True evolution requires measuring the whole trajectory: forward transfer, forgetting, adaptation speed, cost, safety drift, and regression under a concealed task stream.

The most promising frontier is therefore not an unconstrained agent rewriting itself. It is a versioned, modular evolution stack in which memories, tools, skills, prompts, policies, and meta-optimizers can change under independent evaluation, capability-scoped permissions, regression tests, and rollback.

1. Scope and definition

1.1 Evidence window and method

The main window is 12 August 2023–12 August 2026. Three earlier 2023 papers—Reflexion, Self-Refine, and Voyager—are included as essential precursors because the later literature directly inherits their feedback, memory, and skill-library loops. The evidence base contains 46 primary papers with verified arXiv metadata and locally rendered full text. Technical claims are weighted by source status: peer-reviewed papers and reproducible artifacts receive more weight than recent preprints; 2026 results are treated as provisional unless an accepted venue is verified.

The survey uses an evidence-first reading standard: separate authors' claims from demonstrated results, state the adaptation substrate, identify whether learning persists across episodes, map claims to experiments, and record threats from evaluator coupling, benchmark leakage, missing controls, compute, and safety.

1.2 Operational definition

Let an agent at episode $t$ be

$$A_t = (\theta_t, M_t, P_t, T_t, W_t, C_t, U_t),$$

where $\theta$ denotes model parameters, $M$ memory, $P$ prompts, $T$ tools, $W$ workflow or communication graph, $C$ executable agent code, and $U$ the update procedure. After observing trajectory $\tau_t$ and feedback $f_t$, an evolution operator produces

$$A_{t+1} = \mathcal{E}(A_t, \tau_t, f_t).$$

This survey calls the system self-evolving only when at least one change persists and affects later episodes. A single generate–critique–revise loop is within-episode self-refinement: scientifically important, but not lifelong evolution [@madaan_2023_self]. Likewise, an offline optimizer that finds one prompt or workflow is agent design automation, not necessarily a deployed agent learning over its lifetime.

This boundary exposes a spectrum rather than a binary category:

Level	Persistent change	Representative mechanism	What it does not establish
0. Within-episode correction	None	Self-Refine	Cross-task learning
1. External experience	Memory, rules, workflows, skills	Reflexion, ExpeL, Voyager	Parameter learning
2. Policy consolidation	Model weights or learned critic	ReST–ReAct, Agent-R, Q-Evolve	Architecture evolution
3. Agent-system evolution	Prompts, tools, graph, code	Symbolic Learning, ADAS, DGM	Safe/open-ended progress
4. Meta-evolution	Update procedure itself	Hyperagents, MetaSkill-Evolve	Unbounded recursive improvement

2. The development trajectory

flowchart LR
  A["2023: Feedback loops<br/>reflection · experience · skills"] --> B["2024: Optimizable stack<br/>weights · graphs · workflows · code"]
  B --> C["2025: Co-evolution<br/>archives · world models · proposer–solver"]
  C --> D["2026: Meta-evolution<br/>memory architecture · consolidation · update rules"]
  V["External verification"] -. "enables" .-> B
  V -. "selects" .-> C
  G["Governance and rollback"] -. "emerging constraint" .-> D

2.1 2023: feedback becomes reusable experience

The first stage converted model feedback into persistent, human-readable state. Reflexion stores verbal feedback in an episodic buffer and conditions future attempts on it; on HumanEval it reported 91% pass@1 versus 80% for GPT-4, without updating weights [@shinn_2023_reflexion]. ExpeL goes beyond a task-local reflection: it extracts reusable insights from successful and failed trajectories and retrieves both insights and concrete experiences on later tasks [@zhao_2023_expel]. CLIN makes memory more causal and selective, reporting a 23-point advantage over Reflexion in repeated ScienceWorld trials and additional transfer after continual memory updates [@majumder_2023_clin].

Voyager demonstrated the same principle in an embodied environment. Its automatic curriculum proposes progressively novel goals; successful programs enter an executable skill library; environment errors and self-verification repair failures. In Minecraft, it reported 3.3 times as many unique items, 2.3 times the travel distance, and up to 15.3 times faster milestone acquisition than prior baselines [@wang_2023_voyager]. The key conceptual step is that experience becomes actionable code, not merely retrieved prose.

Parallel work made optimization itself linguistic. OPRO asks an LLM to propose new prompts from previously scored candidates [@yang_2023_large]. Promptbreeder evolves both task prompts and the mutation prompts that generate them, an early self-referential population loop [@fernando_2023_promptbreeder]. SELF and ReST-meets-ReAct begin using self-generated feedback or successful reasoning trajectories to update weights [@lu_2023_self; @aksitov_2023_rest].

The period established the canonical loop—act, evaluate, reflect, retain—but the agent body remained mostly fixed. Gains came from a small memory buffer, a skill library, or an offline training pass. Open-endedness and long-term stability were not tested.

2.2 2024: the whole agent stack becomes an optimization target

The second stage widened what could evolve. SPIN and AlphaLLM use self-play or MCTS-like imagination plus critics to generate training signal for model parameters [@chen_2024_self; @tian_2024_toward]. AgentGym combines multiple interactive environments, a trajectory set, and AgentEvol to move beyond a single benchmark [@xi_2024_agentgym]. OpenWebVoyager iteratively explores the live web, filters trajectories with feedback, and updates a multimodal policy [@he_2024_openwebvoyager].

At the system level, GPTSwarm represents agents as computational graphs and optimizes prompts and connectivity [@zhuge_2024_language]. Symbolic Learning treats prompts, tools, and pipelines as symbolic weights, with language analogues of gradients and back-propagation [@zhou_2024_symbolic]. AutoFlow learns natural-language workflow programs [@li_2024_autoflow]. ADAS formalizes automated design of agentic systems and demonstrates Meta Agent Search, where a meta-agent writes new agent implementations into an archive [@hu_2024_automated]. Gödel Agent makes the running agent's own code and logic editable, reporting an 11-point MGSM improvement over Meta Agent Search and a rewrite that reached 100% on Game of 24 after earlier optimization attempts failed [@yin_2024_g].

Memory also becomes procedural. Agent Workflow Memory induces reusable workflows from trajectories and reports 24.6% relative improvement on Mind2Web and 51.1% on WebArena [@wang_2024_agent]. This illustrates a recurrent finding: raw trajectories are often too detailed; abstract procedures transfer better.

The important discontinuity is that researchers stop treating the foundation model as the whole agent. The optimizable object becomes a compound program. However, most methods are still run as offline search against a visible development benchmark. They discover an agent, then freeze it. That is automated engineering, adjacent to but not identical with lifelong deployed evolution.

2.3 2025: archive search, co-evolution, and zero-data curricula

In 2025, three streams converge.

First, recursive code evolution becomes empirically strong. Darwin Gödel Machine (DGM) maintains an archive of coding agents. A foundation model samples a parent, edits its Python implementation, evaluates the child, and retains diverse stepping stones. On the reported setup, SWE-bench performance rose from 20.0% to 50.0% and Polyglot from 14.2% to 30.7% [@zhang_2025_darwin]. The archive matters because some modifications are not immediately best but enable later descendants. Yet the external model is frozen, the fitness function is a fixed coding benchmark, and offspring—not one continuously learning identity—form the unit of evolution.

Second, reflection becomes an optimizer. GEPA turns full traces into natural-language diagnoses, proposes prompt changes, and retains a Pareto frontier. Across six tasks it reported a six-point average advantage over GRPO, up to 20 points on an individual task, with as much as 35-fold fewer rollouts [@agrawal_2025_gepa]. Agent-R uses MCTS to construct error-correction trajectories and iteratively trains the policy to recognize and repair its own mistakes [@yuan_2025_agent].

Third, agent and environment co-evolve. WebEvolver trains a world model that both generates imagined trajectories and supports look-ahead, reporting roughly a ten-percent improvement over a strong self-evolving web baseline [@fang_2025_webevolver]. R-Zero co-evolves a problem proposer and solver using verifiable reasoning rewards [@huang_2025_r]. Agent0 extends this idea to tool-integrated reasoning; its Qwen3-8B base reportedly improves about 18% on mathematics and 22–24% on general reasoning without curated external data [@xia_2025_agent]. EvolveR combines trajectory-to-principle distillation, online retrieval, and policy reinforcement [@wu_2025_evolver].

Meanwhile, SiriuS bootstraps successful and repaired multi-agent reasoning traces [@zhao_2025_sirius], A-MEM reorganizes memories into linked, evolving notes [@xu_2025_mem], ReasoningBank distills successful and failed trajectories into reusable reasoning strategies [@ouyang_2025_reasoningbank], and MemEvolve makes the memory program itself the search object [@zhang_2025_memevolve]. The field now has enough diversity for a systematic taxonomy—what evolves, when, how, and where [@gao_2025_survey]—and enough autonomy for a distinct failure mode: misevolution. Experiments across model, memory, tool, and workflow pathways find safety degradation after memory accumulation and vulnerabilities introduced through tool creation and reuse [@shao_2025_your].

2.4 2026: evolving how the agent learns

The 2026 frontier is meta-level adaptation.

For memory, MemSkill evolves a repertoire of memory operations; EvolveMem diagnoses failures and changes both stored knowledge and retrieval configuration, with guarded rollback; Mem²Evolve lets experience memory and capability assets such as tools mutually generate one another [@zhang_2026_memskill; @liu_2026_evolvemem; @cheng_2026_mem]. FORGE adds population broadcast so successful memory artifacts can spread across agents without gradients [@bogdanov_2026_forge]. CODESKILL learns a compact procedural skill bank and a policy for maintaining it [@li_2026_codeskill].

For parameters and supervision, Self-Consolidation converts failure-aware textual experience into learned parameters [@yu_2026_self]. Q-Evolve jointly trains an in-distribution process critic and the agent policy, addressing the distribution shift that occurs when a fixed critic scores a changing policy [@zhang_2026_self]. Training Language Agents to Learn from Experience trains the reflector itself rather than treating reflection as a prompt [@shalev_2026_training]. Dr. Zero and Tool-R0 extend proposer–solver co-evolution to search and real-world tool calls [@yue_2026_dr; @acikgoz_2026_tool].

For recursive improvement, Hyperagents makes both task behavior and the meta-agent program editable [@zhang_2026_hyperagents]. MetaSkill-Evolve separates a fast task-skill loop from a slower loop that changes the analysis, retrieval, proposal, and evolution procedure [@wang_2026_metaskill]. SEA moves in a complementary direction: it confines modification to a steering adapter and versioned harness, then admits changes through anytime-valid statistical gates and auditable certificates [@sengupta_2026_self].

This stage also produces sharper negative science. A controlled ALFWorld/BabyAI study finds that external memory does not eliminate continual learning's stability–plasticity dilemma; it relocates it into representation and retrieval. Abstract procedural memories transfer better than detailed trajectories, while fine-grained organization can improve forward transfer and simultaneously worsen forgetting [@hu_2026_when].

3. Technical approaches and current state of the art

3.1 Persistent memory and experience distillation

Mechanism. A frozen model stores trajectories, reflections, rules, workflows, or graph-linked notes. Retrieval conditions later actions. More advanced systems summarize, consolidate, forget, reorganize, or evolve the retrieval program.

State of the art. The strongest mature evidence is in bounded interactive environments: Reflexion, ExpeL, CLIN, Voyager, and Agent Workflow Memory show that abstracted experience and executable procedures improve repeated-task performance [@shinn_2023_reflexion; @zhao_2023_expel; @majumder_2023_clin; @wang_2023_voyager; @wang_2024_agent]. Recent systems make memory structure adaptive—A-MEM, ReasoningBank, MemSkill, MemEvolve, and EvolveMem—but many 2025–2026 results remain preprint-level [@xu_2025_mem; @ouyang_2025_reasoningbank; @zhang_2026_memskill; @zhang_2025_memevolve; @liu_2026_evolvemem].

Strengths. It is training-free, interpretable, reversible, and compatible with closed models. Memories can retain provenance and be deleted. Transfer is strongest when experiences are compressed into procedures rather than replayed verbatim [@majumder_2023_clin; @wang_2024_agent; @hu_2026_when].

Limitations. Context and retrieval become the bottleneck. Old and new experiences compete, negative transfer concentrates on difficult tasks, and memory poisoning or stale procedures persist. “No catastrophic forgetting” is false: forgetting reappears as retrieval interference [@hu_2026_when].

3.2 Self-training, reinforcement learning, and consolidation

Mechanism. The agent generates trajectories, critiques, preference pairs, or curricula, then updates model parameters. Verifiable outcomes, learned critics, or self-play provide rewards.

State of the art. ReST-meets-ReAct and SPIN established iterative self-training [@aksitov_2023_rest; @chen_2024_self]. AlphaLLM adds search and multiple critics [@tian_2024_toward]. Agent-R trains error correction [@yuan_2025_agent]. R-Zero, Agent0, Dr. Zero, and Tool-R0 co-evolve proposers and solvers without curated data [@huang_2025_r; @xia_2025_agent; @yue_2026_dr; @acikgoz_2026_tool]. Q-Evolve addresses critic–policy distribution mismatch, while Self-Consolidation connects external memory to parameter learning [@zhang_2026_self; @yu_2026_self].

Strengths. Parameters provide compact, fast reuse without expanding the prompt. In verifier-rich domains, automatic curricula can track the solver's competence frontier and reduce dependence on annotated datasets.

Limitations. The approach is expensive and difficult to reverse. Self-generated data and rewards can amplify correlated errors, collapse tail knowledge, or alter safety behavior. Most evidence comes from mathematics, coding, or APIs with deterministic correctness; open-ended rewards remain unreliable.

3.3 Prompt, workflow, graph, and tool evolution

Mechanism. A meta-optimizer edits symbolic components—system prompts, node prompts, communication edges, role assignments, tools, or natural-language workflows—while a frozen executor model runs candidates.

State of the art. OPRO and Promptbreeder optimize prompts [@yang_2023_large; @fernando_2023_promptbreeder]. GPTSwarm optimizes agent graphs, Symbolic Learning propagates textual gradients through the stack, AutoFlow generates workflows, and ADAS searches agent programs [@zhuge_2024_language; @zhou_2024_symbolic; @li_2024_autoflow; @hu_2024_automated]. GEPA is a strong recent reflective optimizer with favorable rollout efficiency [@agrawal_2025_gepa].

Strengths. Changes are inspectable and portable across model APIs. The search can discover interactions humans overlook, and it avoids full fine-tuning.

Limitations. Many systems are better described as automated agent design than continual evolution: they search on a development set and deploy one static result. They are highly evaluator-dependent and vulnerable to prompt or workflow overfitting.

3.4 Executable skill and agent-code evolution

Mechanism. The evolving object is executable code: task skills, tools, the agent harness, or the mutation algorithm. Sandboxed execution and tests provide fitness.

State of the art. Voyager proves the skill-library pattern in Minecraft [@wang_2023_voyager]. Gödel Agent rewrites its own logic [@yin_2024_g]. DGM supplies the clearest quantitative evidence for archive-based recursive code improvement [@zhang_2025_darwin]. Hyperagents and MetaSkill-Evolve push recursion into the update procedure [@zhang_2026_hyperagents; @wang_2026_metaskill].

Strengths. Code can create genuinely new operations rather than only changing language context. Executable tests give denser, more objective feedback than LLM judgments. Archives preserve diverse stepping stones and rollback points.

Limitations. Self-modified code expands the attack surface and can manipulate tests, resource accounting, or the evaluator. Coding benchmarks align the modification language with the task language unusually well; this advantage may not transfer to social, scientific, or physical domains. Sandboxing, reproducible compute budgets, and held-out evaluation are essential.

3.5 Co-evolutionary agents, environments, and populations

Mechanism. Two or more adaptive processes create each other's curriculum or selection pressure: proposer/solver, agent/world-model, player/opponent, or a population sharing artifacts.

State of the art. WebEvolver couples policy and world model [@fang_2025_webevolver]. R-Zero and Agent0 couple task generation with solution learning [@huang_2025_r; @xia_2025_agent]. FORGE broadcasts memory artifacts across a population [@bogdanov_2026_forge]. DGM's archive preserves divergent lineages [@zhang_2025_darwin].

Strengths. Co-evolution can continually create problems near the agent's competence frontier and maintain diversity that a single greedy lineage loses.

Limitations. Coupled learners can collude on an easy private language, exploit the same evaluator, or drift away from the real environment. World-model errors become training data. Diversity metrics and external anchor tasks are needed to show genuine open-endedness.

3.6 Cross-approach comparison

Approach	Persistent substrate	Best-supported benefit	Principal failure mode	Evidence maturity
Memory/experience	Text, graphs, workflows, skills	Cheap cross-episode reuse and transfer [@majumder_2023_clin; @wang_2024_agent]	Retrieval interference, poisoning, context growth [@hu_2026_when]	Medium–high
Parameter learning	Policy and critic weights	Compact reuse; strong verified-task gains [@yuan_2025_agent; @xia_2025_agent]	Forgetting, drift, synthetic-data feedback	Medium
Prompt/workflow/graph	Symbolic agent specification	Inspectable system optimization [@zhou_2024_symbolic; @agrawal_2025_gepa]	Dev-set/evaluator overfit	Medium
Code/harness	Tools, skills, agent program	New operations and large coding gains [@zhang_2025_darwin]	Test manipulation, unsafe code, domain specificity	Medium
Co-evolution/populations	Curriculum, world model, archive	Diversity and frontier-tracking tasks [@fang_2025_webevolver; @huang_2025_r]	Collusion, simulator drift, compute	Low–medium
Meta-evolution	Update procedure	Adaptation of how learning occurs [@zhang_2026_hyperagents; @wang_2026_metaskill]	Recursive validation and governance	Early

4. Key scientific findings: claims mapped to evidence

Finding	Evidence	Strength	Main risk to interpretation
Linguistic feedback can improve later attempts without weight updates.	Reflexion reports HumanEval 91% pass@1 vs GPT-4 80%; ExpeL and CLIN show accumulating experience and transfer [@shinn_2023_reflexion; @zhao_2023_expel; @majumder_2023_clin].	Strong for bounded tasks	Repeated trials, context budget, verifier quality
Executable skills transfer more directly than raw reflections.	Voyager's skill library yields 3.3× items and transfers across worlds; AWM's induced procedures improve web navigation [@wang_2023_voyager; @wang_2024_agent].	Moderate–strong	Minecraft/web specificity; proprietary models
Agent structure can be optimized, not just model weights.	GPTSwarm, Symbolic Learning, AutoFlow, ADAS, and Gödel Agent improve graph/workflow/code designs [@zhuge_2024_language; @zhou_2024_symbolic; @li_2024_autoflow; @hu_2024_automated; @yin_2024_g].	Moderate	Mostly offline benchmark search
Archive-based code evolution can produce large cumulative gains.	DGM reports SWE-bench 20→50% and Polyglot 14.2→30.7%, with ablations supporting self-modification and open-ended archives [@zhang_2025_darwin].	Strong within coding	Fixed benchmark fitness; frozen foundation model; compute
Zero-data improvement works when rewards are verifiable.	R-Zero, Agent0, Dr. Zero, and Tool-R0 generate adaptive curricula and improve reasoning/search/tool use [@huang_2025_r; @xia_2025_agent; @yue_2026_dr; @acikgoz_2026_tool].	Moderate	Recent preprints; narrow verifier-rich tasks
Reflection is useful as search information, not a guarantee of correctness.	GEPA uses trace reflection effectively; AlphaLLM combines critics and MCTS [@agrawal_2025_gepa; @tian_2024_toward].	Moderate	Same-model critique and evaluator bias
External memory does not solve continual learning.	Sequential ALFWorld/BabyAI experiments reveal transfer–forgetting trade-offs and retrieval competition [@hu_2026_when].	Strong but recent	Two environment families; preprint
The improvement procedure is becoming an explicit learnable object.	Hyperagents, MetaSkill-Evolve, MemEvolve, and EvolveMem modify meta-programs or memory architectures [@zhang_2026_hyperagents; @wang_2026_metaskill; @zhang_2025_memevolve; @liu_2026_evolvemem].	Emerging	Short horizons, evaluator dependence, limited replication
Self-evolution creates a distinct safety surface.	Misevolution observes safety degradation in memory and vulnerabilities in created tools/workflows [@shao_2025_your]. SEA shows one possible versioned-gate design [@sengupta_2026_self].	Emerging but important	Synthetic scenarios; mitigation evidence limited

What has not been established

• No reviewed system demonstrates indefinite, monotonic, domain-general improvement.
• No current benchmark cleanly separates memorization, extra inference compute, search over public tests, and genuine learning.
• No widely accepted safety protocol covers simultaneous changes to memory, weights, tools, workflow, code, and meta-optimizer.
• “Zero data” usually means no curated task dataset, not zero prior knowledge: the pretrained foundation model, tool documentation, simulator, and verifier are substantial external data and inductive bias.
• “Self-referential” does not imply the Gödel-machine theorem's proof-based optimal self-modification. Modern systems use empirical benchmark selection, not proofs of global utility improvement.

5. Evaluation: why endpoint accuracy is insufficient

A credible evaluation should expose an ordered task stream $\mathcal{S}=(D_1,\ldots,D_T)$ and measure more than final success. At minimum:

$$\mathrm{ForwardTransfer}_t = R(A_{t-1}, D_t)-R(A_0,D_t),$$

$$\mathrm{Forgetting}_{i,t} = \max_{k<t}R(A_k,D_i)-R(A_t,D_i),$$

and

$$\mathrm{Efficiency} = \frac{\Delta R}{\text{tokens} + \lambda_c\,\text{compute} + \lambda_h\,\text{human time}}.$$

The protocol should include:

1. hidden future tasks and a strictly separate evolution set;
2. explicit reset versus persistent state semantics;
3. no-evolution, extra-inference, random-search, greedy-search, and oracle baselines;
4. multiple seeds and confidence intervals over stochastic trajectories;
5. forward transfer, backward transfer, forgetting, regret, and time-to-adapt curves;
6. total generation, evaluation, storage, execution, and human-review cost;
7. safety, privacy, and privilege regression suites at every accepted modification;
8. adversarial evaluators and held-out verifier families to detect proxy gaming;
9. artifact lineage: exact parent, diff, data, evaluator version, and reason for acceptance;
10. rollback tests and deletion semantics for memory and consolidated parameters.

The recent SEA architecture is noteworthy because it treats admission of a change as a sequential statistical decision with an error budget, while logging versions and preventing regressions [@sengupta_2026_self]. It is not a complete answer—its reported runs are small and base-model capability dominates—but it points toward evaluation as part of the agent architecture rather than an after-the-fact leaderboard.

6. Frontier research directions

Rank	Frontier problem	Why it matters	Most useful next experiment	Urgency
1	Causal lifelong-evolution benchmarks	Without hidden streams and matched compute, claimed evolution may be memory, repeated inference, or test search [@hu_2026_when; @sengupta_2026_self].	A year-like stream with concealed distribution shifts, revisited old tasks, cost/safety telemetry, and multi-seed baselines	Critical
2	Verifier robustness and anti-reward-hacking	Every autonomous loop is limited by the integrity of its feedback [@tian_2024_toward; @shao_2025_your].	Cross-family judges plus executable checks and randomized hidden metamorphic tests	Critical
3	Safe modular self-modification	Mutable code, tools, and workflows can expand capability and attack surface together [@zhang_2025_darwin; @shao_2025_your].	Capability-scoped sandbox with immutable monitor, signed diffs, canary tasks, and automatic rollback	Critical
4	Memory governance	Persistent memory must update facts, resolve contradictions, forget selectively, honor deletion, and resist poisoning [@xu_2025_mem; @zhang_2026_memskill; @hu_2026_when].	Provenance-aware memory benchmark mixing updates, adversarial inserts, privacy deletion, and long-range transfer	High
5	Non-parametric-to-parametric consolidation	Text memory is interpretable but unbounded; weights are compact but irreversible [@yu_2026_self].	Gated consolidation with replay anchors, safety tests, uncertainty estimates, and reversible adapters	High
6	Meta-evolution with bounded recursion	Fixed improvement loops become the next bottleneck, but changing them weakens validation assumptions [@zhang_2026_hyperagents; @wang_2026_metaskill].	Evolve one update component at a time under an outer immutable constitution and off-policy audit	High
7	Co-evolution without collusion or drift	Proposers and solvers can become mutually easy rather than externally competent [@huang_2025_r; @fang_2025_webevolver].	Periodically inject concealed real tasks and independent world-model audits; measure diversity and external regret	High
8	Evolutionary scaling laws	Search cost is often hidden, preventing comparison of memory, RL, and population methods [@agrawal_2025_gepa; @zhang_2025_darwin].	Budget sweeps over trajectories, evaluator calls, population/archive size, and wall-clock compute	Medium
9	Open-ended scientific and embodied feedback	Current successes concentrate where correctness is cheap; real-world value is delayed and uncertain.	Longitudinal robot or scientific-analysis sandbox with human escalation and causal outcome checks	Medium

A likely winning architecture

The evidence favors a hybrid, not a single adaptation mechanism:

flowchart TD
  E["Environment trajectory"] --> X["Independent evaluator ensemble"]
  X --> M["Provenance-aware episodic memory"]
  M --> S["Procedure / skill distillation"]
  S --> P["Reversible parameter adapter"]
  P --> A["Versioned agent candidate"]
  A --> G{"Hidden regression + safety gate"}
  G -- pass --> R["Registry / diverse archive"]
  G -- fail --> B["Rollback + failure memory"]
  R --> E
  B --> S

The fast loop writes inspectable memory; the medium loop distills stable procedures and tools; the slow loop consolidates repeatedly validated knowledge into reversible adapters; an even slower meta-loop may modify how these stages operate. Independent evaluation, privilege boundaries, version lineage, and rollback remain outside the editable core. This structure combines the reversibility of memory, expressiveness of code, efficiency of parameter learning, and diversity of population search while limiting correlated failure.

7. Research outlook

The field's near-term trajectory is clear. Agent research is moving from better prompts for a fixed model toward learning systems whose components and update rules are explicit objects. The most scientifically valuable work will not be the paper that reports the largest one-shot gain under a self-evolving label. It will be work that demonstrates, under concealed and shifting tasks, that an agent:

• improves faster because of prior experience;
• retains earlier capabilities;
• transfers to genuinely new domains;
• accounts for all compute and human feedback;
• detects and rolls back harmful changes;
• preserves provenance and deletion guarantees;
• remains externally corrigible as its internal mechanisms change.

On present evidence, bounded self-evolution is real; open-ended safe recursive self-improvement is not yet demonstrated. The frontier is increasingly less about inventing another reflection prompt and more about building the science and infrastructure of controlled change.

Selected primary-source reading path

1. Reflexion, ExpeL, CLIN, and Voyager for the 2023 memory/skill foundation [@shinn_2023_reflexion; @zhao_2023_expel; @majumder_2023_clin; @wang_2023_voyager].
2. ADAS, Symbolic Learning, Agent Workflow Memory, and Gödel Agent for the 2024 expansion of the optimization substrate [@hu_2024_automated; @zhou_2024_symbolic; @wang_2024_agent; @yin_2024_g].
3. DGM, GEPA, Agent0, and ReasoningBank for 2025 archive, prompt, curriculum, and memory frontiers [@zhang_2025_darwin; @agrawal_2025_gepa; @xia_2025_agent; @ouyang_2025_reasoningbank].
4. Memory Continual Learning, Hyperagents, MetaSkill-Evolve, and SEA for 2026 bottlenecks and governance [@hu_2026_when; @zhang_2026_hyperagents; @wang_2026_metaskill; @sengupta_2026_self].
5. Misevolution as the essential safety counterpoint [@shao_2025_your].

The complete verified bibliography and full-text research harness are stored in .knowledge/ at the repository root.