The Compounding Loop

§0 · The puzzleThree systems, three different things evolving

Three of the most-discussed self-evolving systems of the past year: AlphaEvolve, Meta-Harness, and the Darwin Gödel Machine. They share the same building blocks: LLMs as intelligent operators, automated evaluators, iterative loops of modify-score-select. On paper, they are the same kind of system.

They are doing categorically different things.

AlphaEvolve evolves solutions. The LLM is frozen; a population of candidate programs evolves through mutation-and-selection, filtered by automated evaluators. What improves is the population. AlphaEvolve discovered an algorithm for 4×4 complex matrix multiplication that Strassen missed for 56 years.

Meta-Harness evolves the harness. The code wrapping an LLM (context assembly, retrieval, prompt construction, tool orchestration) is iteratively rewritten by an agentic proposer. The model stays fixed. What improves is the operating system around it.

The Darwin Gödel Machine evolves the agent itself. The agent rewrites its own code (scaffolding, tools, control flow), keeps an archive of every variant, and selects on benchmark score. The evolver and the target are the same system. What improves is the agent that is doing the improving.

Three systems. Three different targets of evolution: a population of solutions, the harness around a fixed model, and the agent's own code. The surface similarity hides the differences. A casual reader calls them all "self-evolving." This blog is about seeing the differences, and about two questions that decide whether a self-evolving system actually compounds: how is the loop wired, and what keeps it from collapsing.

§1 · FoundationsModel, harness, context, environment

The classical RL picture

A standard RL setup has three parts: an agent, an environment, and the context flowing between them. At each step, the env emits an observation o_t and a reward r_t; the agent emits an action a_t. The context at time t is just the trajectory so far:

c_t = (o₁, a₁, r₁, o₂, a₂, r₂, …, o_t)

Context is the information flowing through the agent during execution. Keep this idea; we generalize it.

Modern agents have internal structure

A 2024-era LLM agent is not a single policy network. It has at least two parts: the Model (weights) and the Harness (code, prompts, tool schemas, retrieval, memory, orchestration, scaffolding). If you've built an agent beyond a chatbot wrapper, the harness is where most of the work lives. The model is the CPU; the harness is the OS.

Harness vs Context: a temporal split

A load-bearing distinction. The Harness is what's fixed during a run: system prompts, tool catalogues, scaffolding code, retrieval logic, all compiled in before deployment. (Weights are also fixed at runtime, but conceptually they're their own thing, the Model, distinct from the orchestration code around them.)

The Context is what changes during a run: conversation history, scratchpad, skills written this session, reflections stored. In this broad sense, context is the direct generalization of RL's trajectory.

Two systems with identical outputs can have very different learning dynamics based on what lives in harness vs context. A prompt optimized offline (harness) is a fundamentally different object from a reflection written mid-run (context), even when the resulting string looks similar. Most confusions in this field come from conflating these two.

Four components

What "self-evolving" means

A system is self-evolving if, during operation, it causes at least one of its four components (Model, Harness, Context, or Environment) to systematically change, based on its own outputs and the signal it receives, in order to better achieve a specified goal. The change is directed: components are updated toward an objective, not altered at random.

Why it took LLMs: the shift that made this a distinct field

The framework just laid out applies to classical RL as readily as to LLM agents. That raises a question worth making explicit: what changed to make self-evolving agents a distinct field rather than a flavor of RL?

The answer is narrow and causal. LLMs made the agent's internals (code, prompts, trajectories) legible and editable by the evolver itself. That one capability, applied to the framework's components, produces every structural novelty of the modern wave. The novelties are not independent; they are consequences of a single change in what the evolver can read and write.

The harness becomes an addressable target, because the evolver can now edit code; in RL there is no harness, a policy is monolithic. Context becomes a durable, inspectable, composable artifact (a skill library, a reflection buffer, a workspace), whereas RL's only runtime memory is recurrent hidden state or an opaque replay buffer. The evolver itself can be an agent (E3) that reads source, traces, and documentation; RL's outer loops are fixed procedures that consume only gradients or fitness scores. Code generation (M3) becomes a mechanism, replacing Gaussian-noise mutation with semantic mutation. And LLM-as-judge becomes a new weak-but-scalable grounding source, sitting between environment reward and human feedback.

Topology (T1–T5) and physics (retention / entropy / grounding) carry over unchanged from RL. The structural novelties live entirely on the target and evolver axes. Appendix §C works out the correspondence concept-by-concept.

§2 · ClassificationTarget and evolver

Every self-evolving system has two parts worth describing: a target (what gets modified) and an evolver (the thing doing the modifying). Target is one concept. The evolver has several facets. Tagging both is where most classifications start, and where they stop.

Target: what gets evolved

When a paper calls itself "prompt evolution" and the prompt is optimized by an outer loop then deployed fixed (OPRO, DSPy, TextGrad, PromptBreeder), it is evolving the harness, not the context. A surprising amount of writing slips here.

The evolver: the thing doing the evolving

The evolver is itself a system. Applied recursively, the framework from §1 says: the evolver has its own Model, Harness, and Context. Three properties of it are worth describing separately: what kind of system it is (capacity), what it reads each iteration (reference), and what operation it performs on what it reads (mechanism).

These three are correlated — a richer evolver tends to support richer references and more sophisticated mechanisms — but not redundant. They are pragmatic vocabulary for what papers actually do, not a carved-at-the-joints ontology. A full set of named tag-codes for each, plus the master classified-papers table, lives in §B.

Capacity is what kind of system the evolver is. It spans a single LLM call with a fixed prompt (OPRO, Self-Refine, APE), a frozen LLM reading a structured archive of past attempts (AlphaEvolve, FunSearch, DGM, Eureka), and a full agent in its own right — with source-reading, retrieval, and tool use (Meta-Harness, AIDE, CORAL). Within a target type, capacity often decides whether a system produces real work or only reshuffles.

Reference is what the evolver consults each iteration to produce its next variant — separate from any memory the deployed agent uses at inference. The richness varies enormously: nothing at all (STaR, Self-Rewarding LMs); an ordered log of past attempts (OPRO, TextGrad); a flat or structured population with quality-diversity bins or island sub-populations (AlphaEvolve, DGM, ADAS, OMNI-EPIC); or a content-indexed library retrieved by semantic match (Reflexion, Voyager, CORAL). This is where most of the design work in strong self-evolving systems lives. Two systems with the same target and same capacity can behave categorically differently based on archive structure: an ordered log can only hill-climb; a structured archive maintains diversity and escapes local optima.

Mechanism is what the evolver actually does with the reference. The options span per-task search like MCTS or beam (AFlow, AIDE), iterative LLM editing of text (OPRO, Reflexion), LLM code generation (DGM, Voyager), evolutionary search across iterations (AlphaEvolve, FunSearch — almost always paired with code generation), and gradient training (STaR, Self-Rewarding, RAGEN). The strongest systems combine two: DGM and AlphaEvolve combine evolutionary search with code generation; Agent Q combines gradient training with per-task search.

How the three fit together

A clarifying case study is DGM vs Meta-Harness. Both evolve the agent's own scaffolding — same target — yet their evolvers sit in different places on all three properties:

• Capacity: DGM uses a frozen LLM reading an archive; Meta-Harness's evolver is itself an agent (think Claude Code) with filesystem access, execution traces, and tool use.
• Reference: DGM consults a structured archive of every past agent variant; Meta-Harness consults an ordered log of recent edits.
• Mechanism: DGM wraps code-rewriting in evolutionary search across the archive; Meta-Harness rewrites code without that population structure.

Same target, categorically different evolvers. The wider information pipe and richer reasoning of the agentic evolver are part of what makes Meta-Harness's rewrites qualitatively different from DGM's: the evolver can notice things a simpler mutator cannot.

This is where most classifications stop. You now know what every paper does. You still can't tell why AlphaEvolve discovers mathematics while Self-Rewarding asymptotes after a few iterations. At this resolution their structural similarities are real and their dynamical differences are invisible. That's what Part 3 is for.

§3 · Two lensesTopology and Physics

The dimensions above classify a system's structure. The two lenses below ask about its dynamics: whether the loop is well-formed, and whether it compounds or just reshuffles.

Lens 1 · Loop topology

The shape of the feedback wiring. The first question to ask of any self-evolving system: draw the loop. Most papers don't do this explicitly, and most of the confusion in the field traces to that omission.

There are five topologies. Each strictly subsumes the previous.

T1 · open loop

One pass, no feedback. Most "use an LLM to write a better prompt" setups. Useful as a baseline.

T2 · single-target loop

Fixed evolver iteratively modifies a target, informed by a signal. Where most of the field lives: OPRO, AlphaEvolve, FunSearch, Reflexion, Voyager, Eureka, ADAS, STaR, Self-Rewarding, all T2. The differences between them are inside the loop, not in the topology.

The T2 ceiling. A T2 system is bounded by its evolver. AlphaEvolve is only ever as clever as Gemini; make Gemini better and AlphaEvolve improves, but AlphaEvolve cannot independently transcend Gemini. Most "self-evolving" systems have a hard ceiling they cannot cross without a base-model upgrade.

T3 · self-referential loop

The evolver is the target. DGM rewrites its own code. STOP improves its own improver. This is what people usually mean by recursive self-improvement: each modification changes both what you produce and how you produce the next one.

Why T3 is fragile. A self-modifier can break its own self-modification ability. T3 works only with strong grounding (to distinguish good modifications from bad) and a retention mechanism that survives bad modifications (an archive to roll back to). DGM's archive of every historical variant is not decorative. It's what keeps the system alive.

The open question. Does T3 transcend the base model, or does it asymptote near the base model's capability after exploiting low-hanging self-modifications? Nobody has demonstrated transcendence convincingly.

T4 · dual co-evolution

Two agents drive each other. Often proposer ↔ solver: Absolute Zero, SPIN, WebRL. In T4, problem difficulty is endogenous: it scales with solver capability. That's what gave AlphaGo Zero its magic. Without a game-like dual structure, it's hard to get this property.

T4's risk. Collapse into uninformative equilibria. The two agents can find a mutually-satisfying-but-useless local optimum. Stability is an art.

T5 · nested / multi-scale

A loop operating on another loop. PromptBreeder evolves task-prompts (inner) and mutation-prompts (outer): it gets better at getting better. T5 is the one topology where the outer loop could accelerate the inner; whether existing systems achieve durable acceleration is still an open empirical question (see §4).

What defines T5 is the outer meta-loop, not what sits inside it. The diagram nests a T3 for concreteness, but the inner loop can be any of the previous topologies (T2, T3, or T4) — PromptBreeder, for instance, nests a T2 inner loop.

Lens 2 · The physics: retention, entropy, grounding

Even with a good topology and a real information source, loops can stall, collapse, or drift. For a loop to compound (reliably get better rather than plateauing or degenerating), it needs three forces in balance.

I call this the physics of self-evolution because the three forces correspond cleanly to physical concepts that happen to carve the field exactly right. Each has a physical analogue, a specific role, and a predictable failure mode when absent. Remove any one and the loop dies in a characteristic way.

Retention: what persists across iterations?

The mechanism that makes good modifications stick and bad ones wash out — asymmetrically, so the loop keeps gains instead of rediscovering them each round (the cultural ratchet effect). Without that asymmetry, gains and losses are symmetric and capability random-walks.

Common retention mechanisms, weakest to strongest: gradient descent on weights (permanent but compressed); quality-diversity archive (best-in-niche); append-only textual memory or skill library (gated by verification); version-controlled archive of all history (perfect; storage-heavy). Failure mode when weak: catastrophic forgetting.

Entropy: what new information enters the loop per iteration

Not variation alone. A loop with high temperature but no information entering per iteration is a closed thermodynamic system. Iterate it N times and you just resample the base model's distribution. That is test-time scaling dressed as evolution, not evolution. Real entropy is information flowing across the loop's boundary, and it requires two preconditions coupled: variation (so there are candidates to distinguish: sampling temperature, mutation, quality-diversity pressure, self-play, novelty rewards) and a grounding signal that certifies which variants add something the base model couldn't have produced on its own.

Failure modes. Weak variation → the loop freezes (mode collapse). Weak delta → the loop iterates but extracts nothing beyond what a best-of-N base-model control already produces (the less recognized, probably more common one).

Grounding: what anchors to reality?

A signal from outside the loop that the loop cannot manipulate. In a self-evolving system, grounding is what keeps candidate quality tied to the world rather than to the loop's own self-assessment. Strongest to weakest: deterministic verifier (code, proofs); environment reward (strong but gameable); human oversight (strong, doesn't scale); self-judgment (weakest; doesn't bring in anything the base model didn't already have). Failure mode when weak: drift, reward hacking, self-delusion.

This is a design checklist. When you read a self-evolving-agent paper, name the three forces explicitly. Where is the retention? What is the entropy source? What is the grounding? If you can't answer any one clearly, either you don't understand the system or the system doesn't work the way the paper implies. The second case is common.

Strong grounding is where the field's biggest wins live. AlphaEvolve, FunSearch, DGM, STaR, Absolute Zero, Eureka all operate in domains with cheap verifiers and strong generators. That one asymmetry is the engine driving most of modern self-evolution: LLMs propose, a cheap evaluator filters, and the loop extracts from the generator what no single inference pass could have surfaced. Extending the field to domains without cheap automated grounding is the hard open problem.

§4 · Open challengesWhere the frontier actually is

Running the two lenses across the literature produces a fairly clear map of where the frontier is.

Extending beyond verifier-rich domains

The verifier-generator asymmetry is why the field works. It's also narrow: it applies cleanly to code, math, formal reasoning, and not much else. Extending self-evolution to domains without cheap verification is the biggest open problem. Two paths: (a) find new grounding that carries similar weight (scalable human-in-the-loop, multi-judge consensus, adversarial probing); (b) decompose un-verifiable tasks into verifiable sub-tasks and compose.

Stabilizing T3 and T4 at scale

T3 and T4 can in principle transcend their starting capability. Both are fragile. We have existence proofs (DGM, Absolute Zero) but no theory of when they're stable.

Internalizing hindsight

A self-evolving system gets much of its signal from what didn't work. Two questions decide how much of that signal compounds.

Does hindsight stick across iterations? The basic compounding question: whether insights from one attempt make the next attempt better. The answer depends on where hindsight lands. Into context (Reflexion, ExpeL, Voyager): works, but is bounded by retrieval and context capacity. Into weights (STaR, Self-Rewarding, Absolute Zero, and recent 2026 work like SDPO and SDFT): escapes that bound in principle, but several documented cases plateau within a few iterations. Whether self-distillation sustains gains across many rounds without external fresh signal is an open empirical question.

Does hindsight transfer across domains? AlphaEvolve discovers algorithms on one problem. Those algorithms do not become primitives for AlphaEvolve's next campaign on a different problem; each run starts fresh. Voyager composes skills within one Minecraft environment, but those skills do not carry to a new environment. Closing this loop, so that insights compound across problems and not just across iterations of the same problem, would add a second compounding mechanism on top of the verifier-generator engine.

Does T5 actually accelerate?

PromptBreeder shows T5 tooling works at small scale: an outer loop that evolves the mutation-prompts driving an inner loop. But "works" here means the nested loop runs and improves, not that the outer loop measurably accelerates the inner one. No system has yet demonstrated durable, repeated meta-level acceleration across many rounds of one continuous loop. Whether T5 delivers compounding speedup at scale, rather than a one-time tooling gain, is the question to watch.

§5 · ClosingIf you internalize one thing

The field is in a confusing phase. Every paper uses some subset of the same tooling (LLMs as mutation operators, archives, automated evaluators), and every paper claims some flavor of "self-improvement." The surface similarity hides real structural difference.

The structural difference is what this blog is about. A T2 system can be brilliant and a T3 system can be trivial; the topology determines what's possible, not the cleverness of the components. A loop with strong retention and grounding but no entropy will look impressive for a few iterations and then freeze. A loop with weak grounding will reshuffle latent knowledge into apparent novelty and then plateau.

Distinguishing these failure modes from genuine compounding is the work the field has yet to do.

§A · MethodHow to analyze a paper

A repeatable checklist. Apply in order.

1. Draw the agent/environment boundary. What's deployed? What's external?
2. Target. Which of the nine categories changes across iterations?
3. Evolver. What is the evolver itself? E1, E2, or E3?
4. Reference. What does the outer loop look at? R1–R6.
5. Mechanism. What procedure produces the variant? M1–M5.
6. Draw the loop. Identify the topology. T1–T5. Does the evolver modify itself?
7. Name the three forces. Retention. Entropy. Grounding. Explicitly.
8. Ceiling. What bounds this system? What would raise it?

§B · ReferencePapers classified

Each work in the master table below is tagged on four axes: evolver capacity (E1–E3), reference (R1–R6), mechanism (M1–M5), and topology (T1–T5). The legend immediately below defines each tag; the master table follows.

Tag legend

Evolver capacity · what kind of system the evolver is

Reference · what the evolver reads each iteration

Listed roughly from weakest to richest reference structure. Scope (whether the reference persists across iterations or is built and discarded per-task) is an orthogonal axis: R1 holds nothing, R2 is per-task by definition, R3–R6 are persistent.

Mechanism · what operation produces the variant

Listed roughly from weakest to strongest in compounding power. The M2/M3 boundary (text vs code) is the main place these tags overlap, which is fine — they're labels, not disjoint bins.

Topology · the shape of the feedback wiring

Master table

§C · RL referenceConcepts, and where they differ

The framework is paradigm-agnostic: most concepts in §2 and §3 have direct RL antecedents. This appendix maps the concepts most readers will bring from an RL background onto the framework's cells. Rows run from the most basic MDP primitives to the more structured methods. (The five places where self-evolving agents do something RL had no vocabulary for are covered in §1.)

The pattern is that topology (T1–T5), mechanism (M1–M5), and the three physics forces accommodate almost every RL method without strain. The differences live on the target and evolver axes — the five specific moves the LLM substrate enabled, which classical RL had no direct name for, are enumerated in §1.