# Reflexio — full blog text

> The learning platform for AI agents. Reflexio observes real agent-user interactions and
> turns corrections into scoped, inspectable behavioral rules.

Index: https://www.reflexio.ai/llms.txt
Documentation: https://www.reflexio.ai/docs/llms-full.txt

---

# Everyone Is Building Self-Improving Agents. Almost Nobody Means the Same Thing.

Source: https://www.reflexio.ai/blog/what-self-improving-agent-means
Published: 2026-09-04
Author: Reflexio Team

Two teams tell you their agent learns from experience.

The first retrains model weights every night on its own successful trajectories. The second appends a line to a markdown file when a user corrects it.

Both are telling the truth. Neither description tells you anything useful about the other, and the phrase they share has stopped carrying information.

That is a problem worth fixing, because the choice between those two systems is not a matter of sophistication. They fail differently, cost different amounts, need different evidence to justify themselves, and are appropriate in different situations. A team that picks one because it read a paper about the other has not made a decision — it has made a guess.

This post is the map. The five that follow it walk the territory.

## One loop, four parts

Strip away the vocabulary and every method in this space is the same loop.

**Something proposes a change.** A reflection on a failed attempt, a rewritten instruction, a gradient step, a new entry for a memory store.

**Something decides whether to keep it.** This is the part that varies most and gets discussed least.

**Something stores it.** The change has to land somewhere that outlives the current session, or nothing has been learned.

**Something measures whether the agent is now better.** Not whether the change looks reasonable. Whether the agent's outcomes improved.

The first three are present in every system. The fourth is frequently absent, and its absence is almost never stated — a system with no measurement step still runs, still produces changes, still generates the pleasant sensation of progress. It just cannot tell you whether any of it worked.

Three questions locate any method on this landscape.

## What changes

The most visible axis, and the one papers are organized around.

**Weights.** The model itself is updated. Reinforcement learning from verifiable rewards, self-training on the model's own filtered outputs, and self-edit methods that let a model write its own training data all live here.

**Context.** The instructions, prompt, or system message the model runs under. Automated prompt optimizers rewrite this text; so, more crudely, does a human editing a system prompt after a bad demo.

**Memory.** A persistent store of facts, preferences, and past events, retrieved at runtime and injected before the model answers.

**Tools and skills.** The agent's action space. An agent that writes a function, verifies it runs, and files it in a library for reuse has extended what it is able to do, without any change to the model or the prompt.

**The harness.** The scaffolding around the model — how it plans, when it spawns subagents, how it manages context, what it does on failure. This is the newest category to be taken seriously, and the one with the most room left in it.

These are not exclusive. Production systems usually touch two or three.

## What decides

The axis that actually determines whether a method works.

**The model's own judgment.** The system generates a change and asks a model whether it is good. Cheap, universally applicable, and structurally the weakest option available.

**A verifiable reward.** Unit tests pass or fail. The proof checks or it does not. The arithmetic is right or wrong. Where this signal exists it is close to unimpeachable, which is why the most convincing results in the field come from domains where it does.

**A human.** Someone reads a diff and accepts or rejects it. Expensive, slow, does not scale, and considerably more reliable than the first option.

**A measured outcome.** Run the old version and the new version against the same held-out cases, compare what each produced, keep the change only if it wins. Expensive in a different way — it requires the ability to re-run your agent faithfully — and the only option on this list that answers the question the loop was built to ask.

## When it changes

**Within an episode.** The agent notices a problem and corrects course mid-task. Nothing persists; the next session starts fresh.

**Between episodes.** After a task ends, something is written down that the next task can use.

**Offline, in batches.** A separate process reads accumulated history when the agent is idle and rewrites some persistent state. Consolidation passes, prompt optimizers, and nightly fine-tunes all sit here.

## The grid

Placing the well-known methods on all three axes at once makes the shape of the field legible in a way that reading them one at a time does not.

| Method | What changes | What decides | When |
| --- | --- | --- | --- |
| Self-Refine | The answer in progress | The model's own critique | Within an episode |
| Reflexion | Reflection text in episodic memory | Task feedback, then self-critique | Between attempts |
| Voyager | A library of executable skills | Whether the code actually runs | Between episodes |
| ExpeL, Agent Workflow Memory | Distilled insights and reusable workflows | The model's own judgment | Offline |
| Agentic Context Engineering | A structured playbook | The model's own judgment | Offline and online |
| Automated prompt optimizers | Instruction text | Score on a held-out set | Offline |
| RL from verifiable rewards | Weights | A checkable ground truth | Offline |
| Self-edit methods | Weights | Downstream task performance | Offline |
| Self-modifying coding agents | The agent's own source code | Benchmark score | Offline |
| Memory consolidation, "dreaming" | The memory store | Review, or recall statistics | Offline |

Two things fall out of the table immediately.

The first is that **the "what changes" column is diverse and the "what decides" column is not.** Six of these ten methods are decided, wholly or partly, by a model's opinion of its own work. That concentration is not a coincidence — self-judgment is the only decider available in every domain, so it becomes the default anywhere a verifiable reward does not exist.

The second is that **the methods with the most convincing published results are the ones with the strongest deciders.** Skill libraries gate on whether code executes. Verifiable-reward training gates on ground truth. These are not the most sophisticated proposal mechanisms in the table; they are the ones whose acceptance test cannot be talked out of a rejection.

## The axis that gets read, and the axis that decides

Almost all the public conversation runs on the first axis. It is where the interesting engineering is, it is how papers are titled, and it is what a demo shows.

But the failures cluster on the second.

The clearest demonstration of this came from training a model against its own reference-free judgments and watching what happened. The judge's pass rate climbed from 0.72 to 0.94. True accuracy stayed at 0.20. The policy had not learned to be correct; it had learned to be convincing, because a judge shown a candidate answer scores how plausible it looks, not whether it is right. The effect transferred across model families, and a strict panel of three judges still accepted more than half of the bad answers.

That is not a bug in a particular judge, and a better prompt does not fix it. It is what happens when the thing being optimized is also the thing doing the grading.

This is the reason the series is organized the way it is. It is easy to write about self-improvement as a story about increasingly clever ways to change an agent. The more useful story is about evidence: what each method accepts as proof that a change was an improvement, and what that choice costs.

## What the rest of this series covers

**Learning without touching the weights** — reflection, experience reuse, skill libraries, playbooks, and automated prompt optimization. The largest and most practical family, and the one where a small team can get real results this quarter.

**Learning in the weights** — verifiable rewards, self-training, and self-edits. What becomes possible when you have a checkable ground truth, and what it costs when you do not.

**Memory as the substrate** — the memory architectures everything else is built on, what their benchmarks actually measure, and the failure modes that arrive with scale.

**The signal problem** — credit assignment, self-authored verification, reward hacking, and the theoretical limits on how far a system can bootstrap itself. The spine of the series.

**What to actually build** — a decision procedure, what to measure, and an honest account of which parts of this are solved.

A closing note on posture. This is a young field with a serious replication problem: many reported gains are measured against weak baselines, several headline numbers come from the vendor selling the system, and the benchmarks are new enough that they are still being argued about in public. Where a number is contested in the posts that follow, both sides get named. Where something is unresolved, it is described as unresolved rather than smoothed over.

---

We build the measurement half of this loop at [Reflexio](https://www.reflexio.ai) — corrections held as rules a person can read, scoped to where they earned their evidence, tested before they ship. That work is why we read this literature closely, and it is a bias worth knowing about while reading a survey we wrote.

For the implementation boundary, the [playbook optimization documentation](https://www.reflexio.ai/docs/concepts/playbook-optimization) describes supported candidate search, while the [impact measurement guide](https://www.reflexio.ai/docs/portal/measuring-reflexio-impact) explains how to inspect results.

Related reading on this blog: [Your Agent Remembers Everything and Learns Nothing](https://www.reflexio.ai/blog/remembers-everything-learns-nothing), on why memory layers and learning layers get mistaken for each other, and [Your Agent Dreams. It Still Can't Tell You If It Got Better.](https://www.reflexio.ai/blog/agent-dreaming-vs-replay-gated-tuning), on what consolidation passes actually validate. [Reflexio](https://github.com/ReflexioAI/reflexio) is on GitHub, along with our open-source coding-agent plugin [claude-smart](https://github.com/reflexioAI/claude-smart).

---

# Your Agent Dreams. It Still Can't Tell You If It Got Better.

Source: https://www.reflexio.ai/blog/agent-dreaming-vs-replay-gated-tuning
Published: 2026-08-28
Author: Reflexio Team

Overnight, the agent consolidated. Duplicates merged. A contradiction between two notes resolved in favor of the newer one. A stale line about a deprecated endpoint replaced with the current path. The memory file came out shorter, better organized, and by every available measure cleaner than it was the day before.

This morning it made the same mistake it made yesterday.

Nothing malfunctioned. Consolidation did exactly what consolidation does.

## The nightly ritual

Agents write memory the way people take notes in a meeting: mid-task, under time pressure, without knowing what will matter later. That produces exactly what you would expect — the same lesson recorded four times in slightly different words, two entries that flatly contradict each other, a note that was true in March, and a file that grows until the useful part no longer fits in the context window.

**Dreaming** is the fix that several teams arrived at independently. It is an offline pass: read the accumulated memory alongside past session transcripts, and write out a reorganized memory state. Merge the duplicates. Let the newer fact win. Drop what has gone stale. Surface the pattern that was implicit across a dozen sessions but never written down once.

The sleep metaphor is doing real work here, and it is worth being precise about which part. The claim is not that the agent is resting. It is that **consolidation is a separate process from experience** — it runs when the agent is not busy, and it operates on the *trace* of what happened rather than on what is happening. That is a genuine architectural idea, not just branding.

## One mechanism, five products

What makes this interesting is the convergence. Different teams, very different surfaces, the same shape.

**Anthropic's [Dreams](https://platform.claude.com/docs/en/managed-agents/dreams)** is an asynchronous job on the managed-agents platform, currently a research preview behind a beta header. You hand it an existing memory store — the file or database the agent reads from — plus somewhere between 1 and 100 past sessions. It returns a *new* store, and never touches the input — so you can diff the two and discard the result if you dislike it.

**[OpenClaw's dreaming](https://docs.openclaw.ai/concepts/dreaming)** is the most explicit about the biology. It runs on a cron schedule — 3 AM by default — in three named phases. The *light* phase ingests recent recall and deduplicates candidates. The *REM* phase builds theme and reflection summaries across recent traces. Only the *deep* phase may touch `MEMORY.md`, and only after a candidate clears a weighted score over six signals: relevance carries the most weight, then how often the entry was recalled, how many distinct queries reached it, recency, consolidation, and conceptual richness. Three thresholds gate promotion. Entries that came from somewhere untrusted are dropped outright. The previous `MEMORY.md` is retained, and a human-readable dream diary records what was added, merged, and superseded.

**ChatGPT** runs the idea continuously rather than on a schedule, synthesizing across past conversations into a memory state injected at the start of each new chat. Entries revise themselves as facts age — a planned trip quietly becoming a past one. OpenAI documents less of the internals than the other two, so this comes from reporting rather than a specification.

Two more landed the same mechanism inside coding agents, which is where most readers will meet it. **OpenAI's [Codex](https://developers.openai.com/codex/memories)** runs at session start, consolidating in the background — and only over sessions idle long enough that it is not summarizing work still in progress. It redacts secrets and can skip chats that touched external context, but nothing gates promotion. **[Gemini CLI's Auto Memory](https://github.com/google-gemini/gemini-cli/blob/main/docs/cli/auto-memory.md)** mines idle sessions and drops each proposal into an inbox as a diff you apply or reject.

| System | Consolidation runs | What gates promotion |
| --- | --- | --- |
| Anthropic Dreams | An async job you start | You review the output store |
| OpenClaw | Cron, 3 AM by default | Weighted score past three thresholds |
| ChatGPT | Continuously | Not publicly documented |
| Codex | Background, on startup once idle | No approval step |
| Gemini CLI | Background, on startup | You apply or reject a diff |

Note the name collision: Claude Code ships an [auto memory](https://code.claude.com/docs/en/memory) too, but that one is notes Claude writes *during* a session — not an offline pass.

## What every dream validates

These are not sloppy systems. OpenClaw's gates in particular are deterministic, auditable and reversible — better engineering than most memory layers ship.

But look at what each gate actually measures.

Anthropic's is human review: you read the output store and decide whether you like it. Gemini CLI's is you reading a diff. OpenClaw's is a six-signal score over how well entries matched, how often they were recalled, from how many queries, how recently. Codex has no promotion gate. ChatGPT does not document one.

Every gate here answers the same question — **is this store in good shape?** Is it deduplicated, current, appropriately sized, sourced from somewhere trustworthy?

None of them answers a different question: **does the agent produce better outcomes with the new store than it did with the old one?**

OpenClaw's scoring makes the gap easiest to see, precisely because it is the most rigorous. Recall frequency counts how often an entry was retrieved — but an entry can be retrieved constantly and be actively harmful. A confidently worded, subtly wrong rule fires every single time its trigger matches, and racks up an excellent frequency score doing damage. Relevance, the heaviest signal, has the same defect: it scores how well an entry matched the queries that reached it, and a confidently wrong rule matches its queries beautifully. Frequency and relevance both measure *salience*. Neither measures *correctness*. That is this morning's mistake, scoring well.

This is not a flaw, and it is not an oversight. It is the correct validation for the job being done. Bookkeeping deserves bookkeeping validation. A librarian who reshelves your books is not making a claim about whether they are any good, and it would be strange to grade them as though they were.

It does mean something specific: a dream cannot tell you whether your agent got better. It was never built to, and tuning the scoring weights will not change that — the necessary evidence is not in the inputs.

## Replay-gated tuning

If you want to know whether a change improved behavior, you have to change one thing and measure the difference. That is an experiment, not an edit.

Why it has to be one is visible in the logs: they contain only what the rule already in place did. The counterfactual was never run, so it was never recorded. Historical evidence can establish that a rule is worth reconsidering. It cannot establish that a rewrite would be better.

Reflexio's **offline tuner** is built around that constraint. It proposes exactly one candidate, changing only the guidance text, to be run against a frozen set of cases held apart from the evidence that produced it — and accepts only strict improvement with no regression. A single case where the incumbent wins rejects it. Anything inconclusive rejects it. Most attempts should end with no change at all.

That is the deciding half, and it is built and tested. The running half — a qualified harness that executes both variants and returns a signed, scored result — is a contract we have not shipped. It is ours to build, not yours to configure, and until it exists the tuner stays off.

Better is the wrong axis, though. Dreaming answers whether the memory is in good shape; replay answers whether a change to it improved behavior. And replay fails in a way dreaming does not: a held-out set freezes a world that keeps moving, so it will defend a rule that has since gone wrong about an API that changed last week — exactly the case a consolidation pass catches for free.

| | Dreaming | Replay-gated tuning |
| --- | --- | --- |
| Unit of change | The memory store | One rule's wording |
| Evidence establishes | What is worth keeping | What is worth reconsidering |
| Decided by | Review, or recall statistics | Measured outcome on held-out cases |
| Default outcome | A rewritten store | No change |

The summary worth carrying away: **dreaming makes an agent's memory better. Only measurement makes the agent better.** Those are two different sentences, and a lot of current writing uses the first to mean the second.

## The question to ask

Whatever you are running, one diagnostic separates the two:

**When your consolidation pass rewrites a rule, what would have to be true for it to be rejected?**

If the answer is "someone reads it and doesn't like it," you have bookkeeping — well-built, genuinely useful bookkeeping. If the answer is "the agent performed no better with it than without it," you have an experiment.

You do not need anyone's infrastructure to find out which you have. Before your next consolidation run, freeze twenty past cases. Re-run them against the old memory and against the new one, and read the pairs. If you cannot tell which won, that is the finding — and it is the same finding whether you spend an afternoon on it or build the industrial version.

Both are worth having. Only one of them will tell you tomorrow whether last night changed anything.

---

Making the second one real is what we work on at [Reflexio](https://www.reflexio.ai) — corrections held as rules a person can read, scoped to where they earned their evidence, tested before they ship, and reversible when they stop paying. The replay-gated offline tuner described here is a decision procedure without an executor yet, and we would rather publish that state than a closed loop we do not have. If you can already re-run your agent faithfully on past cases — or you think our acceptance rule is wrong — those are the two conversations we most want to have.

For the supported candidate-search mechanism, see [playbook optimization](https://www.reflexio.ai/docs/concepts/playbook-optimization). That documentation describes its gates and configuration; it is distinct from the unshipped offline loop discussed here.

Earlier in this series: [Your Agent Remembers Everything and Learns Nothing](https://www.reflexio.ai/blog/remembers-everything-learns-nothing), on why memory layers and learning layers get mistaken for each other. [Reflexio](https://github.com/ReflexioAI/reflexio) is on GitHub, along with our open-source coding-agent plugin [claude-smart](https://github.com/reflexioAI/claude-smart).

---

# How Creao Turned Agent Exploration Into Repeatable Execution

Source: https://www.reflexio.ai/blog/creao-repeatable-agent-execution
Published: 2026-08-20
Author: Reflexio Team

[Creao](https://creao.ai/) helps people turn successful conversations into reusable AI agents and workflows. But getting one result right is only the beginning. The harder challenge is making the next run start with what the agent has already learned.

Reflexio gave Creao a learning platform for that transition: profiles preserve useful context, playbooks capture repeatable procedures, and evaluation shows whether retrieved guidance helped or hurt the session.

## The impact at a glance

![Three headline outcomes: 75% fewer model calls, 36% fewer task failures, and 47% positive impact from retrieved learnings with 1.2% negative impact.](https://www.reflexio.ai/blog/creao-reflexio/impact-at-a-glance.svg "The three outcomes combine Creao’s internal comparison with Reflexio’s retrieved-learning evaluation.")

The headline result is a shift from exploration to execution. When the platform recognizes a task it has already learned, it can reuse a saved procedure and configuration instead of rediscovering the path every time.

## The challenge: successful sessions did not automatically compound

Agent sessions contain valuable operational knowledge: the format a user expects, the sequence that reliably completes a task, the verification step that prevents a bad output, or the fallback that works when a preferred tool is unavailable.

Without a structured learning platform, that knowledge is easy to lose. The next session may repeat the same exploration, spend more model calls rebuilding context, or omit an action that previously proved necessary.

Creao needed a way to carry forward what mattered without turning every prior conversation into permanent prompt baggage.

## The solution: retrieve the right learning at the right moment

Reflexio separates two kinds of knowledge:

- [**Profiles**](https://www.reflexio.ai/docs/concepts/user-profiles) retain durable context about the user and their environment, such as output conventions or production constraints.
- [**Playbooks**](https://www.reflexio.ai/docs/concepts/agent-playbook) retain procedures: the ordered actions, checks, and fallbacks that make a class of task repeatable.

At runtime, the agent [retrieves only the learnings relevant to the current task](https://www.reflexio.ai/docs/build/search). Reflexio then [evaluates the effect of those learnings](https://www.reflexio.ai/docs/portal/measuring-reflexio-impact) so the team can see whether they improved the outcome, had no material effect, or introduced a regression.

This creates a practical loop: observe what worked, structure it as reusable guidance, retrieve it selectively, and measure the result.

## Less exploration, fewer model calls

Across Creao’s 28-day internal evaluation, scheduled playbook runs used a median of **2 model calls per delivered outcome**, compared with **8 calls for freeform chat tasks**. This is an observed comparison across different task cohorts, not a causal estimate.

That is a 75% difference in median model calls. The result is consistent with playbooks reducing repeated exploration, but the comparison does not isolate how much of the difference was caused by the playbook itself.

The same evaluation covered 465,305 runs. Among 463,617 settled playbook runs, Creao reported a 99.0% success rate. Most of those runs—95.2%—executed without a human trigger.

> A learned procedure gives the agent a stronger starting point. The value is not more context; it is less rediscovery.

## Better outcomes, not just shorter paths

Efficiency only matters if quality holds. Creao used an automated quality judge’s “Poor” verdict as the task-failure measure. Across 230,477 evaluations, Creao reported a **36% reduction in “Poor” verdicts** for playbook threads, calculated from unrounded rates; the displayed 5.9% and 3.7% figures are rounded to one decimal place.

![Horizontal bars compare 5.9% Poor verdicts for freeform threads with 3.7% for playbook threads, alongside Creao’s reported 36% reduction.](https://www.reflexio.ai/blog/creao-reflexio/failure-rate-comparison.png "Playbook threads received fewer “Poor” automated quality verdicts than freeform threads.")

The comparison is directional rather than a randomized causal estimate: playbook tasks are narrower than open-ended chat by design. That narrowing is also part of the product value—turning a path that once required exploration into a repeatable operation.

## Retrieved learnings usually helped—or stayed out of the way

Reflexio’s evaluation of Creao’s retrieved learnings tells a complementary story across 2,176 impact verdicts covering 1,228 target interactions.

![A one-hundred-dot distribution shows 47.4% positive, 51.4% neutral, and 1.2% negative impact. A ring chart shows 82.2% of learnings with known relevance were relevant.](https://www.reflexio.ai/blog/creao-reflexio/learning-impact.svg "Retrieved learnings were much more likely to help than hurt, and most were relevant to the task.")

Neutral impact is expected. A retrieved learning can be applicable without becoming decisive in every session. The small negative share matters because it shows that selective retrieval was far more likely to help than to introduce harmful guidance.

## What this looked like in successful sessions

The aggregate metrics become clearer in the work itself. These are selected lines from two playbooks that were actually retrieved in successful Creao sessions, followed by the recorded evaluation—not a reconstructed example.

### Preserve the original image, pixel for pixel

The user wanted an image background removed without changing the subject, its colors, text, edges, or fine details. The retrieved playbook gave the agent a concrete, verifiable procedure:

> **Retrieved playbook · selected lines**
>
> Do NOT call the `generate_image` tool for background-removal tasks that require pixel-perfect preservation of the subject.
>
> 1. Open the source image with PIL and inspect its mode, size, and corner/background pixel values.
> 2. Apply a tool like `rembg` or a NumPy color-distance mask to produce an alpha channel.
> 3. Save as PNG.
> 4. Verify by sampling the result: a corner pixel should have `alpha=0` and a center foreground pixel should have `alpha=255`.

| Recorded evaluation | Result |
| --- | --- |
| Relevance | Relevant |
| Impact | Positive |
| Session outcome | Successful in 1 user turn, with 0 corrections |

The evaluator found that the agent followed the playbook precisely: it inspected the pixels, built the color-distance mask, preserved the source, and verified transparency at both background and foreground points. The evaluator described this as the **core guidance that drove the successful approach**.

### Carry a generated image all the way to a delivered video

In another session, the task was not complete when an image existed; the user needed a finished short video. The retrieved playbook preserved the production sequence and the definition of “done”:

> **Retrieved playbook · selected lines**
>
> After `generate_image` returns a saved image path, pass that exact path as the `firstImage` parameter to `generate_video`.
>
> Poll the returned task ID with `query_video_generation_status` repeatedly—generation typically takes 1–5 minutes, so an initial “still pending” response is normal and must not be treated as a failure.
>
> The video is ready when the status returns a `FILE_DOWNLOAD` entry; use the local file path inside that entry as the final deliverable to the user.
>
> Avoid re-running `generate_image`…reuse the already-saved image so the video stays visually consistent with what the user just saw.

| Recorded evaluation | Result |
| --- | --- |
| Relevance | Relevant |
| Impact | Positive |
| Session outcome | Successful in 2 user turns, with 0 corrections |

The evaluator confirmed that the agent reused the saved image, initiated video generation, kept polling through the pending state, and delivered the completed file. The playbook helped turn an intermediate asset into the outcome the user had actually requested.

These are observational examples, not controlled causal measurements on their own. The retrieved-learning verdicts and Creao’s aggregate comparison provide the broader evidence.

## The result: an agent that compounds from experience

For Creao, Reflexio made previous work operational. Useful context could persist, successful procedures could repeat, and the effect of retrieved guidance could be measured instead of assumed.

The result was a system that used fewer model calls, produced fewer poor outcomes, and received positive impact from retrieved learnings far more often than negative impact.

That is the practical difference between an agent that remembers and an agent that improves: the next session starts closer to the outcome.

---

### Measurement notes

Creao’s internal comparison covered 28 days and 465,305 runs; the quality comparison covered 230,477 automated evaluations. The model-call figures compare different task cohorts and are observational rather than causal. The reported 36% reduction uses Creao’s unrounded “Poor” verdict rates, while 5.9% and 3.7% are rounded display values. Reflexio’s retrieved-learning analysis covered a separate 30-day export from July 4 through August 3, 2026. System-level failures such as insufficient credit or no response were not treated as evidence that a profile or playbook helped or hurt task execution.

---

# Your Agent Remembers Everything and Learns Nothing

Source: https://www.reflexio.ai/blog/remembers-everything-learns-nothing
Published: 2026-08-19
Author: Reflexio Team

An agent with a memory layer will remember every correction it has ever been given. It will still repeat the mistakes those corrections were meant to prevent.

The pattern is easy to reproduce. A support agent quotes a customer a delivery date. The customer corrects it: *never quote a date without checking stock first.* The correction is stored, and the next week that customer returns to a perfect answer.

That same week, a different customer asks the same question — and gets the same wrong answer.

Nothing malfunctioned. The memory system did exactly what memory systems do.

## Two systems in the same slot

Teams install a memory layer expecting that gap to close. It doesn't, and the reason is architectural — obscured by the fact that the system which *would* close it looks almost identical from outside.

Both sit between the application and the model, reading before it answers and writing after. Both persist text across sessions. Both use a model in the write path to decide what is worth keeping. Both are sold against the same failure: the agent that starts every conversation from zero.

Same slot, same substrate, same pitch. The difference is one question: **what has to happen before the system writes anything down?**

## What memory architectures optimize for

Give a memory system a transcript and it extracts the facts worth keeping, compares each against what it already holds, and picks an operation — add, update, or retire a fact that was just contradicted.

Better designs keep the superseded fact retrievable as memory rather than only in a change log, and record when it stopped being true. Move cities, and last year's address is marked valid until a Tuesday in March — and the store can still answer questions about last March.

That is hard engineering and it works. But note what all of it is for: deduplication, contradiction resolution, retiring stale facts, deciding which of two conflicting statements is current. Every mechanism serves one job — maintaining a clean, current picture of the person being served.

*What is true about this user right now?* Memory systems answer that well. It is not the only question a production agent needs answered.

## The write trigger

**Memory systems write on observation. Learning systems write on judgment.**

A memory system commits a fact because it was stated. A learning system commits a rule because an outcome was evaluated.

![Two loops that look like one: memory writes when something was said, learning writes when something was judged.](https://www.reflexio.ai/blog/remembers-everything-learns-nothing/two-loops.png "Same trigger point, same destination — but the write conditions, the scope, and the failure modes all differ.")

| | Memory | Learning |
| --- | --- | --- |
| Writes when | Something is said | Something is judged |
| Unit | A proposition | A behavioral rule |
| Answers | What is true about this user? | What should the agent do differently? |
| Verified by | Checking it | Testing it |
| Fails by | Going stale, self-contradicting | Overfitting, eroding under rewrite |
| Ships only after | Nothing that tests the outcome | Approval, held-out evaluation, rollback |

Adding or retiring an entry resolves a bookkeeping question: does this fact contradict one already held? Memory systems are very good at that, and they have no opinion about whether the agent's answer was any good.

Some platforms accept feedback on a stored memory — useful, not useful. That grades the *note*, not the *answer*. Nothing in the loop connects a bad outcome to the behavior that produced it.

Making that connection is the hard part. A learning system needs a signal that something went wrong — a correction, a retry, an escalation — and must then determine which of the agent's behaviors caused it. That is credit assignment, and in the general case it remains unsolved.

The consequence is concrete: an agent can retrieve "this user prefers short answers" and still produce six paragraphs. Retrieval does not oblige a model to act.

## Facts, rules, and the evidence each needs

Return to the support desk. The two things that agent needed are different kinds of object.

*This customer's orders ship from the Rotterdam warehouse* is either true or false. You check it.

*Check stock before quoting a delivery date* is neither. It is effective or it is not, and the only way to find out is to deploy it and measure. Most teams have tooling for the first case and nothing for the second.

The second difference usually gets stated backwards. Memory is described as per-user and learning as cross-user, but memory stores cross user boundaries routinely — shared stores are an ordinary feature. What differs is *what* crosses, and on what evidence.

A fact propagates because someone asserted it. A rule should propagate only once it has been shown to work, and a bookkeeping loop has no outcome evidence to gate on because it never collected any.

That asymmetry cuts both ways: one customer's bad correction becomes every customer's bug. So a rule needs approval before it goes live, validation against cases it has not seen, and a way to roll back — not conveniences, but requirements for something that applies to people who never generated it.

Rules also decay in a way facts do not. Rewrite a rule set wholesale and it erodes: the specifics that made it useful get summarized away, leaving something shorter, tidier, and worse. Extending a playbook beats regenerating one.

## What the benchmarks measure

Each field reveals its priorities in how it scores itself.

Memory is evaluated on recall: information appeared earlier, and the system is asked to produce it later. The benchmarks have grown more demanding — current ones test temporal reasoning, whether a system registers that a fact has been updated, and whether it knows to abstain. The question underneath is still *did you produce the right answer?*

Self-improvement is evaluated on task completion. *Did the agent finish the job?*

A system can score near-perfectly on recall while the agent it serves repeats the same mistake for the thousandth time. Remembering a mistake and not repeating it are different capabilities, measured on different axes.

## Learning in text, not in weights

"Self-improving agent" suggests an expensive retraining loop. It does not require one.

One of the strongest recent results improves an agent by rewriting its instructions in natural language — reflecting on failures and editing the guidance. No weight updates, no training run. It outperformed a standard reinforcement-learning method while using **up to 35× fewer runs at the task**.

A scalar reward tells a system that it lost. A sentence tells it what to do differently. Language carries the *reason*; a number cannot.

## The blurry edge

One class of system complicates this line.

Background processes now review an agent's recent conversations and fold what they find back into its memory, without a per-write prompt. Consolidating lessons from experience is not clearly bookkeeping, and not clearly learning. Whether the write is triggered by observation or by judgment depends on implementation details vendors do not consistently document.

The adjacent case is a grader that scores an agent's output against a rubric and sends it back for another pass. That is a system with an opinion about whether the answer was good — but it is an evaluation loop sitting beside the agent, not a memory system changing what the agent does next. The line holds there. It is less clear in the first case.

That case is genuinely unsettled, and the distinction drawn here may prove to be a gradient rather than a line. We walk through five implementations of it in [Dreaming, and what its gates actually check](https://www.reflexio.ai/blog/agent-dreaming-vs-replay-gated-tuning).

## A diagnostic

None of this argues against memory. It argues against expecting one thing from a system built for another. Both questions need answering — *who is this person and what is true about them*, and *what should this agent do differently tomorrow* — and at runtime, retrieval should pull from both.

One question is worth taking to your team:

**Does your agent's error rate on repeated mistake classes decline over time?**

Not whether it recalls the correction. Whether the same class of failure becomes rarer across customers, without anyone editing a prompt by hand. Whether the second customer stops receiving the answer the first one already corrected.

If that question has no answer, what is in place is not a learning system. It is an excellent record of every time the agent got it wrong.

For most teams the answer is not yet. The obstacle is usually attribution — determining which behavior to blame — with evaluation cost and rule-approval workflow close behind. If any of the three is behind you, that is worth writing up.

## What answering it takes

Nothing above needs a bigger model. It needs a second loop — one that writes on outcomes rather than utterances, and produces rules a person can read rather than weights they can't.

For the implementation distinction, see how [user profiles](https://www.reflexio.ai/docs/concepts/user-profiles) retain personal context while [playbooks](https://www.reflexio.ai/docs/concepts/agent-playbook) carry behavioral guidance. The [memory-versus-learning comparison](https://www.reflexio.ai/ai-agent-memory-vs-learning) puts the two approaches side by side.

That is the layer we work on at [Reflexio](https://www.reflexio.ai). Not a memory store, and not a claim to have solved attribution — that problem is exactly as open as this post says it is. What's buildable today is narrower: corrections held as rules, scoped to where they earned their evidence, tested before they ship, and reversible when they stop paying. We've written up the shape of that loop as [LGRO](https://www.reflexio.ai/blog/lgro-self-improving-agents); closing it cut agent planning iterations by more than 80% in our own evaluations.

[Reflexio](https://github.com/ReflexioAI/reflexio) is on GitHub, along with our open-source coding-agent plugin [claude-smart](https://github.com/reflexioAI/claude-smart). If you're running a memory layer and wondering why the error rate is flat, we'd love to compare notes.

---

# From Loop Engineering to Self-Improvement: The Loop Over Loops

Source: https://www.reflexio.ai/blog/loop-engineering-and-self-improvement
Published: 2026-07-01
Author: Reflexio Team

Two ideas are converging in how people build with AI agents right now, and it’s worth slowing down to look at how they fit together.

The first is **loop engineering** — a phrase that took off after mentions by Boris Cherny (Claude Code’s creator) and Peter Steinberger (OpenClaw’s creator) went viral. The second is **agent self-improvement** — the idea that an agent should get better from use, not just complete the task in front of it.

These are usually discussed separately. But they’re not two different topics. Self-improvement is what loop engineering becomes when you take it one level up. This post is a field note on why.

## What loop engineering actually is

The core insight of loop engineering is that you don’t get real work out of an agent by prompting it once. You get it by putting the agent inside a **loop**: it acts, it checks its own work against a goal, and it iterates until the work meets the goal.

That last part — *checking against a goal* — is the whole game. A loop without a way to measure “am I there yet?” is just repetition. A loop *with* one is a search that converges. Give an agent a specification and, ideally, a set of evals, and it can write code, run the tests, read the browser or the logs, notice it’s wrong, and try again — for an hour at a stretch, without you.

A useful way to see the landscape is that there isn’t one loop, there are several, nested and running at different speeds:

- **The agentic coding loop** runs in *minutes*. Act, test, read the result, fix, repeat — until the code meets the spec.
- **The developer feedback loop** runs in *tens of minutes to hours*. A human looks at what got built and steers it: what to build, where the UI is wrong, how the flow should feel.
- **The external feedback loop** runs in *hours to weeks*. Real users, alpha testers, A/B tests. This reshapes the vision, which reshapes the spec, which drives the coding loop.

Loop engineering, as a craft, is the work of designing these loops well: picking the goal, wiring up the signal that tells the agent whether it’s meeting the goal, and setting how long each loop should run before it hands off. When people say they’re “finding new ways to engineer more effective loops,” this is what they’re tuning.

## The thing every loop has in common — and the thing they all forget

Notice what all three loops share. Each one is a converging process: a goal, a signal, and iteration until the two line up. That’s a genuinely powerful pattern, and it’s why agents can now work productively for long stretches.

But notice what happens *after* a loop converges.

The coding loop lands on green, and the task is done. The next task begins — and the agent starts from zero. It rediscovers that this repo’s tests hang without a non-watch flag. It re-learns that “done” here means running the integration check. It re-encounters the exact quirk a user explained yesterday.

Every loop generates a rich stream of signal along the way: the corrections, the dead ends, the path that finally worked. And when the loop closes, that signal evaporates. The loop converges on the *output*. It changes nothing about the *agent* that produced it.

This is the ceiling of loop engineering on its own. You can make each loop tighter and faster, but a loop that converges and then forgets will reclimb the same hill every time.

## Self-improvement is a loop over loops

Here’s the shift. Ask a different question about the loops:

> Loop engineering asks: *how do I make this loop converge on the goal?*
>
> Self-improvement asks: *how do I make the next loop start closer to the goal than this one did?*

The second question is answered by wrapping *another loop around the first*. The inner loop iterates on the task. The outer loop iterates on the agent — watching many runs of the inner loop, extracting what actually changed behavior, and folding it back in so the next run begins further ahead.

That’s the whole idea in one line: **self-improvement is a loop over loops.** Same machinery — a goal, a signal, iteration — but the thing being optimized is no longer the output. It’s the policy that produces outputs.

This is why self-improvement is best understood as the *next stage* of loop engineering, not a separate discipline. Loop engineering got us agents that converge within a task. The natural next move — the one you reach for the moment you’ve run the same kind of task fifty times — is to stop letting each convergence go to waste.

Concretely, the outer loop does four things:

1. **Learn** — from a single run, extract a durable, procedural lesson. Not “the test failed,” but “for code fixes, verify before claiming done; prefer a targeted test or the runtime log depending on the failure.”
2. **Generalize** — when the same lesson shows up across many runs and many users, promote it from a one-off into a shared behavior.
3. **Reflect** — check whether a learning still helps when it’s applied. Learnings decay; the outer loop retires the ones that stop earning their place.
4. **Optimize** — across many similar runs, find the better *procedure* for a whole class of tasks, not just an isolated fix.

We’ve written about this framework in more depth as [LGRO — Learn, Generalize, Reflect, Optimize](https://www.reflexio.ai/blog/lgro-self-improving-agents). The point for this post is narrower: each of those four is itself a loop, and together they’re the loop that sits *above* the loops loop engineering already gave you.

## Why the connection runs through the human

There’s one loop in the stack that resists automation, and it’s the most revealing one: the developer feedback loop. It resists because the human knows things the agent does not — the users, the domain, the environment the product lives in. Call it a **context advantage**.

Watch what happens each time a human uses that advantage to steer the agent — “run the test before you say it works,” “this customer never wants confirmation prompts,” “a migration isn’t safe until integration checks pass.” In pure loop engineering, that correction is *ephemeral*. It steers this run and vanishes. The human pays the same context tax again on the next task.

Self-improvement is precisely the mechanism that makes the transfer stick. The outer loop captures that correction as a scoped, reusable lesson — so the thing a human taught once keeps applying without them teaching it again. The human stays in the loop (the context advantage is real and won’t vanish soon), but every pass through it now *narrows the gap* instead of resetting it.

That is the difference between an agent you steer and an agent that learns from being steered — and it’s the clearest place to see loop engineering and self-improvement as one continuous idea.

## Not everything should compound the same way

One caution, because “make it a loop over loops” is easy to over-apply. The loops run at different speeds for a reason, and their lessons deserve different treatment:

- A fast coding-loop fact (“this repo’s test command needs `--no-watch`”) is high-confidence and narrow — capture it, apply it locally.
- A developer-loop preference (“this user likes end-to-end autonomy”) is real but *conditional* — it may reverse as the agent grows more capable, so it needs reflection, not permanence.
- An external-loop signal (an A/B result, a wave of similar feedback) is where careful *generalization* belongs — one correction becoming a shared behavior, without leaking across users or customers.

The failure mode of a naive outer loop is turning a throwaway workaround into a permanent global rule. That’s why self-improvement needs *structure* — scope, provenance, confidence, and evaluation — rather than a growing pile of remembered text. Reflection is what keeps a compounding loop from compounding the wrong things.

For an implementation of this outer loop, see [how Reflexio collects and uses playbooks](https://www.reflexio.ai/docs/build/agent-playbook), then [how to measure their effect on agent outcomes](https://www.reflexio.ai/docs/portal/measuring-reflexio-impact).

## Where this is heading

This is the layer we work on at [Reflexio](https://www.reflexio.ai): the outer loop that makes agents improve non-parametrically — by evolving the skills, playbooks, and context around a fixed model, so every learned behavior stays inspectable, scoped, and reversible. In teams running agents across many users, that outer loop is also where the hard boundaries live — one customer’s lesson generalizing to their organization, but never bleeding into another’s. In our internal evaluations, closing this loop reduced agent planning iterations by more than 80% on GDPVal-style knowledge-work tasks. But the framing matters more than any one product: self-improvement is what you get when you point loop engineering at itself.

So the takeaway is simple.

Loop engineering was the breakthrough that let agents iterate toward a goal without us watching. It works because a loop with a goal converges.

The next stage is a loop with a longer memory — one wrapped around the others, turning each convergence into a starting point instead of an ending. Not just iterating toward the goal, but compounding toward it.

If you’re engineering loops today, the most interesting question is no longer *how tight can this loop get?* It’s *what loop should sit above it?*

You can check out [Reflexio](https://github.com/ReflexioAI/reflexio) on GitHub and try our open-source coding-agent plugin, [claude-smart](https://github.com/reflexioAI/claude-smart). If you’re thinking about the loop over your loops, we’d love to compare notes.

---

# Learn, Generalize, Reflect and Optimize (LGRO): A Practical Guide for Self-Improving AI Agents

Source: https://www.reflexio.ai/blog/lgro-self-improving-agents
Published: 2026-06-16
Author: Reflexio Team

The most important question for AI agents is no longer just:

**How do we build a better agent?**

It is becoming:

**How does the agent improve from use?**

Every production agent generates a stream of useful learning signals: user corrections, failed tool calls, repeated mistakes, successful execution paths, domain-specific workflows, and preferences. Today, most of that signal disappears. The agent may solve the current task, but it rarely becomes meaningfully better for the next one.

That is the practical version of recursive self-improvement we care about at [Reflexio](https://www.reflexio.ai).

It is a controllable system where agents learn from real interactions, generalize what works, reflect on whether the learning still helps, and optimize their execution paths over time.

We call this framework **LGRO: Learn, Generalize, Reflect, Optimize.**

## The production problem: agents do not compound from experience

When you watch agents in production, the same failure modes appear again and again.

An agent repeats a mistake that a user already corrected.

It takes an inefficient path to a task it has solved before.

It lacks procedural knowledge for a specific domain, repo, product, or support workflow.

It fails to adapt to how a particular user wants work done.

It lacks the right tool or validation step, so it guesses instead of verifying.

These are not just “memory” problems. They are behavior problems.

A memory system might store:

> “The user was unhappy because the agent claimed the code worked without running it.”
> 

That is useful context, but it does not necessarily change future behavior.

A learning system should turn the same event into a procedural improvement:

> “Before claiming a code fix is complete, run the relevant test or verification command. If execution is not possible, clearly state what was not verified.”
> 

This distinction is the core of agent self-improvement.

**Memory remembers what happened. Learning changes what the agent does next time.**

## Why this should be non-parametric

One way to make agents improve is to update the model itself through training or fine-tuning. That may be useful in some cases, but it is often the wrong tool for production agent behavior.

Production teams need learned behavior to be inspectable. They need to know why the agent acted a certain way. They need to roll back bad learnings, delete user-specific data, scope behavior to the right customer or environment, and evaluate whether a learning is still helping.

Model-weight updates make those things difficult.

A more practical path is non-parametric self-improvement: keep the model fixed, but improve the skills, playbooks, tools, policies, and context injected around it.

Modern LLMs are already highly steerable through context. The hard part is not whether the model can follow guidance. The hard part is building the system that decides:

What should be learned?

How should it be represented?

Who should it apply to?

When should it be retrieved?

When should it be updated or forgotten?

And how do we know it actually improves the agent?

That is where learn, generalize, reflect and optimize (LGRO) framework comes in.

![The LGRO framework: Learn, Generalize, Reflect, Optimize](https://www.reflexio.ai/blog/lgro/lgro-framework.png)

## Learn: extract local procedural improvements

The first step is learning from individual user-agent trajectories.

A trajectory contains the request, the agent’s plan, tool calls, errors, corrections, final answer, and user feedback. From that trajectory, the system should extract a local procedural improvement.

For example, a coding agent modifies a function and says the issue is fixed. The user replies:

> “Did you actually run the test? Run it locally and check the server log before claiming it works.”
> 

The local learning is not simply “this user cares about tests.”

The useful learning is:

> “For code fixes, verify the change before claiming completion. Prefer targeted tests, local reproduction, build output, or runtime logs depending on the failure mode. Report exactly what was verified.”
> 

This learning may initially apply only to one user, one repo, or one type of task. That is fine. The first step is to capture the lesson in a form that can change future behavior.

Most of existing approaches stop here (such as self-improving skill, or user initiated learning capture into memory), the agent only improves one user at a time. 

Useful, but not enough.

## Generalize: let one user’s lesson help many users

![Generalizing many users' preferences into a shared behavior expectation](https://www.reflexio.ai/blog/lgro/generalize-shared-behavior.png)

Most production agents serve many users. If one user corrects the agent, that correction may reveal a pattern that should benefit everyone.

Suppose multiple users give different versions of the same feedback:

- “Run the test before saying it works.”
- “Check the logs before claiming the deployment succeeded.”
- “Do not call the migration safe without running integration checks.”
- “Verify the API response before telling the customer the issue is fixed.”

The surface details differ, but the underlying lesson is the same:

> “For implementation or operational tasks, do not claim success based only on reasoning or code changes. Verify with the strongest available execution signal.”
> 

This is where self-improvement starts to compound.

A new user should not need to personally teach the agent every lesson that previous users already discovered. The system should identify repeated local learnings and turn them into shared skills.

But generalization must be careful.

A user preference should not become a global rule. A company-specific workflow should not leak into another customer’s agent. A temporary workaround should not become permanent behavior.

So every learning needs structure: scope, trigger conditions, provenance, confidence, version history, and evaluation results.

Without structure, learnings become prompt bloat.

With structure, they become an evolving behavior layer.

## Reflect: keep learnings from becoming stale

Learnings decay.

The product changes. The agent changes. Tools change. User expectations change. A rule that helped three months ago may hurt today.

For example, early users may want the agent to ask for confirmation before every step because the agent is not reliable yet. Later, once the agent becomes more capable, the same users may prefer it to complete routine tasks end to end.

If the system blindly preserves old learnings, the agent gets worse.

Reflection is the process of checking whether a learning still helps when it is applied.

Every time a learning is used, the system should ask:

Did it improve task success?

Did it reduce retries?

Did it save planning steps or tool calls?

Did it make the response more accurate?

Did it make the agent unnecessarily cautious?

Would the agent have done better without it?

This last question is important. A learning should not survive because it sounds reasonable. It should survive because it continues to improve behavior.

Reflection also helps resolve conflicts. One learning may say, “Ask for confirmation before taking action.” Another may say, “Avoid unnecessary confirmation for routine tasks.” Both can be right under different conditions.

Reflection sharpens those boundaries.

The goal is not to accumulate infinite memories. The goal is to maintain a living set of behavioral improvements.

## Optimize: move from local fixes to better execution paths

![A sandboxed optimization loop: observe, reward, learn, simulate](https://www.reflexio.ai/blog/lgro/optimization-loop.png)

Most learning systems stop after reflection. They extract a rule, retrieve it later, and update it when it fails.

That is useful, but it can still leave the agent stuck in a local optimum.

A local learning answers:

> “What would have helped this one trajectory?”
> 

Optimization asks:

> “Across many similar trajectories, what is the best way to solve this class of tasks?”
> 

That distinction matters because the best procedure may not be visible from any single interaction.

One trajectory teaches the agent to run tests. Another teaches it to inspect logs. Another teaches it to check environment variables. Another reveals that a command hangs unless a non-watch flag is used.

Individually, each learning helps.

Together, they may form a better playbook:

1. Reproduce the issue.
2. Identify the smallest relevant subsystem.
3. Run the targeted test.
4. Inspect runtime logs if behavior depends on the server.
5. Apply the fix.
6. Re-run verification.
7. Report what was verified and what remains uncertain.

This is more than memory retrieval. It is execution-path optimization.

To do this well, the system needs sandboxed evaluation. Similar tasks can be replayed or simulated. Different skill variants can be injected into the agent context. The agent attempts the task, and the system evaluates success using tests, tool outputs, environment signals, user feedback, or LLM judges.

In this setup, the optimization target is not only the final answer. It is the procedure the agent follows.

The output is a better way to act.

## The LGRO loop

LGRO is our framework for practical, non-parametric agent self-improvement:

**Learn** from individual trajectories and extract local procedural improvements.

**Generalize** repeated lessons into shared skills while preserving scope and avoiding cross-user contamination.

**Reflect** on whether each learning still helps, and update or retire it when needed.

**Optimize** across similar trajectories to discover better execution paths, not just isolated fixes.

This loop gives agent builders a controllable way to make agents improve with use. The improvements are inspectable, editable, deletable, scoped, and measurable.

That is what makes it practical for production.

## What we are building at Reflexio

At Reflexio, we are building this learning platform for AI agents. The [playbook concepts guide](https://www.reflexio.ai/docs/concepts/agent-playbook) explains how corrections become reusable guidance, and the [playbook optimization guide](https://www.reflexio.ai/docs/concepts/playbook-optimization) describes supported candidate search and its evaluation gates. For an observed deployment, read [how Creao turned exploration into repeatable execution](https://www.reflexio.ai/blog/creao-repeatable-agent-execution).

Our goal is to help agents improve from every interaction: learning from corrections, generalizing useful behavior across users, reflecting on whether learnings still work, and optimizing execution paths over time.

In our internal evaluations, Reflexio reduced agent planning iterations by more than 80% on GDPVal-style knowledge-work tasks. 

The broader point is simple:

The future of agents is not just bigger models.

It is agents that compound from experience.

Not just remembering what happened.

Not just retrieving old context.

But improving how they behave.

That is the shift from memory to learning — and from static agents to self-improving systems.

You can check out [Reflexio](https://github.com/ReflexioAI/reflexio) on GitHub and try our open-source coding-agent plugin, [claude-smart](https://github.com/reflexioAI/claude-smart). We would love feedback from other practitioners building agents in production.