Overnight, the agent consolidated. Duplicates merged. A contradiction between two notes resolved in favor of the newer one. A stale line about a deprecated endpoint replaced with the current path. The memory file came out shorter, better organized, and by every available measure cleaner than it was the day before.
This morning it made the same mistake it made yesterday.
Nothing malfunctioned. Consolidation did exactly what consolidation does.
The nightly ritual
Agents write memory the way people take notes in a meeting: mid-task, under time pressure, without knowing what will matter later. That produces exactly what you would expect — the same lesson recorded four times in slightly different words, two entries that flatly contradict each other, a note that was true in March, and a file that grows until the useful part no longer fits in the context window.
Dreaming is the fix that several teams arrived at independently. It is an offline pass: read the accumulated memory alongside past session transcripts, and write out a reorganized memory state. Merge the duplicates. Let the newer fact win. Drop what has gone stale. Surface the pattern that was implicit across a dozen sessions but never written down once.
The sleep metaphor is doing real work here, and it is worth being precise about which part. The claim is not that the agent is resting. It is that consolidation is a separate process from experience — it runs when the agent is not busy, and it operates on the trace of what happened rather than on what is happening. That is a genuine architectural idea, not just branding.
One mechanism, five products
What makes this interesting is the convergence. Different teams, very different surfaces, the same shape.
Anthropic's Dreams is an asynchronous job on the managed-agents platform, currently a research preview behind a beta header. You hand it an existing memory store — the file or database the agent reads from — plus somewhere between 1 and 100 past sessions. It returns a new store, and never touches the input — so you can diff the two and discard the result if you dislike it.
OpenClaw's dreaming is the most explicit about the biology. It runs on a cron schedule — 3 AM by default — in three named phases. The light phase ingests recent recall and deduplicates candidates. The REM phase builds theme and reflection summaries across recent traces. Only the deep phase may touch MEMORY.md, and only after a candidate clears a weighted score over six signals: relevance carries the most weight, then how often the entry was recalled, how many distinct queries reached it, recency, consolidation, and conceptual richness. Three thresholds gate promotion. Entries that came from somewhere untrusted are dropped outright. The previous MEMORY.md is retained, and a human-readable dream diary records what was added, merged, and superseded.
ChatGPT runs the idea continuously rather than on a schedule, synthesizing across past conversations into a memory state injected at the start of each new chat. Entries revise themselves as facts age — a planned trip quietly becoming a past one. OpenAI documents less of the internals than the other two, so this comes from reporting rather than a specification.
Two more landed the same mechanism inside coding agents, which is where most readers will meet it. OpenAI's Codex runs at session start, consolidating in the background — and only over sessions idle long enough that it is not summarizing work still in progress. It redacts secrets and can skip chats that touched external context, but nothing gates promotion. Gemini CLI's Auto Memory mines idle sessions and drops each proposal into an inbox as a diff you apply or reject.
| System | Consolidation runs | What gates promotion |
|---|---|---|
| Anthropic Dreams | An async job you start | You review the output store |
| OpenClaw | Cron, 3 AM by default | Weighted score past three thresholds |
| ChatGPT | Continuously | Not publicly documented |
| Codex | Background, on startup once idle | No approval step |
| Gemini CLI | Background, on startup | You apply or reject a diff |
Note the name collision: Claude Code ships an auto memory too, but that one is notes Claude writes during a session — not an offline pass.
What every dream validates
These are not sloppy systems. OpenClaw's gates in particular are deterministic, auditable and reversible — better engineering than most memory layers ship.
But look at what each gate actually measures.
Anthropic's is human review: you read the output store and decide whether you like it. Gemini CLI's is you reading a diff. OpenClaw's is a six-signal score over how well entries matched, how often they were recalled, from how many queries, how recently. Codex has no promotion gate. ChatGPT does not document one.
Every gate here answers the same question — is this store in good shape? Is it deduplicated, current, appropriately sized, sourced from somewhere trustworthy?
None of them answers a different question: does the agent produce better outcomes with the new store than it did with the old one?
OpenClaw's scoring makes the gap easiest to see, precisely because it is the most rigorous. Recall frequency counts how often an entry was retrieved — but an entry can be retrieved constantly and be actively harmful. A confidently worded, subtly wrong rule fires every single time its trigger matches, and racks up an excellent frequency score doing damage. Relevance, the heaviest signal, has the same defect: it scores how well an entry matched the queries that reached it, and a confidently wrong rule matches its queries beautifully. Frequency and relevance both measure salience. Neither measures correctness. That is this morning's mistake, scoring well.
This is not a flaw, and it is not an oversight. It is the correct validation for the job being done. Bookkeeping deserves bookkeeping validation. A librarian who reshelves your books is not making a claim about whether they are any good, and it would be strange to grade them as though they were.
It does mean something specific: a dream cannot tell you whether your agent got better. It was never built to, and tuning the scoring weights will not change that — the necessary evidence is not in the inputs.
Replay-gated tuning
If you want to know whether a change improved behavior, you have to change one thing and measure the difference. That is an experiment, not an edit.
Why it has to be one is visible in the logs: they contain only what the rule already in place did. The counterfactual was never run, so it was never recorded. Historical evidence can establish that a rule is worth reconsidering. It cannot establish that a rewrite would be better.
Reflexio's offline tuner is built around that constraint. It proposes exactly one candidate, changing only the guidance text, to be run against a frozen set of cases held apart from the evidence that produced it — and accepts only strict improvement with no regression. A single case where the incumbent wins rejects it. Anything inconclusive rejects it. Most attempts should end with no change at all.
That is the deciding half, and it is built and tested. The running half — a qualified harness that executes both variants and returns a signed, scored result — is a contract we have not shipped. It is ours to build, not yours to configure, and until it exists the tuner stays off.
Better is the wrong axis, though. Dreaming answers whether the memory is in good shape; replay answers whether a change to it improved behavior. And replay fails in a way dreaming does not: a held-out set freezes a world that keeps moving, so it will defend a rule that has since gone wrong about an API that changed last week — exactly the case a consolidation pass catches for free.
| Dreaming | Replay-gated tuning | |
|---|---|---|
| Unit of change | The memory store | One rule's wording |
| Evidence establishes | What is worth keeping | What is worth reconsidering |
| Decided by | Review, or recall statistics | Measured outcome on held-out cases |
| Default outcome | A rewritten store | No change |
The summary worth carrying away: dreaming makes an agent's memory better. Only measurement makes the agent better. Those are two different sentences, and a lot of current writing uses the first to mean the second.
The question to ask
Whatever you are running, one diagnostic separates the two:
When your consolidation pass rewrites a rule, what would have to be true for it to be rejected?
If the answer is "someone reads it and doesn't like it," you have bookkeeping — well-built, genuinely useful bookkeeping. If the answer is "the agent performed no better with it than without it," you have an experiment.
You do not need anyone's infrastructure to find out which you have. Before your next consolidation run, freeze twenty past cases. Re-run them against the old memory and against the new one, and read the pairs. If you cannot tell which won, that is the finding — and it is the same finding whether you spend an afternoon on it or build the industrial version.
Both are worth having. Only one of them will tell you tomorrow whether last night changed anything.
Making the second one real is what we work on at Reflexio — corrections held as rules a person can read, scoped to where they earned their evidence, tested before they ship, and reversible when they stop paying. The replay-gated offline tuner described here is a decision procedure without an executor yet, and we would rather publish that state than a closed loop we do not have. If you can already re-run your agent faithfully on past cases — or you think our acceptance rule is wrong — those are the two conversations we most want to have.
Earlier in this series: Your Agent Remembers Everything and Learns Nothing, on why memory layers and learning layers get mistaken for each other. Reflexio is on GitHub, along with our open-source coding-agent plugin claude-smart.