The most important question for AI agents is no longer just:
How do we build a better agent?
It is becoming:
How does the agent improve from use?
Every production agent generates a stream of useful learning signals: user corrections, failed tool calls, repeated mistakes, successful execution paths, domain-specific workflows, and preferences. Today, most of that signal disappears. The agent may solve the current task, but it rarely becomes meaningfully better for the next one.
That is the practical version of recursive self-improvement we care about at Reflexio.
It is a controllable system where agents learn from real interactions, generalize what works, reflect on whether the learning still helps, and optimize their execution paths over time.
We call this framework LGRO: Learn, Generalize, Reflect, Optimize.
The production problem: agents do not compound from experience
When you watch agents in production, the same failure modes appear again and again.
An agent repeats a mistake that a user already corrected.
It takes an inefficient path to a task it has solved before.
It lacks procedural knowledge for a specific domain, repo, product, or support workflow.
It fails to adapt to how a particular user wants work done.
It lacks the right tool or validation step, so it guesses instead of verifying.
These are not just “memory” problems. They are behavior problems.
A memory system might store:
“The user was unhappy because the agent claimed the code worked without running it.”
That is useful context, but it does not necessarily change future behavior.
A learning system should turn the same event into a procedural improvement:
“Before claiming a code fix is complete, run the relevant test or verification command. If execution is not possible, clearly state what was not verified.”
This distinction is the core of agent self-improvement.
Memory remembers what happened. Learning changes what the agent does next time.
Why this should be non-parametric
One way to make agents improve is to update the model itself through training or fine-tuning. That may be useful in some cases, but it is often the wrong tool for production agent behavior.
Production teams need learned behavior to be inspectable. They need to know why the agent acted a certain way. They need to roll back bad learnings, delete user-specific data, scope behavior to the right customer or environment, and evaluate whether a learning is still helping.
Model-weight updates make those things difficult.
A more practical path is non-parametric self-improvement: keep the model fixed, but improve the skills, playbooks, tools, policies, and context injected around it.
Modern LLMs are already highly steerable through context. The hard part is not whether the model can follow guidance. The hard part is building the system that decides:
What should be learned?
How should it be represented?
Who should it apply to?
When should it be retrieved?
When should it be updated or forgotten?
And how do we know it actually improves the agent?
That is where learn, generalize, reflect and optimize (LGRO) framework comes in.
Learn: extract local procedural improvements
The first step is learning from individual user-agent trajectories.
A trajectory contains the request, the agent’s plan, tool calls, errors, corrections, final answer, and user feedback. From that trajectory, the system should extract a local procedural improvement.
For example, a coding agent modifies a function and says the issue is fixed. The user replies:
“Did you actually run the test? Run it locally and check the server log before claiming it works.”
The local learning is not simply “this user cares about tests.”
The useful learning is:
“For code fixes, verify the change before claiming completion. Prefer targeted tests, local reproduction, build output, or runtime logs depending on the failure mode. Report exactly what was verified.”
This learning may initially apply only to one user, one repo, or one type of task. That is fine. The first step is to capture the lesson in a form that can change future behavior.
Most of existing approaches stop here (such as self-improving skill, or user initiated learning capture into memory), the agent only improves one user at a time.
Useful, but not enough.
Generalize: let one user’s lesson help many users
Most production agents serve many users. If one user corrects the agent, that correction may reveal a pattern that should benefit everyone.
Suppose multiple users give different versions of the same feedback:
- “Run the test before saying it works.”
- “Check the logs before claiming the deployment succeeded.”
- “Do not call the migration safe without running integration checks.”
- “Verify the API response before telling the customer the issue is fixed.”
The surface details differ, but the underlying lesson is the same:
“For implementation or operational tasks, do not claim success based only on reasoning or code changes. Verify with the strongest available execution signal.”
This is where self-improvement starts to compound.
A new user should not need to personally teach the agent every lesson that previous users already discovered. The system should identify repeated local learnings and turn them into shared skills.
But generalization must be careful.
A user preference should not become a global rule. A company-specific workflow should not leak into another customer’s agent. A temporary workaround should not become permanent behavior.
So every learning needs structure: scope, trigger conditions, provenance, confidence, version history, and evaluation results.
Without structure, learnings become prompt bloat.
With structure, they become an evolving behavior layer.
Reflect: keep learnings from becoming stale
Learnings decay.
The product changes. The agent changes. Tools change. User expectations change. A rule that helped three months ago may hurt today.
For example, early users may want the agent to ask for confirmation before every step because the agent is not reliable yet. Later, once the agent becomes more capable, the same users may prefer it to complete routine tasks end to end.
If the system blindly preserves old learnings, the agent gets worse.
Reflection is the process of checking whether a learning still helps when it is applied.
Every time a learning is used, the system should ask:
Did it improve task success?
Did it reduce retries?
Did it save planning steps or tool calls?
Did it make the response more accurate?
Did it make the agent unnecessarily cautious?
Would the agent have done better without it?
This last question is important. A learning should not survive because it sounds reasonable. It should survive because it continues to improve behavior.
Reflection also helps resolve conflicts. One learning may say, “Ask for confirmation before taking action.” Another may say, “Avoid unnecessary confirmation for routine tasks.” Both can be right under different conditions.
Reflection sharpens those boundaries.
The goal is not to accumulate infinite memories. The goal is to maintain a living set of behavioral improvements.
Optimize: move from local fixes to better execution paths
Most learning systems stop after reflection. They extract a rule, retrieve it later, and update it when it fails.
That is useful, but it can still leave the agent stuck in a local optimum.
A local learning answers:
“What would have helped this one trajectory?”
Optimization asks:
“Across many similar trajectories, what is the best way to solve this class of tasks?”
That distinction matters because the best procedure may not be visible from any single interaction.
One trajectory teaches the agent to run tests. Another teaches it to inspect logs. Another teaches it to check environment variables. Another reveals that a command hangs unless a non-watch flag is used.
Individually, each learning helps.
Together, they may form a better playbook:
- Reproduce the issue.
- Identify the smallest relevant subsystem.
- Run the targeted test.
- Inspect runtime logs if behavior depends on the server.
- Apply the fix.
- Re-run verification.
- Report what was verified and what remains uncertain.
This is more than memory retrieval. It is execution-path optimization.
To do this well, the system needs sandboxed evaluation. Similar tasks can be replayed or simulated. Different skill variants can be injected into the agent context. The agent attempts the task, and the system evaluates success using tests, tool outputs, environment signals, user feedback, or LLM judges.
In this setup, the optimization target is not only the final answer. It is the procedure the agent follows.
The output is a better way to act.
The LGRO loop
LGRO is our framework for practical, non-parametric agent self-improvement:
Learn from individual trajectories and extract local procedural improvements.
Generalize repeated lessons into shared skills while preserving scope and avoiding cross-user contamination.
Reflect on whether each learning still helps, and update or retire it when needed.
Optimize across similar trajectories to discover better execution paths, not just isolated fixes.
This loop gives agent builders a controllable way to make agents improve with use. The improvements are inspectable, editable, deletable, scoped, and measurable.
That is what makes it practical for production.
What we are building at Reflexio
At Reflexio, we are building this learning layer for AI agents.
Our goal is to help agents improve from every interaction: learning from corrections, generalizing useful behavior across users, reflecting on whether learnings still work, and optimizing execution paths over time.
In our internal evaluations, Reflexio reduced agent planning iterations by more than 80% on GDPVal-style knowledge-work tasks.
The broader point is simple:
The future of agents is not just bigger models.
It is agents that compound from experience.
Not just remembering what happened.
Not just retrieving old context.
But improving how they behave.
That is the shift from memory to learning — and from static agents to self-improving systems.
You can check out Reflexio on GitHub and try our open-source coding-agent plugin, claude-smart. We would love feedback from other practitioners building agents in production.