Playbook Tuning
Automatic playbook improvement with a 14-day trial and an automatic fallback. Built and shipped, but not yet enabled in any deployment.
Hosted Enterprise
Playbook tuning improves your user playbooks from what actually happened in your sessions. It reads the sessions a playbook was used in, and when the evidence points to a specific wording change worth making, it writes a new version of that playbook.
Not available in any deployment yet
The tuning capability reports Unavailable everywhere today, so the switch on Settings is permanently disabled and no playbook can be put on trial. The screens described on this page are built and shipped, but the tuning engine behind them is not switched on. Nothing is required from you, and there is nothing to configure while this is the case.
Managed and self-hosted deployments alike report config_reset_incomplete
until their configuration reset has run, and tuner_not_composed afterwards.
deployment_unsupported means only that DEPLOYMENT_MODE is not a recognised
value and needs correcting.
What Tuning Does
Tuning is not a review queue and not a suggestion box. When it acts, it acts:
- It reads finished sessions in which a playbook was used.
- If those sessions support a specific change, it publishes a new version of the playbook automatically. There is no approval step.
- The new version goes on trial for 14 days. During the trial it is the version your agents retrieve.
- Reflexio watches the sessions that happen during those 14 days.
- At the end of the trial — or earlier, if the sessions clearly go worse — it decides whether the new version stays.
If the sessions during the trial do not support the new version, the previous version comes back on its own. You do not have to revert it, and you are not asked to approve the revert. The same is true if several sessions come out worse partway through: the new version is pulled before the trial ends.
If the sessions do support it, the new version simply stays and becomes the standard version.
Your Edits Always Win
If you edit a playbook while one of its versions is on trial, your edit wins. The trial ends immediately, your version is what is kept, and tuning does not restore anything over the top of it.
This is a designed guarantee of the tuning path, not a setting: there is no per-playbook opt-out because an edit already is one. In the version history the trial's outcome reads Replaced by your edit.
Turning It On
Tuning is off by default. It is a single organization-wide switch, at
Settings → Tuning (/settings#tuning-settings).
That section shows three things:
| Row | What it tells you |
|---|---|
| Status | Whether tuning can run at all, and why not when it cannot. |
| On trial | How many of your playbooks currently have a version on trial, with a link to that filtered list. |
| Last check | When tuning last looked at your sessions. It reads Not yet until tuning has run for your organization. |
There is nothing else to set. There is no threshold to tune, no model to pin, no schedule to choose, and no per-playbook opt-out — tuning is either on for the organization or off for it.
Turning it on can be refused
Because the capability is unavailable, saving Settings with tuning switched on is rejected and the switch returns to off. The Status row explains which of the three reasons applied.
Reading What Happened
Every version of a playbook, and every decision made about it, is readable from Playbooks. Open the row's actions menu and choose Version history.
The drawer shows the playbook's versions newest-first, each with a badge saying where it stands — On trial, Trial ended, Kept, Reverted, In use, Replaced, or Removed — and a plain-language sentence explaining that outcome. Expanding a published version adds:
- Trial — when it was published and when its trial ends.
- Sessions — how many sessions during the trial supported it, disagreed with it, or were not relevant, plus a tally of sessions that could not be compared and why.
- What changed — a diff of the playbook text against the version it replaced.
- Technical detail — the identifiers and digests, for a support conversation.
Below the versions, the drawer lists checks that produced no new version: the times something looked at this playbook and stopped without changing it, and the reason. These entries are not attributed to tuning specifically, because other optimization work on the same playbook is recorded in the same place.
The Playbooks table also gains a Trial column while tuning is available. It answers exactly one question — is this playbook on trial right now? A badge means yes; an em dash means no. It does not report anything else about tuning.
The Settings On trial link opens this same table filtered to just those
playbooks, at /playbooks?tuning=on_trial.
What Tuning Will Not Do
These are deliberate absences, not gaps waiting to be filled:
- It will not ask you to approve or reject a version. Publication and revert are both automatic.
- It will not let you retry a check, revert a version early, extend a trial, or publish one immediately.
- It will not show the analyst's own written notes about your sessions. Those notes are not returned to the portal at all.
- It will not change agent playbooks. Tuning works on user playbooks only; agent-wide rules keep their human approval gate.
Version numbers are display only
v1, v2 and so on are assigned by the portal for readability, in creation
order. They are not stored anywhere and mean nothing outside this screen. Quote
the playbook ID in a support thread, not the version label.
Related
- Playbooks — what user and agent playbooks are
- Playbook Optimization — the separate, GEPA-backed refinement step for pending agent playbook wording, which is configured, approval-gated, and unrelated to tuning