Read traces from Braintrust

Connect your existing Braintrust traces to Reflexio, map your fields, preview the result, and explicitly activate collection. Your agent keeps logging to Braintrust and calling Reflexio search. It does not need a second interaction-publish flow for the selected traffic. The existing Braintrust evaluation-score integration remains separate.

Set up a source

Start by choosing one of two setup methods:

• Use a coding agent: prepare your Reflexio and Braintrust API keys, enter your Braintrust project, and copy the setup prompt to your coding agent. The agent connects the source and configures traffic and mappings, then gives you a link to review and activate in Reflexio.
• Set up manually: follow Connect → Select traffic → Map fields in Reflexio.

Both paths finish at Review & activate. You only need one path, and no traffic imports until you confirm activation. Use Choose another method to return to this choice before connecting. Once a source exists, the page shows its row so you can resume setup or open the source.

Each Reflexio project has one connected trace source, including a setup draft. Once connected, resume that source or open its dashboard; the additional Connect button is hidden. Pausing or disabling a source keeps its slot. Rotate its credential in Connection, or delete it before connecting a replacement. Expired setup drafts do not block a new connection. Deleting a source does not erase imported interactions or learnings.

To discard an in-progress setup, select Delete setup beside Resume setup on the connected source row. Confirm deletion to remove the saved connection, key, mappings and sampled traces from Reflexio and free the source slot. The original traces in Braintrust are not deleted.

1. Select the destination Reflexio project and open Data sources → Set up manually.
2. Connect: enter a source name and Braintrust key with read access to the intended projects and logs. Choose Enable read. The key is cleared from the form.
3. Select traffic: in one card, choose a discovered workspace/project, optional metadata/operation/tag filters, and the Traffic time range. No filters are prefilled. The coding-agent path saves this same selection; you do not configure another filter for importing. Backfill defaults to seven days; adjust the dates or turn it off to collect only new traffic. Read sample applies those choices and shows up to five examples in the same card. Sampling uses the chosen historical window; with backfill off, it reads the last seven days for mapping only. Changed choices require a fresh sample. Continue to Map fields saves the configuration and advances; no separate save buttons are needed. Dates use your local time zone.
4. Map fields: Reflexio drafts a mapping from representative examples. Review the mapped values beside each original trace. Issues identify the affected field and sampled traces; Review field selects an example and focuses its control. Field options contains conversions and fallbacks. Conversation format contains controls for your selected conversation structure. Use Custom mapping rules only for different response layouts. Apply mapping changes refreshes examples; Continue to Review & activate saves, previews, and validates. Ready traces can import while unresolved traces remain held.
5. Review & activate: review the saved project, traffic filters, historical import dates, and read-only examples. Issue links return to the relevant mapping control or traffic step. Counts describe sampled examples, not all matching traffic. Detailed sample coverage remains in Select traffic.
6. Activate: review the historical period and ongoing collection behavior, acknowledge the import scope and publish-flow cutover, then choose Activate source. A custom historical end excludes the gap before activation. Existing learning settings determine whether imported conversations produce learnings.

After activation, historical backfill initially imports up to 5,000 individual messages. One interaction is one message; a trace can contain several. It reads oldest-first batches and imports ready conversations as it goes. Profile learning can begin while later historical batches are still being read, subject to your extraction settings. Queued records wait for their turn or required context; unresolved records remain held for review. New matching traffic is collected independently, including while history is importing or has reached its limit.

Open the lightbulb beside your username (top right on mobile) to check import progress from any page. The panel updates automatically for the selected project, showing imported conversations and messages, the historical scan position, and any provider delays or records needing attention. View source details opens that source; the attention link opens its held records. Counts cover the dataset, including earlier imports, and are separate from the number of profiles generated. For historical backfill, a background Braintrust aggregate query estimates the number of source records matching the source query in the selected date range without downloading their payloads. The panel and source overview show percent scanned, source records scanned, and estimated remaining to scan when that count is available. The same saved traffic filters apply to sampling, backfill, ongoing collection and counts. Supported filters run in Braintrust before download. Some checks (including existence and numeric/boolean coercion) run locally to preserve their meaning; the review identifies filters that are partially or entirely checked after download, and progress labels these records as candidates. When a mapping needs related spans, collection reads the selected answers and their required context together where supported; missing or incomplete evidence can require follow-up reads. Records may still be held; these counts are not a promise of how many conversations will import. Import starts before counting finishes. If counting is unavailable, the UI keeps showing scanned records and dates. The estimate is cached and timestamped because upstream logs can change. Reaching the estimate does not finish the scan; completion requires exhausting the selected historical range, and queued conversations may still be processing afterward.

Add learning attribution after setup stays in the final review. Reflexio already configures /metadata/reflexio/retrieved_learnings on response spans, with /kind and /learning_id inside each reference. It is not an editable setup field and missing attribution does not produce a setup warning. Copy the instructions, add logging to your agent, and deploy: future matching traces capture attribution without another mapping step. Instructions use the saved mapping, including custom paths and per-assistant-message paths for conversation histories and events.

Back preserves mapping edits during setup. Only saved drafts survive a page reload. Source URLs retain the selected source and dashboard section; an activation page reload returns to the shared review. Expired samples are refreshed using their saved sampling window before validation.

A missing reliable user/session identity holds the record. Reflexio never creates an anonymous placeholder to make import succeed. Initial mapping sends up to three representative masked examples and definitions of every target field to the configured generation model. The trace includes message content. Every returned path is checked against its cited example and its value is validated. Different layouts can produce separate mapping rules. Technical details retain the evidence used for the draft; unavailable sibling spans require manual clarification. Different trace layouts may need manual mapping adjustments. Each example is limited to 64 KB; oversized examples are skipped with an explanation. It runs automatically when first entering Map fields; saved mappings are preserved. Regenerate mapping from examples explicitly replaces a saved draft, checks its revision, and preserves concurrent edits on conflict. If the initial model request fails, choose Retry automatic mapping or fill in the blank draft manually. Save manual edits before requesting regeneration. Mapping does not import data; review and validation are still required.

Configure efficient reads from the first setup

Filter for the span that contains the final user-facing answer. Map fields already on that span directly: the actual question and answer text, stable user and session IDs, and completion. Avoid mapping an entire metadata object, prompt, or search result bundle as a message.

If the question lives on another span, keep the traffic filter on the answer. Configure a named related source with the question span's observed operation name and a verified message/turn ID. Add session matching when both spans have that field. For example, a verified core-q&a answer may use follow-up-question.metadata.latest_user_message from the same trace. These names are examples, not universal defaults. The coding-agent setup skill can prepare this mapping; no custom BQL or extra publish call is needed.

Reflexio uses the saved mapping to request the required fields and supporting spans with each bounded page, then associates and publishes the conversations locally. It preserves pagination, duplicate detection, and complete-context checks. Some fallback mappings need larger objects, and late or incomplete traces can still require follow-up reads. Provider rate limits still apply.

Before activation, preview several turns and check that each question belongs to its answer, supporting spans are not additional interactions, and user IDs identify people across conversations. Resolve missing or ambiguous required fields before expanding the historical import. Sampling gathers richer evidence to discover the layout; its request cost is different from collection after mapping.

Different field layouts

Paths use JSON Pointer syntax, such as /metadata/customer/id. Root-trace and response-span fields are explicit. A fallback applies to a missing field, not a present null. Conflicting identities hold until you choose authority. Constants are permitted for source or agent version, not user identity or dialogue.

Use separate selector rules for different response layouts. Exactly one rule must match. Choose a completed response span, a turn-only message array, or one of the conversation contracts below. Each Agent message carries its own learning references. Root-level attribution is supported for individual response spans only when it identifies that exact response.

Growing histories and message events

Choose Conversation snapshot with stable message IDs when every message has an immutable ID. Set messages path (for example /messages), message ID, role, content and reference paths. Prefer this contract when your logger provides IDs. Choose Declared append-only conversation snapshot only when your application guarantees that existing messages never change, reorder or disappear. Message positions then supply identity. Equal text at separate positions remains separate.

Both contracts require stable user/session fields and either a branch JSON Pointer or an explicit This conversation never branches declaration. Groups are scoped to the destination project, Braintrust dataset, user, session and branch. Trace IDs alone are not inferred to be conversation IDs. A repeated snapshot imports nothing; [A,B] followed by [A,B,C,D] imports only C,D. Preview shows the accepted prefix, new messages and any conflicting prefix. Truncations, edits, reordered messages, unknown branches and incompatible mapping contracts hold for review. Mapping changes cannot reset an accepted prefix. Do not reuse a conversation identity for a rewritten history.

Choose Completed message events when each log record contains one stable-ID message, completion evidence, and a contiguous integer sequence beginning at zero. Set the event-message and sequence JSON Pointers. Events settle for 15 seconds by default (configurable 0–300). Missing predecessors hold subsequent messages; a timeout never skips a gap. Automatic reordering supports gaps up to 64 positions. Late edits conflict with accepted evidence. Unseen historical events outside the initial scan require the existing explicit append-as-late review.

For histories, log references on each Agent message before it completes:

{
  "metadata": {"user_id": "customer-1", "session_id": "chat-1", "agent_version": "v3"},
  "messages": [
    {"id": "m1", "role": "user", "content": "Please keep answers concise."},
    {"id": "m2", "role": "assistant", "content": "Understood.",
     "retrieved_learnings": [{"kind": "profile", "learning_id": "profile-id"}]}
  ]
}

Also supply and map your record's creation timestamp and completion evidence. The example is an application-defined logging shape, not a Braintrust automatic schema. Use /messages and /retrieved_learnings relative to each message. Only log IDs actually retained in that response's context. Missing, explicit empty, malformed and inaccessible references remain distinct; no trace-level reference list is copied onto older responses. System/tool/developer messages hold instead of being silently converted to Agent responses.

Snapshots are limited to 100 messages per observation. Event conversations accept at most 1,000 messages. Longer or ambiguous histories require a new, correctly bounded conversation contract; do not truncate a snapshot under an existing ID. Accepted manifests store hashes and canonical IDs, not dialogue text, and survive source reconnection. User erasure clears manifests and suppresses future revisions.

Save and validate the current mapping, then apply it to future records. Editing or validating a draft leaves the active mapping unchanged. Pending records retain their saved mapping until you explicitly preview and retry a selected set against the current validated mapping. Admitted evidence is never silently rewritten by a mapping edit. Previous mappings are not retained for comparison or rollback.

Log the learnings actually injected into each response

Existing traces often lack retrieved_learnings. Adding it requires a code change and deployment in your agent repository, so treat it as an optional follow-up after data source setup. Activation does not require it. After deploying and generating new responses, use Check new traces for attribution under Mapping to read recent matching traces, to inspect the captured references. The default mapping already reads this field; no additional mapping step is required. Historical traces remain unchanged. This checks parsing, not evaluation results.

Add it to your existing Braintrust response-span metadata before that span completes:

{
  "metadata": {
    "user_id": "your-application-user",
    "session_id": "your-conversation-id",
    "agent_version": "support-agent-v3",
    "reflexio": {
      "retrieved_learnings": [
        {"kind": "profile", "learning_id": "profile-id"},
        {"kind": "user_playbook", "learning_id": "42"}
      ]
    }
  }
}

If search returned three candidates and you injected two into the model context, record those two retained references on that response. Supported kinds are profile, user_playbook, and agent_playbook; IDs are strings. This is separate from citations that identify which learnings influenced the answer.

Use [] when no learning was injected. An absent field means unknown coverage; malformed or ambiguous references are diagnostics, not invented attribution. Missing references do not block otherwise valid dialogue import. Instrument future responses and verify the exact pairs under Latest imports on the source Overview. Editing already admitted message content or references creates a revision conflict; it does not retroactively change the canonical response.

Evaluation uses your existing sampling, configuration and session-inactivity rules. Ongoing imports can postpone inactivity even when event timestamps are old. A missing/unqualified agent version is shown honestly. A parsed reference is not a completed grade, and an inaccessible learning ID is not a neutral verdict. Use the existing evaluation controls for explicit on-demand grading and its normal cost behavior.

Operate and recover

An active source opens on a header that names the source, its Braintrust project and last read time, and shows its state as a badge (Active, Active · backfilling, Paused, Disabled, plus a warning badge while waiting on Braintrust or after a read error). The header carries the day-to-day controls: Pause collection / Resume collection and a ⋯ menu with Cancel remaining backfill, Disable source / Re-enable source and Delete source….

Below the header, four tabs:

• Overview — collection status, historical-scan progress and counts, followed by Latest imports: the five most recent admitted records with canonical interaction links, per-response references and extraction diagnostics. Show all expands to the latest 20.
• Mapping — the editor, validation, Check new traces for attribution, and an Apply mapping action when a newly validated mapping is ready for future records.
• Attention records — held records with reasons and expiry. Fix and validate mapping, select the records, preview the retry, then confirm. Late historical records require explicit append-as-late; consumed learning windows are not rewritten.
• Settings — credential rotation, and a Danger zone with Disable / Re-enable and Delete….

What each control does:

• Pause collection / Resume collection stop and resume new collection/admission. Already admitted downstream work follows existing policy.
• Disable source / Re-enable source retain configuration, mapping, history and import position. Re-enable checks current source access. Rotation keeps the dataset identity and invalidates old samples/in-flight credentials.
• Delete source asks for confirmation, then removes configuration, its dedicated key and pending payloads. Canonical interactions/learnings, minimal receipts and erasure suppression remain. Reconnecting the same dataset to the same destination does not duplicate admissions. Source deletion is separate from user-data erasure.

Existing governance erasure also removes matching pending source evidence and conservatively clears unresolved pending/sample copies in the affected project. Its impact includes erasure_scope_cleanup. Minimal suppression prevents erased records from returning through refetch or a later mapping fix. Other projects and resolved unrelated users are preserved.

Extend historical backfill

In Overview → Historical backfill, choose Extend backfill. Enter a new cumulative total, such as 20,000, to allow up to 15,000 more messages after the initial 5,000. Or choose All matching history in this date range to remove the message cap. Both choices retain your saved dates and traffic filters.

Backfill resumes from its saved position without importing accepted messages again. Complete imports stay intact: if 4,998 messages have been imported and the next import contains 6 messages, the UI asks for a total of at least 5,004. It does not split the import or exceed the cap. Replays and live messages do not consume the historical allowance. Reaching the limit is different from finishing the selected history. Source scan estimates remain separate from imported message counts.

Increasing the allowance does not resume a paused source or reopen cancelled backfill. Each new activation starts with the 5,000-message allowance.

Limits and deployment

Selected historical traffic is read in date-bounded windows with saved page checkpoints. Large ranges continue across worker runs without a total page ceiling. Ongoing collection uses overlapping recent fetch snapshots, not a lossless change feed, bounded to 100 pages per ongoing scan. Changes outside that scanned horizon or provider retention may be missed; upstream deletions are not mirrored. The UI reports incomplete coverage and backlogs. Cancel remaining backfill without undoing already admitted history or stopping new-traffic collection.

Detailed sample coverage is shown only in Select traffic. Samples contain at most 50 retained events. Longer windows use up to seven time buckets; recent/single-day samples read up to five pages. The traffic screen shows at most five examples. The chosen sampling window is applied by Braintrust before pagination and retained with the sample. Samples expire after 24 hours. Draft connections expire after seven days; activated configurations do not expire as drafts. Pending payloads expire after seven days and cannot be retried once expired. Provider pages are capped at 1 MB and expanded staged contexts at 8 MB per page. Live collection pauses at a roughly 2,000-record pending backlog. Historical evidence is staged before importing in event order, so operators need disk capacity for the selected range; the seven-day retention still applies.

Sampling and importing coordinate their requests on the same connection. When Braintrust rate-limits or temporarily fails, Reflexio saves progress, waits for the provider cooldown, and retries automatically. The source overview shows the retry time, historical pages saved, and conversations waiting for related spans separately from records requiring review. Ambiguous relationships, missing identity, expired evidence, and hard evidence limits still need attention.

Keys and payloads are encrypted on the destination data plane. Self-host operators need valid FERNET_KEYS and outbound HTTPS to api.braintrust.dev; self-host source data does not pass through managed persistence. Custom/private Braintrust endpoints are not supported. Managed/Supabase, self-host/Supabase and self-host/Postgres share the connector contracts.

Erase a user's data

User erasure is organization-wide, not source-scoped, so it is not offered on the source page. Use the governance API: preview with POST /api/projects/{project_id}/data-sources/streams/{stream_id}/purge-preview, then execute with .../purge-user (or POST /api/governance/erase-user directly). Both require an admin session and the governance entitlement. The preview includes erasure_scope_cleanup, covering unresolved staged records and sample copies in the selected project. Known user evidence is erased across the organization by the existing governance service, and shared learning follows its rebuild rules. Counts are estimates rechecked during erasure. See the governance reference. Deleting or disabling a source alone does not perform this operation.

Set up with your coding agent

Data sources supports setup in Reflexio or through a coding agent. Both create the same saved configuration. Choose Set up with your coding agent. Before copying the prompt, make two keys available through your coding agent's environment or secret manager: a full-access Reflexio API key for the currently selected Reflexio destination project, and a Braintrust API key that can read your source projects and traces. You can create the Reflexio key from API Keys. Fill in Braintrust source project (name or ID) before choosing Copy prompt and paste it into your agent in your application's repository. The prompt links to a published version of the open-source setup guide and includes your Reflexio endpoint and the Braintrust source project you entered, without credentials. Your Reflexio API key determines the Reflexio destination project; its ID is not included in the prompt. The agent starts with GET /api/data-sources/setup-context using Bearer authentication and follows the returned api_base_path. No Reflexio project ID or project header is required. Copying is disabled until you enter a Braintrust project name or ID. The copy action is available after Reflexio verifies that the linked guide is reachable.

Supply a full-access Reflexio API key for the destination project and a Braintrust read key securely. Your agent resolves the Braintrust project you specified and helps select traffic and time range, inspects representative examples, and saves and validates mappings through Reflexio's APIs. Limited Reflexio keys cannot manage data sources.

The agent asks about missing required fields and can offer a patch to your existing Braintrust logging for the retrieved learnings actually used in responses. Missing attribution does not block conversation import; missing reliable identity does. No additional interaction publishing is needed.

Finish in Reflexio using the review link returned by your agent. Inspect the saved configuration, examples, ready/held counts and blockers. Use an issue link or Back to mapping to make changes, then acknowledge cutover and click Activate source when ready. Agent setup and draft validation never start importing automatically.

Review historical coverage and missing fields

For ranges longer than one day, Reflexio samples up to seven time windows with a shared limit of 50 retained records. Sample coverage shows requested and observed dates, retained counts, empty windows and windows that could not be read. Busy days cannot consume another window's allowance. Counts describe sampled evidence, not all traffic in your selected range. Provider rate limits can leave partial coverage; new sampling jobs retain successful evidence and retry temporary failures automatically until the sample expires. Existing completed samples remain snapshots; read a new sample to refresh an older provider failure.

The mapping model prioritizes candidate answer layouts over internal helper traffic. The review still holds unmatched records until you map or explicitly exclude them. Use Inspect another sampled trace to replace a displayed example; five examples remain visible at a time. A transient mapping timeout or connection failure is retried once automatically. Rate limits and invalid model output require an explicit retry; your saved edits are preserved.

If Reflexio suggests a field such as metadata.visitor_id, confirm it identifies the end user. A merchant ID, generic task label or invented anonymous ID is not a substitute. An unresolved root means its evidence is unavailable, not that its fields are absent. When a response trace lacks a required field, ask your coding agent to add the actual user identity, conversation ID and current message to your existing response logging. Do not map earlier serialized chat history as the current user message. No separate Reflexio publishing flow is needed, and missing retrieved_learnings remains nonblocking.

Fields recorded on related spans

Braintrust may record the customer question on one span and the answer on another. Reflexio reads related spans in the same trace and can map fields across them. In the mapping review, inspect the original spans beside the five conversation examples. Each mapped value shows its source span and field path.

Use a span selector and a verified message/turn correlation to identify the related span. Add session correlation when available on both spans. Exactly one span must match. Recording order is not sufficient: a question-processing span may be logged after the answer. Missing or ambiguous required fields remain held, and incomplete provider reads are shown separately. Missing retrieved-learning IDs do not block conversation import.

A coding agent following the connection guide can discover these capabilities, inspect expanded examples, save the mapping, and return the same review page. Setup never activates importing: review the saved configuration, then click Activate yourself.