Data Sources API

Use these enterprise REST endpoints to import existing Braintrust, Langfuse Cloud, and LangSmith Cloud traces. They do not require a second interaction-publish flow in your agent. For the dashboard workflow, see Connect existing traces. Request and response field tables are in Data-source schemas.

Authentication and project scope

There are two equivalent route prefixes:

Limited API keys cannot manage sources. A project header cannot redirect an API key to another project. Send JSON bodies with Content-Type: application/json, including revision-checked DELETE requests. Provider credentials belong in connection request bodies, never in the Reflexio authorization header.

User-purge endpoints additionally require governance authorization; ordinary API keys cannot erase users. Use their project-scoped routes with a governance-authorized session. See Governance API.

export REFLEXIO_URL="https://www.reflexio.ai"
# Set REFLEXIO_API_KEY privately to a full-access key for the destination project.
curl --fail-with-body "$REFLEXIO_URL/api/data-sources" \
  -H "Authorization: Bearer $REFLEXIO_API_KEY"

IDs, revisions, and timestamps

A connection owns provider credentials. Its stream selects one external project and its traffic filters. A sample is temporary mapping evidence. Activating the stream creates the collection lifecycle. Do not interchange these IDs: the dashboard URL's source parameter is the connection ID, while collection API paths use the stream ID.

Re-read the relevant resource after a 409; do not blindly retry a stale revision. A mapping's validated_revision identifies the validated draft, while status's mapping_revision identifies the active mapping. Saving a draft is not promotion. Stream.status is the setup resource's draft marker; use collection status for current lifecycle state.

Lifecycle and status timestamps are Unix seconds. SampleWindow.start and end use timezone-aware ISO 8601 strings. Historical ranges are start-inclusive and end-exclusive; activation requires a past range with 0 < start < end <= now.

Endpoint index

Append these paths to either prefix above. A successful response is 200 unless another status is shown. A 202 accepts asynchronous work; it does not mean that records have been published.

Setup context and capabilities

GET /setup-context?connection_id={connection_id} returns a non-secret contract for setup clients. Omit the query for general capabilities; a sole connection is selected automatically. An unknown explicit connection ID returns 404.

Use this contract to discover supported behavior without transmitting provider credentials to a model or embedding them in a review URL.

Connect and select traffic

Each Reflexio project supports ten connections, including unexpired setup drafts. Each connection selects one external project. Pausing or disabling retains its slot. Connections never return their stored secret.

Provider defaults are Braintrust and US. Private provider endpoints are not supported. For example, POST this JSON to /connections using your own read credential:

{"name": "Support traces", "provider": "braintrust", "region": "us", "api_key": "<provider-read-key>"}

Discover projects, then PUT /connections/{connection_id}/stream:

{
  "revision": 1,
  "external_project_id": "<ID returned by discovery>",
  "filters": [{"path": "span_attributes.name", "op": "eq", "values": ["answer"]}]
}

revision is the connection revision. The workspace is resolved from discovery. Selection also advances the connection revision; read the connection again before rotating credentials, reselecting, or deleting. Before activation, stream selection can change. After activation, provider, region, external project/workspace, and original activation timestamp stay fixed. Use the live-filter endpoint for supported active edits instead of selecting a new stream.

Filter contract

Filters are combined with AND. An empty list selects all candidate traffic. At most ten filters are accepted. Supported dotted paths are metadata.*, span_attributes.name, and tags (maximum 160 characters).

Each value is 1–512 characters. Preserve commas and whitespace literally; values are separate JSON array elements, not a comma-separated string. Local checks can compare numbers through their string representation and booleans as "true" or "false".

Braintrust applies supported filters and eligibility time bounds before pagination, with local checks for correctness. filter_execution reports before_download, partial, or after_download. Langfuse and LangSmith filter after download. Supporting spans can be read outside the candidate filters only as context for an eligible interaction, not as additional import candidates.

Credential rotation and deletion

PUT /connections/{connection_id}/credential with the connection revision and replacement credential fields for that provider. The new key must reach the selected project. Rotation invalidates retained sample evidence; fetch a fresh sample before mapping validation. The selected provider dataset remains unchanged.

DELETE accepts {"revision": 2} for a setup draft. For an activated source, also supply its current lifecycle_revision. Deletion removes credentials, configuration, and pending payloads. Imported interactions and deduplication receipts remain. It does not delete the provider's traces and is not a user-data erasure operation.

Sample and prepare a mapping

1. Optionally PUT /streams/{stream_id}/history-draft with the stream revision, history (default true), and optional Unix-second bounds. This saves setup choices only; it neither activates collection nor changes an activated source.
2. POST /streams/{stream_id}/samples with the stream revision and, if desired, an explicit sample_window.
3. Poll GET /samples/{sample_id} until complete, failed, or expired.
4. Initialize a mapping from examples or save one manually.
5. Preview, inspect blockers and example output, then validate the exact saved draft.
6. Explicitly activate or promote the validated revision.

{
  "revision": 1,
  "sample_window": {
    "start": "2026-09-01T00:00:00Z",
    "end": "2026-09-08T00:00:00Z"
  }
}

Samples retain up to 50 records, expire after 24 hours, and are bounded evidence, not a count of all matching traffic. Braintrust samples up to seven time buckets; other providers use bounded pages. Inspect truncated, coverage, buckets, code, and retry_at. Sampling without historical import may still inspect older traffic to help configure mappings; this does not make that traffic eligible for import.

Manual mapping, initialization, and advisory suggestions

GET /mapping returns the current draft, its revision, and validated_revision. PUT /mapping takes expected_revision and a complete definition; use revision 0 for a new draft. Saves increment the draft revision and invalidate its validation. They do not replace an active mapping. For example, this complete MappingWrite selects the first text block while keeping identity and completion explicit:

{
  "expected_revision": 0,
  "definition": {
    "version": 1,
    "rules": [{
      "name": "Answer",
      "fields": {
        "user": {"path": "/metadata/user_id"},
        "session": {"path": "/metadata/session_id"},
        "input": {"path": "/input"},
        "output": {"path": "/output/message/content", "transform": "first_text_block"},
        "version": {"path": "/metadata/agent_version"},
        "references": {"path": "/metadata/reflexio/retrieved_learnings"},
        "source": {"path": "/metadata/environment"},
        "timestamp": {"path": "/created"},
        "completion": {"path": "/metrics/end"}
      }
    }]
  }
}

Adapt these pointers to observed fields; the example does not create missing outputs.

POST /mapping/initialize takes sample_id and expected_revision (default 0). It sends up to three representative masked examples, including message content, to the configured generation model and saves a generated draft. Each example is bounded to 64 KB. Calling with revision 0 preserves an already-saved draft; pass its current revision to explicitly regenerate. Review, validation, and promotion remain separate. Model failures do not authorize ingestion.

GET /suggestions/model returns available, model, data_sent, and usage. The separate POST /suggestions operation requires a saved draft, sample_id, expected_revision, sample_digest from preview, and consent: true. It sends a bounded inventory of field names and types, withholding dialogue values. Its response includes candidates, missing, revision/sample provenance, model, prompt_version, measured_coverage, and automatic_application: false. Apply chosen changes by saving a mapping explicitly. This endpoint does not regenerate or promote an active definition.

Preview and validation

POST /preview with:

{"sample_id": "<sample-id>", "expected_revision": 3}

An optional definition previews an unsaved proposal without saving it. The result includes records, ready, held, excluded, sourceRecords, messages, sample_digest, mapping_revision, mapping_digest, policy_digest, blockers, rule_coverage, field_guidance, and advisory: true.

Each proposal includes resolved fields, proposed messages, disposition, errors, diagnostics, provenance, learning eligibility and attribution capture. Counts apply only to this sample. Ready in preview does not mean published.

To validate, POST the same sample and revision to /mapping/validate, adding the returned sample_digest. Omit definition when validating: this endpoint only validates the saved draft and rejects a supplied definition, even if it matches. Validation requires usable ready evidence (or an already-imported history replay) and rejects overlapping rule matches. Some other sampled records can remain held. Read validated_revision from the returned MappingDocument before activation or promotion.

First matching text and existing draft upgrades

For input/output mappings, first_text_block targets one message's content array:

{"scope": "current", "path": "/output/message/content", "transform": "first_text_block"}

It returns the first direct, non-whitespace-only string text from an untyped block or a block with type: "text", in array order. It skips reasoning, tool, image, and nonmatching entries. It does not recurse or search across conversation messages. It rejects conversation-shaped arrays, arrays longer than 100 blocks, and selected text longer than 100,000 characters. An oversized first match fails rather than falling through to a later answer. No usable answer remains held.

text_blocks explicitly joins all usable direct text blocks with newlines. Existing indexed mappings and active definitions retain their existing behavior. New suggestions prefer first_text_block.

POST /mapping/upgrade-text with {"expected_revision": 3} to revise existing indexed input/output paths and applicable fallback paths in the draft. The upgrade uses the containing content array and removes corresponding index-specific existence conditions, preserving unrelated conditions. An unchanged mapping is a no-op. Preview and validate the result, then explicitly PUT /active-mapping; retry existing held records separately.

Activate and control collection

POST /streams/{stream_id}/activate:

{
  "revision": 1,
  "mapping_revision": 3,
  "operation_key": "support-source-activation-1",
  "history": false
}

revision is the stream revision. operation_key is a caller-chosen 1–100 character idempotency key: repeating the same key and activation payload returns status; a conflicting activation returns 409. With history: true (default), omitted bounds select the seven days ending at activation. With history: false, omit both bounds. A custom historical end leaves any gap before activation excluded.

Braintrust uses selected history → live traffic. History reads only the selected interval, oldest first. It switches to live after the historical scan finishes, its message budget is reached, or remaining history is cancelled. With history disabled, its first live query starts at activation, without paging through excluded history. Live reads retain the original activation lower bound, including traffic that arrived during backfill, until an explicit live-filter edit establishes a later bound.

Live imports have no total message limit. Bounded requests and provider cooldowns still apply. The initial historical budget is 5,000 individual messages, not 5,000 traces or conversations. A complete interaction is never partially imported to fill the remaining allowance. An interaction that does not fit is deferred; unfinished history can resume after extending the budget.

Successful live cycles normally schedule their next run about 30 seconds after completion. Collection is asynchronous, with provider pacing, retries, and backpressure; there is no guaranteed fixed polling interval or import latency. Check read and import status separately. Provider retention and incomplete traces can limit completeness. Langfuse and LangSmith keep independent live collection during historical backfill; ongoing scans revisit seven days for late or edited records. Upstream deletion mirroring is unsupported.

Lifecycle operations

POST /operations with the current lifecycle revision:

{"revision": 4, "action": "pause"}

Invalid transitions and stale revisions return 409. Pausing does not reset budgets. Cancelling is distinct from completing the historical date range.

Extend historical allowance

PUT /backfill-limit with {"revision": 4, "message_limit": 20000}. The limit is a new cumulative total, not an increment. It must exceed the current limit and meet backfill.required_total when present. A JSON null limit selects all remaining matching history; the field itself is required. Valid finite limits are positive integers no greater than 9,007,199,254,740,991.

The operation retains dates, original historical filters, deduplication, and lifecycle. For Braintrust it resumes unfinished/deferred history before returning to live. History that was not selected, was cancelled, is already unlimited, or is fully scanned without deferred work is not extendable. A paused source stays paused. Historical usage never includes live messages.

Edit live Braintrust filters

Read status, then PUT /traffic-filters:

{
  "revision": 4,
  "stream_revision": 2,
  "filters": [{"path": "metadata.environment", "op": "in", "values": ["staging", "production"]}]
}

Use the lifecycle revision and stream_revision from that status response. Only activated Braintrust sources support this operation; paused/disabled sources retain their lifecycle. An actual change increments both revisions, invalidates incompatible live pagination and in-flight collection, and schedules work subject to lifecycle. The response includes saved live_filters, original historical_filters, filter_effective_at, live_created_from, and updated revisions.

The fixed lower bound is max(activation_at, filter_effective_at), inclusive. Older unread traffic and later updates to older spans are excluded from the new live scan, even if the filters were broadened. Polling does not move this bound. Equivalent filters are a no-op: revisions and the effective timestamp do not advance.

Historical scans, counts, deferred records, and later budget extensions use the original historical filters. Existing staged records keep their processing contract; held records are not bulk retried or reclassified. Published interactions are unchanged. Actual edits invalidate samples and draft validation while retaining draft content and the active mapping. Read a fresh sample before validating again.

Read collection status

Draft status contains lifecycle: "draft", activation_available, activation_draft, and provider_read. Activated status adds the following fields. GET /status also adds evaluation_settings; mutation responses need not include that augmentation.

backfill.state is one of disabled, cancelled, complete, running, limit_reached, needs_higher_limit, or needs_review. Budget exhaustion is not historical completion. Estimated count exhaustion does not finish a scan; the date range must be exhausted, and queued/held records may still need processing.

Attention records, evidence, and retries

GET /held?after={cursor} returns up to 50 rows in ID order. Continue with next_cursor until null. Despite its name, it can include held, conflict, expired, erased, and temporarily settling ready records. Returned fields are id, event_id, disposition, reason, capture, event_time, available_at, expires_at, source_digest, mapping_revision, canonical_request_id, history_diagnostics, retry_eligible, and source_url.

Attribution warnings are different from import blockers. A published response can have missing, empty, malformed, ambiguous, or unresolvable learning attribution. Use /evidence to confirm publication rather than treating every warning as a failed import. Missing input/output or reliable user/session identity remains held; the connector does not manufacture those values.

GET /evidence returns {"records": [...]} for the latest 20 receipts. Each includes event_id, request_id, interaction_ids, admission, message_captures, history_diagnostics, capture, source_url, extraction_status, user_id, session_id, agent_version, and per-agent responses with interaction_id, capture, and references. Imported messages, extraction, and evaluation are separate stages; a receipt does not prove a profile was generated.

Promote and retry deliberately

PUT /active-mapping with {"revision": 4, "mapping_revision": 3} to promote a validated draft. The response advances lifecycle revision. New records use that mapping; queued records retain their saved mapping unless explicitly retried.

Then POST /retry-preview with current revisions and inbox record IDs from /held, not provider event IDs:

{"revision": 5, "mapping_revision": 3, "record_ids": ["<held-record-id>"], "append_late": false}

The response contains preview_digest, queued: false, and records. Each proposal contains id, disposition, errors, capture, messages, and history_group. Preview may fetch missing provider evidence. It does not publish or enqueue admission. To execute, POST the exact selection to /retry, adding that preview_digest. The 202 response has queued: true; poll status/evidence for actual admission.

Select 1–50 retained, unexpired held records. Admitted, erased, expired, or otherwise changed selections are rejected. Late historical or cancelled-backfill records require append_late: true, explicit preview, then execution; consumed learning windows are not rewritten. A changed record, mapping, or lifecycle can invalidate the digest; preview again after a 409. A paused source remains paused after retry.

User erasure

POST /purge-preview or /purge-user with user_id and an audit request_id, using the additional governance authorization described above. The preview returns estimated counts and a human-readable scope; execution uses the existing organization-level user-erasure workflow and returns UserEraseResult.

The stream is an authorization/context anchor, not a source-only erasure boundary. User erasure covers the organization; unresolved source payloads and samples are conservatively cleared in the selected project. Read the preview scope and Governance API before executing. Deleting a connection is a separate operation and does not erase admitted user data.

Errors and recovery

Source business errors use this shape (request-validation and authentication errors can have different detail shapes):

{"detail": {"code": "conflict", "message": "Mapping changed; reload its draft.", "retry_after_seconds": 0}}

Do not trigger discovery every time you read status or render a source page. It makes a live provider request and can fail independently of viewing saved configuration. The dashboard discovers during setup or an explicit project-name lookup, not when opening an activated source's Settings. Invalid credentials still surface when a provider read is actually requested.