All

Concepts

Key concepts like Shadow Content and Expert Content used in the Reflexio API.

Concepts

Shadow Content

The shadow_content field allows you to capture an alternative agent response alongside the production response, enabling A/B testing and comparison of different agent behaviors.

How it works:

Field	Purpose
`content`	The actual production interaction between user and agent. Used for storage, semantic search, and retrieval.
`shadow_content`	An alternative response (e.g., from a different agent version or with different user profiles). Used for per-turn head-to-head evaluation when provided.

This separation allows you to:

Compare agent versions: Test how a new agent version would respond without affecting production
Evaluate profile effectiveness: See how different user profile configurations affect agent responses
A/B test playbook integration: Compare responses with and without specific agent playbook entries applied

When to use shadow_content:

Testing a new agent version's responses against the current production version
Evaluating how different user profile configurations affect agent behavior
Comparing responses with different playbook/prompt improvements applied
Running shadow evaluations without impacting the user experience

Example: Comparing Agent Responses

# Agent v1.0 is in production, testing v2.0 response in shadow
client.publish_interaction(
    user_id="user_123",
    interactions=[
        {
            "role": "User",
            "content": "I would like to get a refund"
        },
        {
            "role": "Agent",
            # Production response (v1.0) - generic, asks for info
            "content": "Happy to help you with that. What is your email or phone number?",
            # Shadow response (v2.0) - personalized, uses known user data
            "shadow_content": "Happy to assist you with that. I can see your phone number is 123-456-7890. Is that the right number to send the refund confirmation to?"
        }
    ],
    session_id="support_session_001",
    agent_version="v1.0"
)

curl -X POST "${REFLEXIO_URL:-https://www.reflexio.ai}/api/publish_interaction" \
  -H "User-Agent: my-agent-reflexio" \
  -H "Authorization: Bearer $REFLEXIO_API_KEY" \
  -H "Content-Type: application/json" \
  --data @- <<'JSON'
{
  "user_id": "user_123",
  "interaction_data_list": [
    {
      "role": "User",
      "content": "I would like to get a refund"
    },
    {
      "role": "Agent",
      "content": "Happy to help you with that. What is your email or phone number?",
      "shadow_content": "Happy to assist you with that. I can see your phone number is 123-456-7890. Is that the right number to send the refund confirmation to?"
    }
  ],
  "session_id": "support_session_001",
  "agent_version": "v1.0"
}
JSON

In this example:

content stores the production response from agent v1.0
shadow_content captures how agent v2.0 (with access to user profiles) would have responded
Per-turn evaluation uses shadow_content to assess the v2.0 approach
You can compare evaluation results to decide whether to promote v2.0 to production

The expert_content field lets you provide a human expert's ideal response alongside the agent's actual response. Reflexio compares the two and generates actionable playbook entries -- SOPs (Situation/Trigger, Instruction, Pitfall) -- so your agent learns to match expert behavior.

How it works:

Field	Purpose
`content`	The agent's actual production response.
`expert_content`	A human expert's ideal response for the same user query. Used for comparison-based playbook extraction.

When expert_content is present on any interaction in a session, Reflexio automatically switches to an expert playbook extraction pipeline that:

Pairs each expert response with the agent's response and the preceding user question
Analyzes alignment gaps between the agent and expert
Generates structured SOPs: what triggered the gap, what the agent should do instead, and what pitfalls to avoid

When to use expert_content:

Training from gold-standard responses: When domain experts review agent conversations and provide ideal answers
Quality assurance workflows: Compare production agent output against expert-vetted responses
Bootstrapping new agents: Seed playbook entries from existing expert knowledge before you have enough user interactions
Compliance or accuracy-critical domains: Ensure agent responses meet expert standards (legal, medical, financial)

Example: Learning from Expert Responses

client.publish_interaction(
    user_id="user_123",
    interactions=[
        {
            "role": "User",
            "content": "What is your return policy for electronics?"
        },
        {
            "role": "Agent",
            # Agent's actual response — incomplete
            "content": "You can return electronics within 30 days of purchase.",
            # Expert's ideal response — comprehensive
            "expert_content": (
                "Electronics can be returned within 30 days of purchase with the original receipt. "
                "Items must be in original packaging and include all accessories. "
                "Opened software and personal care electronics are final sale. "
                "For defective items, the return window extends to 90 days. "
                "Refunds are processed to the original payment method within 5-7 business days."
            )
        }
    ],
    session_id="expert_review_001",
    source="expert_review",
    agent_version="v1.0"
)

curl -X POST "${REFLEXIO_URL:-https://www.reflexio.ai}/api/publish_interaction" \
  -H "User-Agent: my-agent-reflexio" \
  -H "Authorization: Bearer $REFLEXIO_API_KEY" \
  -H "Content-Type: application/json" \
  --data @- <<'JSON'
{
  "user_id": "user_123",
  "interaction_data_list": [
    {
      "role": "User",
      "content": "What is your return policy for electronics?"
    },
    {
      "role": "Agent",
      "content": "You can return electronics within 30 days of purchase.",
      "expert_content": "Electronics can be returned within 30 days of purchase with the original receipt. Items must be in original packaging and include all accessories. Opened software and personal care electronics are final sale. For defective items, the return window extends to 90 days. Refunds are processed to the original payment method within 5-7 business days."
    }
  ],
  "session_id": "expert_review_001",
  "source": "expert_review",
  "agent_version": "v1.0"
}
JSON

In this example:

content stores the agent's actual (incomplete) response
expert_content provides the expert's comprehensive ideal response
Reflexio compares the two and extracts playbook entries like: "Agent should mention packaging requirements, exceptions for opened software, extended window for defective items, and refund processing timeline"
The extracted entries enter the standard aggregation pipeline, improving the agent for all users