API Reference

initialize

Creates a new experiment with the specified variants. Returns an experimentId that must be used in all subsequent calls.

Parameters

• variants (array of strings, required): List of variant identifiers to test. Minimum 2 variants required. Must be unique (case-sensitive).
• experimentId (string UUID, optional): Custom experiment ID. If not provided, a UUID will be generated.

MCP Tool Call

const result = await client.callTool({
  name: "initialize",
  arguments: { variants: ["A", "B", "C"] }
});
const { experimentId } = JSON.parse(result.content[0].text);

Response

{
  "experimentId": "550e8400-e29b-41d4-a716-446655440000"
}

Save this experimentId — you'll need it for all subsequent operations.

select

Returns the next variant to test using Thompson Sampling. This is a read-only operation that doesn't modify experiment state.

Parameters

• experimentId (string UUID, required): The experiment ID returned from initialize.

MCP Tool Call

const result = await client.callTool({
  name: "select",
  arguments: { experimentId }
});
const { nextVariant } = JSON.parse(result.content[0].text);

Response

{
  "nextVariant": "B"
}

Selection is stochastic — repeated calls with identical state may return different variants by design.

record

Records the outcome of a variant evaluation. Updates Bayesian statistics and checks for convergence.

Parameters

• experimentId (string UUID, required): The experiment ID returned from initialize.
• variant (string, required): The variant identifier being evaluated. Must match one from initialize.
• score (number, required): Normalized outcome score between 0.0 and 1.0 (inclusive). Higher is better. Values outside this range are rejected.

MCP Tool Call

const result = await client.callTool({
  name: "record",
  arguments: { experimentId, variant: "B", score: 0.9 }
});
const response = JSON.parse(result.content[0].text);
if (response.winner) {
  console.log("Winner:", response.winner);
}

Response (No Winner)

{}

Response (Winner Declared)

{
  "winner": "B"
}

The winner field is only present when the experiment has converged with sufficient confidence (typically 95%+ probability that one variant is best).

status

Returns comprehensive experiment statistics including progress, convergence metrics, and winner information. This is a read-only operation.

Parameters

• experimentId (string UUID, required): The experiment ID returned from initialize.

MCP Tool Call

const result = await client.callTool({
  name: "status",
  arguments: { experimentId }
});
const status = JSON.parse(result.content[0].text);

Response (Early Stage)

{
  "experimentId": "550e8400-e29b-41d4-a716-446655440000",
  "totalEvaluations": 12,
  "nextVariant": "A"
}

Response (Winner Declared)

{
  "experimentId": "550e8400-e29b-41d4-a716-446655440000",
  "totalEvaluations": 165,
  "nextVariant": "B",
  "convergenceProgress": 1.0,
  "remainingEvaluations": 0,
  "winner": {
    "variantName": "B",
    "evaluations": 62,
    "meanScore": 0.847,
    "stdDev": 0.042,
    "confidenceInterval": [0.763, 0.931]
  },
  "variants": [
    {
      "name": "A",
      "evaluations": 48,
      "meanScore": 0.653,
      "posteriorMean": 0.655,
      "posteriorStdDev": 0.038
    },
    {
      "name": "B",
      "evaluations": 62,
      "meanScore": 0.847,
      "posteriorMean": 0.848,
      "posteriorStdDev": 0.029
    },
    {
      "name": "C",
      "evaluations": 55,
      "meanScore": 0.712,
      "posteriorMean": 0.713,
      "posteriorStdDev": 0.035
    }
  ]
}

Response Fields

• experimentId: The experiment UUID
• totalEvaluations: Total number of recorded evaluations across all variants
• nextVariant: Recommended variant to evaluate next (same as select tool)
• convergenceProgress (optional): Progress toward winner declaration (0.0-1.0). Only present when a winner has been identified.
• remainingEvaluations (optional): Estimated evaluations until convergence. Only present when a winner has been identified.
• winner (optional): Winner details, only present when experiment has converged with sufficient confidence
• variants (optional): Per-variant statistics. Only present when a winner has been identified.

Error Handling

All tools return error messages in the MCP tool result format. Check the content[0].text field for error objects:

{
  "error": "Variant B not found in experiment 550e8400-e29b-41d4-a716-446655440000"
}

Common Errors

• Variant identifiers must be unique - Duplicate variant names in initialize
• Experiment already initialized for this instance - Only one experiment per experimentId
• No experiment initialized - Tool called before initialize or with wrong experimentId
• Experiment ID mismatch - experimentId doesn't match the current experiment
• Variant [name] not found - Variant name doesn't match any from initialize
• Score validation errors - Score value outside [0, 1] range