Power Calculator

Calculate required sample sizes and estimated durations for A/B experiments based on statistical power analysis.

POST /api/v1/experiments/power-calculator

Calculates the required sample size and estimated experiment duration for an A/B test. Uses a two-proportion z-test formula with traffic estimates derived from the tenant’s recent interaction history.

Request Body

Field	Type	Required	Description
baselineConversionRate	number	Yes	Current conversion rate (0.001-1.0)
minimumDetectableEffect	number	Yes	Smallest effect size to detect (0.001-1.0)
significanceLevel	number	No	Alpha level for the test (0.01-0.20, default: 0.05)
power	number	No	Statistical power (0.5-0.99, default: 0.8)
trafficSplit	number	No	Proportion of traffic in the treatment group (0.1-0.9, default: 0.5)

Example

curl -X POST https://playground.kaireonai.com/api/v1/experiments/power-calculator \
  -H "Content-Type: application/json" \
  -H "X-Tenant-Id: my-tenant" \
  -d '{
    "baselineConversionRate": 0.05,
    "minimumDetectableEffect": 0.02,
    "significanceLevel": 0.05,
    "power": 0.8,
    "trafficSplit": 0.5
  }'

Response

{
  "requiredSampleSize": 7850,
  "perVariant": 3925,
  "estimatedDuration": {
    "days": 16,
    "basedOnDailyTraffic": 500
  },
  "recommendation": "With 500 daily decisions, you need ~16 days to reach statistical significance.",
  "sensitivityTable": [
    { "effect": 0.01, "sampleSize": 31400, "days": 63 },
    { "effect": 0.02, "sampleSize": 7850, "days": 16 },
    { "effect": 0.05, "sampleSize": 1260, "days": 3 },
    { "effect": 0.10, "sampleSize": 320, "days": 1 }
  ]
}

Response Fields

Field	Type	Description
requiredSampleSize	integer	Total sample size needed across both variants
perVariant	integer	Sample size per variant
estimatedDuration.days	integer or null	Estimated days to reach the required sample size
estimatedDuration.basedOnDailyTraffic	integer	Average daily decisions from the last 14 days
recommendation	string	Human-readable guidance
sensitivityTable	array	Sample sizes for different effect sizes

Sensitivity Table

The sensitivity table shows how the required sample size changes with different minimum detectable effects. This helps teams understand the trade-off between experiment duration and the ability to detect small effects.

If estimatedDuration.basedOnDailyTraffic is 0, the tenant has no recent decision traffic. Start sending decisions through the Recommend API to get accurate duration estimates.

Roles

admin, editor, viewer

See also: Experiments | Algorithm Models