A/B Testing Service-Level Agreement Calculator
Estimate whether an A/B test meets its agreed minimum sample size and run-duration service levels.
Overview
Use this A/B Testing Service-Level Agreement Calculator to estimate whether a live experiment has met its minimum run-time and per-variant traffic commitments. Enter daily eligible traffic, the test duration, traffic split, and the sample and duration targets defined in your SLA or testing plan.
How it works
The calculator estimates total eligible traffic by multiplying daily visitors by the number of live test days. It then splits that traffic between the variant and control using the selected allocation. Because both groups must reach the sample target, it evaluates the smaller group against the required per-group sample. Overall SLA completion is the lower of the sample-size completion and run-time completion. The recommended duration is the longer of the minimum SLA duration and the estimated time for the smaller group to reach its sample target.
How to use this calculator
- 1Enter the average number of visitors eligible for the experiment each day.
- 2Enter how many complete days the test has been running.
- 3Set the percentage of traffic allocated to the variant.
- 4Add the minimum sample required in each experiment group.
- 5Enter the SLA minimum test duration and review the completion and recommended duration.
Example Calculation
Eligible visitors per day
1000
Days test has run
14
Variant traffic allocation
50%
Required sample per variant
5000
Minimum required run time
14
SLA completion
100%
After 14 days, each group is estimated to have 7,000 visitors. Both the 5,000-visitor sample target and the 14-day minimum duration are met, so SLA completion is 100%.
Frequently asked questions
What does an A/B testing SLA usually include?
A testing SLA commonly sets a minimum live-test duration, a minimum sample size for each group, and operational expectations such as tracking quality or reporting timing. This calculator focuses on the duration and sample-size components.
Why does the calculator use the smaller experiment group?
Both the control and variant need sufficient data. With an uneven traffic split, the smaller group reaches the required sample later, so it determines the sample-size completion result.
Does reaching 100% SLA completion mean the result is statistically significant?
No. It means the entered minimum duration and sample targets have been met. Statistical significance, confidence intervals, practical impact, and data quality must still be assessed separately.
How should I choose the required sample per variant?
Set it from your experiment design, usually using the baseline conversion rate, the smallest effect worth detecting, confidence level, and desired statistical power. Use the sample requirement agreed for the test.
Can I use an uneven traffic split?
Yes. Enter the percentage assigned to the variant. An uneven split can be useful operationally, but it generally means the smaller group takes longer to reach the same per-group sample target.
Why might actual test duration be longer than the estimate?
Actual duration can increase if eligible traffic falls, visitors are excluded, tracking is incomplete, allocation differs from plan, or the test is paused. The calculation assumes steady eligible traffic.
Explore Related Calculators
Assumptions and warnings
Assumptions
- Eligible visitor traffic is reasonably consistent from day to day.
- Visitors are assigned once to either the control or variant group, with no overlap between groups.
- The stated required sample per variant has been set using an appropriate experiment design or analysis plan.
- SLA completion is based only on minimum traffic and duration requirements, not on statistical significance or business impact.
- The result is an operational estimate and does not account for traffic outages, exclusions, bot filtering, or tracking loss.
Warnings
- This calculator provides an experiment-planning estimate only; meeting a traffic and duration SLA does not by itself establish a statistically reliable result.
- Do not stop or declare a test winner solely because this completion measure reaches 100%; use your agreed analysis method and guardrail checks.