A/B Testing Service-Level Agreement Formula
Learn how A/B test SLA completion is calculated from traffic, allocation, sample targets, and minimum run time.
This calculation estimates operational progress against an A/B testing service-level agreement. It checks whether both experiment groups have enough estimated visitors and whether the test has been live for the required number of days.
- 100% Free
- No Sign-Up Required
- Private & Secure
- Mobile Friendly
SLA completion
Where:
The SLA result is the lower of progress toward the per-group sample target and progress toward the minimum test duration. It reaches 100% only when both requirements are met.
Variables Explained
| Variable | What It Means | Unit |
|---|---|---|
| dailyVisitors - Eligible visitors per day | Average daily visitors who are eligible to enter the experiment. | visitors |
| testDays - Days test has run | Number of complete days the experiment has been live. | days |
| variantAllocation - Variant traffic allocation | Percentage of eligible traffic assigned to the variant; the rest goes to the control. | percent |
| requiredSamplePerVariant - Required sample per variant | Minimum visitor count required in each experiment group. | visitors |
| minimumTestDays - Minimum required run time | Minimum live-test duration specified in the SLA or test plan. | days |
| smallerGroupVisitors - Smaller group size | The lower estimated visitor count of the control and variant groups. | visitors |
Step-by-Step Calculation
Calculate total eligible test visitors
Multiply average eligible daily traffic by completed live days.
totalTestVisitors = dailyVisitors * testDays
Estimate variant visitors
Apply the selected traffic allocation to total eligible test visitors.
variantVisitors = totalTestVisitors * variantAllocation / 100
Estimate control visitors
The remaining eligible traffic is assigned to the control group.
controlVisitors = totalTestVisitors - variantVisitors
Find the smaller experiment group
Both groups must meet the target, so the smaller group controls sample progress.
smallerGroupVisitors = min(variantVisitors, controlVisitors)
Calculate sample-size completion
Compare the smaller group with the required per-group sample and cap the result at 100%.
sampleCompletion = min(smallerGroupVisitors / requiredSamplePerVariant * 100, 100)
Calculate duration completion
Compare completed live days with the SLA minimum duration and cap the result at 100%.
durationCompletion = min(testDays / minimumTestDays * 100, 100)
Calculate overall SLA completion
The lower completion percentage is the operational SLA completion result.
slaCompletion = min(sampleCompletion, durationCompletion)
Estimate the recommended total duration
Use the longer of the stated minimum duration and the estimated days needed for the smaller group to reach its target.
recommendedTestDays = max(minimumTestDays, ceil(requiredSamplePerVariant / min(dailyVisitors * variantAllocation / 100, dailyVisitors * (100 - variantAllocation) / 100)))
Example: 50/50 website experiment
Total eligible visitors
1,000 * 14
14,000 visitors
Variant visitors
14,000 * 50 / 100
7,000 visitors
Control visitors
14,000 - 7,000
7,000 visitors
Sample-size completion
min(7,000 / 5,000 * 100, 100)
100%
Duration completion
min(14 / 14 * 100, 100)
100%
Recommended total duration
max(14, ceil(5,000 / 500))
14 days
Final Result
SLA completion is 100%. The estimated test meets both the sample-size and minimum-duration requirements.
Assumptions
- ✓Eligible traffic is reasonably steady from day to day.
- ✓Each eligible visitor is assigned to either control or variant, not both.
- ✓The required per-group sample was set using an appropriate experiment design or analysis plan.
- ✓The SLA is based on traffic and run duration rather than statistical significance or business impact.
- ✓The test has no material pauses, tracking losses, exclusions, or allocation failures.
Limitations
- !The calculation estimates visitor exposure; it does not determine statistical significance, confidence intervals, or practical impact.
- !Daily traffic can fluctuate, so the actual time to reach a target can differ from the estimate.
- !It does not evaluate conversion tracking quality, bot filtering, duplicate users, or audience eligibility changes.
- !A test can meet its SLA while still needing further analysis under the agreed evaluation method.
Common Mistakes to Avoid
Using total test traffic as the sample size when the requirement applies separately to each group.
Ignoring the smaller group when using an uneven traffic allocation.
Entering visits rather than eligible experiment visitors.
Treating 100% SLA completion as proof that a variant has won.
Forgetting to adjust the estimate after a traffic outage, pause, or major eligibility change.
Related Formulas
Frequently Asked Questions
What is the A/B testing SLA completion formula?
It is the lower of sample-size completion and duration completion. Both values are capped at 100%, so the result reaches 100% only when both requirements are met.
Why is the smaller experiment group used?
A per-variant target must be met by both control and variant. The smaller group reaches that target later, particularly under an uneven split.
How are days needed for the sample calculated?
The required per-group sample is divided by the smaller group's expected daily traffic, then rounded up to a whole day.
Does 100% SLA completion mean statistical significance?
No. It only means the entered traffic and duration thresholds have been met. Statistical analysis and data-quality checks remain separate.
Ready to calculate your result?
Use the calculator to get instant results with your own inputs.