A/B Testing Service-Level Agreement Formula
Learn how users per variation, total traffic, and estimated test duration are calculated for a conversion-rate A/B test.
This calculator estimates the qualifying-user sample needed to detect a chosen relative conversion lift. The result helps set a realistic experiment service-level agreement by linking statistical requirements to available traffic.
- 100% Free
- No Sign-Up Required
- Private & Secure
- Mobile Friendly
Required Users per Variation
Where:
The calculation estimates users needed in each equally sized group by combining the baseline rate, target variant rate, confidence threshold, statistical power, and absolute conversion-rate difference.
Variables Explained
| Variable | What It Means | Unit |
|---|---|---|
| pc - Control conversion rate | Baseline conversion rate expressed as a decimal. | percent |
| pv - Expected variant conversion rate | Baseline rate after applying the target relative lift. | percent |
| p̄ - Average conversion rate | Average of the control and expected variant rates. | percent |
| zα - Significance-level value | Two-sided normal-distribution threshold selected for the confidence level. | N/A |
| zβ - Statistical-power value | Normal-distribution value corresponding to selected power. | N/A |
| n - Users per variation | Required qualifying users in each of the two groups. | users |
| d - Conversion-rate difference | Absolute difference between the expected variant and control rates. | percent |
Step-by-Step Calculation
Convert the baseline rate
The entered baseline percentage is converted to a decimal rate.
controlRate = baselineConversionRate / 100
Apply the target lift
The target relative lift creates the conversion rate the test is designed to detect.
variantRate = controlRate * (1 + minimumDetectableLift / 100)
Find the absolute difference
Sample size depends on the absolute change in conversion rate, not only the relative lift.
conversionRateDifference = variantRate - controlRate
Calculate users per variation
The two-proportion approximation estimates the required sample in each equally sized group.
usersPerVariation = ceil(pow(significanceLevel * sqrt(2 * averageRate * (1 - averageRate)) + statisticalPower * sqrt(controlRate * (1 - controlRate) + variantRate * (1 - variantRate)), 2) / pow(conversionRateDifference, 2))
Calculate total sample
There is one control group and one variant group.
totalRequiredUsers = usersPerVariation * 2
Estimate duration
The total sample is divided by expected daily test traffic and rounded up to whole days.
estimatedTestDays = ceil(totalRequiredUsers / (dailyEligibleUsers * trafficSplit / 100))
Example: 10% baseline and 10% relative lift
Control rate
10 / 100
0.10
Target variant rate
0.10 × (1 + 10 / 100)
0.11
Absolute difference
0.11 - 0.10
0.01
Users per variation
ceil([(1.96 × sqrt(2 × 0.105 × 0.895)) + (0.842 × sqrt(0.10 × 0.90 + 0.11 × 0.89))²] / 0.01²)
14,751 users
Total required users
14,751 × 2
29,502 users
Estimated duration
ceil(29,502 / (2,000 × 100 / 100))
15 days
Final Result
Plan for approximately 14,751 users per variation, 29,502 users overall, and 15 days of test traffic.
Assumptions
- ✓The test has two independent, equally sized variations: a control and one variant.
- ✓The selected significance threshold is two-sided.
- ✓Eligible traffic and conversion behavior stay reasonably stable during the experiment.
- ✓The conversion metric is binary, such as sign-up completed or purchase completed.
- ✓The calculation uses a standard normal approximation for two conversion rates.
Limitations
- !Unequal allocation, more than two variants, and clustered users require a different design or adjustment.
- !The estimate does not account for bot traffic, tracking errors, duplicate users, or eligibility changes.
- !Repeatedly checking results and stopping based on interim outcomes can affect inference unless the test plan accounts for it.
- !The traffic estimate does not ensure that the test covers a representative business cycle or seasonality.
Common Mistakes to Avoid
Entering a relative lift as though it were an absolute percentage-point increase.
Using total site visitors instead of users who actually qualify for and enter the experiment.
Treating 100% traffic allocation as if all users will be eligible or correctly tracked.
Changing the target lift after observing early results.
Using this binary-conversion calculation for revenue, order value, or other continuous metrics.
Related Formulas
Frequently Asked Questions
What is the formula for A/B test sample size per variation?
It is a two-proportion sample-size estimate that uses the control rate, expected variant rate, two-sided significance value, power value, and squared absolute difference between conversion rates.
Why does a low conversion rate need more users?
At low rates, a given relative lift becomes a very small absolute difference, which is harder to distinguish from random variation.
How does minimum detectable lift affect sample size?
A smaller target lift generally requires substantially more users because the expected absolute gap is smaller.
Why is the result rounded up?
A fraction of a user is not possible, so the required sample and duration are rounded up to avoid falling short of the estimate.
Ready to calculate your result?
Use the calculator to get instant results with your own inputs.