A/B Testing Service-Level Agreement Calculator Examples
Worked A/B test sample-size and duration examples for different conversion rates, lifts, and traffic levels.
These scenarios show how the target lift, baseline conversion rate, confidence, power, and eligible traffic affect a per-user A/B testing SLA. Each uses two equally sized variations and a binary conversion metric.
Signup test with moderate traffic
10% baseline, 10% relative lift, 95% confidence, 80% power, and 2,000 eligible users per day at full traffic.
Input Summary
Baseline conversion rate
10%
Target relative lift
10%
Daily eligible users
2,000
Test traffic
100%
Calculation Breakdown
- 1Expected variant rate10% × 1.1011%
- 2Required users per variationTwo-proportion sample-size calculation14,751 users
- 3Total users14,751 × 229,502 users
- 4Durationceil(29,502 / 2,000)15 days
Result Summary
Total users
29,502 users
A/B Testing Service-Level Agreement Calculator
The estimated SLA is 14,751 users in each variation over about 15 days.
Low-conversion purchase test
2% baseline, 20% relative lift, 95% confidence, 80% power, 5,000 daily eligible users, and 50% traffic allocation.
Input Summary
Baseline conversion rate
2%
Target relative lift
20%
Daily eligible users
5,000
Test traffic
50%
Calculation Breakdown
- 1Expected variant rate2% × 1.202.4%
- 2Daily test users5,000 × 50%2,500 users/day
- 3Required users per variationTwo-proportion sample-size calculation19,245 users
- 4Durationceil((19,245 × 2) / 2,500)16 days
Result Summary
Duration
16 days
A/B Testing Service-Level Agreement Calculator
The estimated SLA is 19,245 users per variation and about 16 days.
High-traffic onboarding test
25% baseline, 15% relative lift, 95% confidence, 80% power, 20,000 daily eligible users, and 100% traffic allocation.
Input Summary
Baseline conversion rate
25%
Target relative lift
15%
Daily eligible users
20,000
Test traffic
100%
Calculation Breakdown
- 1Expected variant rate25% × 1.1528.75%
- 2Required users per variationTwo-proportion sample-size calculation2,363 users
- 3Total users2,363 × 24,726 users
- 4Durationceil(4,726 / 20,000)1 day
Result Summary
Total users
4,726 users
A/B Testing Service-Level Agreement Calculator
The statistical sample is about 2,363 users per variation, or one day at the stated traffic level.
How to Read Your Results
Users per variation is the estimated minimum qualifying-user sample for both the control and the variant.
Total required users is twice the per-variation result because this calculator assumes two equal groups.
Estimated days uses only the eligible traffic and traffic share entered; it is a planning estimate, not a launch guarantee.
The target variant rate is the baseline rate plus the requested relative lift.
A shorter mathematical duration does not by itself address tracking quality, weekday effects, or unusual events.
Assumptions & Important Notes
- All examples use a two-sided 95% confidence threshold and 80% power.
- Traffic is split equally between the control and variant after entering the test.
- Users are counted when they qualify for the experiment, not merely when they visit the site.
- The primary metric is a binary conversion event.
Related Examples
Frequently Asked Questions
Can I use these A/B test examples for a 50/50 split?
Yes. The examples assume the test audience is split equally after traffic enters the experiment.
What happens if I send only 25% of traffic to a test?
Daily test users fall to one quarter of eligible traffic, so the estimated duration generally becomes about four times longer than at full traffic.
Do I need more users for 99% confidence?
Yes. A higher confidence threshold raises the sample needed, all else equal.
Can a test finish in one day?
It can meet the calculated sample target in one day at high traffic, but representativeness and operational checks may still matter.
Ready to calculate your own result?
Use the live calculator with your own inputs, timing, and preferences.