A/B Testing Sample Size Formula
Learn how an A/B testing sample size calculator estimates visitors per variation, total traffic, and test duration for a conversion-rate test.
This formula estimates the traffic needed to compare a control and one variant when the outcome is a binary conversion, such as a purchase or sign-up. It helps set a realistic test length before a test begins by combining the baseline rate, target uplift, confidence level, and statistical power.
- 100% Free
- No Sign-Up Required
- Private & Secure
- Mobile Friendly
Required Visitors per Variation
Where:
The calculation estimates the visitors needed in each group by comparing the expected control and variant conversion rates. Smaller differences require more visitors, while higher confidence and power also increase the required sample.
Variables Explained
| Variable | What It Means | Unit |
|---|---|---|
| n - Sample size per variation | Estimated number of eligible visitors required in the control group and in the variant group. | visitors |
| baselineRate - Baseline conversion probability | Current control conversion rate expressed as a decimal probability. | number |
| variantRate - Target variant conversion probability | Expected variant conversion probability after applying the relative uplift. | number |
| averageRate - Average conversion probability | Average of the expected control and variant conversion probabilities. | number |
| conversionRateDifference - Absolute conversion-rate difference | Difference between the target variant probability and baseline probability. | number |
| confidenceZ - Confidence Z-score | Z-score selected for the confidence level, such as 1.96 for 95% confidence. | number |
| powerZ - Power Z-score | Z-score selected for statistical power, such as 0.84 for 80% power. | number |
Step-by-Step Calculation
Convert the baseline percentage to a probability
A percentage such as 10% becomes 0.10 for use in the sample-size calculation.
baselineRate = baselineConversionRate / 100
Calculate the target variant rate
The minimum detectable uplift is relative. A 20% uplift on a 10% baseline produces a 12% target conversion rate.
variantRate = baselineRate * (1 + minimumDetectableEffect / 100)
Find the average expected rate
The two-proportion approximation uses the average expected conversion probability in one part of the formula.
averageRate = (baselineRate + variantRate) / 2
Calculate the absolute difference
This is the conversion-rate gap the test is designed to detect.
conversionRateDifference = variantRate - baselineRate
Estimate visitors per variation
The result is rounded up because each variation needs a whole number of visitors.
sampleSizePerVariation = ceil(pow((confidenceZ * sqrt(2 * averageRate * (1 - averageRate))) + (powerZ * sqrt((baselineRate * (1 - baselineRate)) + (variantRate * (1 - variantRate)))), 2) / pow(conversionRateDifference, 2))
Calculate total traffic and duration
The total covers both groups. Dividing it by average daily eligible traffic estimates the calendar time needed.
totalSampleSize = sampleSizePerVariation * 2; estimatedTestDays = ceil(totalSampleSize / dailyVisitors)
Example: 10% baseline rate and 20% relative uplift
Convert the baseline rate
10 / 100
0.10
Find the target variant rate
0.10 * (1 + 20 / 100)
0.12 or 12%
Find the average rate
(0.10 + 0.12) / 2
0.11
Find the absolute difference
0.12 - 0.10
0.02 or 2 percentage points
Calculate visitors per variation
ceil(((1.96 * sqrt(2 * 0.11 * 0.89)) + (0.84 * sqrt((0.10 * 0.90) + (0.12 * 0.88))))^2 / 0.02^2)
3,837 visitors
Estimate total traffic and duration
(3,837 * 2) / 1,000, rounded up
7,674 visitors and 8 days
Final Result
Plan for about 3,837 visitors in each variation, 7,674 visitors in total, and roughly 8 days at 1,000 eligible visitors per day.
Assumptions
- ✓The experiment has exactly two independent variations: a control and one variant.
- ✓Eligible traffic is allocated evenly, with approximately 50% of visitors in each variation.
- ✓The measured outcome is binary, such as converted or did not convert.
- ✓The baseline rate and traffic mix remain broadly stable during the experiment.
- ✓The selected uplift is relative to the baseline conversion rate.
- ✓The calculation uses a standard two-sided two-proportion sample-size approximation.
Limitations
- !The estimate does not adjust for more than two variants, multiple metrics, or multiple comparisons.
- !It does not account for repeated significance checks or an early-stopping method.
- !Daily traffic may fluctuate, so actual duration can be longer or shorter than the estimate.
- !A conversion rate can vary by device, channel, day of week, season, and audience mix.
- !This approach is not designed for continuous metrics such as revenue per visitor or average order value.
Common Mistakes to Avoid
Treating a relative uplift as percentage points; a 20% uplift from 10% is 12%, not 30%.
Using all site visits instead of visitors who are actually eligible to enter the experiment.
Choosing an MDE that is too small to be practically meaningful, which can create an unworkably long test.
Stopping when an interim result looks favorable rather than following a defined measurement plan.
Forgetting that total required traffic is twice the per-variation sample for a 50/50 A/B test.
Related Formulas
Frequently Asked Questions
What formula is used for A/B test sample size?
This calculator uses a standard two-proportion sample-size approximation. It combines the expected control and variant conversion rates with Z-scores for confidence and power.
Why is sample size based on an absolute difference if I enter relative uplift?
The uplift input first converts the baseline rate into a target variant rate. The formula then tests the absolute probability gap between those two rates.
How does baseline conversion rate affect sample size?
At a given relative uplift, a lower baseline often produces a smaller absolute rate difference, which generally requires more visitors to detect.
Does 99% confidence require more traffic than 95% confidence?
Yes. A higher confidence level uses a larger Z-score, so the calculation requires more observations per variation.
What does 80% power mean in this calculation?
It means that if the target effect truly exists under the model assumptions, the test has an estimated 80% chance of detecting it using the chosen threshold.
Ready to calculate your result?
Use the calculator to get instant results with your own inputs.