A/B Testing Sample Size (Annual) Calculator
Estimate the visitor sample and annual traffic needed to run an A/B test with a chosen conversion rate, detectable lift, confidence level and power.
Overview
This A/B testing sample size calculator estimates how many eligible visitors you need over a year to detect a planned conversion-rate lift. Enter your current conversion rate, the minimum relative improvement that matters, confidence and power settings, and expected annual traffic.
How it works
The calculator first converts the baseline conversion rate and target relative lift into expected control and variant conversion rates. It then uses a standard two-proportion sample-size approximation. Higher confidence or power requires more visitors, while a larger detectable lift requires fewer visitors. The required total sample is compared with your annual eligible traffic to estimate how much of the year the test may take.
How to use this calculator
- 1Enter the current conversion rate for your control experience.
- 2Set the minimum relative conversion lift that would be meaningful to your business.
- 3Use confidence and power z-scores that match your testing standard.
- 4Enter the eligible visitors you expect in one year.
- 5Review the total sample, sample per group, and estimated test duration.
Example Calculation
Baseline conversion rate
5%
Minimum detectable lift
10%
Confidence z-score
1.96
Statistical power z-score
0.84
Annual eligible visitors
120000
Traffic allocated to variant
50%
Total required sample
62,397 visitors
With a 5% baseline rate and a goal of detecting a 10% relative lift, the test needs roughly 61,000 visitors in total. At 120,000 eligible visitors per year, this is approximately six months of traffic.
Frequently asked questions
What is sample size in A/B testing?
Sample size is the number of eligible visitors or observations needed before comparing a control experience with a variant at the chosen confidence and power levels.
What does minimum detectable effect mean?
The minimum detectable effect is the smallest change you want the test to reliably identify. In this calculator, it is entered as a relative lift from the baseline conversion rate.
Why does a smaller expected lift require more traffic?
Small changes are harder to distinguish from normal random variation, so more observations are needed to detect them with the same confidence and power.
What z-score should I use for 95% confidence?
A z-score of 1.96 is commonly used for approximately 95% two-sided confidence. For about 99% confidence, a common value is 2.58.
What power z-score should I use?
A z-score of 0.84 corresponds to about 80% power, while 1.28 corresponds to about 90% power. Higher power increases the required sample.
Does this calculator support unequal traffic splits?
You can enter your planned variant allocation for planning context, but the core sample-size estimate assumes an equal control-versus-variant split because it is the most statistically efficient arrangement.
Can I stop an A/B test as soon as results look significant?
Stopping early can make results less reliable. Plan the sample size in advance and use a testing approach that accounts for any interim checks.
Explore Related Calculators
Assumptions and warnings
Assumptions
- The calculation uses a normal approximation for comparing two independent conversion rates.
- The confidence z-score represents a two-sided significance threshold.
- Traffic and conversion behavior are assumed to remain reasonably stable during the test.
- The main sample estimate assumes an even 50/50 split between control and variant.
- Visitors are assumed to be independent, and each visitor is counted once in the experiment.
Warnings
- This calculator provides a statistical planning estimate, not a guarantee of a test outcome.
- Seasonality, multiple variants, repeat visitors, tracking errors, and early stopping can affect the sample needed and the reliability of results.