A/B Testing Sample Size (Annual) Calculator Examples
Worked examples showing how baseline rate, detectable lift, confidence, power, and annual visitors affect A/B test sample size.
These examples show how different conversion rates and testing targets change the visitors required and the likely time needed to collect them. All figures are planning estimates based on an equal control-versus-variant split.
Established page with a 5% conversion rate
A landing page converts at 5%, and the team has 120,000 eligible visitors per year.
Input Summary
Baseline conversion rate
5%
Minimum detectable lift
10% relative
Confidence
95% two-sided (z = 1.96)
Power
80% (z = 0.84)
Annual eligible visitors
120,000
Calculation Breakdown
- 1Expected variant rate5% × 1.105.5%
- 2Sample per groupTwo-proportion sample-size approximation31,197 visitors
- 3Total sample31,197 × 262,394 visitors
- 4Estimated duration62,394 / 120,000 × 126.2 months
Result Summary
Estimated duration
6.2 months
A/B Testing Sample Size (Annual) Calculator
The experiment needs about 62,394 visitors in total and uses roughly 52.0% of annual eligible traffic.
Low-conversion ecommerce funnel
A checkout funnel converts at 1%, with 300,000 eligible visitors per year and a target 10% relative lift.
Input Summary
Baseline conversion rate
1%
Minimum detectable lift
10% relative
Confidence
95% two-sided (z = 1.96)
Power
80% (z = 0.84)
Annual eligible visitors
300,000
Calculation Breakdown
- 1Expected variant rate1% × 1.101.1%
- 2Sample per groupTwo-proportion sample-size approximation147,429 visitors
- 3Total sample147,429 × 2294,858 visitors
- 4Estimated duration294,858 / 300,000 × 1211.8 months
Result Summary
Estimated duration
11.8 months
A/B Testing Sample Size (Annual) Calculator
The test requires about 294,858 eligible visitors and would take approximately 11.8 months.
Higher baseline rate with a larger meaningful lift
A product page converts at 12%, and a 20% relative lift is the minimum change worth detecting.
Input Summary
Baseline conversion rate
12%
Minimum detectable lift
20% relative
Confidence
95% two-sided (z = 1.96)
Power
80% (z = 0.84)
Annual eligible visitors
180,000
Calculation Breakdown
- 1Expected variant rate12% × 1.2014.4%
- 2Sample per groupTwo-proportion sample-size approximation3,626 visitors
- 3Total sample3,626 × 27,252 visitors
- 4Estimated duration7,252 / 180,000 × 120.5 months
Result Summary
Estimated duration
0.5 months
A/B Testing Sample Size (Annual) Calculator
The experiment needs about 7,252 visitors in total and could reach the estimate in about half a month.
Higher confidence and power requirement
A team tests a 5% baseline rate and a 10% relative lift but chooses 99% confidence and 90% power.
Input Summary
Baseline conversion rate
5%
Minimum detectable lift
10% relative
Confidence
99% two-sided (z = 2.58)
Power
90% (z = 1.28)
Annual eligible visitors
120,000
Calculation Breakdown
- 1Expected variant rate5% × 1.105.5%
- 2Sample per groupTwo-proportion sample-size approximation54,679 visitors
- 3Total sample54,679 × 2109,358 visitors
- 4Estimated duration109,358 / 120,000 × 1210.9 months
Result Summary
Estimated duration
10.9 months
A/B Testing Sample Size (Annual) Calculator
The stricter design needs about 109,358 total visitors and takes approximately 10.9 months.
How to Read Your Results
Total required sample is the estimated number of eligible visitors needed across control and variant.
Sample per group assumes an efficient equal 50/50 split.
Estimated test months compares the required total sample with annual eligible traffic; it is not a guaranteed calendar duration.
Annual traffic required above 100% means the stated yearly traffic is unlikely to reach the planned sample within one year.
Expected baseline conversions per group provide a useful check on how much conversion data the experiment may produce.
Assumptions & Important Notes
- Examples use a two-sided comparison of independent conversion rates.
- Traffic is assumed to arrive evenly throughout the year.
- The examples assume one control and one variant with an even split.
- Actual traffic eligibility, tracking, and conversion behavior may differ during a live experiment.
Related Examples
Frequently Asked Questions
Can an A/B test finish in less than a month?
It can when eligible traffic is high enough, the conversion rate is sufficiently high, or the minimum detectable lift is relatively large.
Why can a 1% baseline test take almost a year?
A 10% relative lift at a 1% baseline is only a 0.1 percentage-point change, which requires many visitors to detect.
How does annual traffic affect sample size?
Annual traffic does not change the statistical sample requirement. It changes the estimated time needed to reach that requirement.
What happens if I increase the planned lift?
A larger planned lift creates a larger absolute difference between rates, which generally lowers the sample needed.
Are these examples guaranteed outcomes?
No. They are planning estimates and depend on the selected assumptions and the quality and stability of the actual experiment data.
Ready to calculate your own result?
Use the live calculator with your own inputs, timing, and preferences.