A/B Testing Sample Size (Monthly) Calculator FAQ
Answers to common questions about A/B test sample size, conversion uplift, confidence, power, eligible traffic, and estimated test duration.
Use these answers to understand the inputs and results of an A/B testing sample size estimate. The calculator is intended for experiment planning and uses a two-version conversion-rate comparison.
General sample size questions
Core concepts behind visitor requirements for conversion experiments.
What does A/B test sample size mean?
It is the estimated number of eligible visitors needed in each version before comparing their conversion rates at the selected planning settings.
Why does the calculator show visitors per variant?
The calculation assumes an even split, so each group needs its own target sample. Total required visitors combines both groups.
What is a minimum detectable uplift?
It is the smallest relative conversion improvement the test is designed to detect. It should represent a difference meaningful to the decision being tested.
Is a larger sample always better?
A larger sample can improve precision for a fixed design, but it also takes longer. The appropriate target depends on the planned effect, confidence, power, and practical constraints.
Inputs and settings
How the calculator interprets conversion rate, z-scores, and traffic inputs.
Should baseline conversion rate be entered as 3 or 0.03?
Enter 3 for a 3% baseline rate. The calculator converts the percentage to a decimal probability internally.
Which z-score represents 95% confidence?
For a standard two-sided 95% confidence setting, use 1.96.
Which z-score represents 80% power?
Use 0.84 for 80% power. A z-score of 1.282 corresponds to 90% power.
What counts as eligible monthly visitors?
Eligible visitors are people who can actually enter the experiment after exclusions such as geography, device, logged-in status, or campaign targeting.
What should I enter for traffic included in the test?
Enter the percentage of monthly visitors expected to be eligible and assigned to the experiment. Use 100% only when all stated visitors can be included.
Duration and interpretation
How to interpret the monthly traffic estimate.
How is estimated test duration calculated?
The calculator divides total required visitors by eligible monthly visitors. It assumes the eligible traffic is split evenly between control and variant.
Why might the actual test take longer than the estimate?
Traffic may decline, eligibility may be lower than expected, allocation may be uneven, tracking may need review, or the baseline rate may change.
Does one month mean 30 days exactly?
The output is based on the monthly traffic figure supplied. It is a traffic-based estimate rather than a fixed calendar-day forecast.
Can a test finish in less than one month?
The estimate can be below one month when eligible traffic exceeds the total visitor requirement, but practical run length may still need consideration for normal time-based variation.
Scope and accuracy
Important boundaries of this simplified planning model.
Does this calculator support more than two variants?
No. It estimates a control-versus-one-variant test with an even split. Multi-variant testing generally needs additional planning adjustments.
Does the calculator account for revenue or average order value?
No. It is based on a binary conversion-rate outcome and does not model monetary outcomes or other metrics.
Can I check results repeatedly and stop when they look significant?
Repeated unplanned checking can affect false-positive risk. This calculator does not model sequential testing or stopping rules.
Are the results guaranteed?
No. They are statistical planning estimates based on the values entered and assumptions about stable, accurately measured traffic and conversion behavior.
What is sample size in A/B testing?
It is the estimated eligible visitor count required in each version to compare conversion rates using the chosen confidence and power settings.
Explore Related Questions
Ready to see what you can calculate?
Open the calculator and get personalized results in seconds.