A/B Testing Screen Resolution Per-User Calculator FAQ
Answers to common questions about A/B test sample size, users per variation, minimum detectable effect, confidence, and power.
Use these answers to understand the inputs and results of an A/B conversion sample-size estimate. The calculator is intended for planning two-variation tests with an equal split of eligible users.
General A/B Test Planning Questions
Basic questions about what the calculator estimates.
What does this A/B testing calculator estimate?
It estimates the number of eligible users needed in each of two variations to detect a selected conversion-rate difference.
What is the total sample size?
For an equal two-variation split, total sample size is users per variation multiplied by two.
What counts as an eligible user?
An eligible user is a person who can be assigned to a variation and has a genuine opportunity to complete the defined conversion.
Is the result a required test duration?
No. It is a user-count estimate. Duration depends on the rate at which eligible users arrive.
Inputs and Effect Size
How to enter the baseline rate and selected conversion change.
How should I enter the baseline conversion rate?
Enter the current control conversion rate as a percentage, such as 10 for a 10% conversion rate.
What is a minimum detectable effect?
It is the smallest absolute conversion-rate difference the test is intended to detect.
Are percentage points and relative lift different?
Yes. Moving from 10% to 11% is one percentage point and a 10% relative lift.
Can I plan for a decrease rather than an increase?
The sample-size magnitude is based on the size of the absolute difference. The test is two-sided, so it can assess either direction when analyzed appropriately.
Confidence and Statistical Power
How the selected statistical thresholds affect traffic requirements.
What does 95% confidence mean here?
It corresponds to a two-sided significance threshold represented by a z-value of 1.96 in the calculation.
What is statistical power?
Power is the planned probability of detecting the selected effect when that effect is truly present, under the calculation assumptions.
Does higher power require more users?
Yes. Increasing power increases the per-variation sample-size estimate.
Does 99% confidence require more users than 95% confidence?
Yes. A stricter confidence threshold increases the amount of evidence required and therefore the estimated sample size.
Accuracy and Use Limits
Important boundaries of this planning estimate.
Is this calculation suitable for revenue metrics?
No. It is designed for binary conversion outcomes. Revenue and average order value need methods that account for metric variability.
Can I use it for an uneven traffic allocation?
Not directly. The calculation assumes equal allocation; an uneven split generally needs more total traffic.
Does the calculator account for repeated result checks?
No. Repeated interim decisions can change false-positive risk and may require a different testing approach.
Does reaching the estimated sample size prove that a test result is valid?
No. Data quality, assignment integrity, metric definitions, and the chosen analysis still matter.
What does users per variation mean in an A/B test?
It is the estimated number of eligible users needed in control and separately in treatment.
Explore Related Questions
Ready to see what you can calculate?
Open the calculator and get personalized results in seconds.