A/B Testing Uptime (Per-User) Calculator
Estimate the uptime, downtime per user, and availability difference when A/B test users are split between two service variants.
Overview
This A/B testing uptime calculator estimates the availability experienced by users when traffic is split between two variants with different uptime levels. Enter each variant's uptime, the traffic allocation, test length, and expected number of users to compare per-user downtime and overall exposure.
How it works
The calculator converts the test duration into hours and estimates downtime for a user assigned to each variant using one minus that variant's uptime. It then weights those downtime and uptime values by the selected traffic split. For example, a 50/50 test gives each variant equal influence on the average user result. Total lost user-hours is the average downtime per user multiplied by the number of test users.
How to use this calculator
- 1Enter the expected uptime percentage for Variant A.
- 2Enter the expected uptime percentage for Variant B.
- 3Set the percentage of users assigned to Variant A.
- 4Add the planned test duration and total number of test users.
- 5Review the weighted per-user uptime, expected downtime, and difference between variants.
Example Calculation
Variant A uptime
100%
Variant B uptime
100%
Traffic assigned to Variant A
50%
Test duration
30
Total test users
10000
Average per-user uptime
99.700%
With 99.9% uptime for Variant A and 99.5% uptime for Variant B, split equally over 30 days, average per-user uptime is 99.7%. Expected downtime is about 2.16 hours per user, and Variant B has 0.4 percentage points less uptime than Variant A.
Frequently asked questions
What is per-user uptime in an A/B test?
Per-user uptime is the availability a typical user is expected to experience based on the variant they are assigned to and the share of traffic sent to each variant.
How is average A/B test uptime calculated?
It is calculated as a weighted average of Variant A and Variant B uptime, using the percentage of test traffic assigned to each variant.
Why does a small uptime difference matter?
A small percentage-point difference can create meaningful user downtime over a long test period or across a large user base.
What are lost user-hours?
Lost user-hours are the combined estimated downtime across all test users. They equal average downtime per user multiplied by the number of users.
Does this calculator measure incident impact?
No. It estimates availability time only. It does not measure the severity of failures, affected user journeys, revenue impact, or recovery quality.
Should uptime be the only A/B test reliability metric?
No. Consider error rates, latency, conversion effects, support contacts, and the impact of failures on key user actions as well.
Explore Related Calculators
Assumptions and warnings
Assumptions
- Users are assigned to one variant for the relevant part of the test according to the selected traffic split.
- Each variant's stated uptime is assumed to apply consistently throughout the test period.
- Downtime is treated as equally likely to affect users over time.
- The result is an availability estimate and does not model the severity, timing, or business impact of individual incidents.
Warnings
- Availability estimates do not capture differences in outage severity, user behavior, regional routing, or error rates.
- Use observed monitoring and incident data alongside this estimate when evaluating production reliability.