A/B Testing GPU Performance Formula
Learn how GPU benchmark A/B results are converted into FPS change, frame-time change, and a variability-adjusted performance difference score.
This calculator compares two sets of repeated GPU benchmark results. It estimates how much GPU B changes average FPS relative to GPU A, translates that change into average frame time, and scales the observed FPS gap against measured run-to-run variation.
- 100% Free
- No Sign-Up Required
- Private & Secure
- Mobile Friendly
GPU B Performance Change
Where:
Subtract GPU A's average FPS from GPU B's average FPS, divide by GPU A's FPS, and multiply by 100. A positive result means GPU B was faster.
Variables Explained
| Variable | What It Means | Unit |
|---|---|---|
| gpuAFps - GPU A average FPS | Mean frames per second for the baseline GPU or setup across comparable runs. | FPS |
| gpuBFps - GPU B average FPS | Mean frames per second for the comparison GPU or setup across comparable runs. | FPS |
| gpuAStdDev - GPU A FPS standard deviation | Run-to-run spread in GPU A FPS results. | FPS |
| gpuBStdDev - GPU B FPS standard deviation | Run-to-run spread in GPU B FPS results. | FPS |
| gpuARuns - GPU A benchmark runs | Number of comparable benchmark runs for GPU A. | runs |
| gpuBRuns - GPU B benchmark runs | Number of comparable benchmark runs for GPU B. | runs |
| standardError - Difference standard error | Estimated uncertainty in the difference between the two average FPS values. | FPS |
Step-by-Step Calculation
Find the FPS difference
This is the absolute average-FPS gap between the two tested setups.
fpsDifference = gpuBFps - gpuAFps
Calculate percentage performance change
The FPS difference is measured against GPU A, making results comparable across starting frame rates.
performanceChange = (fpsDifference / gpuAFps) * 100
Convert GPU A FPS to frame time
Dividing 1,000 milliseconds by FPS estimates the average time required to render one frame.
gpuAFrameTime = 1000 / gpuAFps
Convert GPU B FPS to frame time
A lower result means GPU B is delivering frames more quickly on average.
gpuBFrameTime = 1000 / gpuBFps
Find the frame-time change
A negative value indicates that GPU B has a shorter average frame time than GPU A.
frameTimeChange = gpuBFrameTime - gpuAFrameTime
Estimate variation in the FPS difference
This combines both setups' run-to-run variation while accounting for the number of runs.
standardError = sqrt((gpuAStdDev * gpuAStdDev / gpuARuns) + (gpuBStdDev * gpuBStdDev / gpuBRuns))
Calculate the performance difference score
A larger score means the measured FPS gap is larger relative to the reported benchmark variation.
zScore = abs(fpsDifference) / standardError
Worked example: 120 FPS versus 135 FPS
FPS difference
135 - 120
15 FPS
Performance change
(15 / 120) * 100
12.5%
GPU A frame time
1000 / 120
8.33 ms
GPU B frame time
1000 / 135
7.41 ms
Frame-time change
7.41 - 8.33
-0.93 ms
Standard error
sqrt((3 * 3 / 10) + (3.5 * 3.5 / 10))
1.46 FPS
Performance difference score
abs(15) / 1.46
10.27
Final Result
GPU B averages 15 FPS more than GPU A, a 12.5% improvement, with an average frame-time reduction of about 0.93 ms and a performance difference score of 10.27.
Assumptions
- ✓Both setups use the same game version, scene, resolution, graphics settings, and benchmark method.
- ✓The supplied averages and standard deviations come from representative repeated runs.
- ✓Each run is treated as broadly independent of the others.
- ✓FPS is an appropriate primary performance measure for the benchmark being compared.
- ✓The difference score is a simplified variability-adjusted indicator rather than a complete experimental analysis.
Limitations
- !Average FPS does not show minimum FPS, percentile FPS, stutter, or frame-time spikes.
- !Driver state, shader compilation, temperatures, power limits, background activity, and run order can affect results.
- !A large statistical score does not prove that every game or workload will show the same improvement.
- !A measurable FPS gain may not be noticeable when both results already exceed the display refresh rate or another system component is limiting performance.
Common Mistakes to Avoid
Comparing runs made with different resolutions, presets, game versions, or benchmark scenes.
Entering a single-run result as an average and guessing a standard deviation.
Using zero or near-zero variation without confirming that the measurements are valid.
Reading a positive FPS difference as proof of smoother gameplay without checking frame-time consistency.
Comparing results from a CPU-limited scene when the purpose is to isolate GPU performance.
Related Formulas
Frequently Asked Questions
How do you calculate GPU FPS improvement?
Subtract GPU A average FPS from GPU B average FPS, divide by GPU A average FPS, and multiply by 100.
Why is frame time calculated as 1000 divided by FPS?
There are 1,000 milliseconds in one second, so 1,000 divided by frames per second gives the average milliseconds per frame.
What does a negative frame-time change mean?
It means GPU B has a lower average frame time and is rendering frames faster than GPU A.
How is the GPU performance difference score calculated?
It divides the absolute FPS difference by a combined standard error based on both standard deviations and run counts.
Is a score above 2 statistically significant?
A score near or above 2 can indicate a clearer measured difference under comparable, independent runs. It is only a rough guide and not a full statistical conclusion.
Ready to calculate your result?
Use the calculator to get instant results with your own inputs.