You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Interpreting benchmark results can be hard; in SwiftUI (and I think this was taken from something done for Swift's benchmarks) the benchmark tool would run a baseline state and new state of the code in the same process, interleaving runs, and report how much speedup or slowdown there is. Also the reports would be adorned with a mark that indicated definite speedup, definite slowdown, or statistically insignificant result. These facilities definitely helped us understand the effects we were seeing and also whether we needed to increase the iteration count or manually kill off processes, etc., to get a useful measurement. IIRC there were three ways to measure the mean result and we mostly ignored everything but the geometric mean. I can probably find out a bit more about the specifics on request. It's also very valuable to have this kind of report from CI.
The text was updated successfully, but these errors were encountered:
Interpreting benchmark results can be hard; in SwiftUI (and I think this was taken from something done for Swift's benchmarks) the benchmark tool would run a baseline state and new state of the code in the same process, interleaving runs, and report how much speedup or slowdown there is. Also the reports would be adorned with a mark that indicated definite speedup, definite slowdown, or statistically insignificant result. These facilities definitely helped us understand the effects we were seeing and also whether we needed to increase the iteration count or manually kill off processes, etc., to get a useful measurement. IIRC there were three ways to measure the mean result and we mostly ignored everything but the geometric mean. I can probably find out a bit more about the specifics on request. It's also very valuable to have this kind of report from CI.
The text was updated successfully, but these errors were encountered: