Skip to content

Commit a1f04a5

Browse files
committed
[GR-19691] Improve documentation about benchmarking
1 parent ccfde36 commit a1f04a5

File tree

1 file changed

+17
-2
lines changed

1 file changed

+17
-2
lines changed

doc/user/benchmarking.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,14 @@ The Native configuration provides better startup and warmup but has slower peak
3232

3333
Of course you can also benchmark both configurations and see which one is better for what you are benchmarking.
3434

35+
### Run with Enough Warmup
36+
37+
TruffleRuby like other runtimes with a just-in-time compiler needs some time (called warmup) to reach peak performance,
38+
because it takes time to just-in-time compile the relevant methods of the benchmark.
39+
The easiest way to check if there was enough warmup is to run the benchmark workload repeatedly inside a process and print the times of each run.
40+
The times should be very stable once it's warmed up and conversely keep changing while it is not warmed up yet.
41+
See [this documentation](reporting-performance-problems.md) for more details about warmup.
42+
3543
### Consider Disabling the Global C-Extension Lock
3644

3745
On TruffleRuby, C extensions by default use a global lock for maximum compatibility with CRuby.
@@ -85,6 +93,13 @@ Restricting it to a single core for benchmarking does not make sense, it would c
8593

8694
### Avoid Benchmarking on macOS
8795

88-
macOS's memory management is sub-par and can cause unnecessary memory swapping even when there is enough memory.
96+
macOS's memory management is subpar and can cause unnecessary memory swapping even when there is enough memory (e.g. it sometimes keeps terminated processes in memory needlessly).
97+
98+
macOS's TCP stack is also subpar, see the [Passenger docs](https://www.phusionpassenger.com/library/config/apache/optimization/#operating-system-recommendations) on this subject.
8999

90-
macOS's TCP stack is also sub-par, see the [Passenger docs](https://www.phusionpassenger.com/library/config/apache/optimization/#operating-system-recommendations) on this subject.
100+
If you have no choice but to benchmark on macOS then mention that with the results
101+
and ensure there is plenty free memory and no swap while benchmarking.
102+
Use `vm_stat` (`Pages free`) or `top` (`PhysMem` -> `unused`) to check the amount of free memory.
103+
Unfortunately neither of these tools show the swap usage, illustrating how difficult it is to ensure there is no swapping going on when benchmarking on macOS.
104+
Use `Activity Monitor` before and after benchmarking to ensure there is no swap (`Swap Used`) as a workaround.
105+
`Activity Monitor` is too heavy to be used during benchmarking.

0 commit comments

Comments
 (0)