You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/user/benchmarking.md
+17-2Lines changed: 17 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -32,6 +32,14 @@ The Native configuration provides better startup and warmup but has slower peak
32
32
33
33
Of course you can also benchmark both configurations and see which one is better for what you are benchmarking.
34
34
35
+
### Run with Enough Warmup
36
+
37
+
TruffleRuby like other runtimes with a just-in-time compiler needs some time (called warmup) to reach peak performance,
38
+
because it takes time to just-in-time compile the relevant methods of the benchmark.
39
+
The easiest way to check if there was enough warmup is to run the benchmark workload repeatedly inside a process and print the times of each run.
40
+
The times should be very stable once it's warmed up and conversely keep changing while it is not warmed up yet.
41
+
See [this documentation](reporting-performance-problems.md) for more details about warmup.
42
+
35
43
### Consider Disabling the Global C-Extension Lock
36
44
37
45
On TruffleRuby, C extensions by default use a global lock for maximum compatibility with CRuby.
@@ -85,6 +93,13 @@ Restricting it to a single core for benchmarking does not make sense, it would c
85
93
86
94
### Avoid Benchmarking on macOS
87
95
88
-
macOS's memory management is sub-par and can cause unnecessary memory swapping even when there is enough memory.
96
+
macOS's memory management is subpar and can cause unnecessary memory swapping even when there is enough memory (e.g. it sometimes keeps terminated processes in memory needlessly).
97
+
98
+
macOS's TCP stack is also subpar, see the [Passenger docs](https://www.phusionpassenger.com/library/config/apache/optimization/#operating-system-recommendations) on this subject.
89
99
90
-
macOS's TCP stack is also sub-par, see the [Passenger docs](https://www.phusionpassenger.com/library/config/apache/optimization/#operating-system-recommendations) on this subject.
100
+
If you have no choice but to benchmark on macOS then mention that with the results
101
+
and ensure there is plenty free memory and no swap while benchmarking.
102
+
Use `vm_stat` (`Pages free`) or `top` (`PhysMem` -> `unused`) to check the amount of free memory.
103
+
Unfortunately neither of these tools show the swap usage, illustrating how difficult it is to ensure there is no swapping going on when benchmarking on macOS.
104
+
Use `Activity Monitor` before and after benchmarking to ensure there is no swap (`Swap Used`) as a workaround.
105
+
`Activity Monitor` is too heavy to be used during benchmarking.
0 commit comments