Skip to content

Commit 8b2865d

Browse files
committed
Update documentation of runtime benchmarks directory
1 parent 1205e06 commit 8b2865d

File tree

1 file changed

+19
-4
lines changed

1 file changed

+19
-4
lines changed

collector/runtime-benchmarks/README.md

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,23 @@
22
This directory contains various pieces of code for which we measure how fast do they execute
33
when they are compiled with a specific version of `rustc`.
44

5-
The benchmarks are located in crates that are part of the `runtime-benchmarks` workspace. Each crate
6-
contains a set of benchmarks defined using named closures.
5+
The benchmarks are located in several crates (`benchmark groups`) located in this directory. Each
6+
group defines a set of benchmarks using named closures. Each group should have a single,
7+
default, binary target, which will be executed by `collector`, and it should use the
8+
`run_benchmark_group` function from [`benchlib`](../benchlib) to define the benchmarks.
79

8-
Benchmarks are divided into sub-crates so that some benchmarks can use various versions of dependency
9-
crates and also so that they are grouped together by a relevant area (e.g. hashmap benchmarks).
10+
Runtime benchmarks are divided into groups so that some benchmarks can use different versions of
11+
dependency crates and also so that they are grouped together by a relevant area
12+
(e.g. hashmap benchmarks).
13+
14+
## How are benchmarks executed
15+
The `collector` compiles each benchmark group and then invokes it with the `list` argument to list
16+
all benchmarks contained in the group.
17+
18+
Then it executes each group with the `run` argument, which will cause `benchlib` to actually perform
19+
the benchmarks and output the results to `stdout` in JSON. That means that the benchmarked code should
20+
not print anything to `stdout`!
21+
22+
Note that each benchmark group binary will be thus invoked twice per benchmarking collection. Keep this
23+
in mind so that the `main` function of the group doesn't perform too much work, which will be thrown
24+
away when it is invoked with the `list` argument.

0 commit comments

Comments
 (0)