Skip to content

Commit dafe4e1

Browse files
committed
doc: fix arguments to ab_test.py in README
It's --pytest-opts, not --test anymore. Fixes: 316d955 ("test(ab): generalize --test to --pytest-opts") Signed-off-by: Patrick Roy <roypat@amazon.co.uk>
1 parent 4a02df4 commit dafe4e1

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

tests/README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ function to [`.buildkite/pipeline_perf.py`](../.buildkite/pipeline_perf.py). To
184184
manually run an A/B-Test, use
185185

186186
```sh
187-
tools/devtool -y test --ab [optional arguments to ab_test.py] run <dir A> <dir B> --test <test specification>
187+
tools/devtool -y test --ab [optional arguments to ab_test.py] run <dir A> <dir B> --pytest-opts <test specification>
188188
```
189189

190190
Here, _dir A_ and _dir B_ are directories containing firecracker and jailer
@@ -198,7 +198,7 @@ branch and the `HEAD` of your current branch, run
198198
```sh
199199
tools/devtool -y build --rev main --release
200200
tools/devtool -y build --rev HEAD --release
201-
tools/devtool -y test --no-build --ab -- run build/main build/HEAD --test integration_tests/performance/test_boottime.py::test_boottime
201+
tools/devtool -y test --no-build --ab -- run build/main build/HEAD --pytest-opts integration_tests/performance/test_boottime.py::test_boottime
202202
```
203203

204204
#### How to Write an A/B-Compatible Test and Common Pitfalls
@@ -213,9 +213,9 @@ dimension to match up data series between two test runs. It only matches up two
213213
data series with the same name if their dimensions match.
214214

215215
Special care needs to be taken when pytest expands the argument passed to
216-
`tools/ab_test.py`'s `--test` option into multiple individual test cases. If two
217-
test cases use the same dimensions for different data series, the script will
218-
fail and print out the names of the violating data series. For this reason,
216+
`tools/ab_test.py`'s `--pytest-opts` option into multiple individual test cases.
217+
If two test cases use the same dimensions for different data series, the script
218+
will fail and print out the names of the violating data series. For this reason,
219219
**A/B-Compatible tests should include a `performance_test` key in their
220220
dimension set whose value is set to the name of the test**.
221221

0 commit comments

Comments
 (0)