Skip to content

Commit 48d7161

Browse files
committed
chore: ignore boot test resume_time metric in A/B
This metrics has very small numbers which makes it too volatile for A/B tests. Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
1 parent 119f129 commit 48d7161

File tree

2 files changed

+3
-1
lines changed

2 files changed

+3
-1
lines changed

tests/framework/ab_test.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ def git_ab_test_host_command(
160160

161161

162162
def set_did_not_grow_comparator(
163-
set_generator: Callable[[CommandReturn], set]
163+
set_generator: Callable[[CommandReturn], set],
164164
) -> Callable[[CommandReturn, CommandReturn], bool]:
165165
"""Factory function for comparators to use with git_ab_test_command that converts the command output to sets
166166
(using the given callable) and then checks that the "B" set is a subset of the "A" set

tools/ab_test.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@
4949
# block latencies if guest uses async request submission
5050
{"fio_engine": "libaio", "metric": "clat_read"},
5151
{"fio_engine": "libaio", "metric": "clat_write"},
52+
# boot time metrics
53+
{"performance_test": "test_boottime", "metric": "resume_time"},
5254
]
5355

5456

0 commit comments

Comments
 (0)