Skip to content

Commit 785486a

Browse files
committed
Merge bitcoin/bitcoin#31489: fuzz: Fix test_runner error reporting
fa0e30b fuzz: Fix test_runner error reporting (MarcoFalke) Pull request description: The error reporting is confusing, because right now it prints: https://cirrus-ci.com/task/4846031060336640?logs=ci#L4931 ``` ... Traceback (most recent call last): File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 411, in <module> main() File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 199, in main run_once( File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 376, in run_once assert len(done_stat) == 1 ^^^^^^^^^^^^^^^^^^^ AssertionError ``` This is harmless, but confusing. Fix it by collecting statistics only when the program has not aborted. (Can be reviewed with `--color-moved=dimmed-zebra`) Also, reword the error message to align it with error messages in other test_runners in this repo. ACKs for top commit: dergoegge: utACK fa0e30b brunoerg: code review ACK fa0e30b marcofleon: Tested ACK fa0e30b. Prints out the error for the target that crashed. Much clearer than the current error message. Tree-SHA512: 5e8d3fc0e4837b3264ff0c3cb322fe7fe2ec7af48d35e2a14f82080d03ace793963c3314611b0a170a38e200497d7ba703d9c35c9a7ed3272d93e43f0f0e4c2b
2 parents 1251a23 + fa0e30b commit 785486a

File tree

1 file changed

+5
-5
lines changed

1 file changed

+5
-5
lines changed

test/fuzz/test_runner.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -371,19 +371,19 @@ def job(t, args):
371371
for future in as_completed(jobs):
372372
output, result, target = future.result()
373373
logging.debug(output)
374-
if using_libfuzzer:
375-
done_stat = [l for l in output.splitlines() if "DONE" in l]
376-
assert len(done_stat) == 1
377-
stats.append((target, done_stat[0]))
378374
try:
379375
result.check_returncode()
380376
except subprocess.CalledProcessError as e:
381377
if e.stdout:
382378
logging.info(e.stdout)
383379
if e.stderr:
384380
logging.info(e.stderr)
385-
logging.info(f"Target {result.args} failed with exit code {e.returncode}")
381+
logging.info(f"⚠️ Failure generated from target with exit code {e.returncode}: {result.args}")
386382
sys.exit(1)
383+
if using_libfuzzer:
384+
done_stat = [l for l in output.splitlines() if "DONE" in l]
385+
assert len(done_stat) == 1
386+
stats.append((target, done_stat[0]))
387387

388388
if using_libfuzzer:
389389
print("Summary:")

0 commit comments

Comments
 (0)