Skip to content

Conversation

c-hagem
Copy link
Contributor

@c-hagem c-hagem commented Aug 28, 2025

Adds two features to the autogroup.py script, namely

  • the possibility to print either the numbers of all runs of a category (ordered descendingly by throughput), when specifying --runs=all, or just to print max, median and min run numbers --runs=rep (for representative)
  • makes default order of throughputs consistent with sorting order if runs are specified
  • adds possibility to use print the latest run, which is used automatically when no directory is specified

Only changes behaviour of the benchmarking autogroup script, so no Changelog entries / version bumps needed.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:26 — with GitHub Actions Failure
@c-hagem c-hagem force-pushed the autogroup-latest-option branch from 12e26af to b9172e4 Compare August 28, 2025 06:33
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 28, 2025 06:33 — with GitHub Actions Failure
@c-hagem c-hagem temporarily deployed to PR integration tests August 29, 2025 07:44 — with GitHub Actions Inactive
@c-hagem c-hagem requested review from muddyfish and sahityadg August 29, 2025 10:52
Adds two features to the autogroup.py script, namely
 - the possibility to print either the numbers of all runs of a category (ordered descendingly by throughput),
   when specifying `--runs=all`, or just to print max, median and min run numbers `--runs=tri`
-  makes default order of throughputs consistent with sorting order if runs are specified
-  adds possibility to use print the latest run (by default latest is inferred from the run number, but can also be switched to modification time). Additionally, with `--latest=K` the k-th latest run acording to the specified order is picked.

Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
@c-hagem c-hagem force-pushed the autogroup-latest-option branch from ac44bf2 to 137139b Compare August 29, 2025 12:13
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests August 29, 2025 12:13 — with GitHub Actions Failure

# Add run numbers column if requested
if args.runs:
sorted_by_throughput = sorted(zip(throughputs, run_numbers), reverse=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't zip(throughputs, run_numbers) equivalent to throughput_data?

selected_runs = [max_run, median_run, min_run]
# Remove duplicates while preserving order using dict.fromkeys()
# (works in python > 3.7)
unique_runs = list(dict.fromkeys(selected_runs))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think your previous approach was more readable 😅
Perhaps just move it to a function instead?


row.append(",".join(unique_runs))
else:
all_runs = [r for _, r in sorted_by_throughput]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this run_numbers?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants