Skip to content

Commit 9c224ba

Browse files
fixed bench statements (#895)
Co-authored-by: Spencer Bryngelson <shb@gatech.edu>
1 parent 7bb8ed3 commit 9c224ba

File tree

2 files changed

+15
-6
lines changed

2 files changed

+15
-6
lines changed

.github/workflows/bench.yml

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
name: 'Benchmark'
22

3-
on: pull_request
3+
on:
4+
pull_request_review:
5+
types: [submitted]
6+
workflow_dispatch:
47

58
jobs:
69
file-changes:
@@ -20,11 +23,12 @@ jobs:
2023

2124
self:
2225
name: Georgia Tech | Phoenix (NVHPC)
23-
if: github.repository == 'MFlowCode/MFC' && needs.file-changes.outputs.checkall == 'true'
26+
if: github.repository == 'MFlowCode/MFC' && needs.file-changes.outputs.checkall == 'true' && ${{ github.event.review.state == 'approved' }}
2427
needs: file-changes
2528
strategy:
2629
matrix:
2730
device: ['cpu', 'gpu']
31+
fail-fast: false
2832
runs-on:
2933
group: phoenix
3034
labels: gt
@@ -56,6 +60,12 @@ jobs:
5660
(cd pr && . ./mfc.sh load -c p -m g)
5761
(cd pr && ./mfc.sh bench_diff ../master/bench-${{ matrix.device }}.yaml ../pr/bench-${{ matrix.device }}.yaml)
5862
63+
- name: Print Logs
64+
if: always()
65+
run: |
66+
cat pr/bench-${{ matrix.device }}.* 2>/dev/null || true
67+
cat master/bench-${{ matrix.device }}.* 2>/dev/null || true
68+
5969
- name: Archive Logs
6070
uses: actions/upload-artifact@v4
6171
if: always()

toolchain/mfc/bench.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,6 @@ def diff():
125125
if not math.isfinite(lhs_summary[target.name]["exec"]) or not math.isfinite(rhs_summary[target.name]["exec"]):
126126
err = 1
127127
cons.print(f"lhs_summary or rhs_summary reports non-real exec time for {target.name} - Case: {slug}")
128-
129128
try:
130129
exec_time_value = lhs_summary[target.name]["exec"] / rhs_summary[target.name]["exec"]
131130
if exec_time_value < 0.9:
@@ -139,10 +138,10 @@ def diff():
139138
grind_time_value = lhs_summary[target.name]["grind"] / rhs_summary[target.name]["grind"]
140139
speedups[i] += f" & Grind: {grind_time_value:.2f}"
141140
if grind_time_value <0.95:
142-
raise MFCException(f"Benchmarking failed since grind time speedup for {target.name} below acceptable threshold (<0.98) - Case: {slug}")
141+
cons.print(f"[bold red]Error[/bold red]: Benchmarking failed since grind time speedup for {target.name} below acceptable threshold (<0.95) - Case: {slug}")
142+
err = 1
143143
except Exception as _:
144-
err = 1
145-
cons.print(f"lhs_summary or rhs_summary reports non-real grind time for {target.name} - Case: {slug}")
144+
pass
146145

147146
table.add_row(f"[magenta]{slug}[/magenta]", *speedups)
148147

0 commit comments

Comments
 (0)