Skip to content

Commit c22ecc9

Browse files
mlcommons-botmlcommons-bot
committed
Results from R50 GH action on
1 parent bffcea1 commit c22ecc9

File tree

11 files changed

+208
-207
lines changed

11 files changed

+208
-207
lines changed

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
22
|-----------|------------|------------|--------------|-------------------|
3-
| retinanet | offline | 49.593 | 0.421 | - |
3+
| retinanet | offline | 49.593 | 0.422 | - |

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,10 @@
22

33
## Host platform
44

5-
* OS version: Linux-6.11.0-1015-azure-x86_64-with-glibc2.39
5+
* OS version: Linux-6.11.0-1015-azure-x86_64-with-glibc2.34
66
* CPU version: x86_64
7-
* Python version: 3.12.11 (main, Jun 4 2025, 04:14:08) [GCC 13.3.0]
7+
* Python version: 3.8.18 (default, Dec 12 2024, 19:15:30)
8+
[GCC 13.2.0]
89
* MLC version: unknown
910

1011
## CM Run Command
@@ -16,7 +17,7 @@ pip install -U mlcflow
1617

1718
mlc rm cache -f
1819

19-
mlc pull repo GATEOverflow@mlperf-automations --checkout=1ae202ac0f0fca0cb1a976c850cb69aeba0733f2
20+
mlc pull repo GATEOverflow@mlperf-automations --checkout=81b61cfd6fe1d811fad454a890813b4540c09bea
2021

2122

2223
```
@@ -40,4 +41,4 @@ Model Precision: fp32
4041
`mAP`: `49.593`, Required accuracy for closed division `>= 37.1745`
4142

4243
### Performance Results
43-
`Samples per second`: `0.420907`
44+
`Samples per second`: `0.422334`

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/accuracy_console.out

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
User Conf path: /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/c6da6c9e2cb646839fd0b77e559b80d1.conf
2-
Dataset Preprocessed path: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_dea2dee8
3-
Dataset List filepath: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_dea2dee8/annotations/openimages-mlperf.json
1+
User Conf path: /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/9689bd97555041108a1f44c52734fde2.conf
2+
Dataset Preprocessed path: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_4f1a333c
3+
Dataset List filepath: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_4f1a333c/annotations/openimages-mlperf.json
44
Scenario: Offline
55
Mode: AccuracyOnly
66
Batch size: 1

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/mlc-deps.mmd

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@ graph TD
88
app-mlperf-inference,d775cac873ee4231_(_cpp,_retinanet,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline_) --> get,mlcommons,inference,src,_branch.master
99
get-mlperf-inference-utils,e341e5f86d8342e5 --> get,mlperf,inference,src,_branch.master
1010
app-mlperf-inference,d775cac873ee4231_(_cpp,_retinanet,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline_) --> get,mlperf,inference,utils
11-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> detect,os
11+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> detect,os
1212
detect-cpu,586c8a43320142f7 --> detect,os
13-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> detect,cpu
14-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,sys-utils-cm
13+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> detect,cpu
14+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,sys-utils-cm
1515
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> detect,os
1616
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> get,python3
1717
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> get,mlcommons,inference,src
@@ -43,27 +43,27 @@ graph TD
4343
get-generic-python-lib,94b62a682bc44791_(_pip_) --> get,python3
4444
get-generic-python-lib,94b62a682bc44791_(_package.setuptools_) --> get,generic-python-lib,_pip
4545
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> get,generic-python-lib,_package.setuptools
46-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,loadgen,_wg-inference
47-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,mlcommons,inference,src,_branch.master
48-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,lib,onnxruntime,lang-cpp,_cpu
49-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,dataset,preprocessed,openimages,_validation,_NCHW,_50
50-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> get,ml-model,retinanet,_onnx,_fp32
46+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,loadgen,_wg-inference
47+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,mlcommons,inference,src,_branch.master
48+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,lib,onnxruntime,lang-cpp,_cpu
49+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,dataset,preprocessed,openimages,_validation,_NCHW,_50
50+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> get,ml-model,retinanet,_onnx,_fp32
5151
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> detect,os
5252
detect-cpu,586c8a43320142f7 --> detect,os
5353
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> detect,cpu
5454
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,python
5555
get-mlperf-inference-sut-configs,c2fbf72009e2445b --> get,cache,dir,_name.mlperf-inference-sut-configs
5656
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,sut,configs
5757
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,mlcommons,inference,src,_branch.master
58-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> generate,user-conf,mlperf,inference,_wg-inference
58+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> generate,user-conf,mlperf,inference,_wg-inference
5959
detect-cpu,586c8a43320142f7 --> detect,os
6060
compile-program,c05042ba005a4bfa --> detect,cpu
6161
compile-program,c05042ba005a4bfa --> get,compiler,gcc
6262
detect-cpu,586c8a43320142f7 --> detect,os
6363
get-compiler-flags,31be8b74a69742f8 --> detect,cpu
6464
compile-program,c05042ba005a4bfa --> get,compiler-flags
65-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> compile,cpp-program
65+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> compile,cpp-program
6666
detect-cpu,586c8a43320142f7 --> detect,os
6767
benchmark-program,19f369ef47084895 --> detect,cpu
6868
benchmark-program-mlperf,cfff0132a8aa4018 --> benchmark-program,program
69-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_retinanet,_onnxruntime,_cpu_) --> benchmark-mlperf
69+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_offline,_onnxruntime,_cpu,_retinanet_) --> benchmark-mlperf

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/mlc-deps.png

-120 Bytes
Loading

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/mlc-version-info.json

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@
6666
"script_alias": "get-mlperf-inference-src",
6767
"script_tags": "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons",
6868
"script_variations": "branch.master",
69-
"version": "custom-git-0a3570efb0309b5581f2831d84c05fe5483b5ef7",
69+
"version": "custom-git-fdac4dcc0bcae68ec29462bb80a68e54067ab7a9",
7070
"parent": "app-mlperf-inference,d775cac873ee4231 ( cpp,_retinanet,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline )"
7171
}
7272
},
@@ -76,7 +76,7 @@
7676
"script_alias": "get-mlperf-inference-src",
7777
"script_tags": "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons",
7878
"script_variations": "branch.master",
79-
"version": "custom-git-0a3570efb0309b5581f2831d84c05fe5483b5ef7",
79+
"version": "custom-git-fdac4dcc0bcae68ec29462bb80a68e54067ab7a9",
8080
"parent": "get-mlperf-inference-utils,e341e5f86d8342e5"
8181
}
8282
},
@@ -97,7 +97,7 @@
9797
"script_tags": "detect-os,detect,os,info",
9898
"script_variations": "",
9999
"version": "",
100-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
100+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
101101
}
102102
},
103103
{
@@ -117,7 +117,7 @@
117117
"script_tags": "detect,cpu,detect-cpu,info",
118118
"script_variations": "",
119119
"version": "",
120-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
120+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
121121
}
122122
},
123123
{
@@ -127,7 +127,7 @@
127127
"script_tags": "get,sys-utils-cm,sys-utils-mlc",
128128
"script_variations": "",
129129
"version": "",
130-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
130+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
131131
}
132132
},
133133
{
@@ -156,7 +156,7 @@
156156
"script_alias": "get-mlperf-inference-src",
157157
"script_tags": "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons",
158158
"script_variations": "",
159-
"version": "master-git-0a3570efb0309b5581f2831d84c05fe5483b5ef7",
159+
"version": "master-git-fdac4dcc0bcae68ec29462bb80a68e54067ab7a9",
160160
"parent": "get-mlperf-inference-loadgen,64c3d98d0ba04950 ( wg-inference )"
161161
}
162162
},
@@ -447,7 +447,7 @@
447447
"script_tags": "get,loadgen,inference,inference-loadgen,mlperf,mlcommons",
448448
"script_variations": "wg-inference",
449449
"version": "master",
450-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
450+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
451451
}
452452
},
453453
{
@@ -456,8 +456,8 @@
456456
"script_alias": "get-mlperf-inference-src",
457457
"script_tags": "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons",
458458
"script_variations": "branch.master",
459-
"version": "custom-git-0a3570efb0309b5581f2831d84c05fe5483b5ef7",
460-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
459+
"version": "custom-git-fdac4dcc0bcae68ec29462bb80a68e54067ab7a9",
460+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
461461
}
462462
},
463463
{
@@ -467,7 +467,7 @@
467467
"script_tags": "install,onnxruntime,get,prebuilt,lib,lang-c,lang-cpp",
468468
"script_variations": "cpu",
469469
"version": "",
470-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
470+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
471471
}
472472
},
473473
{
@@ -477,7 +477,7 @@
477477
"script_tags": "get,dataset,openimages,open-images,object-detection,preprocessed",
478478
"script_variations": "validation,NCHW,50",
479479
"version": "",
480-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
480+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
481481
}
482482
},
483483
{
@@ -487,7 +487,7 @@
487487
"script_tags": "get,ml-model,raw,resnext50,retinanet,object-detection",
488488
"script_variations": "onnx,fp32",
489489
"version": "",
490-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
490+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
491491
}
492492
},
493493
{
@@ -556,7 +556,7 @@
556556
"script_alias": "get-mlperf-inference-src",
557557
"script_tags": "get,src,source,inference,inference-src,inference-source,mlperf,mlcommons",
558558
"script_variations": "branch.master",
559-
"version": "custom-git-0a3570efb0309b5581f2831d84c05fe5483b5ef7",
559+
"version": "custom-git-fdac4dcc0bcae68ec29462bb80a68e54067ab7a9",
560560
"parent": "generate-mlperf-inference-user-conf,3af4475745964b93 ( wg-inference )"
561561
}
562562
},
@@ -567,7 +567,7 @@
567567
"script_tags": "generate,mlperf,inference,user-conf,inference-user-conf",
568568
"script_variations": "wg-inference",
569569
"version": "",
570-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
570+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
571571
}
572572
},
573573
{
@@ -637,7 +637,7 @@
637637
"script_tags": "compile,program,c-program,cpp-program,compile-program,compile-c-program,compile-cpp-program",
638638
"script_variations": "",
639639
"version": "",
640-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
640+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
641641
}
642642
},
643643
{
@@ -677,7 +677,7 @@
677677
"script_tags": "mlperf,benchmark-mlperf",
678678
"script_variations": "",
679679
"version": "",
680-
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_retinanet,_onnxruntime,_cpu )"
680+
"parent": "app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf ( offline,_onnxruntime,_cpu,_retinanet )"
681681
}
682682
}
683683
]

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/os_info.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,6 @@
2828
"MLC_HOST_PYTHON_BITS": "64",
2929
"MLC_HOST_SYSTEM_NAME": "pkrvmbietmlfzoi",
3030
"+PATH": [
31-
"/home/runner/MLC/repos/local/cache/install-python-src_9818ad19/install/bin"
31+
"/home/runner/MLC/repos/local/cache/install-python-src_26358a05/install/bin"
3232
]
3333
}

‎open/MLCommons/measurements/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline/performance_console.out

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
User Conf path: /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/e6a5edc91faf44df999a0dede8c69bfb.conf
2-
Dataset Preprocessed path: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_dea2dee8
3-
Dataset List filepath: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_dea2dee8/annotations/openimages-mlperf.json
1+
User Conf path: /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/2b2ac05c1ee843d6b37ef472964dae15.conf
2+
Dataset Preprocessed path: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_4f1a333c
3+
Dataset List filepath: /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-openimages_4f1a333c/annotations/openimages-mlperf.json
44
Scenario: Offline
55
Mode: PerformanceOnly
66
Batch size: 1

0 commit comments

Comments
 (0)