Skip to content

Commit 9288741

Browse files
mlcommons-botmlcommons-bot
committed
Auto-merge updates from auto-update branch
2 parents f060589 + db0b7fc commit 9288741

File tree

48 files changed

+1216
-1130
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+1216
-1130
lines changed

‎open/MLCommons/measurements/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
22
|----------|------------|------------|--------------|-------------------|
3-
| resnet50 | offline | 76 | 9.393 | - |
3+
| resnet50 | offline | 76 | 8.084 | - |

‎open/MLCommons/measurements/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline/README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,16 +16,16 @@ pip install -U mlcflow
1616

1717
mlc rm cache -f
1818

19-
mlc pull repo mlcommons@mlperf-automations --checkout=a94e9eb5369c706613bb99bf0b1b4a443ffbcd24
19+
mlc pull repo GATEOverflow@mlperf-automations --checkout=ed88c46883545f2db0fe0b604d162882d8b18f37
2020

2121

2222
```
2323
*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf,
24-
you should simply reload mlcommons@mlperf-automations without checkout and clean MLC cache as follows:*
24+
you should simply reload GATEOverflow@mlperf-automations without checkout and clean MLC cache as follows:*
2525

2626
```bash
27-
mlc rm repo mlcommons@mlperf-automations
28-
mlc pull repo mlcommons@mlperf-automations
27+
mlc rm repo GATEOverflow@mlperf-automations
28+
mlc pull repo GATEOverflow@mlperf-automations
2929
mlc rm cache -f
3030

3131
```
@@ -40,4 +40,4 @@ Model Precision: fp32
4040
`acc`: `76.0`, Required accuracy for closed division `>= 75.6954`
4141

4242
### Performance Results
43-
`Samples per second`: `9.39264`
43+
`Samples per second`: `8.08387`

‎open/MLCommons/measurements/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline/accuracy_console.out

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
User Conf path: /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/e9c90cc1afa943fdb7eb525a1f31d975.conf
2-
Dataset Preprocessed path: /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_473de91c
1+
User Conf path: /Users/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/0d6d5b968bce4c128cb72727d8c1077d.conf
2+
Dataset Preprocessed path: /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_4c00390f
33
Dataset List filepath:
44
Scenario: Offline
55
Mode: AccuracyOnly

‎open/MLCommons/measurements/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline/mlc-deps.mmd

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -9,18 +9,20 @@ graph TD
99
app-mlperf-inference,d775cac873ee4231_(_cpp,_resnet50,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline_) --> get,mlperf,inference,utils
1010
download-file,9cdc8dc41aae437e_(_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> detect,os
1111
get-generic-sys-util,bb0393afa8404a11_(_md5sha1sum_) --> detect,os
12+
get-generic-sys-util,bb0393afa8404a11_(_md5sha1sum_) --> detect,sudo
1213
get-generic-sys-util,bb0393afa8404a11_(_install,_md5sha1sum_) --> detect,os
14+
get-generic-sys-util,bb0393afa8404a11_(_install,_md5sha1sum_) --> detect,sudo
1315
get-generic-sys-util,bb0393afa8404a11_(_md5sha1sum_) --> get,generic-sys-util,_install,_md5sha1sum
1416
download-file,9cdc8dc41aae437e_(_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
1517
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> download,file,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz
16-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_83e826c6/caffe_ilsvrc12.tar.gz_) --> detect,os
17-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_83e826c6/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
18-
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_83e826c6/caffe_ilsvrc12.tar.gz
18+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_7ea9ec6c/caffe_ilsvrc12.tar.gz_) --> detect,os
19+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_7ea9ec6c/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
20+
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_7ea9ec6c/caffe_ilsvrc12.tar.gz
1921
get-dataset-imagenet-aux,bb2c6dd8c8c64217 --> download-and-extract,_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz
2022
app-mlperf-inference,d775cac873ee4231_(_cpp,_resnet50,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline_) --> get,dataset-aux,imagenet-aux
21-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> detect,os
23+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> detect,os
2224
detect-cpu,586c8a43320142f7 --> detect,os
23-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> detect,cpu
25+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> detect,cpu
2426
get-sys-utils-cm,bc90993277e84b8e --> detect,os
2527
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> detect,os
2628
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> get,python3
@@ -55,18 +57,18 @@ graph TD
5557
get-generic-python-lib,94b62a682bc44791_(_pip_) --> get,python3
5658
get-generic-python-lib,94b62a682bc44791_(_package.setuptools_) --> get,generic-python-lib,_pip
5759
get-mlperf-inference-loadgen,64c3d98d0ba04950_(_wg-inference_) --> get,generic-python-lib,_package.setuptools
58-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> get,loadgen,_wg-inference
59-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> get,mlcommons,inference,src
60+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> get,loadgen,_wg-inference
61+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> get,mlcommons,inference,src
6062
get-onnxruntime-prebuilt,be02c84ff57c4244_(_cpu_) --> detect,os
61-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> get,lib,onnxruntime,lang-cpp,_cpu
63+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> get,lib,onnxruntime,lang-cpp,_cpu
6264
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> get,python3
6365
get-dataset-imagenet-val,7afd58d287fe4f11_(_size.500,_2012_) --> detect,os
6466
download-file,9cdc8dc41aae437e_(_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> detect,os
6567
download-file,9cdc8dc41aae437e_(_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
6668
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> download,file,_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar
67-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_e04041d8/ILSVRC2012_img_val_500.tar_) --> detect,os
68-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_e04041d8/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
69-
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_e04041d8/ILSVRC2012_img_val_500.tar
69+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_6372acea/ILSVRC2012_img_val_500.tar_) --> detect,os
70+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_6372acea/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
71+
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_6372acea/ILSVRC2012_img_val_500.tar
7072
get-dataset-imagenet-val,7afd58d287fe4f11_(_size.500,_2012_) --> download-and-extract,file,_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar
7173
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> get,dataset,image-classification,original,_size.500,_2012
7274
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> get,dataset-aux,image-classification,imagenet-aux
@@ -86,31 +88,33 @@ graph TD
8688
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> get,generic-python-lib,_pillow
8789
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> mlperf,mlcommons,inference,source,src
8890
get-generic-sys-util,bb0393afa8404a11_(_libgl_) --> detect,os
91+
get-generic-sys-util,bb0393afa8404a11_(_libgl_) --> detect,sudo
8992
get-generic-sys-util,bb0393afa8404a11_(_install,_libgl_) --> detect,os
93+
get-generic-sys-util,bb0393afa8404a11_(_install,_libgl_) --> detect,sudo
9094
get-generic-sys-util,bb0393afa8404a11_(_libgl_) --> get,generic-sys-util,_install,_libgl
9195
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_NCHW_) --> get,generic-sys-util,_libgl
92-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> get,dataset,preprocessed,imagenet,_NCHW
96+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> get,dataset,preprocessed,imagenet,_NCHW
9397
download-file,9cdc8dc41aae437e_(_cmutil,_url.https://armi.in/files/resnet50_v1_op11/resnet50_v1.onnx_) --> detect,os
9498
download-file,9cdc8dc41aae437e_(_cmutil,_url.https://armi.in/files/resnet50_v1_op11/resnet50_v1.onnx_) --> get,generic-sys-util,_md5sha1sum
9599
download-and-extract,c67e81a4ce2649f5_(_url.https://armi.in/files/resnet50_v1_op11/resnet50_v1.onnx_) --> download,file,_cmutil,_url.https://armi.in/files/resnet50_v1_op11/resnet50_v1.onnx
96100
get-ml-model-resnet50,56203e4e998b4bc0_(_onnx_) --> download-and-extract,_url.https://armi.in/files/resnet50_v1_op11/resnet50_v1.onnx
97-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> get,ml-model,raw,resnet50,_onnx
101+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> get,ml-model,raw,resnet50,_onnx
98102
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> detect,os
99103
detect-cpu,586c8a43320142f7 --> detect,os
100104
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> detect,cpu
101105
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,python
102106
get-mlperf-inference-sut-configs,c2fbf72009e2445b --> get,cache,dir,_name.mlperf-inference-sut-configs
103107
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,sut,configs
104108
generate-mlperf-inference-user-conf,3af4475745964b93_(_wg-inference_) --> get,mlcommons,inference,src
105-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> generate,user-conf,mlperf,inference,_wg-inference
109+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> generate,user-conf,mlperf,inference,_wg-inference
106110
detect-cpu,586c8a43320142f7 --> detect,os
107111
compile-program,c05042ba005a4bfa --> detect,cpu
108112
compile-program,c05042ba005a4bfa --> get,compiler
109113
detect-cpu,586c8a43320142f7 --> detect,os
110114
get-compiler-flags,31be8b74a69742f8 --> detect,cpu
111115
compile-program,c05042ba005a4bfa --> get,compiler-flags
112-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> compile,cpp-program
116+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> compile,cpp-program
113117
detect-cpu,586c8a43320142f7 --> detect,os
114118
benchmark-program,19f369ef47084895 --> detect,cpu
115119
benchmark-program-mlperf,cfff0132a8aa4018 --> benchmark-program,program
116-
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_cpu,_resnet50,_offline,_onnxruntime_) --> benchmark-mlperf
120+
app-mlperf-inference-mlcommons-cpp,bf62405e6c7a44bf_(_onnxruntime,_cpu,_offline,_resnet50_) --> benchmark-mlperf

‎open/MLCommons/measurements/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline/mlc-deps.png

336 KB
Loading

0 commit comments

Comments
 (0)