Results from R50 GH action on ubuntu-latest

Author: mlcommons-bot

open/MLCommons/measurements/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/README.md

@@ -1,3 +1,3 @@
 | Model    | Scenario   |   Accuracy |   Throughput | Latency (in ms)   |
 |----------|------------|------------|--------------|-------------------|
-| resnet50 | offline    |         76 |       23.163 | -                 |
+| resnet50 | offline    |         76 |       23.056 | -                 |

open/MLCommons/measurements/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline/README.md

@@ -16,7 +16,7 @@ pip install -U mlcflow
 
 mlc rm cache -f
 
-mlc pull repo GATEOverflow@mlperf-automations --checkout=1d1da6cf718c171bb8c76f4fa96edf2ed534659a
+mlc pull repo GATEOverflow@mlperf-automations --checkout=85ae32f3d6aec5e2b1a03cf032d05ccf77a8133c
 
 
 ```
@@ -40,4 +40,4 @@ Model Precision: fp32
 `acc`: `76.0`, Required accuracy for closed division `>= 75.6954`
 
 ### Performance Results 
-`Samples per second`: `23.1628`
+`Samples per second`: `23.0561`

open/MLCommons/measurements/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline/accuracy_console.out

@@ -1,6 +1,6 @@
-python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_ml-model-resnet_009ac81a/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_167230ea --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_f494b3fd/test_results/gh_ubuntu-latest x86-reference-cpu-onnxruntime-v1.22.0-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ed317ee80d47480d9f9714c971097080.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_167230ea --dataset-list /home/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_5cebd4d7/val.txt
-INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_167230ea', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_5cebd4d7/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_ml-model-resnet_009ac81a/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_f494b3fd/test_results/gh_ubuntu-latest x86-reference-cpu-onnxruntime-v1.22.0-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_167230ea', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ed317ee80d47480d9f9714c971097080.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
+python3 python/main.py --profile resnet50-onnxruntime --model "/home/runner/MLC/repos/local/cache/download-file_ml-model-resnet_bc15cffa/resnet50_v1.onnx" --dataset-path /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_d8efe128 --output "/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_16fe31d9/test_results/gh_ubuntu-latest x86-reference-cpu-onnxruntime-v1.22.0-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 4 --user_conf /home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/cbbf670eed9e4d8eba5a2620b6d1c9f4.conf --accuracy --use_preprocessed_dataset --cache_dir /home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_d8efe128 --dataset-list /home/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_78b26921/val.txt
+INFO:main:Namespace(dataset='imagenet', dataset_path='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_d8efe128', dataset_list='/home/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_78b26921/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/home/runner/MLC/repos/local/cache/download-file_ml-model-resnet_bc15cffa/resnet50_v1.onnx', output='/home/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_16fe31d9/test_results/gh_ubuntu-latest x86-reference-cpu-onnxruntime-v1.22.0-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=4, qps=None, cache=0, cache_dir='/home/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_d8efe128', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/home/runner/MLC/repos/GATEOverflow@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/cbbf670eed9e4d8eba5a2620b6d1c9f4.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
 INFO:imagenet:Loading 500 preprocessed images using 4 threads
 INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=0.0sec
 INFO:main:starting TestScenario.Offline
-TestScenario.Offline qps=0.74, mean=13.5946, time=21.674, acc=76.000%, queries=16, tiles=50.0:13.7374,80.0:21.0466,90.0:21.2050,95.0:21.2831,99.0:21.4402,99.9:21.4756
+TestScenario.Offline qps=0.74, mean=13.6547, time=21.702, acc=76.000%, queries=16, tiles=50.0:13.8638,80.0:21.4606,90.0:21.4984,95.0:21.5136,99.0:21.5364,99.9:21.5415