Skip to content

Commit 77220d5

Browse files
authored
updated mixtral dataset download based on latest inference readme (#284)
1 parent 3cb1135 commit 77220d5

File tree

1 file changed

+35
-1
lines changed
  • script/get-dataset-mlperf-inference-mixtral

1 file changed

+35
-1
lines changed

script/get-dataset-mlperf-inference-mixtral/meta.yaml

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,48 @@ prehook_deps:
1616
update_tags_from_env_with_prefix:
1717
_url.:
1818
- MLC_PACKAGE_URL
19+
names:
20+
- download-file
1921
tags:
2022
- get
2123
- dataset-mixtral
2224
- openorca-mbxp-gsm8k-combined
2325
uid: 89e7c91444804775
2426
variations:
25-
mlcommons-storage:
27+
rclone:
28+
group: download-tool
29+
default: true
30+
adr:
31+
download-file:
32+
tags: _rclone
33+
env:
34+
MLC_DOWNLOAD_TOOL: rclone
35+
MLC_RCLONE_COPY_USING: copyurl
36+
wget:
37+
group: download-tool
38+
adr:
39+
download-file:
40+
tags: _wget
41+
env:
42+
MLC_DOWNLOAD_TOOL: wget
43+
validation:
2644
default: true
45+
group: dataset-type
46+
adr:
47+
download-file:
48+
extra_cache_tags: mixtral,get-mixtral-dataset,validation
49+
env:
50+
ML_MODEL_STARTING_WEIGHTS_FILENAME: https://github.com/mlcommons/inference/tree/master/language/mixtral-8x7b#preprocessed
51+
MLC_DOWNLOAD_URL: https://inference.mlcommons-storage.org/mixtral_8x7b/09292024_mixtral_15k_mintoken2_v1.pkl
52+
calibration:
53+
group: dataset-type
54+
adr:
55+
download-file:
56+
extra_cache_tags: mixtral,get-mixtral-dataset,calibration
57+
env:
58+
ML_MODEL_STARTING_WEIGHTS_FILENAME: https://github.com/mlcommons/inference/tree/master/language/mixtral-8x7b#calibration-dataset
59+
MLC_DOWNLOAD_URL: https://inference.mlcommons-storage.org/mixtral_8x7b%2F2024.06.06_mixtral_15k_calibration_v4.pkl
60+
mlcommons-storage:
2761
env:
2862
MLC_DOWNLOAD_CHECKSUM: 78823c13e0e73e518872105c4b09628b
2963
MLC_DOWNLOAD_FILENAME: 2024.06.06_mixtral_15k_v4.pkl

0 commit comments

Comments
 (0)