@@ -133,36 +133,20 @@ docker run \
133
133
The following configurations have been validated to function with
134
134
Gaudi2 devices. Configurations that are not listed may or may not work.
135
135
136
- - [ meta-llama/Llama-2-7b] ( https://huggingface.co/meta-llama/Llama-2-7b )
137
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
138
- datatype with random or greedy sampling
139
- - [ meta-llama/Llama-2-7b-chat-hf] ( https://huggingface.co/meta-llama/Llama-2-7b-chat-hf )
140
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
141
- datatype with random or greedy sampling
142
- - [ meta-llama/Meta-Llama-3-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B )
143
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
144
- datatype with random or greedy sampling
145
- - [ meta-llama/Meta-Llama-3-8B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct )
146
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
147
- datatype with random or greedy sampling
148
- - [ meta-llama/Meta-Llama-3.1-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-8B )
149
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
150
- datatype with random or greedy sampling
151
- - [ meta-llama/Meta-Llama-3.1-8B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct )
152
- on single HPU, or with tensor parallelism on 2x and 8x HPU, BF16
153
- datatype with random or greedy sampling
154
- - [ meta-llama/Llama-2-70b] ( https://huggingface.co/meta-llama/Llama-2-70b )
155
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
156
- - [ meta-llama/Llama-2-70b-chat-hf] ( https://huggingface.co/meta-llama/Llama-2-70b-chat-hf )
157
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
158
- - [ meta-llama/Meta-Llama-3-70B] ( https://huggingface.co/meta-llama/Meta-Llama-3-70B )
159
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
160
- - [ meta-llama/Meta-Llama-3-70B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct )
161
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
162
- - [ meta-llama/Meta-Llama-3.1-70B] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-70B )
163
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
164
- - [ meta-llama/Meta-Llama-3.1-70B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct )
165
- with tensor parallelism on 8x HPU, BF16 datatype with random or greedy sampling
136
+ | Model | TP Size| dtype | Sampling |
137
+ | -------| --------| --------| ----------|
138
+ | [ meta-llama/Llama-2-7b] ( https://huggingface.co/meta-llama/Llama-2-7b ) | 1, 2, 8 | BF16 | Random / Greedy |
139
+ | [ meta-llama/Llama-2-7b-chat-hf] ( https://huggingface.co/meta-llama/Llama-2-7b-chat-hf ) | 1, 2, 8 | BF16 | Random / Greedy |
140
+ | [ meta-llama/Meta-Llama-3-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B ) | 1, 2, 8 | BF16 | Random / Greedy |
141
+ | [ meta-llama/Meta-Llama-3-8B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct ) | 1, 2, 8 | BF16 | Random / Greedy |
142
+ | [ meta-llama/Meta-Llama-3.1-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-8B ) | 1, 2, 8 | BF16 | Random / Greedy |
143
+ | [ meta-llama/Meta-Llama-3.1-8B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct ) | 1, 2, 8 | BF16 | Random / Greedy |
144
+ | [ meta-llama/Llama-2-70b] ( https://huggingface.co/meta-llama/Llama-2-70b ) | 8 | BF16 | Random / Greedy |
145
+ | [ meta-llama/Llama-2-70b-chat-hf] ( https://huggingface.co/meta-llama/Llama-2-70b-chat-hf ) | 8 | BF16 | Random / Greedy |
146
+ | [ meta-llama/Meta-Llama-3-70B] ( https://huggingface.co/meta-llama/Meta-Llama-3-70B ) | 8 | BF16 | Random / Greedy |
147
+ | [ meta-llama/Meta-Llama-3-70B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct ) | 8 | BF16 | Random / Greedy |
148
+ | [ meta-llama/Meta-Llama-3.1-70B] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-70B ) | 8 | BF16 | Random / Greedy |
149
+ | [ meta-llama/Meta-Llama-3.1-70B-Instruct] ( https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct ) | 8 | BF16 | Random / Greedy |
166
150
167
151
## Performance tuning
168
152
0 commit comments