File tree Expand file tree Collapse file tree 1 file changed +4
-4
lines changed
vllm/model_executor/layers Expand file tree Collapse file tree 1 file changed +4
-4
lines changed Original file line number Diff line number Diff line change @@ -176,17 +176,17 @@ class VocabParallelEmbedding(torch.nn.Module):
176
176
Therefore, the tensor format looks like the following:
177
177
TP1, rank 0 (no sharding):
178
178
|< --------BASE-------- >|< -BASE PADDING-- >|< -----LORA------ >|< -LORA PADDING-- >|
179
- corresponding token_id: | 0 | 1 | ... | 1009 | -1 | ... | -1 | 1010 | ... | 1015 | -1 | ... | -1 |
179
+ corresponding token_id: | 0 | 1 | ... | 1009 | -1 | ... | -1 | 1010 | ... | 1025 | -1 | ... | -1 |
180
180
index: | 0 | 1 | ... | 1009 | 1010 | ... | 1023 | 1024 | ... | 1039 | 1040 | ... | 1087 |
181
181
182
182
TP2, rank 0:
183
183
|< --------------------BASE--------------------- >|< -----LORA------ >|< -LORA PADDING- >|
184
- corresponding token_id: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 1000 | ... | 1015 | -1 | ... | -1 |
185
- index: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 512 | ... | 527 | 520 | ... | 543 |
184
+ corresponding token_id: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 1010 | ... | 1025 | -1 | ... | -1 |
185
+ index: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 512 | ... | 527 | 528 | ... | 543 |
186
186
TP2, rank 1:
187
187
|< -----------BASE----------- >|< -BASE PADDING- >|< -----------LORA PADDING----------- >|
188
188
corresponding token_id: | 512 | 513 | 514 | ... | 1009 | -1 | ... | -1 | -1 | ... | -1 | -1 | ... | -1 |
189
- index: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 512 | ... | 519 | 520 | ... | 543 |
189
+ index: | 0 | 1 | 2 | ... | 497 | 498 | ... | 511 | 512 | ... | 527 | 528 | ... | 543 |
190
190
191
191
Args:
192
192
num_embeddings: vocabulary size.
You can’t perform that action at this time.
0 commit comments