File tree Expand file tree Collapse file tree 2 files changed +2
-2
lines changed Expand file tree Collapse file tree 2 files changed +2
-2
lines changed Original file line number Diff line number Diff line change @@ -14,11 +14,11 @@ export FD_MODEL_CACHE=/ssd1/download_models
14
14
15
15
| Model Name | Context Length | Quantization | Minimum Deployment Resources | Notes |
16
16
| :------------------------------------------ | :------------- | :----------- | :--------------------------- | :----------------------------------------------------------------------------------------- |
17
- | baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT2 | 1* 141G GPU VRAM/1T RAM | Chunked Prefill required for 128K |
18
17
| baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT4 | 4* 80G GPU VRAM/1T RAM | Chunked Prefill required for 128K |
19
18
| baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT8 | 8* 80G GPU VRAM/1T RAM | Chunked Prefill required for 128K |
20
19
| baidu/ERNIE-4.5-300B-A47B-Paddle | 32K/128K | WINT4 | 4* 64G GPU VRAM/600G RAM | Chunked Prefill required for 128K |
21
20
| baidu/ERNIE-4.5-300B-A47B-Paddle | 32K/128K | WINT8 | 8* 64G GPU VRAM/600G RAM | Chunked Prefill required for 128K |
21
+ | baidu/ERNIE-4.5-300B-A47B-2Bits-Paddle | 32K/128K | WINT2 | 1* 141G GPU VRAM/600G RAM | Chunked Prefill required for 128K |
22
22
| baidu/ERNIE-4.5-300B-A47B-W4A8C8-TP4-Paddle | 32K/128K | W4A8C8 | 4* 64G GPU VRAM/160G RAM | Fixed 4-GPU setup, Chunked Prefill recommended |
23
23
| baidu/ERNIE-4.5-300B-A47B-FP8-Paddle | 32K/128K | FP8 | 8* 64G GPU VRAM/600G RAM | Chunked Prefill recommended, only supports PD Disaggragated Deployment with EP parallelism |
24
24
| baidu/ERNIE-4.5-300B-A47B-Base-Paddle | 32K/128K | WINT4 | 4* 64G GPU VRAM/600G RAM | Chunked Prefill recommended |
Original file line number Diff line number Diff line change @@ -14,11 +14,11 @@ export FD_MODEL_CACHE=/ssd1/download_models
14
14
15
15
| 模型名 | 上下文长度 | 量化方式 | 最小部署资源 | 说明 |
16
16
| :------------------------------------------ | :--------- | :------- | :-------------------- | :---------------------------------------------- |
17
- | baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT2 | 1卡* 141G显存/1T内存 | 128K需要开启Chunked Prefill |
18
17
| baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT4 | 4卡* 80G显存/1T内存 | 128K需要开启Chunked Prefill |
19
18
| baidu/ERNIE-4.5-VL-424B-A47B-Paddle | 32K/128K | WINT8 | 8卡* 80G显存/1T内存 | 128K需要开启Chunked Prefill |
20
19
| baidu/ERNIE-4.5-300B-A47B-Paddle | 32K/128K | WINT4 | 4卡* 64G显存/600G内存 | 128K需要开启Chunked Prefill |
21
20
| baidu/ERNIE-4.5-300B-A47B-Paddle | 32K/128K | WINT8 | 8卡* 64G显存/600G内存 | 128K需要开启Chunked Prefill |
21
+ | baidu/ERNIE-4.5-300B-A47B-2Bits-Paddle | 32K/128K | WINT2 | 1卡* 141G显存/600G内存 | 128K需要开启Chunked Prefill |
22
22
| baidu/ERNIE-4.5-300B-A47B-W4A8C8-TP4-Paddle | 32K/128K | W4A8C8 | 4卡* 64G显存/160G内存 | 限定4卡,建议开启Chunked Prefill |
23
23
| baidu/ERNIE-4.5-300B-A47B-FP8-Paddle | 32K/128K | FP8 | 8卡* 64G显存/600G内存 | 建议开启Chunked Prefill,仅在PD分离EP并行下支持 |
24
24
| baidu/ERNIE-4.5-300B-A47B-Base-Paddle | 32K/128K | WINT4 | 4卡* 64G显存/600G内存 | 建议开启Chunked Prefill |
You can’t perform that action at this time.
0 commit comments