test model inference benchmark（ChatGLM2-6B，LLaMA2-7b-chat，Baichuan2-7B-chat）
| input token | throughput* | first token耗时（ms） | one token耗时（ms）｜ |
|---|---|---|---|
| 32 | |||
| 64 | |||
| 128 | |||
| 256 | |||
| 512 | |||
| 1024 | |||
| 2048 |
forked from daixd5520/LLM-benchmark
-
Notifications
You must be signed in to change notification settings - Fork 0
ispyu85/LLM-benchmark
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
test model inference benchmark（ChatGLM2-6B，LLaMA2-7b-chat，Baichuan2-7B-chat）
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 99.7%
- Shell 0.3%