Skip to content

Commit f794711

Browse files
authored
Add missing vllm docs link (#50)
1 parent 328c0f5 commit f794711

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ GuideLLM requires an OpenAI-compatible server to run evaluations. [vLLM](https:/
5656
vllm serve "neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16"
5757
```
5858

59+
For more information on starting a vLLM server, see the [vLLM Documentation](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html).
60+
5961
#### 1b. Start an OpenAI Compatible Server (Hugging Face TGI)
6062

6163
GuideLLM requires an OpenAI-compatible server to run evaluations. [Text Generation Inference](https://github.com/huggingface/text-generation-inference) can be used here. To start a TGI server with a Llama 3.1 8B using docker, run the following command:

0 commit comments

Comments
 (0)