Skip to content

Commit 53e28e0

Browse files
Document how to send batched inputs (#222)
Co-authored-by: OlivierDehaene <Olivier.dehaene@gmail.com>
1 parent 3edace2 commit 53e28e0

File tree

1 file changed

+23
-3
lines changed

1 file changed

+23
-3
lines changed

docs/source/en/quick_tour.md

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,12 @@ docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingf
3939

4040
<Tip>
4141

42-
Here we pass a `revision=refs/pr/5`, because the `safetensors` variant of this model is currently in a pull request.
42+
Here we pass a `revision=refs/pr/5` because the `safetensors` variant of this model is currently in a pull request.
4343
We also recommend sharing a volume with the Docker container (`volume=$PWD/data`) to avoid downloading weights every run.
4444

4545
</Tip>
4646

47-
Once you have deployed a model you can use the `embed` endpoint by sending requests:
47+
Once you have deployed a model, you can use the `embed` endpoint by sending requests:
4848

4949
```bash
5050
curl 127.0.0.1:8080/embed \
@@ -72,7 +72,7 @@ volume=$PWD/data
7272
docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision
7373
```
7474

75-
Once you have deployed a model you can use the `rerank` endpoint to rank the similarity between a query and a list
75+
Once you have deployed a model, you can use the `rerank` endpoint to rank the similarity between a query and a list
7676
of texts:
7777

7878
```bash
@@ -101,3 +101,23 @@ curl 127.0.0.1:8080/predict \
101101
-d '{"inputs":"I like you."}' \
102102
-H 'Content-Type: application/json'
103103
```
104+
105+
## Batching
106+
107+
You can send multiple inputs in a batch. For example, for embeddings
108+
109+
```bash
110+
curl 127.0.0.1:8080/embed \
111+
-X POST \
112+
-d '{"inputs":["Today is a nice day", "I like you"]}' \
113+
-H 'Content-Type: application/json'
114+
```
115+
116+
And for Sequence Classification:
117+
118+
```bash
119+
curl 127.0.0.1:8080/predict \
120+
-X POST \
121+
-d '{"inputs":[["I like you."], ["I hate pineapples"]]}' \
122+
-H 'Content-Type: application/json'
123+
```

0 commit comments

Comments
 (0)