Skip to content

Commit b1f6d21

Browse files
committed
Provided instruction for demo
1 parent 78f6cec commit b1f6d21

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,18 @@ python3 -m bulk_chain.infer \
9090
--api_token "<REPLICATE-API-TOKEN>"
9191
```
9292

93+
Or, you can launch **demo mode** to interact with LLM via command line:
94+
> **NOTE:** Demo supports streaming!
95+
```bash
96+
python3 -m bulk_chain.demo \
97+
--schema "test/schema/thor_cot_schema.json" \
98+
--adapter "dynamic:replicate_104.py:Replicate" \
99+
%%m \
100+
--model_name "meta/meta-llama-3-70b-instruct" \
101+
--api_token "<REPLICATE-API-TOKEN>" \
102+
--stream
103+
```
104+
93105
# Embed your LLM
94106

95107
All you have to do is to implement `BaseLM` class, that includes:

0 commit comments

Comments
 (0)