Skip to content

Commit 0df90a8

Browse files
committed
📘 doc(plugin/stream): add fetch proxy strea
1 parent 2b01cb6 commit 0df90a8

File tree

1 file changed

+30
-2
lines changed

1 file changed

+30
-2
lines changed

docs/plugins/stream.md

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,9 +72,9 @@ Below is the example to integrate ChatGPT to Elysia.
7272

7373
```ts
7474
new Elysia()
75-
.post(
75+
.get(
7676
'/ai',
77-
({ body, query: { prompt } }) =>
77+
({ query: { prompt } }) =>
7878
new Stream(
7979
openai.chat.completions.create({
8080
model: 'gpt-3.5-turbo',
@@ -90,6 +90,32 @@ new Elysia()
9090

9191
By default [openai](https://npmjs.com/package/openai) chatGPT completion return `AsyncIterable` so you should be able to wrap the OpenAI in `Stream`.
9292

93+
## Fetch Stream
94+
You can pass a fetch from an endpoint that returns the stream to proxy a stream.
95+
96+
This is useful for those endpoints that use AI text-generation since you can proxy it directly, eg. [Cloudflare AI](https://developers.cloudflare.com/workers-ai/models/llm/#examples---chat-style-with-system-prompt-preferred).
97+
```ts
98+
const model = '@cf/meta/llama-2-7b-chat-int8'
99+
const endpoint = `https://api.cloudflare.com/client/v4/accounts/${process.env.ACCOUNT_ID}/ai/run/${model}`
100+
101+
new Elysia()
102+
.get('/ai', ({ query: { prompt } }) =>
103+
fetch(endpoint, {
104+
method: 'POST',
105+
headers: {
106+
authorization: `Bearer ${API_TOKEN}`,
107+
'content-type': 'application/json'
108+
},
109+
body: JSON.stringify({
110+
messages: [
111+
{ role: 'system', content: 'You are a friendly assistant' },
112+
{ role: 'user', content: prompt }
113+
]
114+
})
115+
})
116+
)
117+
```
118+
93119
## Server Sent Event
94120
Manual mode is triggered when parameter is either `callback` or `undefined`, allowing you to control the stream.
95121

@@ -132,3 +158,5 @@ new Elysia()
132158
return stream
133159
})
134160
```
161+
162+
Both callback-based and value-based stream work in the same way but with just difference syntax for your preference.

0 commit comments

Comments
 (0)