Skip to content

Commit 2ef82b6

Browse files
authored
add standard blueprint for azure embedding ada2 (#3725)
Signed-off-by: Shengbo Ma <shengboma@gmail.com>
1 parent b7e6c20 commit 2ef82b6

File tree

1 file changed

+172
-0
lines changed

1 file changed

+172
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# Azure OpenAI connector standard blueprint example for embedding model
2+
3+
This blueprint demonstrates how to deploy a `text-embedding-ada-002` using the Azure OpenAI connector without pre and post processing functions. This is recommended for version after OS 2.14.0 for models to use the ML inference processor to handle input/output mapping. Note that if using a model that requires pre and post processing functions, you must provide the functions in the blueprint. Please refer to legacy blueprint: [Azure OpenAI connector blueprint example for embedding model](../azure_openai_connector_embedding_blueprint.md)
4+
5+
## 1. Add Azure OpenAI endpoint to trusted URLs:
6+
7+
```json
8+
PUT /_cluster/settings
9+
{
10+
"persistent": {
11+
"plugins.ml_commons.trusted_connector_endpoints_regex": [
12+
"^https://.*\\.openai\\.azure\\.com/.*$"
13+
]
14+
}
15+
}
16+
```
17+
18+
## 2. Create connector for Azure OpenAI embedding model:
19+
20+
Refer to [Azure OpenAI Service REST API reference - Embedding](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#embeddings).
21+
22+
If you are using self-managed OpenSearch, you should supply OpenAI API key:
23+
24+
```jsonc
25+
POST /_plugins/_ml/connectors/_create
26+
{
27+
"name": "<YOUR CONNECTOR NAME>",
28+
"description": "<YOUR CONNECTOR DESCRIPTION>",
29+
"version": "<YOUR CONNECTOR VERSION>",
30+
"protocol": "http",
31+
"parameters": {
32+
"endpoint": "<YOUR RESOURCE NAME>.openai.azure.com/",
33+
"deploy-name": "<YOUR DEPLOYMENT NAME>",
34+
"model": "text-embedding-ada-002",
35+
"api-version": "<YOUR API VERSION>"
36+
},
37+
"credential": {
38+
"openAI_key": "<YOUR API KEY>"
39+
},
40+
"actions": [
41+
{
42+
"action_type": "predict",
43+
"method": "POST",
44+
"url": "https://${parameters.endpoint}/openai/deployments/${parameters.deploy-name}/embeddings?api-version=${parameters.api-version}",
45+
"headers": {
46+
"api-key": "${credential.openAI_key}"
47+
},
48+
"request_body": "{ \"input\": ${parameters.input}, \"input_type\": \"array\"}" // support array of strings
49+
}
50+
]
51+
}
52+
```
53+
54+
> [!NOTE]
55+
> If you need to the input type to be a string instead of array of strings, you can modify the request body to:
56+
> ```json
57+
> "request_body": "{ \"input\": \"${parameters.input}\" }"
58+
> ```
59+
> See [Azure OpenAI API Reference - Request Body - input_type](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#request-body-1)
60+
61+
Sample response:
62+
```json
63+
{
64+
"connector_id": "OyB0josB2yd36FqHy3lO"
65+
}
66+
```
67+
68+
## 3. Create model group:
69+
70+
```json
71+
POST /_plugins/_ml/model_groups/_register
72+
{
73+
"name": "remote_model_group",
74+
"description": "This is an example description"
75+
}
76+
```
77+
78+
Sample response:
79+
```json
80+
{
81+
"model_group_id": "TWR0josByE8GuSOJ629m",
82+
"status": "CREATED"
83+
}
84+
```
85+
86+
## 4. Register model to model group & deploy model:
87+
88+
```json
89+
POST /_plugins/_ml/models/_register
90+
{
91+
"name": "OpenAI embedding model",
92+
"function_name": "remote",
93+
"model_group_id": "TWR0josByE8GuSOJ629m",
94+
"description": "test model",
95+
"connector_id": "OyB0josB2yd36FqHy3lO"
96+
}
97+
```
98+
99+
100+
Sample response:
101+
```json
102+
{
103+
"task_id": "PCB1josB2yd36FqHAXk9",
104+
"status": "CREATED"
105+
}
106+
```
107+
Get model id from task
108+
```json
109+
GET /_plugins/_ml/tasks/PCB1josB2yd36FqHAXk9
110+
```
111+
Deploy model, in this demo the model id is `PSB1josB2yd36FqHAnl1`
112+
```json
113+
POST /_plugins/_ml/models/PSB1josB2yd36FqHAnl1/_deploy
114+
```
115+
116+
## 5. Test model inference
117+
118+
```json
119+
POST /_plugins/_ml/models/PSB1josB2yd36FqHAnl1/_predict
120+
{
121+
"parameters": {
122+
"input": ["What is the meaning of life?", "42"]
123+
}
124+
}
125+
```
126+
127+
Response:
128+
```json
129+
{
130+
"inference_results": [
131+
{
132+
"output": [
133+
{
134+
"name": "response",
135+
"dataAsMap": {
136+
"object": "list",
137+
"data": [
138+
{
139+
"object": "embedding",
140+
"index": 0,
141+
"embedding": [
142+
0.004411249,
143+
-0.029655455,
144+
-0.008198498,
145+
...
146+
]
147+
},
148+
{
149+
"object": "embedding",
150+
"index": 1,
151+
"embedding": [
152+
-0.020884188,
153+
-0.012239939,
154+
0.031366087,
155+
...
156+
]
157+
}
158+
],
159+
"model": "text-embedding-ada-002",
160+
"usage": {
161+
"prompt_tokens": 7,
162+
"total_tokens": 7
163+
}
164+
}
165+
}
166+
],
167+
"status_code": 200
168+
}
169+
]
170+
}
171+
```
172+

0 commit comments

Comments
 (0)