Skip to content

Commit 4e2f31c

Browse files
mingshlmingshl
authored
Add standard blueprint for vector search (#3659)
* add standard blueprint for vector search Signed-off-by: Mingshi Liu <mingshl@amazon.com> * remove add trusted endpoint Signed-off-by: Mingshi Liu <mingshl@amazon.com> * use english sentence in predict Signed-off-by: Mingshi Liu <mingshl@amazon.com> --------- Signed-off-by: Mingshi Liu <mingshl@amazon.com>
1 parent 31f0422 commit 4e2f31c

9 files changed

+1332
-1
lines changed

‎docs/remote_inference_blueprints/cohere_connector_image_embedding_blueprint.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ See above for all the values the `parameters > model` parameter can take.
3131
```json
3232
POST /_plugins/_ml/connectors/_create
3333
{
34-
"name": "Cohere Embed Model",
34+
"name": "Cohere Image Embed Model",
3535
"description": "The connector to Cohere's public embed API",
3636
"version": "1",
3737
"protocol": "http",

‎docs/remote_inference_blueprints/standard_blueprints/README.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Model Blueprints for Vector Search
2+
3+
OpenSearch provides two approaches for implementing embedding models, depending on your needs:
4+
5+
## Standard Blueprints (Using ML Inference Processor)
6+
Recommended for new implementations for version after OS 2.14.0
7+
8+
These blueprints use the ML inference processor to handle input/output mapping, offering:
9+
10+
- Simpler implementation
11+
- Direct model registration
12+
- Flexible input/output mapping through the processor
13+
14+
[List of available models]:
15+
- Bedrock:
16+
- [amazon.titan-embed-text-v1](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/bedrock_connector_titan_embedding_standard_blueprint.md)
17+
- [amazon.titan-embed-image-v1](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/bedrock_connector_titan_multimodal_embedding_standard_blueprint.md)
18+
- [cohere.embed-english-v3](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md)
19+
- [cohere.embed-multilingual-v3](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/bedrock_connector_cohere_cohere.embed-multilingual-v3_standard_blueprint.md)
20+
- Cohere
21+
- text embedding: [embed-english-v3.0 & embed-english-v2.0](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/cohere_connector_text_embedding_standard_blueprint.md)
22+
- image embedding: [embed-multimodal-v3.0 & embed-multimodal-v2.0](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/cohere_connector_image_embedding_blueprint.md)
23+
- OpenAI:
24+
- [text-embedding-ada-002](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/standard_blueprints/openai_connector_embedding_standard_blueprint.md)
25+
26+
## Legacy Blueprints (With Pre/Post Processing)
27+
For existing implementations or specific customization needs
28+
29+
These blueprints include pre- and post-processing functions, suitable when you need:
30+
31+
- Custom preprocessing logic
32+
- Specific output formatting requirements
33+
- Compatibility with existing implementations
34+
35+
[List of available models]:
36+
37+
- Bedrock:
38+
- [amazon.titan-embed-text-v1](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/bedrock_connector_titan_embedding_blueprint.md)
39+
- [amazon.titan-embed-image-v1](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/bedrock_connector_titan_multimodal_embedding_blueprint.md)
40+
- [cohere.embed-english-v3](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_blueprint.md)
41+
- [cohere.embed-multilingual-v3](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/bedrock_connector_cohere_cohere.embed-multilingual-v3_blueprint.md)
42+
- Cohere
43+
- text embedding: [embed-english-v3.0 & embed-english-v2.0](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/cohere_connector_embedding_blueprint.md)
44+
- image embedding: [embed-multimodal-v3.0 & embed-multimodal-v2.0](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/cohere_connector_image_embedding_blueprint.md)
45+
46+
- OpenAI:
47+
- [text-embedding-ada-002](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/openai_connector_embedding_blueprint.md)
48+
49+
- VertexAI
50+
- [embedding](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/gcp_vertexai_connector_embedding_blueprint.md)

‎docs/remote_inference_blueprints/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Bedrock connector standard blueprint example for Cohere text embedding model
2+
This blueprint demonstrates how to deploy a cohere.embed-english-v3 using the Bedrock connector without pre and post processing functions.
3+
This is recommended for version after OS 2.14.0 for models to use the ML inference processor to handle input/output mapping.
4+
Note that if using a model that requires pre and post processing functions, you must provide the functions in the blueprint. Please refer to legacy blueprint: [Bedrock connector blueprint example for Cohere embed-english-v3 model](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_blueprint.md)
5+
6+
## 1. Create connector for Amazon Bedrock:
7+
8+
If you are using self-managed Opensearch, you should supply AWS credentials:
9+
10+
```json
11+
POST /_plugins/_ml/connectors/_create
12+
{
13+
"name": "Amazon Bedrock Connector: Cohere embed-english-v3",
14+
"description": "Test connector for Amazon Bedrock Cohere embed-english-v3",
15+
"version": 1,
16+
"protocol": "aws_sigv4",
17+
"credential": {
18+
"access_key": "<PLEASE ADD YOUR AWS ACCESS KEY HERE>",
19+
"secret_key": "<PLEASE ADD YOUR AWS SECRET KEY HERE>",
20+
"session_token": "<PLEASE ADD YOUR AWS SECURITY TOKEN HERE>"
21+
},
22+
"parameters": {
23+
"region": "<PLEASE ADD YOUR AWS REGION HERE>",
24+
"service_name": "bedrock",
25+
"truncate": "<NONE|START|END>",
26+
"input_type": "<search_document|search_query|classification|clustering>",
27+
"model": "cohere.embed-english-v3"
28+
},
29+
"actions": [
30+
{
31+
"action_type": "predict",
32+
"method": "POST",
33+
"headers": {
34+
"x-amz-content-sha256": "required",
35+
"content-type": "application/json"
36+
},
37+
"url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/invoke",
38+
"request_body": "{ \"texts\": ${parameters.texts}, \"truncate\": \"${parameters.truncate}\", \"input_type\": \"${parameters.input_type}\" }"
39+
}
40+
]
41+
}
42+
```
43+
44+
If using the AWS Opensearch Service, you can provide an IAM role arn that allows access to the bedrock service.
45+
Refer to this [AWS doc](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html)
46+
47+
```json
48+
POST /_plugins/_ml/connectors/_create
49+
{
50+
"name": "Amazon Bedrock Connector: Cohere embed-english-v3",
51+
"description": "Test connector for Amazon Bedrock Cohere embed-english-v3 model",
52+
"version": 1,
53+
"protocol": "aws_sigv4",
54+
"credential": {
55+
"roleArn": "<PLEASE ADD YOUR AWS ROLE ARN HERE>"
56+
},
57+
"parameters": {
58+
"region": "<PLEASE ADD YOUR AWS REGION HERE>",
59+
"service_name": "bedrock",
60+
"truncate": "<NONE|START|END>",
61+
"input_type": "<search_document|search_query|classification|clustering>",
62+
"model": "cohere.embed-english-v3"
63+
},
64+
"actions": [
65+
{
66+
"action_type": "predict",
67+
"method": "POST",
68+
"headers": {
69+
"x-amz-content-sha256": "required",
70+
"content-type": "application/json"
71+
},
72+
"url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/invoke",
73+
"request_body": "{ \"texts\": ${parameters.texts}, \"truncate\": \"${parameters.truncate}\", \"input_type\": \"${parameters.input_type}\" }"
74+
}
75+
]
76+
}
77+
```
78+
79+
Sample response:
80+
```json
81+
{
82+
"connector_id": "hijwwZABNrAVdFa9prf7"
83+
}
84+
```
85+
For more information of the model inference parameters in the connector, please refer to this [AWS doc](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html)
86+
87+
## 2. Create model group(Optional):
88+
89+
```json
90+
POST /_plugins/_ml/model_groups/_register
91+
{
92+
"name": "remote_model_group_cohere",
93+
"description": "model group for cohere models"
94+
}
95+
```
96+
97+
Sample response:
98+
```json
99+
{
100+
"model_group_id": "IMobmY8B8aiZvtEZeO_i",
101+
"status": "CREATED"
102+
}
103+
```
104+
105+
## 3. Register model to model group & deploy model:
106+
107+
```json
108+
POST /_plugins/_ml/models/_register?deploy=true
109+
{
110+
"name": "cohere.embed-english-v3",
111+
"function_name": "remote",
112+
"model_group_id": "IMobmY8B8aiZvtEZeO_i",
113+
"description": "cohere embed-english v3 model",
114+
"connector_id": "hijwwZABNrAVdFa9prf7"
115+
}
116+
```
117+
Sample response:
118+
```json
119+
{
120+
"task_id": "rMormY8B8aiZvtEZIO_j",
121+
"status": "CREATED",
122+
"model_id": "lyjxwZABNrAVdFa9zrcZ"
123+
}
124+
```
125+
126+
## 4. Test model inference
127+
128+
```json
129+
POST /_plugins/_ml/models/lyjxwZABNrAVdFa9zrcZ/_predict
130+
{
131+
"parameters": {
132+
"texts" : ["Hello World", "This is a test"]
133+
}
134+
}
135+
```
136+
137+
Sample response:
138+
```json
139+
{
140+
"inference_results": [
141+
{
142+
"output": [
143+
{
144+
"name": "response",
145+
"dataAsMap": {
146+
"id": "74372fce-e09d-483e-af6a-b45c84e3999e",
147+
"texts": [
148+
"hello world",
149+
"this is a test"
150+
],
151+
"embeddings": [
152+
[
153+
-0.029205322,
154+
-0.02357483,
155+
-0.05987549,
156+
...
157+
],
158+
[
159+
-0.013885498,
160+
0.009994507,
161+
-0.03253174,
162+
...
163+
]
164+
],
165+
"response_type": "embeddings_floats"
166+
}
167+
}
168+
],
169+
"status_code": 200
170+
}
171+
]
172+
}
173+
```
174+

0 commit comments

Comments
 (0)