You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings/onnx.adoc
+14-1Lines changed: 14 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -77,7 +77,7 @@ The complete list of supported properties are:
77
77
| spring.ai.embedding.transformer.cache.directory | Directory path to cache remote resources, such as the ONNX models | ${java.io.tmpdir}/spring-ai-onnx-model
| spring.ai.embedding.transformer.onnx.modelOutputName | The ONNX model's output node name, which we'll use for embedding calculation. | last_hidden_state
80
-
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise. | -1
80
+
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise.(Requires additional onnxruntime_gpu dependency) | -1
81
81
| spring.ai.embedding.transformer.metadataMode | Specifies what parts of the Documents content and metadata will be used for computing the embeddings. | NONE
82
82
|===
83
83
@@ -114,6 +114,19 @@ The `model.onnx_data` is called link:https://onnx.ai/onnx/repo-docs/ExternalData
114
114
Currently the only workaround is to copy the large `model.onnx_data` in the folder you run your Boot applicaiton.
115
115
====
116
116
117
+
[NOTE]
118
+
====
119
+
If you get an error like `ai.onnxruntime.OrtException: Error code - ORT_EP_FAIL - message: Failed to find CUDA shared provider`,
120
+
that means that you are using the GPU parameters `spring.ai.embedding.transformer.onnx.gpuDeviceId` , but the onnxruntime_gpu dependency are missing.
121
+
----
122
+
<dependency>
123
+
<groupId>com.microsoft.onnxruntime</groupId>
124
+
<artifactId>onnxruntime_gpu</artifactId>
125
+
</dependency>
126
+
----
127
+
Please select the appropriate onnxruntime_gpu version based on the CUDA version(link:https://onnxruntime.ai/docs/get-started/with-java.html[ONNX Java Runtime]).
128
+
====
129
+
117
130
== Manual Configuration
118
131
119
132
If you are not using Spring Boot, you can manually configure the Onnx Transformers Embedding Model.
0 commit comments