Skip to content

Commit b3e7dc5

Browse files
Rutik7066Rutik7066
committed
redme and made some fixes
1 parent 664f4c7 commit b3e7dc5

File tree

6 files changed

+192
-5
lines changed

6 files changed

+192
-5
lines changed

‎golem-embed/README.md

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
# golem-embed
2+
3+
WebAssembly Components providing a unified API for various AI embedding and reranking providers.
4+
5+
## Versions
6+
7+
There are 8 published WASM files for each release:
8+
9+
| Name | Description |
10+
|-------------------------------------------|--------------------------------------------------------------------------------------------|
11+
| `golem-embed-openai.wasm` | Embedding implementation for OpenAI, using custom Golem specific durability features |
12+
| `golem-embed-cohere.wasm` | Embedding implementation for Cohere, using custom Golem specific durability features |
13+
| `golem-embed-hugging-face.wasm` | Embedding implementation for Hugging Face, using custom Golem specific durability features|
14+
| `golem-embed-voyageai.wasm` | Embedding implementation for VoyageAI, using custom Golem specific durability features |
15+
| `golem-embed-openai-portable.wasm` | Embedding implementation for OpenAI, with no Golem specific dependencies. |
16+
| `golem-embed-cohere-portable.wasm` | Embedding implementation for Cohere, with no Golem specific dependencies. |
17+
| `golem-embed-hugging-face-portable.wasm` | Embedding implementation for Hugging Face, with no Golem specific dependencies. |
18+
| `golem-embed-voyageai-portable.wasm` | Embedding implementation for VoyageAI, with no Golem specific dependencies. |
19+
20+
Every component **exports** the same `golem:embed` interface, [defined here](wit/golem-embed.wit).
21+
22+
The `-portable` versions only depend on `wasi:io`, `wasi:http` and `wasi:logging`.
23+
24+
The default versions also depend on [Golem's host API](https://learn.golem.cloud/golem-host-functions) to implement
25+
advanced durability related features.
26+
27+
## Provider Capabilities
28+
29+
Each provider supports different functionality and input types:
30+
31+
| Provider |Text Embedding | Image Embedding | Reranking |
32+
|---------------|-----------|------|-------|
33+
| OpenAI | ✅ | ❌ | ❌ |
34+
| Cohere | ✅ | ✅ | ✅ |
35+
| Hugging Face | ✅ | ❌ | ❌ |
36+
| VoyageAI | ✅ | ❌ | ✅ |
37+
38+
39+
## Usage
40+
41+
Each provider has to be configured with an API key passed as an environment variable:
42+
43+
| Provider | Environment Variable |
44+
|---------------|--------------------------|
45+
| OpenAI | `OPENAI_API_KEY` |
46+
| Cohere | `COHERE_API_KEY` |
47+
| Hugging Face | `HUGGING_FACE_API_KEY` |
48+
| VoyageAI | `VOYAGEAI_API_KEY` |
49+
50+
Additionally, setting the `GOLEM_EMBED_LOG=trace` environment variable enables trace logging for all the communication
51+
with the underlying embedding provider.
52+
53+
### Using with Golem
54+
55+
#### Using a template
56+
57+
The easiest way to get started is to use one of the predefined **templates** Golem provides.
58+
59+
**NOT AVAILABLE YET**
60+
61+
#### Using a component dependency
62+
63+
To existing Golem applications the `golem-embed` WASM components can be added as a **binary dependency**.
64+
65+
**NOT AVAILABLE YET**
66+
67+
#### Integrating the composing step to the build
68+
69+
Currently it is necessary to manually add the [`wac`](https://github.com/bytecodealliance/wac) tool call to the
70+
application manifest to link with the selected embedding implementation. The `test` directory of this repository shows an
71+
example of this.
72+
73+
The summary of the steps to be done, assuming the component was created with `golem-cli component new rust my:example`:
74+
75+
1. Copy the `profiles` section from `common-rust/golem.yaml` to the component's `golem.yaml` file (for example in
76+
`components-rust/my-example/golem.yaml`) so it can be customized.
77+
2. Add a second **build step** after the `cargo component build` which is calling `wac` to compose with the selected (
78+
and downloaded) `golem-embed` binary. See the example below.
79+
3. Modify the `componentWasm` field to point to the composed WASM file.
80+
4. Add the `golem-embed.wit` file (from this repository) to the application's root `wit/deps/golem:embed` directory.
81+
5. Import `golem-embed.wit` in your component's WIT file: `import golem:embed/embed@1.0.0;'
82+
83+
Example app manifest build section:
84+
85+
```yaml
86+
components:
87+
my:example:
88+
profiles:
89+
debug:
90+
build:
91+
- command: cargo component build
92+
sources:
93+
- src
94+
- wit-generated
95+
- ../../common-rust
96+
targets:
97+
- ../../target/wasm32-wasip1/debug/my_example.wasm
98+
- command: wac plug --plug ../../golem_embed_openai.wasm ../../target/wasm32-wasip1/debug/my_example.wasm -o ../../target/wasm32-wasip1/debug/my_example_plugged.wasm
99+
sources:
100+
- ../../target/wasm32-wasip1/debug/my_example.wasm
101+
- ../../golem_embed_openai.wasm
102+
targets:
103+
- ../../target/wasm32-wasip1/debug/my_example_plugged.wasm
104+
sourceWit: wit
105+
generatedWit: wit-generated
106+
componentWasm: ../../target/wasm32-wasip1/debug/my_example_plugged.wasm
107+
linkedWasm: ../../golem-temp/components/my_example_debug.wasm
108+
clean:
109+
- src/bindings.rs
110+
```
111+
112+
### Using without Golem
113+
114+
To use the embedding provider components in a WebAssembly project independent of Golem you need to do the following:
115+
116+
1. Download one of the `-portable.wasm` versions
117+
2. Download the `golem-embed.wit` WIT package and import it
118+
3. Use [`wac`](https://github.com/bytecodealliance/wac) to compose your component with the selected embedding implementation.
119+
120+
## Examples
121+
122+
Take the [test application](test/components-rust/test-embed/src/lib.rs) as an example of using `golem-embed` from Rust. The
123+
implemented test functions are demonstrating the following:
124+
125+
| Function Name | Description |
126+
|---------------|--------------------------------------------------------------------------------------------|
127+
| `test1` | Simple text embedding generation |
128+
| `test2` | Demonstrates document reranking functionality |
129+
130+
### Running the examples
131+
132+
To run the examples first you need a running Golem instance. This can be Golem Cloud or the single-executable `golem`
133+
binary
134+
started with `golem server run`.
135+
136+
**NOTE**: `golem-embed` requires the latest (unstable) version of Golem currently. It's going to work with the next public
137+
stable release 1.2.2.
138+
139+
Then build and deploy the _test application_. Select one of the following profiles to choose which provider to use:
140+
| Profile Name | Description |
141+
|--------------|-----------------------------------------------------------------------------------------------|
142+
| `openai-debug` | Uses the OpenAI embedding implementation and compiles the code in debug profile |
143+
| `openai-release` | Uses the OpenAI embedding implementation and compiles the code in release profile |
144+
| `cohere-debug` | Uses the Cohere embedding implementation and compiles the code in debug profile |
145+
| `cohere-release` | Uses the Cohere embedding implementation and compiles the code in release profile |
146+
| `hugging-face-debug` | Uses the Hugging Face embedding implementation and compiles the code in debug profile |
147+
| `hugging-face-release` | Uses the Hugging Face embedding implementation and compiles the code in release profile |
148+
| `voyageai-debug` | Uses the VoyageAI embedding implementation and compiles the code in debug profile |
149+
| `voyageai-release` | Uses the VoyageAI embedding implementation and compiles the code in release profile |
150+
151+
```bash
152+
cd test
153+
golem app build -b openai-debug
154+
golem app deploy -b openai-debug
155+
```
156+
157+
Depending on the provider selected, an environment variable has to be set for the worker to be started, containing the API key for the given provider:
158+
159+
```bash
160+
golem worker new test:embed/debug --env OPENAI_API_KEY=xxx --env GOLEM_EMBED_LOG=trace
161+
```
162+
163+
Then you can invoke the test functions on this worker:
164+
165+
```bash
166+
golem worker invoke test:embed/debug test1 --stream
167+
```
168+
169+
## Development
170+
171+
This repository uses [cargo-make](https://github.com/sagiegurari/cargo-make) to automate build tasks.
172+
Some of the important tasks are:
173+
174+
| Command | Description |
175+
|-------------------------------------|--------------------------------------------------------------------------------------------------------|
176+
| `cargo make build` | Build all components with Golem bindings in Debug |
177+
| `cargo make release-build` | Build all components with Golem bindings in Release |
178+
| `cargo make build-portable` | Build all components with no Golem bindings in Debug |
179+
| `cargo make release-build-portable` | Build all components with no Golem bindings in Release |
180+
| `cargo make unit-tests` | Run all unit tests |
181+
| `cargo make check` | Checks formatting and Clippy rules |
182+
| `cargo make fix` | Fixes formatting and Clippy rules |
183+
| `cargo make wit` | To be used after editing the `wit/golem-embed.wit` file - distributes the changes to all wit directories |
184+
185+
The `test` directory contains a **Golem application** for testing various features of the embedding components.
186+
Check [the Golem documentation](https://learn.golem.cloud/quickstart) to learn how to install Golem and `golem-cli` to
187+
run these tests.

‎golem-embed/embed-cohere/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ edition = "2021"
55
license = "Apache-2.0"
66
homepage = "https://golem.cloud"
77
repository = "https://github.com/golemcloud/golem-llm"
8-
description = "WebAssembly component for working with Ollama APIs, with special support for Golem Cloud"
8+
description = "WebAssembly component for working with Cohere embeding and reranking APIs, with special support for Golem Cloud"
99

1010

1111
[lib]

‎golem-embed/embed-hugging-face/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ edition = "2021"
55
license = "Apache-2.0"
66
homepage = "https://golem.cloud"
77
repository = "https://github.com/golemcloud/golem-llm"
8-
description = "WebAssembly component for working with Ollama APIs, with special support for Golem Cloud"
8+
description = "WebAssembly component for working with Hugging face embeding APIs, with special support for Golem Cloud"
99

1010

1111
[lib]

‎golem-embed/embed-openai/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ edition = "2021"
55
license = "Apache-2.0"
66
homepage = "https://golem.cloud"
77
repository = "https://github.com/golemcloud/golem-llm"
8-
description = "WebAssembly component for working with Ollama APIs, with special support for Golem Cloud"
8+
description = "WebAssembly component for working with OpenAI embeding APIs, with special support for Golem Cloud"
99

1010

1111
[lib]

‎golem-embed/embed-voyageai/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ edition = "2021"
55
license = "Apache-2.0"
66
homepage = "https://golem.cloud"
77
repository = "https://github.com/golemcloud/golem-llm"
8-
description = "WebAssembly component for working with Ollama APIs, with special support for Golem Cloud"
8+
description = "WebAssembly component for working with VoyageAI embeding and reranking APIs, with special support for Golem Cloud"
99

1010

1111
[lib]

‎golem-embed/embed/src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ impl LoggingState {
2525
if !self.logging_initialized {
2626
let _ = wasi_logger::Logger::install();
2727
let max_level: log::LevelFilter =
28-
log::LevelFilter::from_str(&std::env::var("GOLEM_LLM_LOG").unwrap_or_default())
28+
log::LevelFilter::from_str(&std::env::var("GOLEM_EMBED_LOG").unwrap_or_default())
2929
.unwrap_or(log::LevelFilter::Info);
3030
log::set_max_level(max_level);
3131
self.logging_initialized = true;

0 commit comments

Comments
 (0)