Skip to content

Commit 1b22873

Browse files
juntaojuntao
committed
Add database clients
Signed-off-by: Michael Yuan <michael@secondstate.io>
1 parent 0ea42d3 commit 1b22873

File tree

3 files changed

+303
-12
lines changed

3 files changed

+303
-12
lines changed

‎docs/develop/rust/database/postgres_driver.md

Lines changed: 126 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ A database connection is necessary for today's enterprise development. WasmEdge
88

99
<!-- prettier-ignore -->
1010
:::note
11-
Before we start, make sure [you have Rust and WasmEdge installed](../setup.md).
11+
Before we start, [you need to have Rust and WasmEdge installed](../setup.md).
12+
Make sure that you read the [special notes on networking apps](../setup#special-notes) especially if you are compiling Rust programs on a Mac.
1213
:::
1314

1415
## Run the example
@@ -20,15 +21,134 @@ git clone https://github.com/WasmEdge/wasmedge-db-examples
2021
cd wasmedge-db-examples/postgres
2122

2223
# Compile the rust code into WASM
23-
cargo build --target wasm32-wasi --release
24+
RUSTFLAGS="--cfg wasmedge --cfg tokio_unstable" cargo build --target wasm32-wasi --release
2425

2526
# Execute SQL statements against a PostgreSQL database at postgres://user:passwd@localhost/testdb
2627
wasmedge --env "DATABASE_URL=postgres://user:passwd@localhost/testdb" target/wasm32-wasi/release/crud.wasm
2728
```
2829

30+
## Configuration
31+
32+
In order to compile the `tokio-postgres` and `tokio` crates, we will need to apply patches to add WasmEdge-specific socket APIs to those crates in `Cargo.toml`.
33+
34+
```
35+
[patch.crates-io]
36+
tokio = { git = "https://github.com/second-state/wasi_tokio.git", branch = "v1.36.x" }
37+
socket2 = { git = "https://github.com/second-state/socket2.git", branch = "v0.5.x" }
38+
tokio-postgres = { git = "https://github.com/second-state/rust-postgres.git" }
39+
40+
[dependencies]
41+
tokio-postgres = "0.7"
42+
tokio = { version = "1", features = [
43+
"io-util",
44+
"fs",
45+
"net",
46+
"time",
47+
"rt",
48+
"macros",
49+
] }
50+
```
51+
2952
## Code explanation
3053

31-
<!-- prettier-ignore -->
32-
:::info
33-
Work in Progress
34-
:::
54+
We first use a Rust struct to represent the database table.
55+
56+
```rust
57+
#[derive(Debug)]
58+
struct Order {
59+
order_id: i32,
60+
production_id: i32,
61+
quantity: i32,
62+
amount: f32,
63+
shipping: f32,
64+
tax: f32,
65+
shipping_address: String,
66+
}
67+
68+
impl Order {
69+
fn new(
70+
order_id: i32,
71+
production_id: i32,
72+
quantity: i32,
73+
amount: f32,
74+
shipping: f32,
75+
tax: f32,
76+
shipping_address: String,
77+
) -> Self {
78+
Self {
79+
order_id,
80+
production_id,
81+
quantity,
82+
amount,
83+
shipping,
84+
tax,
85+
shipping_address,
86+
}
87+
}
88+
}
89+
```
90+
91+
Then, you can use the `tokio-postgres` API to access the database through its connection URL.
92+
The code below shows how to perform basic CRUD operations using SQL commands.
93+
94+
```rust
95+
#[tokio::main(flavor = "current_thread")]
96+
async fn main() -> Result<(), Error> {
97+
// Connect to the database.
98+
let (client, connection) = tokio_postgres::connect(&*get_url(), NoTls).await?;
99+
100+
// The connection object performs the actual communication with the database,
101+
// so spawn it off to run on its own.
102+
tokio::spawn(async move {
103+
if let Err(e) = connection.await {
104+
eprintln!("connection error: {}", e);
105+
}
106+
});
107+
108+
client.execute("CREATE TABLE IF NOT EXISTS orders (order_id INT, production_id INT, quantity INT, amount REAL, shipping REAL, tax REAL, shipping_address VARCHAR(256));", &[]).await?;
109+
110+
let orders = vec![
111+
Order::new(1, 12, 2, 56.0, 15.0, 2.0, String::from("Mataderos 2312")),
112+
Order::new(2, 15, 3, 256.0, 30.0, 16.0, String::from("1234 NW Bobcat")),
113+
Order::new(3, 11, 5, 536.0, 50.0, 24.0, String::from("20 Havelock")),
114+
Order::new(4, 8, 8, 126.0, 20.0, 12.0, String::from("224 Pandan Loop")),
115+
Order::new(5, 24, 1, 46.0, 10.0, 2.0, String::from("No.10 Jalan Besar")),
116+
];
117+
118+
for order in orders.iter() {
119+
client.execute(
120+
"INSERT INTO orders (order_id, production_id, quantity, amount, shipping, tax, shipping_address) VALUES ($1, $2, $3, $4, $5, $6, $7)",
121+
&[&order.order_id, &order.production_id, &order.quantity, &order.amount, &order.shipping, &order.tax, &order.shipping_address]
122+
).await?;
123+
}
124+
125+
let rows = client.query("SELECT * FROM orders;", &[]).await?;
126+
for row in rows.iter() {
127+
let order_id : i32 = row.get(0);
128+
println!("order_id {}", order_id);
129+
130+
let production_id : i32 = row.get(1);
131+
println!("production_id {}", production_id);
132+
133+
let quantity : i32 = row.get(2);
134+
println!("quantity {}", quantity);
135+
136+
let amount : f32 = row.get(3);
137+
println!("amount {}", amount);
138+
139+
let shipping : f32 = row.get(4);
140+
println!("shipping {}", shipping);
141+
142+
let tax : f32 = row.get(5);
143+
println!("tax {}", tax);
144+
145+
let shipping_address : &str = row.get(6);
146+
println!("shipping_address {}", shipping_address);
147+
}
148+
149+
client.execute("DELETE FROM orders;", &[]).await?;
150+
151+
Ok(())
152+
}
153+
```
154+

‎docs/develop/rust/database/qdrant_driver.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
---
2+
sidebar_position: 4
3+
---
4+
5+
# Qdrant driver
6+
7+
WasmEdge is emerging as a lightweight, portable, and embeddable runtime for large language models (LLMs). LLM inference applications, such as RAG chatbots and AI agents, can be developed on Mac or Windows, compiled to Wasm once, and then deployed across Nvidia / AMD / ARM-powered devices or servers, fully taking advantage of on-device GPUs, NPUs, and accelerators.
8+
9+
Hence, besides the LLM inference runtime, those LLM applications also need to manage embeddings in vector databases. The [qdrant-rest-client](https://crates.io/crates/qdrant_rest_client) crate allows you to access the Qdrant vector database from your portable Wasm apps!
10+
11+
<!-- prettier-ignore -->
12+
:::note
13+
Before we start, [you need to have Rust and WasmEdge installed](../setup.md).
14+
Make sure that you read the [special notes on networking apps](../setup#special-notes) especially if you are compiling Rust programs on a Mac.
15+
:::
16+
17+
## Run the example
18+
19+
The [wasmedge-db-example/qdrant](https://github.com/WasmEdge/wasmedge-db-examples/tree/main/qdrant) is a Qdrant client example written in Rust.
20+
21+
```bash
22+
git clone https://github.com/WasmEdge/wasmedge-db-examples
23+
cd wasmedge-db-examples/qdrant
24+
25+
# Compile the rust code into WASM
26+
RUSTFLAGS="--cfg wasmedge --cfg tokio_unstable" cargo build --target wasm32-wasi --release
27+
28+
# Perform vector data operations against a Qdrant at http://localhost:6333
29+
wasmedge target/wasm32-wasi/release/qdrant_examples.wasm
30+
```
31+
32+
## Configuration
33+
34+
In order to compile the `qdrant_rest_client` and `tokio` crates, we will need to apply patches to add WasmEdge-specific socket APIs to those crates in `Cargo.toml`.
35+
36+
```rust
37+
[patch.crates-io]
38+
socket2 = { git = "https://github.com/second-state/socket2.git", branch = "v0.5.x" }
39+
reqwest = { git = "https://github.com/second-state/wasi_reqwest.git", branch = "0.11.x" }
40+
hyper = { git = "https://github.com/second-state/wasi_hyper.git", branch = "v0.14.x" }
41+
tokio = { git = "https://github.com/second-state/wasi_tokio.git", branch = "v1.36.x" }
42+
43+
[dependencies]
44+
anyhow = "1.0"
45+
serde_json = "1.0"
46+
serde = { version = "1.0", features = ["derive"] }
47+
url = "2.3"
48+
tokio = { version = "1", features = ["io-util", "fs", "net", "time", "rt", "macros"] }
49+
qdrant_rest_client = "0.1.0"
50+
```
51+
52+
## Code explanation
53+
54+
The following program uses the `qdrant_rest_client` crate to access local Qdrant server through its RESTful API.
55+
It first creates several points (vectors), saves those vectors to the Qdrant database, retrieves some vectors,
56+
searches for vectors, and finally deletes them from the database.
57+
58+
```rust
59+
#[tokio::main(flavor = "current_thread")]
60+
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
61+
let client = qdrant::Qdrant::new();
62+
// Create a collection with 10-dimensional vectors
63+
let r = client.create_collection("my_test", 4).await;
64+
println!("Create collection result is {:?}", r);
65+
66+
let mut points = Vec::<Point>::new();
67+
points.push(Point {
68+
id: PointId::Num(1),
69+
vector: vec![0.05, 0.61, 0.76, 0.74],
70+
payload: json!({"city": "Berlin"}).as_object().map(|m| m.to_owned()),
71+
});
72+
points.push(Point {
73+
id: PointId::Num(2),
74+
vector: vec![0.19, 0.81, 0.75, 0.11],
75+
payload: json!({"city": "London"}).as_object().map(|m| m.to_owned()),
76+
});
77+
points.push(Point {
78+
id: PointId::Num(3),
79+
vector: vec![0.36, 0.55, 0.47, 0.94],
80+
payload: json!({"city": "Moscow"}).as_object().map(|m| m.to_owned()),
81+
});
82+
points.push(Point {
83+
id: PointId::Num(4),
84+
vector: vec![0.18, 0.01, 0.85, 0.80],
85+
payload: json!({"city": "New York"})
86+
.as_object()
87+
.map(|m| m.to_owned()),
88+
});
89+
points.push(Point {
90+
id: PointId::Num(5),
91+
vector: vec![0.24, 0.18, 0.22, 0.44],
92+
payload: json!({"city": "Beijing"}).as_object().map(|m| m.to_owned()),
93+
});
94+
points.push(Point {
95+
id: PointId::Num(6),
96+
vector: vec![0.35, 0.08, 0.11, 0.44],
97+
payload: json!({"city": "Mumbai"}).as_object().map(|m| m.to_owned()),
98+
});
99+
100+
let r = client.upsert_points("my_test", points).await;
101+
println!("Upsert points result is {:?}", r);
102+
103+
println!(
104+
"The collection size is {}",
105+
client.collection_info("my_test").await
106+
);
107+
108+
let p = client.get_point("my_test", 2).await;
109+
println!("The second point is {:?}", p);
110+
111+
let ps = client.get_points("my_test", vec![1, 2, 3, 4, 5, 6]).await;
112+
println!("The 1-6 points are {:?}", ps);
113+
114+
let q = vec![0.2, 0.1, 0.9, 0.7];
115+
let r = client.search_points("my_test", q, 2, None).await;
116+
println!("Search result points are {:?}", r);
117+
118+
let r = client.delete_points("my_test", vec![1, 4]).await;
119+
println!("Delete points result is {:?}", r);
120+
121+
println!(
122+
"The collection size is {}",
123+
client.collection_info("my_test").await
124+
);
125+
126+
let q = vec![0.2, 0.1, 0.9, 0.7];
127+
let r = client.search_points("my_test", q, 2, None).await;
128+
println!("Search result points are {:?}", r);
129+
Ok(())
130+
}
131+
```
132+

‎docs/develop/rust/database/redis_driver.md

Lines changed: 45 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ WasmEdge provides a Redis driver for Rust developers, enabling developers to bui
88

99
<!-- prettier-ignore -->
1010
:::note
11-
Before we start, ensure [you have Rust and WasmEdge installed](../setup.md).
11+
Before we start, [you need to have Rust and WasmEdge installed](../setup.md).
12+
Make sure that you read the [special notes on networking apps](../setup#special-notes) especially if you are compiling Rust programs on a Mac.
1213
:::
1314

1415
## Run the example
@@ -20,15 +21,53 @@ git clone https://github.com/WasmEdge/wasmedge-db-examples
2021
cd wasmedge-db-examples/redis
2122

2223
# Compile the rust code into WASM
23-
cargo build --target wasm32-wasi --release
24+
RUSTFLAGS="--cfg wasmedge --cfg tokio_unstable" cargo build --target wasm32-wasi --release
2425

2526
# Execute Redis command against a Redis instance at redis://localhost/
2627
wasmedge --env "REDIS_URL=redis://localhost/" target/wasm32-wasi/release/wasmedge-redis-client-examples.wasm
2728
```
2829

30+
## Configuration
31+
32+
In order to compile the `redis` and `tokio` crates, we will need to apply patches to add WasmEdge-specific socket APIs to those crates in `Cargo.toml`.
33+
34+
```rust
35+
[patch.crates-io]
36+
tokio = { git = "https://github.com/second-state/wasi_tokio.git", branch = "v1.36.x" }
37+
38+
[dependencies]
39+
anyhow = "1.0"
40+
chrono = { version = "0.4", features = ["serde"] }
41+
tokio = { version = "1", features = ["full"] }
42+
redis = { version = "0.25.4", default-features = false, features = [
43+
"tokio-comp",
44+
] }
45+
```
46+
2947
## Code explanation
3048

31-
<!-- prettier-ignore -->
32-
:::info
33-
Work in Progress
34-
:::
49+
The following program uses the `redis` crate to access a Redis server through its connection URL.
50+
It gets the current time, saves the timestamp object to the Redis server, and then reads it back for
51+
display on the console.
52+
53+
```rust
54+
#[tokio::main(flavor = "current_thread")]
55+
async fn main() -> Result<()> {
56+
// connect to redis
57+
let client = redis::Client::open(&*get_url()).unwrap();
58+
let mut con = client.get_multiplexed_async_connection().await.unwrap();
59+
60+
let time = format!("{}", chrono::Utc::now());
61+
// throw away the result, just make sure it does not fail
62+
let _: () = con.set("current_time", time).await.unwrap();
63+
64+
// read back the key and return it. Because the return value
65+
// from the function is a result for String, this will automatically
66+
// convert into one.
67+
let value: String = con.get("current_time").await.unwrap();
68+
println!("Successfully GET `time`: {}", value);
69+
70+
Ok(())
71+
}
72+
```
73+

0 commit comments

Comments
 (0)