Skip to content

feat(wasm): add feature support for indexedb and sqlite to matrix-sdk-ffi #5245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions bindings/matrix-sdk-ffi/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,29 @@ All notable changes to this project will be documented in this file.

## [Unreleased] - ReleaseDate

### Features
- Configuration of the session store has been changed to allow for use of either IndexedDb
or SQLite on relevant platforms. These can be enabled via features, `indexeddb` or `sqlite`.

Previously the code to configure a sqlite session would look something like this in a host language:
```
builder
.sessionPaths("data_path", "cache_path")
.passphrase("foobar")
```
With the new system, a helper object is exposed for either SQLite or IndexedDB to group those settings.
```
builder
.session_store_sqlite(
SqliteSessionStoreBuilder.new({ dataPath: "data_path", cachePath: "cache_path" })
.passphrase("foobar")
)
```

The following methods from `ClientBuilder` have been moved onto `SqliteSessionStoreBuilder`:
`session_paths`, `session_passphrase`, `session_pool_max_size`, `session_cache_size`, and `session_journal_size_limit`.


### Refactor

- Adjust features in the `matrix-sdk-ffi` crate to expose more platform-specific knobs.
Expand Down
12 changes: 8 additions & 4 deletions bindings/matrix-sdk-ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,22 @@ crate-type = [

[features]
default = ["bundled-sqlite", "unstable-msc4274"]
bundled-sqlite = ["matrix-sdk/bundled-sqlite"]
bundled-sqlite = ["sqlite", "matrix-sdk/bundled-sqlite"]
unstable-msc4274 = ["matrix-sdk-ui/unstable-msc4274"]
# Use indexeddb for session storage, supported only on Wasm platforms.
indexeddb = ["matrix-sdk/indexeddb"]
# Use sqlite for session storage, not supported on Wasm platforms.
sqlite = ["matrix-sdk/sqlite"]
# Required when targeting a Javascript environment, like Wasm in a browser.
js = ["matrix-sdk-ui/js"]
js = ["matrix-sdk-ui/js", "uuid/js"]
# Use the TLS implementation provided by the host system, necessary on iOS and Wasm platforms.
native-tls = ["matrix-sdk/native-tls", "sentry?/native-tls"]
# Use Rustls as the TLS implementation, necessary on Android platforms.
rustls-tls = ["matrix-sdk/rustls-tls", "sentry?/rustls"]
# Enable sentry error monitoring, not compatible with Wasm platforms.
# Enable sentry error monitoring, not compatible on Wasm platforms.
sentry = ["dep:sentry", "dep:sentry-tracing"]


[dependencies]
anyhow.workspace = true
as_variant.workspace = true
Expand All @@ -50,7 +55,6 @@ matrix-sdk = { workspace = true, features = [
"experimental-widgets",
"markdown",
"socks",
"sqlite",
"uniffi",
] }
matrix-sdk-common.workspace = true
Expand Down
4 changes: 3 additions & 1 deletion bindings/matrix-sdk-ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Given the number of platforms targeted, we have broken out a number of features
### Functionality
- `sentry`: Enable error monitoring using Sentry, not supports on Wasm platforms.
- `bundled-sqlite`: Use an embedded version of sqlite instead of the system provided one.
- `sqlite`: Use sqlite for session storage, not available on Wasm platforms.
- `indexeddb`: Use IndexedDb for session storage, only available on Wasm platforms.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same suggestions as in the Cargo.toml file apply.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what this means

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These sentences are copied-pasted from the Cargo.toml comment. I made suggestions on Cargo.toml. So please re-apply them here too :-).


### Unstable specs
- `unstable-msc4274`: Adds support for gallery message types, which contain multiple media elements.
Expand All @@ -22,7 +24,7 @@ Each supported target should use features to select the relevant TLS system. He

- Android: `"bundled-sqlite,unstable-msc4274,rustls-tls,sentry"`
- iOS: `"bundled-sqlite,unstable-msc4274,native-tls,sentry"`
- Javascript/Wasm: `"unstable-msc4274,native-tls"`
- Javascript/Wasm: `"indexeddb,unstable-msc4274,native-tls"`

### Swift/iOS sync

Expand Down
26 changes: 25 additions & 1 deletion bindings/matrix-sdk-ffi/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use std::{
env,
error::Error,
path::{Path, PathBuf},
process::Command,
process::{self, Command},
};

use vergen::EmitBuilder;
Expand Down Expand Up @@ -56,7 +56,31 @@ fn get_clang_major_version(clang_path: &Path) -> String {
clang_version.split('.').next().expect("could not parse clang output").to_owned()
}

fn env_is_set(var_name: &str) -> bool {
env::var_os(var_name).is_some()
}

fn ensure(cond: bool, err: &str) {
if !cond {
eprintln!(
"\n\
┏━━━━━━━━{pad}━┓\n\
┃ error: {err} ┃\n\
┗━━━━━━━━{pad}━┛\n\
",
pad = "━".repeat(err.len()),
);
process::exit(1);
}
}

fn main() -> Result<(), Box<dyn Error>> {
let sqlite_set = env_is_set("CARGO_FEATURE_SQLITE");
let indexeddb_set = env_is_set("CARGO_FEATURE_INDEXEDDB");
ensure(
sqlite_set || indexeddb_set,
"one of the features 'sqlite' or 'indexeddb' must be enabled",
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You must also ensure that they are not both activated at the same time, it's a XOR:

Suggested change
ensure(
sqlite_set || indexeddb_set,
"one of the features 'sqlite' or 'indexeddb' must be enabled",
);
ensure(
sqlite_set ^ indexeddb_set,
"one of the features 'sqlite' or (exclusive) 'indexeddb' must be enabled",
);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think they are necessarily exclusive.

At the moment they are because of platform limitations, but the code to enable IndexeDB + SQLite at the same time would work perfectly fine someday when there is a bundled sqlite solution on web (not an impossibility).

Copy link
Member

@Hywan Hywan Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt the Client can have 2 stores, so, yeah, these features are mutually exclusive 🙂.
Edit: except if you use for example IndexedDB for one store, like the state store, and SQLite for the crypto store, but it seems pretty weird. Please, make it mutually exclusive for the moment, as having them two at the same time isn't supported right now, nor tested.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the compilation layer, you can definitely have two stores enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated this as you requested, although I do still disagree with your reasoning.

The builder system written here is explicitly designed to choose between the two, nothing is mutually exclusive about the feature as written. The fact that indexedb and sqlite have platform restrictions is an orthogonal issue entirely, and doesn't seem worth guarding against at the feature level.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the compilation layer, you can have IDB and SQLite if SQLite can be compiled to Wasm. Right now:

$ cargo build --target wasm32-unknown-unknown --package matrix-sdk-sqlite

does not compile.

I prefer to be safe and to prevent someone doing the wrong thing.

If one day a user is asking for SQLite+Wasm support, I'll be happy to revisit this. For the moment, I prefer to provide a satisfying developer experience.

setup_x86_64_android_workaround();
uniffi::generate_scaffolding("./src/api.udl").expect("Building the UDL file failed");
EmitBuilder::builder().git_sha(true).emit()?;
Expand Down
176 changes: 46 additions & 130 deletions bindings/matrix-sdk-ffi/src/client_builder.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use std::{fs, num::NonZeroUsize, path::Path, sync::Arc, time::Duration};
use std::{num::NonZeroUsize, sync::Arc, time::Duration};

use futures_util::StreamExt;
use matrix_sdk::{
Expand All @@ -12,11 +12,10 @@ use matrix_sdk::{
VersionBuilderError,
},
Client as MatrixClient, ClientBuildError as MatrixClientBuildError, HttpError, IdParseError,
RumaApiError, SqliteStoreConfig,
RumaApiError,
};
use ruma::api::error::{DeserializationError, FromHttpResponseError};
use tracing::{debug, error};
use zeroize::Zeroizing;

use super::client::Client;
use crate::{
Expand All @@ -26,6 +25,7 @@ use crate::{
helpers::unwrap_or_clone_arc,
qr_code::{HumanQrLoginError, QrCodeData, QrLoginProgressListener},
runtime::get_runtime_handle,
session_store::{SessionStoreConfig, SessionStoreResult},
task_handle::TaskHandle,
};

Expand Down Expand Up @@ -108,11 +108,7 @@ impl From<ClientError> for ClientBuildError {

#[derive(Clone, uniffi::Object)]
pub struct ClientBuilder {
session_paths: Option<SessionPaths>,
session_passphrase: Zeroizing<Option<String>>,
session_pool_max_size: Option<usize>,
session_cache_size: Option<u32>,
session_journal_size_limit: Option<u32>,
session_store: Option<SessionStoreConfig>,
system_is_memory_constrained: bool,
username: Option<String>,
homeserver_cfg: Option<HomeserverConfig>,
Expand All @@ -138,11 +134,7 @@ impl ClientBuilder {
#[uniffi::constructor]
pub fn new() -> Arc<Self> {
Arc::new(Self {
session_paths: None,
session_passphrase: Zeroizing::new(None),
session_pool_max_size: None,
session_cache_size: None,
session_journal_size_limit: None,
session_store: None,
system_is_memory_constrained: false,
username: None,
homeserver_cfg: None,
Expand Down Expand Up @@ -193,73 +185,6 @@ impl ClientBuilder {
Arc::new(builder)
}

/// Sets the paths that the client will use to store its data and caches.
/// Both paths **must** be unique per session as the SDK stores aren't
/// capable of handling multiple users, however it is valid to use the
/// same path for both stores on a single session.
///
/// Leaving this unset tells the client to use an in-memory data store.
pub fn session_paths(self: Arc<Self>, data_path: String, cache_path: String) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_paths = Some(SessionPaths { data_path, cache_path });
Arc::new(builder)
}

/// Set the passphrase for the stores given to
/// [`ClientBuilder::session_paths`].
pub fn session_passphrase(self: Arc<Self>, passphrase: Option<String>) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_passphrase = Zeroizing::new(passphrase);
Arc::new(builder)
}

/// Set the pool max size for the SQLite stores given to
/// [`ClientBuilder::session_paths`].
///
/// Each store exposes an async pool of connections. This method controls
/// the size of the pool. The larger the pool is, the more memory is
/// consumed, but also the more the app is reactive because it doesn't need
/// to wait on a pool to be available to run queries.
///
/// See [`SqliteStoreConfig::pool_max_size`] to learn more.
pub fn session_pool_max_size(self: Arc<Self>, pool_max_size: Option<u32>) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_pool_max_size = pool_max_size
.map(|size| size.try_into().expect("`pool_max_size` is too large to fit in `usize`"));
Arc::new(builder)
}

/// Set the cache size for the SQLite stores given to
/// [`ClientBuilder::session_paths`].
///
/// Each store exposes a SQLite connection. This method controls the cache
/// size, in **bytes (!)**.
///
/// The cache represents data SQLite holds in memory at once per open
/// database file. The default cache implementation does not allocate the
/// full amount of cache memory all at once. Cache memory is allocated
/// in smaller chunks on an as-needed basis.
///
/// See [`SqliteStoreConfig::cache_size`] to learn more.
pub fn session_cache_size(self: Arc<Self>, cache_size: Option<u32>) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_cache_size = cache_size;
Arc::new(builder)
}

/// Set the size limit for the SQLite WAL files of stores given to
/// [`ClientBuilder::session_paths`].
///
/// Each store uses the WAL journal mode. This method controls the size
/// limit of the WAL files, in **bytes (!)**.
///
/// See [`SqliteStoreConfig::journal_size_limit`] to learn more.
pub fn session_journal_size_limit(self: Arc<Self>, limit: Option<u32>) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_journal_size_limit = limit;
Arc::new(builder)
}

/// Tell the client that the system is memory constrained, like in a push
/// notification process for example.
///
Expand Down Expand Up @@ -425,50 +350,23 @@ impl ClientBuilder {
inner_builder.cross_process_store_locks_holder_name(holder_name.clone());
}

let store_path = if let Some(session_paths) = &builder.session_paths {
// This is the path where both the state store and the crypto store will live.
let data_path = Path::new(&session_paths.data_path);
// This is the path where the event cache store will live.
let cache_path = Path::new(&session_paths.cache_path);

debug!(
data_path = %data_path.to_string_lossy(),
event_cache_path = %cache_path.to_string_lossy(),
"Creating directories for data (state and crypto) and cache stores.",
);

fs::create_dir_all(data_path)?;
fs::create_dir_all(cache_path)?;

let mut sqlite_store_config = if builder.system_is_memory_constrained {
SqliteStoreConfig::with_low_memory_config(data_path)
} else {
SqliteStoreConfig::new(data_path)
};

sqlite_store_config =
sqlite_store_config.passphrase(builder.session_passphrase.as_deref());

if let Some(size) = builder.session_pool_max_size {
sqlite_store_config = sqlite_store_config.pool_max_size(size);
}

if let Some(size) = builder.session_cache_size {
sqlite_store_config = sqlite_store_config.cache_size(size);
}

if let Some(limit) = builder.session_journal_size_limit {
sqlite_store_config = sqlite_store_config.journal_size_limit(limit);
let mut store_path = None;
if let Some(session_store) = builder.session_store {
match session_store.build()? {
#[cfg(feature = "indexeddb")]
SessionStoreResult::IndexedDb { name, passphrase } => {
inner_builder = inner_builder.indexeddb_store(&name, passphrase.as_deref());
}
#[cfg(feature = "sqlite")]
SessionStoreResult::Sqlite { config, cache_path, store_path: data_path } => {
inner_builder = inner_builder
.sqlite_store_with_config_and_cache_path(config, Some(cache_path));
store_path = Some(data_path);
}
}

inner_builder = inner_builder
.sqlite_store_with_config_and_cache_path(sqlite_store_config, Some(cache_path));

Some(data_path.to_owned())
} else {
debug!("Not using a store path.");
None
};
debug!("Not using a session store.")
}

// Determine server either from URL, server name or user ID.
inner_builder = match builder.homeserver_cfg {
Expand Down Expand Up @@ -646,14 +544,32 @@ impl ClientBuilder {
}
}

/// The store paths the client will use when built.
#[derive(Clone)]
struct SessionPaths {
/// The path that the client will use to store its data.
data_path: String,
/// The path that the client will use to store its caches. This path can be
/// the same as the data path if you prefer to keep everything in one place.
cache_path: String,
#[cfg(feature = "sqlite")]
#[matrix_sdk_ffi_macros::export]
impl ClientBuilder {
/// Tell the client to use sqlite to store session data.
pub fn session_store_sqlite(
self: Arc<Self>,
config: Arc<crate::session_store::SqliteSessionStoreBuilder>,
) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_store = Some(SessionStoreConfig::Sqlite(config.as_ref().clone()));
Arc::new(builder)
}
}

#[cfg(feature = "indexeddb")]
#[matrix_sdk_ffi_macros::export]
impl ClientBuilder {
/// Tell the client to use IndexedDb to store session data.
pub fn session_store_indexeddb(
self: Arc<Self>,
config: Arc<crate::session_store::IndexedDbSessionStoreBuilder>,
) -> Arc<Self> {
let mut builder = unwrap_or_clone_arc(self);
builder.session_store = Some(SessionStoreConfig::IndexedDb(config.as_ref().clone()));
Arc::new(builder)
}
}

#[derive(Clone, uniffi::Record)]
Expand Down
1 change: 1 addition & 0 deletions bindings/matrix-sdk-ffi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ mod room_member;
mod room_preview;
mod ruma;
mod runtime;
mod session_store;
mod session_verification;
mod sync_service;
mod task_handle;
Expand Down
Loading