feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli #3917

djarb · 2025-07-01T20:25:09Z

Does your PR solve an issue?

Is this a breaking change?

This is not a breaking change. All new behaviors are toggled by the presence of a field in the sqlx.toml file which did not previously exist.

https://en.wikipedia.org/wiki/Byte_order_mark

simplifies usage while still making parsing optional for less generated code

…qlx.toml`

…ed-crates`

…oes not already exist

…SQLite extensions

Doc formatting tweak Co-authored-by: Josh McKinney <joshka@users.noreply.github.com>

https://en.wikipedia.org/wiki/Byte_order_mark

simplifies usage while still making parsing optional for less generated code

…qlx.toml`

…ed-crates`

…g used and causing a compile error

abonander

Also concerned about the proliferation of the sqlx-toml feature.

The Config type is still defined without it, the parsing is just short-circuited to do nothing. It shouldn't be necessary to cfg a ton of stuff out on it.

sqlx-sqlite/src/lib.rs

sqlx-sqlite/src/any.rs

sqlx-core/src/config/reference.toml

djarb · 2025-07-02T11:19:29Z

Also concerned about the proliferation of the sqlx-toml feature.

The Config type is still defined without it, the parsing is just short-circuited to do nothing. It shouldn't be necessary to cfg a ton of stuff out on it.

No problem, I'll pull out most of the conditional compilation. I was just trying to respect the feature flag.

…flag use

etodanik · 2025-07-02T18:36:25Z

This is incredibly useful! Stuck on exactly this feature missing. Thanks @djarb ❤️

sqlx-core/src/config/reference.toml

abonander · 2025-07-08T08:18:44Z

sqlx-cli/src/lib.rs

+    retry_connect_errors(opts, move |url| {
+        // This only handles the default case. For good support of
+        // the new command line options, we need to work out some
+        // way to make the appropriate ConfigOpt available here. I
+        // suspect that that infrastructure would be useful for
+        // other things in the future, as well, but it also seems
+        // like an extensive and intrusive change.
+        //
+        // On the other hand, the compile-time checking macros
+        // can't be configured to use a different config file at
+        // all, so I believe this is okay for the time being.
+        let config = Some(std::path::PathBuf::from("sqlx.toml")).and_then(|p| {
+            if p.exists() {
+                Some(p)
+            } else {
+                None
+            }
+        });


I've got a better idea of how to do this but it's a nontrivial change so I'm going to go ahead and merge. Just know that this might look a bit different in the final release.

…unchbadge#3917) * feat: create `sqlx.toml` format * feat: add support for ignored_chars config to sqlx_core::migrate * chore: test ignored_chars with `U+FEFF` (ZWNBSP/BOM) https://en.wikipedia.org/wiki/Byte_order_mark * refactor: make `Config` always compiled simplifies usage while still making parsing optional for less generated code * refactor: add origin information to `Column` * feat(macros): implement `type_override` and `column_override` from `sqlx.toml` * refactor(sqlx.toml): make all keys kebab-case, create `macros.preferred-crates` * feat: make macros aware of `macros.preferred-crates` * feat: make `sqlx-cli` aware of `database-url-var` * feat: teach macros about `migrate.table-name`, `migrations-dir` * feat: teach macros about `migrate.ignored-chars` * feat: teach `sqlx-cli` about `migrate.defaults` * feat: teach `sqlx-cli` about `migrate.migrations-dir` * feat: teach `sqlx-cli` about `migrate.table-name` * feat: introduce `migrate.create-schemas` * WIP feat: create multi-tenant database example * SQLite extension loading via sqlx.toml for CLI and query macros * fix: allow start_database to function when the SQLite database file does not already exist * Added example demonstrating migration and compile-time checking with SQLite extensions * remove accidentally included db file * Update sqlx-core/src/config/common.rs Doc formatting tweak Co-authored-by: Josh McKinney <joshka@users.noreply.github.com> * feat: create `sqlx.toml` format * feat: add support for ignored_chars config to sqlx_core::migrate * chore: test ignored_chars with `U+FEFF` (ZWNBSP/BOM) https://en.wikipedia.org/wiki/Byte_order_mark * refactor: make `Config` always compiled simplifies usage while still making parsing optional for less generated code * refactor: add origin information to `Column` * feat(macros): implement `type_override` and `column_override` from `sqlx.toml` * refactor(sqlx.toml): make all keys kebab-case, create `macros.preferred-crates` * feat: make macros aware of `macros.preferred-crates` * feat: make `sqlx-cli` aware of `database-url-var` * feat: teach macros about `migrate.table-name`, `migrations-dir` * feat: teach macros about `migrate.ignored-chars` * feat: teach `sqlx-cli` about `migrate.defaults` * feat: teach `sqlx-cli` about `migrate.migrations-dir` * feat: teach `sqlx-cli` about `migrate.table-name` * feat: introduce `migrate.create-schemas` * fix(postgres): don't fetch `ColumnOrigin` for transparently-prepared statements * feat: progress on axum-multi-tenant example * feat(config): better errors for mislabeled fields * WIP feat: filling out axum-multi-tenant example * feat: multi-tenant example No longer Axum-based because filling out the request routes would have distracted from the purpose of the example. * chore(ci): test multi-tenant example * fixup after merge * fix: CI, README for `multi-tenant` * fix: clippy warnings * fix: multi-tenant README * fix: sequential versioning inference for migrations * fix: migration versioning with explicit overrides * fix: only warn on ambiguous crates if the invocation relies on it * fix: remove unused imports * fix: `sqlx mig add` behavior and tests * fix: restore original type-checking order * fix: deprecation warning in `tests/postgres/macros.rs` * feat: create postgres/multi-database example * fix: examples/postgres/multi-database * fix: cargo fmt * chore: add tests for config `migrate.defaults` * fix: sqlx-cli/tests/add.rs * feat(cli): add `--config` override to all relevant commands * chore: run `sqlx mig add` test with `RUST_BACKTRACE=1` * fix: properly canonicalize config path for `sqlx mig add` test * fix: get `sqlx mig add` test passing * fix(cli): test `migrate.ignored-chars`, fix bugs * feat: create `macros.preferred-crates` example * fix(examples): use workspace `sqlx` * fix: examples * fix: run `cargo fmt` * fix: more example fixes * fix(ci): preferred-crates setup * fix: axum-multi-tenant example locked to specific sqlx version * import anyhow::Context trait in sqlx-cli/src/lib.rs since it was being used and causing a compile error * rebased on upstream/main * make cargo fmt happy * make clippy happy * make clippy happier still * fix: improved error reporting, added parsing test, removed sqlx-toml flag use * switched to kebab-case for the config key * switched to kebab-case for the config key --------- Co-authored-by: Austin Bonander <austin.bonander@gmail.com> Co-authored-by: Josh McKinney <joshka@users.noreply.github.com>

abonander and others added 30 commits July 1, 2025 02:58

feat: create sqlx.toml format

39e5a5a

feat: add support for ignored_chars config to sqlx_core::migrate

fdce6b8

chore: test ignored_chars with U+FEFF (ZWNBSP/BOM)

44db470

https://en.wikipedia.org/wiki/Byte_order_mark

refactor: make Config always compiled

62c0c6f

simplifies usage while still making parsing optional for less generated code

refactor: add origin information to Column

d7b6cd2

feat(macros): implement type_override and column_override from `s…

c0ed389

…qlx.toml`

refactor(sqlx.toml): make all keys kebab-case, create `macros.preferr…

ff3b532

…ed-crates`

feat: make macros aware of macros.preferred-crates

0791e06

feat: make sqlx-cli aware of database-url-var

6b0df6d

feat: teach macros about migrate.table-name, migrations-dir

c377de7

feat: teach macros about migrate.ignored-chars

d41aa07

feat: teach sqlx-cli about migrate.defaults

57c75b4

feat: teach sqlx-cli about migrate.migrations-dir

b4efb9e

feat: teach sqlx-cli about migrate.table-name

528a7d4

feat: introduce migrate.create-schemas

2ec3698

WIP feat: create multi-tenant database example

a8d9ac2

SQLite extension loading via sqlx.toml for CLI and query macros

f415b08

fix: allow start_database to function when the SQLite database file d…

2231769

…oes not already exist

Added example demonstrating migration and compile-time checking with …

6e55a88

…SQLite extensions

remove accidentally included db file

8126ae1

Update sqlx-core/src/config/common.rs

d36b9f9

Doc formatting tweak Co-authored-by: Josh McKinney <joshka@users.noreply.github.com>

feat: create sqlx.toml format

15c837e

feat: add support for ignored_chars config to sqlx_core::migrate

f5ed721

chore: test ignored_chars with U+FEFF (ZWNBSP/BOM)

0a7dfea

https://en.wikipedia.org/wiki/Byte_order_mark

refactor: make Config always compiled

2a35d6a

simplifies usage while still making parsing optional for less generated code

refactor: add origin information to Column

2bed2ee

feat(macros): implement type_override and column_override from `s…

d687017

…qlx.toml`

refactor(sqlx.toml): make all keys kebab-case, create `macros.preferr…

3a22958

…ed-crates`

feat: make macros aware of macros.preferred-crates

a6947b2

feat: make sqlx-cli aware of database-url-var

ff5374e

abonander and others added 15 commits July 1, 2025 04:50

fix: properly canonicalize config path for sqlx mig add test

12b823e

fix: get sqlx mig add test passing

b4cc0d3

fix(cli): test migrate.ignored-chars, fix bugs

2d007c0

feat: create macros.preferred-crates example

5453589

fix(examples): use workspace sqlx

929939a

fix: examples

aa43ced

fix: run cargo fmt

35b6a79

fix: more example fixes

f57ee86

fix(ci): preferred-crates setup

daca7a0

fix: axum-multi-tenant example locked to specific sqlx version

756ee0c

import anyhow::Context trait in sqlx-cli/src/lib.rs since it was bein…

1956256

…g used and causing a compile error

rebased on upstream/main

94c93d8

make cargo fmt happy

89c63ae

make clippy happy

4bf945a

make clippy happier still

6d4db03

djarb changed the title ~~Use sqlx.toml to load extensions while running compile-time checking macros and CLI migrations against SQLite databases~~ Use sqlx.toml to load extensions when running compile-time checking macros and CLI migrations against SQLite databases Jul 1, 2025

abonander requested changes Jul 2, 2025

View reviewed changes

sqlx-sqlite/src/lib.rs Outdated Show resolved Hide resolved

sqlx-sqlite/src/any.rs Outdated Show resolved Hide resolved

sqlx-core/src/config/reference.toml Outdated Show resolved Hide resolved

fix: improved error reporting, added parsing test, removed sqlx-toml …

c04c328

…flag use

djarb requested a review from abonander July 2, 2025 18:40

etodanik reviewed Jul 2, 2025

View reviewed changes

sqlx-core/src/config/reference.toml Outdated Show resolved Hide resolved

abonander requested changes Jul 5, 2025

View reviewed changes

sqlx-core/src/config/reference.toml Outdated Show resolved Hide resolved

djarb added 2 commits July 5, 2025 09:30

switched to kebab-case for the config key

fe9916f

switched to kebab-case for the config key

55567c3

djarb requested a review from abonander July 5, 2025 16:56

abonander reviewed Jul 8, 2025

View reviewed changes

abonander changed the title ~~Use sqlx.toml to load extensions when running compile-time checking macros and CLI migrations against SQLite databases~~ feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli Jul 8, 2025

abonander merged commit f7ef1ed into launchbadge:main Jul 8, 2025
84 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli #3917

feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli #3917

djarb commented Jul 1, 2025 •

edited

Loading

Uh oh!

abonander left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

djarb commented Jul 2, 2025

Uh oh!

etodanik commented Jul 2, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

abonander Jul 8, 2025

Uh oh!

Uh oh!

Uh oh!

feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli #3917

feat(sqlx.toml): support SQLite extensions in macros and sqlx-cli #3917

Conversation

djarb commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Does your PR solve an issue?

Is this a breaking change?

Uh oh!

abonander left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

djarb commented Jul 2, 2025

Uh oh!

etodanik commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abonander Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

djarb commented Jul 1, 2025 •

edited

Loading

etodanik commented Jul 2, 2025 •

edited

Loading