Help with preupdate_hook internals

[dj-jqpub]

Hello!

I'd love to use grdb for my personal project. If I'm not mistaken, grdb makes use of the preupdate_hook extension of sqlite.
I understand that it provides a lot of great functionality. In my case, however, it is a roadblock, since I make use of the hook through a session wrapper I'm writing.

I wouldn't dare ask to make changes to the codebase proper to get around this obstacle (that is, for me). I understand using the session api is a rather rare use case.

Rather, I would like to ask those who are knowledgeable with the internals, to give me some direction/pointers/tips on how I could, perhaps using custom swift flags and/or some other mechanism and where I have to look, to disable preupdate_hook usage, along with, of course, all the functionality it provides so that I can fork grdb. I'm aware of the trade-offs and what will be lost but I don't have much choice in the matter.

Thanks in advance for any help anyone can provide!

[groue]

Hello @dj-jqpub,

You'll find a similar question in #1608

Unless I'm mistaken, the pre-update hook is not used by default in GRDB. But one can't reasonably use the pre-update hook without the commit and rollback hooks, owned by GRDB.

I should also mention the authorization hook. It makes it possible to discard update events that are rollbacked in an intermediate savepoint. Of course GRDB owns it as well.

If you want to install your own hooks with the SQLite C interface, you do indeed find GRDB on your way. Are we on the same track?

---

The root problem is that SQLite APIs do not allow the installation of several hooks. All libraries that need a given hook are, to be short, mutually incompatible.

There is currently no way to build GRBD without its hooks. The work needed to perform a clean cut (i.e. remove or adapt all public and internal apis that depend on those hooks) has to be done from zero (well, it is already done for SQLITE_ENABLE_PREUPDATE_HOOK).

Honestly I'm not sure the authorization hook can be removed. It is used, for example, in the cancellation handling of async database accesses. And as long as GRDB owns it, you won't be able to deal with updates performed within a rollbacked savepoint.

I can answer specific questions if you need help.

Meanwhile, GRDB exposes Swift APIs for accessing the pre-update hook: https://swiftpackageindex.com/groue/grdb.swift/documentation/grdb/transactionobserver. However, building your personal project on top of them would introduce a hard dependency on GRDB.

[dj-jqpub]

Sorry, I should clarify, I don't want to install my own hooks, I'd like to wrap the sqlite session extension api (which makes use of pre update hook).
However, if GRDB does not use this hook by default, then there is something else at play. Going from the session docs, I don't see why any of the other hooks that GRDB implements would cause any issues with wrapping the session extension api. Looks like I'll have to double back and re-check what other issues may be lurking in the background.

Appreciate the quick reply and keep up the great work on GRDB!

[groue]

Thanks for the investigations! I wonder why SQLITE_ENABLE_PREUPDATE_HOOK is enabled in your GRDB build, with the consequence that GRDB installs the pre-update hook. It is not set by default in Package.swift.

[dj-jqpub]

I use the custom sqlite build feature you have because I have to use SQLITE_ENABLE_PREUPDATE_HOOK and SQLITE_ENABLE_SESSION flags to get sesssion api support. What I neglected to notice on the page CustomSQLiteBuilds.md was where it mentions SQLite Pre-Update Hooks, which is used by the TransactionObserver, if the flag is enabled Support-for-SQLite-Pre-Update-Hooks.

So I guess the question is:
Aside from the code then not having access to this function:

func databaseWillChange(with event: DatabasePreUpdateEvent)

which then, as a consequence, will not notify the observer of willChange,
is there anything else that this affects, specifically GRDBQuery? I would love to make
use of that functionality as well, if possible.

Please forgive me if I've omitted anything. :)

[groue]

There are multiple questions here.

I use the custom sqlite build feature you have because I have to use SQLITE_ENABLE_PREUPDATE_HOOK and SQLITE_ENABLE_SESSION flags to get sesssion api support.

OK. The doc says:

SQLiteLib-USER.xcconfig: this file sets the extra SQLite compilation flags.
GRDBCustomSQLite-USER.xcconfig: this file lets GRDB know about extra compilation flags, and enables extra GRDB APIs.

You may thus try activate SQLITE_ENABLE_SNAPSHOT in SQLiteLib-USER.xcconfig, but not in GRDBCustomSQLite-USER.xcconfig. This should compile SQLite with the right option, but prevent GRDB from installing its own pre-update hook.

is there anything else that this affects, specifically GRDBQuery?

I'm afraid that GRDBQuery requires the SPM version of GRDB, and won't work with the GRDBCustom target.

GRDBQuery is not required for developing SwiftUI apps, though. You can look at the GRDB demo app.

[dj-jqpub]

Cool. Thank you, I will give that a shot! I would hate to have to fork the project just for my silly little session wrapper. :)

In regards to GRDBQuery, I have been so caught up with this session issue that I forgot that it actually works. I'll admit, I didn't bring it into the project via SPM but by cloning the repo and referencing it in my project, if I recall correctly. I'll have to take a look at the commit logs though because I don't off-hand remember what else I may have done to make it work! :D

Anyway, thanks again! You've been prompt and extremely helpful! There's a donation heading your way in the coming days!

[groue]

Thank you very much @dj-jqpub 🙏 My main concern is that you find a solution, and that this story can help other GRDB users :-)