Storage model of shared document nodes (subgraphs?) in the database

[michael]

I'm looking for a simple and fast way to store both, self-contained and shared subgraphs in the database.

First, there needs to be a way to mark nodes / or subgraphs as shared, which means they'll not be stored in-place but as its own document (even though they are not supposed to be viewed independently).

The first use-case for this is storing the nav of a page, so it can be reused from all the other pages.

{
  id: 'page_1',
  shared: true,
  type: 'page',
  nav: 'nav_1',
  body: ['hero_1', 'paragraph_1'],
}

For the editable website v2 release, I'll just hardwire this (no UI needed for creating/selecting global navigations), but I still want it to be flexible for later (shared story nodes, etc.).

So importantly, any knowledge about shared nodes (and storage) lives outside of Svedit, the Svedit schema must not know about this. This lives on a separate layer, and I'll first develop this for editable website 2.

Marking nodes as shared

My first thought was to mark certain node types as shared, but that's not flexible enough. E.g. To stick with the nav use-case, I might have pages who's navigation can be shared by other pages. But there might also be pages, who's menu should only be used there and nowhere else. So the nav only lives inside the page. And should also be stored along with the page, rather than a separate document / subgraph.

So I think the easiest way to mark subgraphs as shared is to add a property shared: boolean to all nodes that are sharable (which signals that they will be broken out into their own document on the next save). So technically you are storing a hint in a Svedit node, but for Svedit this is just an abribtray property. However your app (and storage layer) will know how to interpret it.

Lifecycle (example)

Let's assume you create a new page, and let's assume it's empty. No body, no nav. Now, I'd create a nav and add some links, my first menu. I'll add some paragraphs to my page (the actual content). Now I hit save. (So at first everything gets stored as one self-contained document, in one database row, the nodes being serialized as JSON).

Next, you want to create a second page. Now you have to do everything again. Not good.

So you go back to the original page you created, you select the nav and in the contextual toolbar you enable the shared flag. This will make this nav available in other places as well.

Okay, let's create the second page again, now when adding a nav you'll be prompted if you want to reuse an existing nav, or create a new one. Great, so you select the existing one. You can edit this nav in place. If you hit save, the nav will be updated for the current page, but also the other page where it is used.

• NOTE: All the UI-flows related to shared subgraphs are part of the app. Svedit doesn't know about this stuff at all. Svedit just sees a self-contained document and makes it editable.
• NOTE: As you see creating/selecting/maintaining shared nodes involves quite a bit of a UI workflow. Mostly for the reason of scope, I'll with a fixed nav/footer for editable website v2, and deal with a more complete shared nodes solution in editable website v3 **

Storage

I don't think storing each flat node separately in a database row is the best way. It will lead to a lot of lookups just to load a single document, and most nodes will be referenced exactly once (e.g. think of a paragraph, list item, etc.). But we can get the best of both worlds by storing documents as subgraphs. The documents don't need to be complete. E.g. say your page references a shared nav, the nav will be broken out into its own document (created if it doesn't exist, or updated if it's already there). Then the nav parts will miss in your page, but when you load/request a page, those links can be traversed and a single document can be returned that holds all the nodes. This brings database requests to a handful, and it doesn't cost us any flexibility. All rows in a documents table in SQLite are then either a top-level document like page, or a shared node (e.g. nav, or a story)

Limitations/Restrictions

• A shared node/subgraph/document must not reference another shared node/subgraph/document. It will get unecessarily complicated UI-wise, but also hard for the data layer to stay robust.
• No cyclic dependencies, we don't want to point a node that points back to the same node.
  • NOTE: this doesn't mean we can't link to a page that links back to that original page. I'm talking only about referenced nodes (=transclusions)
• You can only address document nodes, not any node on the top level (e.g. you can address a page document, and you can address a nav document, that is embedded in that page. However you can not address a paragraph node inside that page.
• However, these are just restrictions for the app-specific storage model we are choosing here, for practical reasons (readability, and performance - we don't need so many lookups and database queries).

Vocab

• a node a the lowest-level building block (e.g. a paragraph)
• a document is a collection of nodes (a subgraph) (e.g. page, or a nav embedded in a page)
  • on the storage layer a document doesn't include other documents, but merely reference them
  • on the display layer (in Svedid-land) you always view a single doc that contains all sub-documents. So on the client you edit a single-doc which gets split up into multiple docs on the storage layer

[johannesmutter]

I think your approach of extracting subgraphs that are explicitly marked as shared, while keeping all other nodes serialized together within a single document is clever. It strikes a practical balance between composability and simplicity for your current editable website scope.

That said, I wonder if this introduces additional conceptual and implementation overhead.
Once you need to distinguish between “shared” and “non-shared” nodes, you end up maintaining extra logic for:

• detecting and extracting those nodes,
• handling save/merge behaviour,
• and resolving references on load in additional variations.

This adds another layer of schema semantics (“is this node shared?”), which might otherwise be unnecessary in a model where all nodes are inherently addressable and shareable.
In such a model, sharing or embedding becomes a matter of referencing an existing node ID.
No special “shared flag” or extraction pass needed.

---

Performance considerations

The main argument for subgraph-based storage seems to be reducing the number of database lookups when loading a full document.
However, given that SQLite excels at concurrent reads and operates locally (no network latency), the overhead of multiple small lookups is effectively negligible. SQLite can handle tens of thousands of concurrent read queries efficiently, because reads are non-blocking and use shared locks. So even if each node were stored as its own row, fetching nodes for a page would still be very fast.

So performance alone may not justify the added conceptual complexity of distinguishing “shared” vs “inline” storage, especially in an embedded SQLite setup.

---

What we lose when nodes are not individually addressable

By not making individual nodes directly addressable (only full documents), we give up some interesting capabilities:

• In-document navigation such as a generated table of contents.
• Single-page or long-form layouts with top-level anchors for sections.
• Deep linking to specific parts of a page using #fragment URLs or similar.
• Inline transclusions — embedding a specific node (e.g., a quote, figure, or paragraph) across routes without having to promote it to a separate document first.

Some of these can be partially simulated (e.g. text-fragment URLs in browsers), but native addressability would make them straightforward. I don't know of a single user that's aware of the fragment copy command in the context menu.

---

Collaboration and diffing

In Svedit, the editor always operates on a single, fully assembled document state. That's great and makes the editing part easy.

That said, if nodes were individually addressable, collaboration and diffing could be simpler at the application layer.
For example, each node could be independently locked, edited, or streamed, making it easier to implement granular updates, real-time collaboration, and merge logic.
With the current single-document model, queries for these operations are more complex because edits always touch the entire document JSON, even though Svedit itself remains agnostic and continues to operate on a unified view.

---

Transclusions and UX flow

Another small drawback of non-addressable nodes is friction in cross-document reuse.
Imagine editing one route and wanting to embed a paragraph or block from another route: you’d first need to explicitly mark it as “shared” before it becomes referenceable.
That introduces painful extra context switching for the user and more logic for the app layer.

---

I think your current approach makes perfect sense for page-like content where users primarily edit whole documents and where reuse is the exception. If I were you I would in your case not even have shared nodes. e.g. editing the nav would be a separate Svedit document editing session. But one that is loaded and present on all routes.

But if the long-term goal is to support more networked, stream-based, or collaborative content models, treating every node as addressable from the start will drastically simplify the architecture, both conceptually and operationally, without noticeable performance cost in SQLite.