Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 35 additions & 17 deletions modules/ROOT/content-nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -129,25 +129,8 @@
*** xref:database-administration/aliases/manage-aliases-standard-databases.adoc[]
*** xref:database-administration/aliases/manage-aliases-composite-databases.adoc[]
*** xref:database-administration/aliases/remote-database-alias-configuration.adoc[]
** Composite databases
*** xref:database-administration/composite-databases/concepts.adoc[]
*** xref:database-administration/composite-databases/create-composite-databases.adoc[]
*** xref:database-administration/composite-databases/list-composite-databases.adoc[]
*** xref:database-administration/composite-databases/alter-composite-databases.adoc[]
*** xref:database-administration/composite-databases/start-stop-composite-databases.adoc[]
*** xref:database-administration/composite-databases/delete-composite-databases.adoc[]
*** xref:database-administration/composite-databases/querying-composite-databases.adoc[]
*** xref:database-administration/composite-databases/sharding-with-copy.adoc[]
** xref:database-administration/routing-decisions.adoc[]

* xref:database-internals/index.adoc[]
** xref:database-internals/transaction-management.adoc[]
** xref:database-internals/concurrent-data-access.adoc[]
** xref:database-internals/transaction-logs.adoc[]
** xref:database-internals/checkpointing.adoc[]
** xref:database-internals/store-formats.adoc[]
** xref:database-internals/neo4j-admin-store-info.adoc[]

* xref:clustering/index.adoc[]
** xref:clustering/introduction.adoc[]
** Setting up a cluster
Expand Down Expand Up @@ -176,6 +159,41 @@
** xref:clustering/server-syntax.adoc[]
** xref:clustering/glossary.adoc[]

* Scalability
** xref:scalability/concepts.adoc[]
** xref:scalability/scaling-with-neo4j.adoc[]
** Composite databases
*** xref:scalability/composite-databases/concepts.adoc[]
*** xref:scalability/composite-databases/create-composite-databases.adoc[]
*** xref:scalability/composite-databases/list-composite-databases.adoc[]
*** xref:scalability/composite-databases/alter-composite-databases.adoc[]
*** xref:scalability/composite-databases/start-stop-composite-databases.adoc[]
*** xref:scalability/composite-databases/delete-composite-databases.adoc[]
*** xref:scalability/composite-databases/querying-composite-databases.adoc[]
*** xref:scalability/composite-databases/sharding-with-copy.adoc[]
//*** xref:scalability/composite-databases/scaling-with-composite-databases.adoc[]
** Property sharding (Preview feature)
*** xref:scalability/sharded-property-databases/overview.adoc[]
*** xref:scalability/sharded-property-databases/planning-and-sizing.adoc[]
*** xref:scalability/sharded-property-databases/configuration.adoc[]
*** xref:scalability/sharded-property-databases/data-ingestion.adoc[]
*** xref:scalability/sharded-property-databases/starting-stopping-sharded-databases.adoc[]
*** xref:scalability/sharded-property-databases/listing-sharded-databases.adoc[]
*** xref:scalability/sharded-property-databases/altering-sharded-databases.adoc[]
*** xref:scalability/sharded-property-databases/deleting-sharded-databases.adoc[]
*** xref:scalability/sharded-property-databases/role-based-access-control.adoc[]
*** xref:scalability/sharded-property-databases/admin-operations.adoc[]
*** xref:scalability/sharded-property-databases/security.adoc[]
*** xref:scalability/sharded-property-databases/limitations-and-considerations.adoc[]

* xref:database-internals/index.adoc[]
** xref:database-internals/transaction-management.adoc[]
** xref:database-internals/concurrent-data-access.adoc[]
** xref:database-internals/transaction-logs.adoc[]
** xref:database-internals/checkpointing.adoc[]
** xref:database-internals/store-formats.adoc[]
** xref:database-internals/neo4j-admin-store-info.adoc[]

* xref:backup-restore/index.adoc[]
** xref:backup-restore/planning.adoc[]
** xref:backup-restore/modes.adoc[]
Expand Down
Binary file added modules/ROOT/images/scalability/cluster.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hannah has made a slightly better one than this. I wonder if it might be helpful to add some graphics to the property shard databases to show the nodes and relationships with the properties on?

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion modules/ROOT/pages/backup-restore/consistency-checker.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ The following are examples of how to check the consistency of a database, a dump

[NOTE]
====
`neo4j-admin database check` cannot be applied to xref:database-administration/composite-databases/concepts.adoc[Composite databases].
`neo4j-admin database check` cannot be applied to xref:scalability/composite-databases/concepts.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
====

Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/copy-database.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ You can use the `neo4j-admin database copy` command to copy a database, create a
====
* `neo4j-admin database copy` preserves the node IDs (unless `--compact-node-store` is used), but the relationships get new IDs.
* `neo4j-admin database copy` is not supported for use on the `system` database.
* `neo4j-admin database copy` is not supported for use on xref:database-administration/composite-databases/concepts.adoc[Composite databases].
* `neo4j-admin database copy` is not supported for use on xref:scalability/composite-databases/concepts.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
* `neo4j-admin database copy` is an IOPS-intensive process.
For more information, see <<copy-estimating-iops, Estimating the processing time>>.
Expand Down Expand Up @@ -300,7 +300,7 @@ Labels are processed independently, i.e., the filter ignores any node with a lab

[TIP]
====
For a detailed example of how to use `neo4j-admin database copy` to filter out data for sharding a database, see xref:database-administration/composite-databases/sharding-with-copy.adoc[Sharding data with the `copy` command].
For a detailed example of how to use `neo4j-admin database copy` to filter out data for sharding a database, see xref:scalability/composite-databases/sharding-with-copy.adoc[Sharding data with the `copy` command].
====

[[compact-database]]
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/backup-restore/offline-backup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ The command creates a file called _database.dump_ where `database` is the databa

[NOTE]
====
`neo4j-admin database dump` cannot be applied to xref:database-administration/composite-databases/concepts.adoc[Composite databases].
`neo4j-admin database dump` cannot be applied to xref:scalability/composite-databases/concepts.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
====

Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/planning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ The following table summarizes the commands' capabilities and usage.

[NOTE]
====
The Neo4j Admin commands `backup`, `restore`, `dump`, `load`, `copy`, and `check-consistency` are not supported for use on xref:database-administration/composite-databases/concepts.adoc[Composite databases].
The Neo4j Admin commands `backup`, `restore`, `dump`, `load`, `copy`, and `check-consistency` are not supported for use on xref:scalability/composite-databases/concepts.adoc[Composite databases].
They must be run directly on the databases that are associated with that Composite database.
====

Expand All @@ -196,7 +196,7 @@ By default, a database backup includes only the database contents.
If you choose to include metadata, the backup also stores the role-based access control (RBAC) settings associated with the database.

When restoring, you have the flexibility to define the target topology (how many primaries and secondaries are desired for the database), which may differ from the topology at backup time.
The database will then be allocated across the available servers according to that topology.
The database will then be allocated across the available servers according to that topology.
====

[[backup-planning-databases]]
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/backup-restore/restore-dump.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ For more information, see xref:clustering/databases.adoc#cluster-seed[Seed a clu

[NOTE]
====
`neo4j-admin database load` cannot be applied to xref:database-administration/composite-databases/concepts.adoc[Composite databases].
`neo4j-admin database load` cannot be applied to xref:scalability/composite-databases/concepts.adoc[Composite databases].
It must be run directly on the databases that are associated with that Composite database.
====

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Refer to xref:configuration/neo4j-conf.adoc#_configuration_settings[The neo4j.co

For lists of deprecated and removed configuration settings in 2025.x, refer to the page xref:changes-deprecations-removals.adoc[Changes, deprecations, and removals in Neo4j 2025.x].

To list all available configuration settings on a Neo4j server, run the link:https://neo4j.com/docs/cypher-manual/5/clauses/listing-settings[`SHOW SETTINGS`] command.
To list all available configuration settings on a Neo4j server, run the link:{neo4j-docs-base-uri}/cypher-manual/5/clauses/listing-settings[`SHOW SETTINGS`] command.


== Dynamic configuration settings
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/configuration/plugins.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ For more information on configuring the plugins, see the respective documentatio
* link:https://neo4j.com/docs/bloom-user-guide/current/bloom-installation/[Bloom documentation]
* link:https://neo4j.com/docs/aura/fleet-management/setup/[Fleet management documentation]
* link:https://neo4j.com/docs/graph-data-science/current/installation/neo4j-server/[GDS documentation]
* link:https://neo4j.com/docs/cypher-manual/current/genai-integrations/[GenAI documentation]
* link:{neo4j-docs-base-uri}/cypher-manual/current/genai-integrations/[GenAI documentation]
* link:https://neo4j.com/labs/neosemantics/[Neosemantics documentation]

. Restart Neo4j for the plugins to be loaded and available for use.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ For a description of all the returned columns of this command, and for ways in w
[[create-composite-database-alias]]
== Create database aliases in composite databases

Both local and remote database aliases can be part of a xref::database-administration/composite-databases/concepts.adoc[composite database].
Both local and remote database aliases can be part of a xref::scalability/composite-databases/concepts.adoc[composite database].

The database alias consists of two parts, separated by a dot: the namespace and the alias name.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The home database for users can be set to an alias, which will be resolved to th
Starting with Neo4j 2025.04, a database alias can also be set as the DBMS default database.

This page describes managing database aliases for standard databases.
For aliases created as part of a xref:database-administration/composite-databases/concepts.adoc[composite database], see xref:database-administration/aliases/manage-aliases-composite-databases.adoc[].
For aliases created as part of a xref:scalability/composite-databases/concepts.adoc[composite database], see xref:database-administration/aliases/manage-aliases-composite-databases.adoc[].

[[manage-aliases-list]]
== List database aliases
Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/database-administration/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,10 @@ image::manage-dbs-community.svg[title="A default Neo4j installation.",role=popup
.An installation of Neo4j with multiple active databases, named `marketing`, `sales`, and `hr`:
image::manage-dbs-enterprise.svg[title="A multiple database Neo4j installation.",role=popup]

For details about the `system` database in a clustered environment, refer to xref:clustering/databases.adoc#cluster-system-db[Managing databases in a cluster -> The `system` database].
For details about the `system` database in a clustered environment, refer to xref:clustering/databases.adoc#cluster-system-db[Managing databases in a cluster -> The `system` database].

== Composite databases

A Composite database is a logical grouping of multiple graphs contained in other, standard databases.
A Composite database defines an _execution context_ and a (limited) _transaction domain_.
For more information, see xref:database-administration/composite-databases/concepts.adoc[Composite databases].
For more information, see xref:scalability/composite-databases/concepts.adoc[Composite databases].
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Step 2: Reuse open transaction::
* If not, then proceed to step 3.
Step 3: Determine the type of the target database (execution context type)::
* If the target database is a database in this DBMS, then the context type is _Internal_.
* If the target database is a xref::database-administration/composite-databases/concepts.adoc[Composite database], then the context type is _Composite_. +
* If the target database is a xref::scalability/composite-databases/concepts.adoc[Composite database], then the context type is _Composite_. +
+
[NOTE]
====
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/introduction.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ a| APOC 450+ link:https://neo4j.com/docs/apoc/5/[Core Procedures and Functions]
|
| {check-mark}

| xref:database-administration/composite-databases/concepts.adoc[Composite databases]
| xref:scalability/composite-databases/concepts.adoc[Composite databases]
|
| {check-mark}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Local database aliases target databases within the same DBMS, while remote datab
For more information, see xref:database-administration/aliases/manage-aliases-composite-databases.adoc[].

Composite databases are managed using Cypher administrative commands.
For a detailed example of how to create a Composite database and add database aliases to it, see xref:database-administration/composite-databases/querying-composite-databases.adoc[Set up and query composite databases].
For a detailed example of how to create a Composite database and add database aliases to it, see xref:scalability/composite-databases/querying-composite-databases.adoc[Set up and query composite databases].

Composite databases cannot guarantee compatibility between constituents from different Neo4j versions.
Constituents from versions without breaking changes should work fine, apart from newly-added features.
Expand All @@ -26,7 +26,7 @@ Composite databases have the following characteristics:
* Can be deployed in standalone and cluster deployments.
* Managed using Cypher commands, such as `CREATE COMPOSITE DATABASE` and `CREATE ALIAS`.
* You can shard an existing database with the help of the `neo4j-admin copy` command.
See xref:database-administration/composite-databases/sharding-with-copy.adoc[Sharding data with the copy command] for details.
See xref:scalability/composite-databases/sharding-with-copy.adoc[Sharding data with the copy command] for details.
* Use the existing user for local constituents or the user credentials defined by the remote aliases for remote consituents.
* Do not support privileges, index, and constraint management commands.
These must be defined on the constituent target database in the respective DBMS.
Expand All @@ -45,7 +45,7 @@ Data sharding is when you have two graphs that share the *same model* (same labe
For example, you can deploy shards on separate servers, splitting the load on resources and storage.
Or, you can deploy shards in different locations, to be able to manage them independently or split the load on network traffic.
An existing database can be sharded with the help of the `neo4j-admin database copy` command.
For an example, see xref:database-administration/composite-databases/sharding-with-copy.adoc[Sharding data with the copy command].
For an example, see xref:scalability/composite-databases/sharding-with-copy.adoc[Sharding data with the copy command].

Connecting data across graphs::
Because relationships cannot span across graphs, to query your data, you have to federate the graphs by
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ CREATE ALIAS `cineasts.upcoming`
====


For more information about composite databases and database aliases in composite databases, see xref:database-administration/composite-databases/concepts.adoc[], and xref:database-administration/aliases/manage-aliases-composite-databases.adoc[].
For more information about composite databases and database aliases in composite databases, see xref:scalability/composite-databases/concepts.adoc[], and xref:database-administration/aliases/manage-aliases-composite-databases.adoc[].

[[composite-databases-queries-graph-selection]]
== Graph selection
Expand Down
91 changes: 91 additions & 0 deletions modules/ROOT/pages/scalability/concepts.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
:page-role: new-2025.10 enterprise-edition not-on-aura
:description: This page describes the concepts behind scalability with Neo4j.
= Concepts

Scalability is a crucial aspect of database management, allowing a system to handle changing demands by adding and removing resources to meet the demands of a database's workload.
Neo4j supports multiple strategies to achieve scalability, enabling systems to handle larger datasets, more concurrent users, and higher query complexity without compromising performance or availability, i.e. the system's resiliency.
The three main strategies are:

* xref:clustering/setup/analytics-cluster.adoc[Analytics clustering] -- for horizontal read scalability.
* xref:scalability/composite-databases/concepts.adoc[Composite databases] -- for federated queries and distributed data management.
* xref:scalability/sharded-property-databases/overview.adoc[Property sharding] -- for handling massive property-heavy graphs.

== What is scalability?

Scalability is a system's ability to handle an increasing workload without compromising performance.
There are two primary methods to achieve scalability:

.Scaling methods
[options="header", cols="1,1,1a,1a"]
|===
| Method
| Description
| Pros
| Cons

| Vertical Scaling (Scaling Up / Down)
| Increase or decrease the capacity of a single server by adding or removing CPUs, memory, or storage.
| Simple to manage.
| * Physical limits. +
* Difficult to make online changes.

| Horizontal Scaling (Scaling Out / In)
| Distribute the workload by adding more servers or reduce the infrastructure by removing existing servers.
| * Greater scalability and fault tolerance. +
* Easier to make online changes.
| More complex to manage.
|===

== What is database scalability?

Database scalability is the ability of a database management system (DBMS) to handle changing demands.
To scale properly, a database must apply strategies that cover all areas: data access, data manipulation in memory, and database computing.

Strategies include:

* **Vertical Scaling**
** Optimize usage (e.g., granular locks, partitioning)
** Optimize physical resources (multi-threading, tiered storage)

* **Horizontal Scaling** (distributed computing architectures):

** *Shared Everything*: All servers share data and memory.
Flexible, but prone to contention. +
In this model, data is shared between disk and memory across all servers in a cluster.
Requests are satisfied by any combination of servers.
This approach introduces complexity, as the cluster must implement a way to avoid contention when multiple servers try to update the same data simultaneously.

** *Shared Nothing*: Each server manages its own partition (shard).
More fault-tolerant, eliminates single points of failure. +
Every update request is handled by a single cluster member, which eliminates single points of failure.
Each part of the database on a single cluster member is called a *shard*.

image::horizontal-scaling-strategies.png[title="An example of horizontal scaling strategies.", role="middle"]

== What is graph database scalability?

Graph database scalability refers to the ability of a database to handle different amounts of data and workloads without compromising performance.
It includes:

* *Data volume* - involves ensuring a consistent SLA in both query and administration response times, even as the size of the data for storage and retrieval expands. +
Volume depends on data type(s).
Vectors occupy a large data space.

* *Query volume*
** Read queries + write queries.
** Queries and user concurrency -- the aim is to ensure a linear response time during the execution of concurrent queries against the same database.
** Query complexity -- provide response time in line with the complexity of a query. The complexity of a query can be set by the combination of:
*** Steps to execute
*** Rows to retrieve
*** Total DB hits
*** Total memory allocation
*** Total execution time

* *Admin volume*
** Data ingestion/extraction -- When scaling data ingestion/extraction, the goal is to maintain a linear response time when ingesting or extracting an increasing set of data.
This objective remains true regardless of the volume of stored data, provided a similar data structure is used.
** Multi-tenancy -- In SaaS and AaaS environments, the scaling cost for tenants should exhibit linearity.
For more general services, such as DBaaS (e.g., Aura), scalability should also be linear, considering all five scalability factors mentioned here.



Loading
Loading