From a91e7ebfdad8be2b32ce240e0b7f5fd877d866c4 Mon Sep 17 00:00:00 2001 From: Chetan Phirke Date: Fri, 9 May 2025 19:35:50 +0800 Subject: [PATCH 1/2] Add pruning and passive sync documentation --- zq2/docs/nodes/node.md | 5 +++-- zq2/mkdocs.in.yaml | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/zq2/docs/nodes/node.md b/zq2/docs/nodes/node.md index 5effb3d..9012667 100644 --- a/zq2/docs/nodes/node.md +++ b/zq2/docs/nodes/node.md @@ -115,14 +115,15 @@ There are two options you can choose from: settings to start the node from a checkpoint. Detailed instructions for this configuration are available in [syncing-from-checkpoints](../nodes/checkpoints/index.md#syncing-a-node-from-a-checkpointsyncing-a-node-from-a-checkpoint). - - Synchronization from the genesis. + - Synchronization from the genesis(passive syncing). This method initializes the node from the genesis block, ensuring that the node processes the entire blockchain history. However, this process is time-consuming, as the node must download and validate every block from the genesis block to the latest block height. Syncing the node to the latest block may take a considerable amount of time, potentially up to several days to complete fully. - + +Please refer to [Pruning and Passive sync](../nodes/passive-pruning.md) for more information on genesis syncing and pruning. ### [Starting your node](#starting-your-node) Since only full archive nodes need to sync from the genesis block, all other nodes can be started from a checkpoint: diff --git a/zq2/mkdocs.in.yaml b/zq2/mkdocs.in.yaml index e0afece..e949362 100644 --- a/zq2/mkdocs.in.yaml +++ b/zq2/mkdocs.in.yaml @@ -10,6 +10,7 @@ nav: - nodes/checkpoints/index.md - Downloads: nodes/checkpoints/download.md - Node FAQ: nodes/nodefaq.md + - Passive sync and Pruning: nodes/passive-pruning.md - Monitoring: - Validator Stats Agent: nodes/monitoring/statsagent.md - OpenTelemetry: nodes/monitoring/opentelemetry.md From 03acee7cbc12a3ebbd6ab9b350227fadf8d26f89 Mon Sep 17 00:00:00 2001 From: Chetan Phirke Date: Fri, 9 May 2025 19:36:11 +0800 Subject: [PATCH 2/2] Add pruning and passive sync documentation --- zq2/docs/nodes/passive-pruning.md | 99 +++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 zq2/docs/nodes/passive-pruning.md diff --git a/zq2/docs/nodes/passive-pruning.md b/zq2/docs/nodes/passive-pruning.md new file mode 100644 index 0000000..2c6422d --- /dev/null +++ b/zq2/docs/nodes/passive-pruning.md @@ -0,0 +1,99 @@ +--- +id: nodes/passive-pruning +title: Node Pruning and Passive Sync +--- + +# [Node Pruning and Passive Sync](#node-prune-passive-sync) + +## Overview + +This document explains how pruning and passive sync mechanisms work in Zilliqa 2.0 nodes. These features are controlled by two mutually exclusive configuration parameters and help nodes manage database size and block retention based on their roles (e.g., validator vs. archive node). + +--- + +## Configuration Parameters + +Zilliqa nodes support two **mutually exclusive** configuration parameters: + +### `node.sync.prune_interval` + +- **Purpose:** Retains only the most recent `N` blocks in the database. +- **Effect:** Deletes blocks below `TIP - N`. +- **Restriction:** `N` must be **greater than 300**. +- **Use case:** Ideal for **validator nodes** that only need recent block data for performance. +- **Note:** Pruning only starts **after** more than `N` blocks are present. No backward sync is performed to reach count `N`. + +### `node.sync.sync_base_height` + +- **Purpose:** Ensures the lowest block retained is at height `N`. +- **Effect:** Deletes blocks **below** height `N` and **passively syncs** blocks from `TIP` backwards to `N`. +- **Restriction:** `N` must be **less than TIP**. +- **Use case:** Ideal for **archive nodes** to store historical data from `TIP` to `N`. +- **Note:** If `N > lowest_block`, the node prunes until `N`. + +> **⚠️ Only one of these values can be set at a time. Setting both is invalid.** +> Leaving both unset will disable pruning and passive sync. + +--- + +## Example Usage + +### ✅ Step 1: Bootstrap from Checkpoint + +Start the node from a known checkpoint. It will active-sync to the current chain tip. + +### ✅ Step 2a: Pruning Mode (Validator Node) + +Set the following in the node config: + +```toml +node.sync.prune_interval = 10000 +``` + +* Node retains only the latest **10,000** blocks. +* No older block syncing occurs. +* Pruning starts once more than 10,000 blocks exist in the DB. + +### ✅ Step 2b: Passive Sync Mode (Archive Node) + +Set the following in the config: + +```toml +node.sync.sync_base_height = 0 +``` + +* Node begins **passive syncing backwards** toward block `0`. +* Deletes any blocks below the configured height if present. +* Suitable for retaining full history from checkpoint to genesis. + +--- + +## Performance Notes + +* Pruning is done **gradually in the background**. +* On a GCP node: + + * **Pruning rate:** \~300–400 **empty** blocks/sec. + * Rate drops significantly when pruning blocks with **many transactions**, possibly below 1 block/sec. +* **Disk space is not immediately freed** due to SQLite behavior: + + * SQLite does **not shrink** DB file unless `VACUUM` is used. + * Freed space is reused for future inserts. + * This behavior may change when migrating to `redb` in the future. + +--- + +## Summary Table + +| Config | Purpose | Passive Sync | Prune Old Blocks | Use Case | +| ---------------------------- | ------------------------ | ------------ | ---------------- | -------------- | +| `node.sync.prune_interval` | Retain latest `N` blocks | ❌ | ✅ | Validator Node | +| `node.sync.sync_base_height` | Retain from `N` to TIP | ✅ | ✅ | Archive Node | + +--- + +## Notes + +* Always let the node reach chain tip before enabling pruning or passive sync. +* These operations are **non-blocking** and run in the background. +* Use with care to avoid unintentional data loss or sync delays. \ No newline at end of file