You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/influxdb3/clustered/admin/scale-cluster.md
+50-28Lines changed: 50 additions & 28 deletions
Original file line number
Diff line number
Diff line change
@@ -466,19 +466,29 @@ helm upgrade \
466
466
467
467
### Router
468
468
469
-
The Router can be scaled both [vertically](#vertical-scaling) and
469
+
The [Router](/influxdb3/clustered/reference/internals/storage-engine/#router) can be scaled both [vertically](#vertical-scaling) and
470
470
[horizontally](#horizontal-scaling).
471
-
Horizontal scaling increases write throughput and is typically the most
471
+
472
+
- **Recommended**: Horizontal scaling increases write throughput and is typically the most
472
473
effective scaling strategy for the Router.
473
-
Vertical scaling (specifically increased CPU) improves the Router's ability to
474
+
- Vertical scaling (specifically increased CPU) improves the Router's ability to
474
475
parse incoming line protocol with lower latency.
475
476
477
+
#### Router latency
478
+
479
+
Latency of the Router’s write endpoint is directly impacted by:
480
+
481
+
- Ingester latency--the router calls the Ingester during a client write request
482
+
- Catalog latency during schema validation
483
+
476
484
### Ingester
477
485
478
-
The Ingester can be scaled both [vertically](#vertical-scaling) and
486
+
The [Ingester](/influxdb3/clustered/reference/internals/storage-engine/#ingester) can be scaled both [vertically](#vertical-scaling) and
479
487
[horizontally](#horizontal-scaling).
480
-
Vertical scaling increases write throughput and is typically the most effective
481
-
scaling strategy for the Ingester.
488
+
489
+
- **Recommended**: Vertical scaling is typically the most effective scaling strategy for the Ingester.
490
+
Compared to horizontal scaling, vertical scaling not only increases write throughput but also lessens query, catalog, and compaction overheads as well as Object store costs.
491
+
- Horizontal scaling can help distribute write load but comes with additional coordination overhead.
482
492
483
493
#### Ingester storage volume
484
494
@@ -541,50 +551,62 @@ ingesterStorage:
541
551
542
552
### Querier
543
553
544
-
The Querier can be scaled both [vertically](#vertical-scaling) and
554
+
The [Querier](/influxdb3/clustered/reference/internals/storage-engine/#querier) can be scaled both [vertically](#vertical-scaling) and
545
555
[horizontally](#horizontal-scaling).
546
-
Horizontal scaling increases query throughput to handle more concurrent queries.
547
-
Vertical scaling improves the Querier’s ability to process computationally
548
-
intensive queries.
556
+
557
+
- **Recommended**: [Vertical scaling](#vertical-scaling) improves the Querier's ability to process concurrent or computationally
558
+
intensive queries, and increases the effective cache capacity.
559
+
- Horizontal scaling increases query throughput to handle more concurrent queries.
560
+
Consider horizontal scaling if vertical scaling doesn't adequately address
561
+
concurrency demands or reaches the hardware limits of your underlying nodes.
549
562
550
563
### Compactor
551
564
552
-
The Compactor can be scaled both [vertically](#vertical-scaling) and
553
-
[horizontally](#horizontal-scaling).
554
-
Because compaction is a compute-heavy process, vertical scaling (especially
555
-
increasing the available CPU) is the most effective scaling strategy for the
556
-
Compactor. Horizontal scaling increases compaction throughput, but not as
565
+
- **Recommended**: Maintain **1 Compactor pod** and use [vertical scaling](#vertical-scaling) (especially
566
+
increasing the available CPU) for the Compactor.
567
+
- Because compaction is a compute-heavy process, horizontal scaling increases compaction throughput, but not as
557
568
efficiently as vertical scaling.
558
569
559
570
### Garbage collector
560
571
561
-
The Garbage collector is not designed for distributed load and should _not_ be
562
-
scaled horizontally. It is a lightweight process that typically doesn't require
563
-
significant system resources. [Vertical scaling](#vertical-scaling) should only
564
-
be considered if you observe consistently high CPU usage or if the container
572
+
The [Garbage collector](/influxdb3/clustered/reference/internals/storage-engine/#garbage-collector) is a lightweight process that typically doesn't require
573
+
significant system resources.
574
+
575
+
- Don't horizontally scale the Garbage collector; it isn't designed for distributed load.
576
+
- Consider [vertical scaling](#vertical-scaling) only if you observe consistently high CPU usage or if the container
565
577
regularly runs out of memory.
566
578
567
579
### Catalog store
568
580
569
-
The Catalog store is a PostgreSQL-compatible database that persistently stores metadata.
570
-
Scaling strategies depend on your chosen PostgreSQL implementation.
571
-
All support [vertical scaling](#vertical-scaling), and most support
572
-
[horizontal scaling](#horizontal-scaling) for redundancy and failover.
581
+
The [Catalog store](/influxdb3/clustered/reference/internals/storage-engine/#catalog-store) is a PostgreSQL-compatible database that stores critical metadata for your InfluxDB cluster.
582
+
An underprovisioned Catalog store can cause write outages and system-wide performance issues.
583
+
584
+
- Scaling strategies depend on your specific PostgreSQL implementation
585
+
- All PostgreSQL implementations support [vertical scaling](#vertical-scaling)
586
+
- Most implementations support [horizontal scaling](#horizontal-scaling) for improved redundancy and failover
587
+
573
588
574
589
### Catalog service
575
590
576
-
The Catalog service should maintain exactly
577
-
3 replicas for optimal redundancy.
578
-
Additional replicas are discouraged; favor vertical scaling instead if performance improvements are needed.
591
+
The [Catalog service](/influxdb3/clustered/reference/internals/storage-engine/#catalog-service) (iox-shared-catalog statefulset) caches
592
+
and manages access to the Catalog store.
593
+
594
+
- **Recommended**: Maintain **exactly 3 replicas** of the Catalog service for optimal redundancy. Additional replicas are discouraged.
595
+
- If performance improvements are needed, use [vertical scaling](#vertical-scaling).
579
596
580
597
> [!Note]
598
+
> #### Managing Catalog components
599
+
>
581
600
> The [Catalog service](/influxdb3/clustered/reference/internals/storage-engine/#catalog-service) is managed through the
582
601
> `AppInstance` resource, while the [Catalog store](/influxdb3/clustered/reference/internals/storage-engine/#catalog-store)
583
602
> is managed separately according to your PostgreSQL implementation.
584
603
585
604
### Object store
586
605
587
-
Scaling strategies available for the Object store depend on the underlying
588
-
object storage services used to run the object store. Most support
606
+
The [Object store](/influxdb3/clustered/reference/internals/storage-engine/#object-store)
607
+
contains time series data in Parquet format.
608
+
609
+
Scaling strategies depend on the underlying object storage services used.
610
+
Most services support
589
611
[horizontal scaling](#horizontal-scaling) for redundancy, failover, and
0 commit comments