Skip to content

Commit 3b252a6

Browse files
committed
docs: Create Documentation for SPDK Blobstore Cluster Size
Signed-off-by: Bala Harish <bala.harish.ac@datacore.com>
1 parent 3031285 commit 3b252a6

File tree

2 files changed

+85
-0
lines changed

2 files changed

+85
-0
lines changed
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
id: spdk-blobstore
3+
title: Configuring SPDK Blobstore Cluster Size for Replicated PV Mayastor DiskPools
4+
keywords:
5+
- Configuring SPDK Blobstore Cluster Size for Mayastor DiskPools
6+
- SPDK
7+
- SPDK Blobstore
8+
- Blobstore Cluster Size
9+
description: This section explains the recommended practices for better performance.
10+
---
11+
# Configuring SPDK Blobstore Cluster Size for Replicated PV Mayastor DiskPools
12+
13+
## Overview
14+
15+
You can configure the Storage Performance Development Kit (SPDK) blobstore cluster size when creating a Replicated PV Mayastor DiskPool. Adjusting this value lets you optimize the on-disk layout for your specific workloads and device sizes.
16+
17+
- Smaller cluster sizes (default 4 MiB) provide better storage efficiency but generate more metadata overhead.
18+
- Larger cluster sizes (for example, 16 MiB or 32 MiB) reduce metadata, speed up pool creation and imports, and improve performance for large sequential I/O operations.
19+
20+
:::note
21+
Before modifying the default setting, carefully evaluate application I/O patterns and device size. For example: Smaller cluster size provides better storage efficiency and less internal fragmentation, but more metadata overhead. Larger cluster size will provide better performance for large sequential IOs, and low metadata overhead.
22+
:::
23+
24+
### Configuration
25+
26+
1. Per-Pool Configuration (DiskPool CR)
27+
28+
Set the `cluster_size` field directly in your DiskPool custom resource manifest. This provides granular control for specific storage devices.
29+
30+
```
31+
apiVersion: "openebs.io/v1beta3"
32+
kind: DiskPool
33+
metadata:
34+
name: <pool_name>
35+
namespace: <namespace>
36+
spec:
37+
node: <none_name>
38+
disks: ["/disk/path"]
39+
cluster_size: 32MiB
40+
```
41+
42+
2. Global Configuration (Helm Chart)
43+
44+
Set a global cluster size for all new pools that do not specify it in their custom resource. Provide the size in bytes.
45+
46+
```
47+
--set openebs.engines.replicated.mayastor.agents.core.poolClusterSize=33554432
48+
(The value above sets the global cluster size to 32 MiB.)
49+
```
50+
51+
3. Volume Provisioning
52+
53+
A new storageclass parameter named `poolClusterSize` is provided. With this option, only the pools that match requested blobstore cluster size will be used for that volume’s replicas. If enough such pools are not found, to satisfy the volume’s replication factor, the provisioning will fail or in case of an existing volume the rebuilds might not be able to start.
54+
55+
### Recommendations
56+
57+
- This is an advanced user configuration. Proceed with caution and ensure that you fully understand the implications before modifying the default value of `4 MiB`.
58+
59+
- Typically, this configuration has been tested at `32 MiB` for device sizes upto `20 TiB`, where the pool import takes about three minutes on a performant cloud disk.
60+
61+
- For simpler management and more predictable replica scheduling, we recommend minimizing the number of different cluster sizes used in deployment environment. As a general guideline for node clusters utilizing large storage devices, configuring a global blobstore cluster size of `16 MiB` or `32 MiB` provides a strong balance of performance and efficiency.
62+
63+
### Benefits
64+
65+
Choosing a larger cluster size (Example: `16 MiB` or `32 MiB)` significantly reduces the amount of metadata that SPDK needs to manage.
66+
67+
- Faster Pool Creation: When a pool is created, the device is formatted by writing metadata for every cluster. Fewer clusters mean significantly less metadata to write, leading to a significant reduction in the time it takes to create a pool on a large device.
68+
69+
- Faster Pool Imports: During startup or recovery, Replicated PV Mayastor imports existing pools by reading their metadata from disk. A more compact metadata layout (due to larger clusters) requires fewer I/O operations, making the import process much quicker.
70+
71+
The performance of the disk also impacts the apparent benefits of this configuration.
72+
73+
## See Also
74+
75+
- [RDMA Enablement](../configuration/rs-rdma.md)
76+
- [Create DiskPool(s)](../configuration/rs-create-diskpool.md)
77+
- [Create StorageClass(s)](../configuration/rs-create-storageclass.md)
78+
- [Storage Class Parameters](../configuration/rs-storage-class-parameters.md)
79+
- [Topology Parameters](../configuration/rs-topology-parameters.md)
80+
- [Deploy an Application](../configuration/rs-deployment.md)

docs/sidebars.js

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -548,6 +548,11 @@ module.exports = {
548548
id: "user-guides/replicated-storage-user-guide/replicated-pv-mayastor/additional-information/performance-tips",
549549
label: "Performance Tips"
550550
},
551+
{
552+
type: "doc",
553+
id: "user-guides/replicated-storage-user-guide/replicated-pv-mayastor/additional-information/spdk-blobstore",
554+
label: "SPDK Blobstore Cluster Size"
555+
},
551556
{
552557
type: "doc",
553558
id: "user-guides/replicated-storage-user-guide/replicated-pv-mayastor/additional-information/io-path-description",

0 commit comments

Comments
 (0)