|  | 
|  | 1 | +--- | 
|  | 2 | +id: spdk-blobstore | 
|  | 3 | +title: Configuring SPDK Blobstore Cluster Size for Replicated PV Mayastor DiskPools | 
|  | 4 | +keywords: | 
|  | 5 | + - Configuring SPDK Blobstore Cluster Size for Mayastor DiskPools | 
|  | 6 | + - SPDK | 
|  | 7 | + - SPDK Blobstore | 
|  | 8 | + - Blobstore Cluster Size | 
|  | 9 | +description: This section explains the recommended practices for better performance. | 
|  | 10 | +--- | 
|  | 11 | +# Configuring SPDK Blobstore Cluster Size for Replicated PV Mayastor DiskPools | 
|  | 12 | + | 
|  | 13 | +## Overview | 
|  | 14 | + | 
|  | 15 | +You can configure the Storage Performance Development Kit (SPDK) blobstore cluster size when creating a Replicated PV Mayastor DiskPool. Adjusting this value lets you optimize the on-disk layout for your specific workloads and device sizes. | 
|  | 16 | + | 
|  | 17 | +- Smaller cluster sizes (default 4 MiB) provide better storage efficiency but generate more metadata overhead. | 
|  | 18 | +- Larger cluster sizes (for example, 16 MiB or 32 MiB) reduce metadata, speed up pool creation and imports, and improve performance for large sequential I/O operations. | 
|  | 19 | + | 
|  | 20 | +:::note | 
|  | 21 | +Before modifying the default setting, carefully evaluate application I/O patterns and device size. For example: Smaller cluster size provides better storage efficiency and less internal fragmentation, but more metadata overhead. Larger cluster size will provide better performance for large sequential IOs, and low metadata overhead. | 
|  | 22 | +::: | 
|  | 23 | + | 
|  | 24 | +### Configuration | 
|  | 25 | + | 
|  | 26 | +1. Per-Pool Configuration (DiskPool CR) | 
|  | 27 | + | 
|  | 28 | +Set the `cluster_size` field directly in your DiskPool custom resource manifest. This provides granular control for specific storage devices. | 
|  | 29 | + | 
|  | 30 | +``` | 
|  | 31 | +apiVersion: "openebs.io/v1beta3" | 
|  | 32 | +kind: DiskPool | 
|  | 33 | +metadata: | 
|  | 34 | +  name: <pool_name> | 
|  | 35 | +  namespace: <namespace> | 
|  | 36 | +spec: | 
|  | 37 | +  node: <none_name> | 
|  | 38 | +  disks: ["/disk/path"] | 
|  | 39 | +  cluster_size: 32MiB | 
|  | 40 | +``` | 
|  | 41 | + | 
|  | 42 | +2. Global Configuration (Helm Chart) | 
|  | 43 | + | 
|  | 44 | +Set a global cluster size for all new pools that do not specify it in their custom resource. Provide the size in bytes. | 
|  | 45 | + | 
|  | 46 | +``` | 
|  | 47 | +--set openebs.engines.replicated.mayastor.agents.core.poolClusterSize=33554432 | 
|  | 48 | +(The value above sets the global cluster size to 32 MiB.) | 
|  | 49 | +``` | 
|  | 50 | + | 
|  | 51 | +3. Volume Provisioning | 
|  | 52 | + | 
|  | 53 | +A new storageclass parameter named `poolClusterSize` is provided. With this option, only the pools that match requested blobstore cluster size will be used for that volume’s replicas. If enough such pools are not found, to satisfy the volume’s replication factor, the provisioning will fail or in case of an existing volume the rebuilds might not be able to start. | 
|  | 54 | + | 
|  | 55 | +### Recommendations | 
|  | 56 | + | 
|  | 57 | +- This is an advanced user configuration. Proceed with caution and ensure that you fully understand the implications before modifying the default value of `4 MiB`. | 
|  | 58 | + | 
|  | 59 | +- Typically, this configuration has been tested at `32 MiB` for device sizes upto `20 TiB`, where the pool import takes about three minutes on a performant cloud disk. | 
|  | 60 | + | 
|  | 61 | +- For simpler management and more predictable replica scheduling, we recommend minimizing the number of different cluster sizes used in deployment environment. As a general guideline for node clusters utilizing large storage devices, configuring a global blobstore cluster size of `16 MiB` or `32 MiB` provides a strong balance of performance and efficiency. | 
|  | 62 | + | 
|  | 63 | +### Benefits | 
|  | 64 | + | 
|  | 65 | +Choosing a larger cluster size (Example: `16 MiB` or `32 MiB)` significantly reduces the amount of metadata that SPDK needs to manage. | 
|  | 66 | + | 
|  | 67 | +- Faster Pool Creation: When a pool is created, the device is formatted by writing metadata for every cluster. Fewer clusters mean significantly less metadata to write, leading to a significant reduction in the time it takes to create a pool on a large device. | 
|  | 68 | + | 
|  | 69 | +- Faster Pool Imports: During startup or recovery, Replicated PV Mayastor imports existing pools by reading their metadata from disk. A more compact metadata layout (due to larger clusters) requires fewer I/O operations, making the import process much quicker. | 
|  | 70 | + | 
|  | 71 | +The performance of the disk also impacts the apparent benefits of this configuration. | 
|  | 72 | + | 
|  | 73 | +## See Also | 
|  | 74 | + | 
|  | 75 | +- [RDMA Enablement](../configuration/rs-rdma.md) | 
|  | 76 | +- [Create DiskPool(s)](../configuration/rs-create-diskpool.md) | 
|  | 77 | +- [Create StorageClass(s)](../configuration/rs-create-storageclass.md) | 
|  | 78 | +- [Storage Class Parameters](../configuration/rs-storage-class-parameters.md) | 
|  | 79 | +- [Topology Parameters](../configuration/rs-topology-parameters.md) | 
|  | 80 | +- [Deploy an Application](../configuration/rs-deployment.md) | 
0 commit comments