Topology Parameters
The topology parameters defined in storage class helps in determining the placement of volume replicas across different nodes/pools of the cluster. A brief explanation of each parameter is as follows.
note
We support only one type of topology parameter per storage class.
"nodeAffinityTopologyLabel"#
The parameter nodeAffinityTopologyLabel will allow the placement of replicas on the node that exactly matches the labels defined in the storage class.
For the case shown below, the volume replicas will be provisioned on worker-node-1 and worker-node-3 only as they match the labels specified under nodeAffinityTopologyLabel in storage class which is equal to zone=us-west-1.
Command
Apply the labels to the nodes using the below command:
Command
Command (Get nodes)
"nodeHasTopologyKey"#
The parameter nodeHasTopologyKey will allow the placement of replicas on the nodes having a label whose key matches the key specified in the storage class.
Command
Apply the labels on the node using the below command:
Command
In this case, the volume replicas will be provisioned on any two of the three nodes i.e.
worker-node-1andworker-node-2orworker-node-1andworker-node-3orworker-node-2andworker-node-3as the storage class hasrackas the value fornodeHasTopologyKeythat matches the label key of the node.
"nodeSpreadTopologyKey"#
The parameter nodeSpreadTopologyKey will allow the placement of replicas on the node that has label keys that are identical to the keys specified in the storage class but have different values.
Command
Apply the labels to the nodes using the below command:
Command
Command (Get nodes)
In this case, the volume replicas will be provisioned on the below given nodes i.e.
worker-node-1andworker-node-2orworker-node-2andworker-node-3as the storage class haszoneas the value fornodeSpreadTopologyKeythat matches the label key of the node but has a different value.
"poolAffinityTopologyLabel"#
The parameter poolAffinityTopologyLabel will allow the placement of replicas on the pool that exactly match the labels defined in the storage class.
Command
Apply the labels to the pools using the below command:
Command
Command (Get filtered pools based on labels)
For the case shown above, the volume replicas will be provisioned on pool-on-node-0 and pool-on-node-2 only as they match the labels specified under poolAffinityTopologyLabel in the storage class that is equal to zone=us-west-1.
"poolHasTopologyKey"#
The parameter poolHasTopologyKey will allow the placement of replicas on the pool that has label keys same as the keys passed in the storage class.
Command
Command (Get filtered pools based on labels)
In this case, the volume replicas will be provisioned on any two of the three pools i.e.
pool-on-node-1andpool-on-node-2orpool-on-node-1andpool-on-node-3orpool-on-node-2andpool-on-node-3as the storage class haszoneas the value forpoolHasTopologyKeythat matches with the label key of the pool.
"stsAffinityGroup"#
stsAffinityGroup represents a collection of volumes that belong to instances of Kubernetes StatefulSet. When a StatefulSet is deployed, each instance within the StatefulSet creates its own individual volume, which collectively forms the stsAffinityGroup. Each volume within the stsAffinityGroup corresponds to a pod of the StatefulSet.
This feature enforces the following rules to ensure the proper placement and distribution of replicas and targets so that there is not any single point of failure affecting multiple instances of StatefulSet.
Anti-Affinity among single-replica volumes: This is a hard rule. Single-replica volumes in the same affinity group must not be placed on the same node. This prevents a single node failure from impacting multiple StatefulSet pods.
Anti-Affinity among multi-replica volumes: This is a soft rule. While placement is optimized to spread replicas across nodes, the scheduler may relax this rule when necessary.
Anti-affinity among targets: Targets are distributed to avoid a failure domain impacting multiple volumes in the affinity group.
By default, the stsAffinityGroup feature is disabled. To enable it, modify the storage class YAML by setting the parameters.stsAffinityGroup parameter to true.
Volume Affinity Group Scale-Down Restrictions#
When using stsAffinityGroup, replicas of volumes belonging to the same StatefulSet are distributed across different nodes to avoid a single point of failure. Because of these anti-affinity rules, scaling a volume down to 1 replica may be restricted if doing so would place the last remaining replica on a node that already hosts another single-replica volume from the same affinity group.
A scale-down to 1 replica is allowed only when the current replicas are already placed on different nodes. If the replicas end up on the same node, for example, after scaling from 3 replicas to 2, the system may block the scale-down until the placement is improved.
If a scale-down is blocked, you can resolve it by temporarily scaling the volume up to add a replica whilst the volume is published and then scaling down again. This reshuffles the replicas to meet the affinity group’s placement rules.
These restrictions ensure that a single node failure does not impact multiple StatefulSet instances, preserving fault isolation and reliability for applications using affinity-grouped volumes.
"cloneFsIdAsVolumeId"#
cloneFsIdAsVolumeId is a setting for volume clones/restores with two options: true and false. By default, it is set to false.
- When set to
true, the created clone/restore's filesystemuuidwill be set to the restore volume'suuid. This is important because some file systems, like XFS, do not allow duplicate filesystemuuidon the same machine by default. - When set to
false, the created clone/restore's filesystemuuidwill be the same as the original volumeuuid, but it will be mounted using thenouuidflag to bypass duplicateuuidvalidation.
note
This option needs to be set to true when using a btrfs filesystem, if the application using the restored volume is scheduled on the same node where the original volume is mounted, concurrently.