Skip to content

Commit bbcd927

Browse files
authored
Merge branch 'main' into hp_section
2 parents 96fbf0e + cbbc02d commit bbcd927

2 files changed

Lines changed: 95 additions & 19 deletions

File tree

docs/deployments/kubernetes/install-csi.md

Lines changed: 65 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,7 @@ To enable interaction with multiple clusters, there are two key changes:
169169

170170
When the Simplyblock CSI driver is initially installed, only a single cluster can be referenced.
171171

172-
```
172+
```bash title="Install the Simplyblock CSI driver via Helm"
173173
helm install simplyblock-csi ./ \
174174
--set csiConfig.simplybk.uuid=${CLUSTER_ID} \
175175
--set csiConfig.simplybk.ip=${CLUSTER_IP} \
@@ -210,12 +210,14 @@ To add a new cluster, the current secret must be retrieved from Kubernetes, edit
210210
and uploaded to the Kubernetes cluster.
211211

212212

213-
```sh
213+
```bash
214214
# Save cluster secret to a file
215-
kubectl get secret simplyblock-csi-secret-v2 -o jsonpath='{.data.secret\.json}' | base64 --decode > secret.yaml
215+
kubectl get secret simplyblock-csi-secret-v2 \
216+
-o jsonpath='{.data.secret\.json}' |\
217+
base64 --decode > secret.yaml
216218
217-
# Edit the clusters and add the new cluster's cluster_id, cluster_endpoint, cluster_secret
218-
# vi secret.json
219+
# Edit the clusters and add the new cluster's cluster_id,
220+
# cluster_endpoint, cluster_secret vi secret.json
219221
220222
cat secret.json | base64 | tr -d '\n' > secret-encoded.json
221223
@@ -227,12 +229,10 @@ kubectl -n simplyblock edit secret simplyblock-csi-secret-v2
227229

228230
#### Option 1: Cluster ID–Based Method (One StorageClass per Cluster)
229231

230-
In this approach, each SimplyBlock cluster has its own dedicated StorageClass that specifies which cluster to use for provisioning.
232+
In this approach, each simplyblock cluster has its own dedicated StorageClass that specifies which cluster to use for provisioning.
231233
This is ideal for setups where workloads are manually directed to specific clusters.
232234

233-
For example:
234-
235-
```yaml
235+
```yaml title="Example of Cluster ID-Based Selection"
236236
apiVersion: storage.k8s.io/v1
237237
kind: StorageClass
238238
metadata:
@@ -248,7 +248,7 @@ allowVolumeExpansion: true
248248

249249
You can define another StorageClass for a different cluster:
250250

251-
```yaml
251+
```yaml title="Example of selecting another cluster"
252252
apiVersion: storage.k8s.io/v1
253253
kind: StorageClass
254254
metadata:
@@ -263,26 +263,24 @@ allowVolumeExpansion: true
263263
```
264264

265265
Each StorageClass references a unique cluster_id.
266-
The CSI driver uses that ID to determine which SimplyBlock cluster to connect to.
266+
The CSI driver uses that ID to determine which simplyblock cluster to connect to.
267267

268268
#### Option 2: Zone-Aware Method (Automatic Multi-Cluster Selection)
269269

270-
This approach allows a single StorageClass to automatically select the appropriate SimplyBlock cluster based on the Kubernetes zone where the workload runs.
271-
It is recommended for multi-zone Kubernetes deployments that span multiple SimplyBlock clusters.
270+
This approach allows a single StorageClass to automatically select the appropriate simplyblock cluster based on the Kubernetes zone where the workload runs.
271+
It is recommended for multi-zone Kubernetes deployments that span multiple simplyblock clusters.
272272

273273
`storageclass.zoneClusterMap`
274274

275-
Sets the mapping between Kubernetes zones and SimplyBlock cluster IDs.
275+
Sets the mapping between Kubernetes zones and simplyblock cluster IDs.
276276
Each zone is associated with one cluster.
277277

278278
`storageclass.allowedTopologyZones`
279279

280280
Sets the list of zones where the StorageClass is permitted to provision volumes.
281281
This ensures that scheduling aligns with the clusters defined in `zoneClusterMap`.
282282

283-
example:
284-
285-
```yaml
283+
```yaml title="Example of zoneClusterMap usage"
286284
apiVersion: storage.k8s.io/v1
287285
kind: StorageClass
288286
metadata:
@@ -303,4 +301,53 @@ allowedTopologies:
303301
- us-east-1b
304302
```
305303

306-
This method allows Kubernetes to automatically pick the right cluster based on the pod’s scheduling zone.
304+
This method allows Kubernetes to automatically pick the right cluster based on the pod’s scheduling zone.
305+
306+
#### Option 3: Region-Aware Method (Automatic Multi-Cluster Selection)
307+
308+
This approach allows a single StorageClass to automatically select the appropriate simplyblock cluster based on the Kubernetes region where the workload runs.
309+
It’s recommended when:
310+
311+
- your cluster spans multiple regions, and
312+
313+
- each region maps to a different simplyblock backend, or
314+
315+
- you want region-scoped placement rather than zone-scoped placement
316+
317+
`storageclass.regionClusterMap`
318+
319+
Sets the mapping between Kubernetes regions and simplyblock cluster IDs.
320+
Each region is associated with one cluster.
321+
322+
`storageclass.allowedTopologyRegions`
323+
324+
Sets the list of regions where the StorageClass is permitted to provision volumes.
325+
This ensures scheduling aligns with the clusters defined in `regionClusterMap`.
326+
327+
```yaml title="Example of regionClusterMap usage"
328+
apiVersion: storage.k8s.io/v1
329+
kind: StorageClass
330+
metadata:
331+
name: simplyblock-csi-sc
332+
provisioner: csi.simplyblock.io
333+
parameters:
334+
region_cluster_map: |
335+
{"us-east-1":"cluster-uuid-a","us-west-2":"cluster-uuid-b"}
336+
... other parameters
337+
reclaimPolicy: Delete
338+
volumeBindingMode: WaitForFirstConsumer
339+
allowVolumeExpansion: true
340+
allowedTopologies:
341+
- matchLabelExpressions:
342+
- key: topology.kubernetes.io/region
343+
values:
344+
- us-east-1
345+
- us-east-1
346+
```
347+
348+
This method allows Kubernetes to automatically pick the right cluster based on the pod’s scheduling region.
349+
350+
!!! tip
351+
The keys inside `region_cluster_map` must match the region labels present on your Kubernetes nodes
352+
(typically `topology.kubernetes.io/region`). You can include as many regions as needed, each pointing to
353+
the cluster ID defined in `simplyblock-csi-secret-v2`.

docs/deployments/kubernetes/k8s-storage-plane.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -176,8 +176,37 @@ mode. The most important ones are:
176176
| `storagenode.pciAllowed` | Sets the list of allowed NVMe PCIe addresses. | `<empty>` |
177177
| `storagenode.pciBlocked` | Sets the list of blocked NVMe PCIe addresses. | `<empty>` |
178178
| `storagenode.socketsToUse` | Sets the list of NUMA sockets to use. If a worker node has more than 1 NUMA socket, it is possible to deploy more than one simplyblock storage node per host, depending on the distribution of NVMe devices and NICs across NUMA sockets and the resource demand of other workloads. | 1 |
179-
| `storagenode.nodesPerSocket` | Sets the number of storage nodes to be deployed per NUMA socket. It is possible to deploy one or two storage nodes per socket. This improves performance if one each NUMA socket has more than 32 cores. | 1 |
179+
| `storagenode.nodesPerSocket` | Sets the number of storage nodes to be deployed per NUMA socket. It is possible to deploy one or two storage nodes per socket. This improves performance if one each NUMA socket has more than 32 cores.| 1 |
180+
| `storagenode.haJMCount` | Number of HA Journal Managers. | `<empty>` |
180181
| `storagenode.coresPercentage` | Sets the percentage of total cores (vCPUs) available to simplyblock storage node services. It must be ensured that the configured percentage yields at least 8 vCPUs per storage node. For example, if a host has 128 vCPUs on two NUMA sockets (64 each) and `--storagenode.socketsToUse=2` and `--storagenode.nodesPerSocket=1`, at least 13% (as `13% * 64 > 8`) must be set. Simplyblock does not use more than 32 vCPUs per storage node efficiently. | `<empty>` |
182+
| `storagenode.deviceModel` | NVMe SSD model to use for device selection. Must be set together with `storagenode.sizeRange`. | `<empty>` |
183+
| `storagenode.sizeRange` | NVMe SSD device size range expressed as `min-max` (for example `500G-1T`). Used together with `storagenode.deviceModel` to filter eligible devices. | `<empty>` |
184+
| `storagenode.deviceNames` | Comma separated list of NVMe namespace names to be used by the storage node (for example `nvme0n1`, `nvme1n1`). Do **not** include the `/dev/` prefix. | `<empty>` |
185+
| `storagenode.format4k` | If set to `true`, the specified devices will be formatted with a 4K sector size. Recommended for modern NVMe drives when supported. | `false` |
186+
| `storagenode.multiCluster.enable` | Enable multi-cluster storage node support. When enabled, a single Kubernetes cluster can host storage nodes connected to multiple simplyblock clusters. | `false` |
187+
| `storagenode.multiCluster.clusters[].cluster_id` | UUID of the simplyblock cluster this storage node should connect to. | `<empty>` |
188+
| `storagenode.multiCluster.clusters[].secret` | Secret used to authenticate against the specified simplyblock cluster. | `<empty>` |
189+
| `storagenode.multiCluster.clusters[].workers` | List of Kubernetes worker node names assigned to this simplyblock cluster. Storage nodes will only be scheduled on these workers. | `<empty>` |
190+
191+
192+
### Multi-Cluster Storage Node Support
193+
194+
```yaml title="Multi-Cluster definition"
195+
storagenode:
196+
multiCluster:
197+
enable: true
198+
clusters:
199+
- cluster_id: cluster-uuid-1
200+
secret: cluster-secret-id-1
201+
workers:
202+
- worker-a-1
203+
- worker-a-2
204+
- cluster_id: cluster-uuid-2
205+
secret: cluster-secret-id-2
206+
workers:
207+
- worker-b-1
208+
- worker-b-2
209+
```
181210
182211
!!! warning
183212
The resources consumed by simplyblock are exclusively used and have to be aligned with resources required by other

0 commit comments

Comments
 (0)