Add service account based volume access restriction by Rakshith-R · Pull Request #6076 · ceph/ceph-csi

Rakshith-R · 2026-02-17T12:40:35Z

Describe what this PR does

Service Account Based Volume Access Restriction

Ceph-CSI supports optionally restricting RBD/CephFS volume access to specific Kubernetes
service accounts. When configured, only pods running with the allowed service account
can mount the volume. This feature uses RBD image/cephFS subvolume metadata to store the
restriction and the CSI podInfoOnMount mechanism to identify the pod's service
account during mount. Refer https://kubernetes-csi.github.io/docs/pod-info.html#pod-info-on-mount-with-csi-driver-object.

How it works

A user sets the .rbd.csi.ceph.com/serviceaccount metadata on an
RBD image to specify the allowed service account name.
During ControllerPublishVolume, Ceph-CSI reads this metadata and passes
it to the node via publish context.
During NodePublishVolume, Ceph-CSI compares the value against the
pod's service account (provided via volume context by Kubelet).
If the service account was set in metadata and does not match the pod's
service account, the mount is rejected with a PermissionDenied error.

Prerequisites

The podInfoOnMount field must be set to true in the CSIDriver spec so that
Kubelet passes pod information (including service account name) in the volume
context during NodePublishVolume. Without this, the restriction cannot be
enforced and all mounts are allowed.

Setting the restriction on an RBD image/nvmeof

Use the rbd image-meta set command to set the allowed service account:

rbd image-meta set <pool>/<image> .rbd.csi.ceph.com/serviceaccount <service-account-name>

For example, to restrict a volume to the my-app-sa service account:

rbd image-meta set mypool/csi-vol-abc123 .rbd.csi.ceph.com/serviceaccount my-app-sa

Removing the restriction

To remove the restriction and allow any service account to mount the volume:

rbd image-meta remove <pool>/<image> .rbd.csi.ceph.com/serviceaccount

The Deployment should be scaled down completely and then scaled up for this removing
the restriction after removing metadata from the image.

Setting the restriction on a CephFS subvolume/nfs export

Use the ceph fs subvolume metadata set command to set the allowed service account:

ceph fs subvolume metadata set <filesystem> <subvolume> --group_name=<group> \
  .cephfs.csi.ceph.com/serviceaccount <service-account-name>

For example, to restrict a volume to the my-app-sa service account:

ceph fs subvolume metadata set myfs csi-vol-abc123 --group_name=csi \
  .cephfs.csi.ceph.com/serviceaccount my-app-sa

Removing the restriction

To remove the restriction and allow any service account to mount the volume:

ceph fs subvolume metadata rm <filesystem> <subvolume> --group_name=<group> \
  .cephfs.csi.ceph.com/serviceaccount

The Deployment should be scaled down completely and then scaled up for
removing the restriction after removing metadata from the subvolume.

Use Case: Ceph VolSync Plugin Replication Destination PVC Protection

A primary motivator for this feature is the custom
Ceph VolSync Plugin that
performs incremental data replication across clusters. In a disaster recovery
or migration workflow:

A ReplicationDestination controller creates a PVC on the destination
cluster to receive replicated data.
A replication worker pod, running under a dedicated service account (e.g.
volsync-worker-sa), incrementally syncs data from the source cluster into
this destination PVC.
The destination PVC must remain writable only by the replication worker
until the replication is complete and a failover is triggered.

Without service account based restriction, any pod in the namespace with a
reference to the destination PVC could write to it, potentially corrupting the
replicated data or breaking the incremental sync state. By binding the
destination volume to the replication worker's service account, the volume is
protected from unintended writes throughout the replication lifecycle. On
failover, the restriction is removed so the application workload can mount
the volume.

Checklist:

Commit Message Formatting: Commit titles and messages follow
guidelines in the developer
guide.
Reviewed the developer guide on Submitting a Pull
Request
Pending release
notes
updated with breaking and/or notable changes for the next major release.
Documentation has been updated, if necessary.
Unit tests have been added, if necessary.
Integration tests have been added, if necessary.

Show available bot commands

These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)

Rakshith-R · 2026-02-17T12:44:53Z

/test ci/centos/mini-e2e/k8s-1.34

Rakshith-R · 2026-02-17T12:45:12Z

/test ci/centos/mini-e2e-helm/k8s-1.34

Rakshith-R · 2026-02-17T16:03:42Z

/test ci/centos/mini-e2e/k8s-1.34/cephfs

Rakshith-R · 2026-02-17T16:40:29Z

/test ci/centos/mini-e2e/k8s-1.34

Rakshith-R · 2026-02-18T10:06:57Z

/test ci/centos/mini-e2e/k8s-1.34

Rakshith-R · 2026-02-18T11:20:50Z

/test ci/centos/mini-e2e/k8s-1.34/rbd

Rakshith-R · 2026-02-19T05:14:07Z

/test ci/centos/mini-e2e/k8s-1.34/cephfs

Rakshith-R · 2026-02-19T07:52:29Z

rbd alone takes
run e2e

1h 36m 20s

https://jenkins-ceph-csi.apps.ocp.cloud.ci.centos.org/blue/organizations/jenkins/mini-e2e_k8s-1.34-rbd/detail/mini-e2e_k8s-1.34-rbd/6/pipeline/90

cephfs takes

run e2e

55m 27s

https://jenkins-ceph-csi.apps.ocp.cloud.ci.centos.org/blue/organizations/jenkins/mini-e2e_k8s-1.34-cephfs/detail/mini-e2e_k8s-1.34-cephfs/17/pipeline/90

We need to increase e2e timeout for now, and then work on optimising e2e testcases.

Rakshith-R · 2026-02-19T09:13:53Z

/test ci/centos/mini-e2e/k8s-1.34

Rakshith-R · 2026-02-19T10:33:45Z

/test ci/centos/mini-e2e/k8s-1.34

nvmeof testcase passed here: #6097

https://jenkins-ceph-csi.apps.ocp.cloud.ci.centos.org/blue/rest/organizations/jenkins/pipelines/mini-e2e_k8s-1.35-nvmeof/runs/22/nodes/90/log/?start=0

Rakshith-R · 2026-02-19T13:07:41Z

/test ci/centos/mini-e2e/k8s-1.34

Madhu-1 · 2026-02-20T05:03:31Z

@Rakshith-R, what is the usecase this one is trying to cover? its good to have a design first to see how feasible this solution is from a storage admin point of view?

Rakshith-R · 2026-02-20T06:54:30Z

@Rakshith-R, what is the usecase this one is trying to cover? its good to have a design first to see how feasible this solution is from a storage admin point of view?

Added design proposal document.

Use Case: Ceph VolSync Plugin Replication Destination PVC Protection

A primary motivator for this feature is the custom
Ceph VolSync Plugin that
performs incremental data replication across clusters. In a disaster recovery
or migration workflow:

A ReplicationDestination controller creates a PVC on the destination
cluster to receive replicated data.
A replication worker pod, running under a dedicated service account (e.g.
volsync-worker-sa), incrementally syncs data from the source cluster into
this destination PVC.
The destination PVC must remain writable only by the replication worker
until the replication is complete and a failover is triggered.

Without service account based restriction, any pod in the namespace with a
reference to the destination PVC could write to it, potentially corrupting the
replicated data or breaking the incremental sync state. By binding the
destination volume to the replication worker's service account, the volume is
protected from unintended writes throughout the replication lifecycle. On
failover, the restriction is removed so the application workload can mount
the volume.

nixpanic

Went through the design as a 1st step. Looks okayish to me, but I do have a few questions.

docs/design/proposals/sa-based-volume-access-restriction.md

docs/design/proposals/k8s-sa-based-volume-access-restriction.md

docs/design/proposals/sa-based-volume-access-restriction.md

nixpanic · 2026-02-20T08:27:31Z

docs/design/proposals/sa-based-volume-access-restriction.md

+### Metadata Keys
+
+Each driver type uses a driver-specific metadata key to store the allowed
+service account name:


what is the format of the value? Does it include the Kubernetes Namespace of the ServiceAccount? If not, would the Namespace add extra security?

what is the format of the value? Does it include the Kubernetes Namespace of the ServiceAccount? If not, would the Namespace add extra security?

It is just the serviceAccount name.
The pvc and pod are already namespaced and hence NodePublish request is also namespaced.
If a NodePublish request for same volume and with same serviceAccount has to come from another NS,
A user will need to have clusterScoped permission to create a duplicate PV and PVC as well as a a serviceAccount with same name in another namespace, which is very unlikely and another security threat outside the scope here.

But I've mentioned additional parameters (e.g. namespace) validation in future enhancement section.

nixpanic · 2026-02-20T08:31:20Z

docs/design/proposals/k8s-sa-based-volume-access-restriction.md

+   metadata from the Ceph backend. If present, it is included in the publish
+   context passed to the node.
+
+1. **NodePublishVolume**: The node plugin compares the publish context value


This isn't a very safe way. Both the value that is expected and the value that is checked are in the same CSI RPC call? It isn't really how security checks for access restrictions should work.

Why not read the metadata during NodePublishVolume?

This isn't a very safe way. Both the value that is expected and the value that is checked are in the same CSI RPC call? It isn't really how security checks for access restrictions should work.

Why not read the metadata during NodePublishVolume?

Yes, the value that is expected and the value that is checked are in the same CSI RPC call
but the source of these values are from two different places.

expected <- publish_context <- controllerPublishVolume call <- image/subvolume metadata

actual checked <- volume_context <- pod's SA name

The spec and k8s storage framework ensures the values are populated in such manner.

Reading it from metadata on every NodePublishVolume(per pod, re-requested with backoff on errors) is resource intensive and can be totally avoided by making use of publish_context in controllerPublishVolume call (once per pvc per node).

I understand that the flow of the Kubernetes CSI procedures makes this work. But it feels wrong to get the valid value from the publish context which is part of the same RPC as the details from podInfoOnMount.

NodePublishVolume is not in a critical path, doing a security check by getting the metadata is not very invasive. It would be more proper to validate it during the call.

I understand that the flow of the Kubernetes CSI procedures makes this work. But it feels wrong to get the valid value from the publish context which is part of the same RPC as the details from podInfoOnMount.

the source of these values are from two different places.
expected <- publish_context <- controllerPublishVolume call <- image/subvolume metadata
actual checked <- volume_context <- pod's SA name

Kubelet and CSI Spec are trusted entities in this framework.
The entire framework relies on this fact.
Therefore, the content of the RPC is very reliable.

NodePublishVolume is not in a critical path, doing a security check by getting the metadata is not very invasive. It would be more proper to validate it during the call.

It is a critical path, every pod using our volume will be able to come up only after a NodePublish.
CephCSI's nodepublish ops are very light weight and fast, almost zero lookups.

Introducing a metadata lookup here would inherently slow the entire setup.

Combining with the above established fact of Kubelet and CSI Spec being reliable and trustworthy entities, we don't need to look at introducing resource intensive ops in NodePublish call.

That's all fine, but it needs documenting in the design too.

docs/design/proposals/sa-based-volume-access-restriction.md

iPraveenParihar · 2026-03-12T10:10:06Z

/retest ci/centos/mini-e2e/k8s-1.34

nixpanic · 2026-03-13T07:59:39Z

@Mergifyio rebase

Allow restricting RBD volume access to a specific Kubernetes ServiceAccount using ".rbd.csi.ceph.com/serviceaccount" image metadata. During ControllerPublishVolume, the controller reads the ".rbd.csi.ceph.com/serviceaccount" metadata from the backing RBD image and passes it to the node via publish context. During NodePublishVolume, the node validates the Pod's ServiceAccount (provided by Kubelet when `podInfoOnMount` is enabled) against the allowed value, returning PermissionDenied on mismatch. Signed-off-by: Rakshith R <rar@redhat.com>

Allow restricting nvmeof volume access to a specific Kubernetes ServiceAccount using ".rbd.csi.ceph.com/serviceaccount" image metadata. During ControllerPublishVolume, the controller reads the ".rbd.csi.ceph.com/serviceaccount" metadata from the backing RBD image and passes it to the node via publish context. During NodePublishVolume, the node validates the Pod's ServiceAccount (provided by Kubelet when `podInfoOnMount` is enabled) against the allowed value, returning PermissionDenied on mismatch. Signed-off-by: Rakshith R <rar@redhat.com>

Allow restricting CephFS volume access to a specific Kubernetes ServiceAccount using ".cephfs.csi.ceph.com/serviceaccount" subvolume metadata. During ControllerPublishVolume, the controller reads the ".cephfs.csi.ceph.com/serviceaccount" metadata from the backing CephFS subvolume and passes it to the node via publish context. During NodePublishVolume, the node validates the Pod's ServiceAccount(provided by Kubelet when `podInfoOnMount` is enabled) against the allowed value, returning PermissionDenied on mismatch. Signed-off-by: Rakshith R <rar@redhat.com>

Allow restricting nfs volume access to a specific Kubernetes ServiceAccount using ".cephfs.csi.ceph.com/serviceaccount" subvolume metadata. During ControllerPublishVolume, the controller delegates to the CephFS backend to read the ".cephfs.csi.ceph.com/serviceaccount" metadata from the backing CephFS subvolume and passes it to the node via publish context. During NodePublishVolume, the node validates the Pod's ServiceAccount (provided by Kubelet when `podInfoOnMount` is enabled) against the allowed value, returning PermissionDenied on mismatch. Signed-off-by: Rakshith R <rar@redhat.com>

…triction Signed-off-by: Rakshith R <rar@redhat.com>

Signed-off-by: Rakshith R <rar@redhat.com>

mergify · 2026-03-13T08:00:00Z

rebase

✅ Branch has been successfully rebased

ceph-csi-bot · 2026-03-13T08:00:18Z

/test ci/centos/k8s-e2e-external-storage/1.33

ceph-csi-bot · 2026-03-13T08:00:18Z

/test ci/centos/upgrade-tests-cephfs

ceph-csi-bot · 2026-03-13T08:00:18Z

/test ci/centos/mini-e2e-helm/k8s-1.33

ceph-csi-bot · 2026-03-13T08:00:18Z

/test ci/centos/upgrade-tests-rbd

ceph-csi-bot · 2026-03-13T08:00:19Z

/test ci/centos/mini-e2e/k8s-1.33

ceph-csi-bot · 2026-03-13T08:00:19Z

/test ci/centos/k8s-e2e-external-storage/1.34

ceph-csi-bot · 2026-03-13T08:00:19Z

/test ci/centos/k8s-e2e-external-storage/1.35

ceph-csi-bot · 2026-03-13T08:00:20Z

/test ci/centos/mini-e2e-helm/k8s-1.34

ceph-csi-bot · 2026-03-13T08:00:20Z

/test ci/centos/mini-e2e-helm/k8s-1.35

ceph-csi-bot · 2026-03-13T08:00:21Z

/test ci/centos/mini-e2e/k8s-1.34

ceph-csi-bot · 2026-03-13T08:00:21Z

/test ci/centos/mini-e2e/k8s-1.35

mergify · 2026-03-13T10:51:33Z

Merge Queue Status

✅ Entered queue — 2026-03-13 10:51 UTC · Rule: default
✅ Checks passed · in-place
✅ Merged — 2026-03-13 10:51 UTC · at ee282df0efe269c89b05d7fb7cdd80baeeaf4fd3

This pull request spent 12 seconds in the queue, with no time running CI.

Required conditions to merge

#approved-reviews-by >= 2 [🛡 GitHub branch protection]
- Add service account based volume access restriction #6076
#changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
- Add service account based volume access restriction #6076

Rakshith-R force-pushed the restrict-volume-by-sa branch from 47467b0 to 713d6e4 Compare February 17, 2026 12:40

Rakshith-R force-pushed the restrict-volume-by-sa branch from 713d6e4 to 7ea4af4 Compare February 17, 2026 16:39

Rakshith-R force-pushed the restrict-volume-by-sa branch from 7ea4af4 to c128ac9 Compare February 18, 2026 10:04

Rakshith-R mentioned this pull request Feb 19, 2026

ci: increase e2e run timeout with 30 minutes #6093

Merged

6 tasks

Rakshith-R force-pushed the restrict-volume-by-sa branch from c128ac9 to c85ac0e Compare February 19, 2026 09:12

nixpanic mentioned this pull request Feb 19, 2026

Test nvmeof SA restriction #6097

Closed

Rakshith-R force-pushed the restrict-volume-by-sa branch from c85ac0e to 8a460ae Compare February 19, 2026 13:06

Rakshith-R force-pushed the restrict-volume-by-sa branch 2 times, most recently from c03ed0c to 8f75d14 Compare February 20, 2026 06:54

Rakshith-R marked this pull request as ready for review February 20, 2026 06:54

Rakshith-R requested review from a team February 20, 2026 06:54

Rakshith-R force-pushed the restrict-volume-by-sa branch from 8f75d14 to c5cfd44 Compare February 20, 2026 07:14

nixpanic reviewed Feb 20, 2026

View reviewed changes

Rakshith-R force-pushed the restrict-volume-by-sa branch 2 times, most recently from 3cfbc63 to b26cb70 Compare February 20, 2026 10:26

Rakshith-R requested a review from nixpanic February 20, 2026 10:31

Rakshith-R requested a review from a team March 13, 2026 05:27

Madhu-1 approved these changes Mar 13, 2026

View reviewed changes

Rakshith-R added 7 commits March 13, 2026 07:59

doc: add PendingReleaseNotes for k8s ServiceAccount volume access res…

872937f

…triction Signed-off-by: Rakshith R <rar@redhat.com>

e2e: account for custom volumeNamePrefix in getImageInfoFromPVC

70f7e31

Signed-off-by: Rakshith R <rar@redhat.com>

doc: add design proposal for K8s SA based volume access restriction

ee282df

Signed-off-by: Rakshith R <rar@redhat.com>

ceph-csi-bot force-pushed the restrict-volume-by-sa branch from cb100ba to ee282df Compare March 13, 2026 07:59

nixpanic added the ok-to-test Label to trigger E2E tests label Mar 13, 2026

ceph-csi-bot removed the ok-to-test Label to trigger E2E tests label Mar 13, 2026

mergify bot added the queued label Mar 13, 2026

mergify bot merged commit 9011880 into ceph:devel Mar 13, 2026
38 checks passed

mergify bot removed the queued label Mar 13, 2026

Conversation

Rakshith-R commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe what this PR does

Service Account Based Volume Access Restriction

How it works

Prerequisites

Setting the restriction on an RBD image/nvmeof

Removing the restriction

Setting the restriction on a CephFS subvolume/nfs export

Removing the restriction

Use Case: Ceph VolSync Plugin Replication Destination PVC Protection

Uh oh!

Rakshith-R commented Feb 17, 2026

Uh oh!

Rakshith-R commented Feb 17, 2026

Uh oh!

Rakshith-R commented Feb 17, 2026

Uh oh!

Rakshith-R commented Feb 17, 2026

Uh oh!

Rakshith-R commented Feb 18, 2026

Uh oh!

Rakshith-R commented Feb 18, 2026

Uh oh!

Rakshith-R commented Feb 19, 2026

Uh oh!

Rakshith-R commented Feb 19, 2026

Uh oh!

Rakshith-R commented Feb 19, 2026

Uh oh!

Rakshith-R commented Feb 19, 2026

Uh oh!

Rakshith-R commented Feb 19, 2026

Uh oh!

Madhu-1 commented Feb 20, 2026

Uh oh!

Rakshith-R commented Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Use Case: Ceph VolSync Plugin Replication Destination PVC Protection

Uh oh!

nixpanic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nixpanic Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Rakshith-R Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

nixpanic Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Rakshith-R Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

nixpanic Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Rakshith-R Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

nixpanic Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

iPraveenParihar commented Mar 12, 2026

Uh oh!

nixpanic commented Mar 13, 2026

Uh oh!

mergify bot commented Mar 13, 2026

✅ Branch has been successfully rebased

Uh oh!

ceph-csi-bot commented Mar 13, 2026

Uh oh!

Rakshith-R commented Feb 17, 2026 •

edited

Loading

Rakshith-R commented Feb 20, 2026 •

edited

Loading