e2e: add acceptance minikube quick e2e smoke gate by Rakshith-R · Pull Request #6437 · ceph/ceph-csi

Rakshith-R · 2026-07-29T13:24:23Z

Summary

Add a lightweight acceptance e2e workflow that runs on every PR via
GitHub Actions using minikube + Rook Ceph. This gates basic
provisioning, snapshot, and clone operations across RBD, CephFS, and
NFS drivers (12 specs total) before the heavier CentOS mini-e2e suite.

Deploy ceph-csi via ceph-csi-operator (CRDs: Driver, OperatorConfig)
instead of manual manifests
Tag 12 core specs with Label("acceptance") — contributors can add
the label to new feature specs for quick e2e verification loops in
their fork
Add KUBE_VERSION to build.env as single source of truth
Add scripts/github-action-helper.sh with install_minikube_prereqs,
prepare_disk, and collect_logs functions
Add skip-vault flag to skip Vault KMS deployment for faster runs
Tolerate pre-existing KMS configmap in operator deployment mode
Document acceptance suite in e2e/README.md and AGENTS.md

Acceptance specs (12 total)

Driver	Specs
RBD	PVC→app, snapshot→clone, PVC-PVC clone, block PVC
CephFS	health check, PVC→app, snapshot→clone, PVC-PVC clone
NFS	health check, PVC→app, snapshot→clone, PVC-PVC clone

Test plan

All lints pass (golangci-lint, lint-extras, codespell, link-check)
Unit tests pass (go-test, go-test-api, e2e-build-test)
mod-check passes
commitlint + DCO pass
e2e-acceptance passes with operator deployment (3 consecutive
green runs verified on upstream PR)

🤖 Generated with Claude Code

Rakshith-R · 2026-07-30T10:06:35Z

/test ci/centos/mini-e2e/k8s-1.36

Rakshith-R · 2026-07-31T11:50:05Z

/test ci/centos/mini-e2e/k8s-1.36

Rakshith-R · 2026-08-03T07:31:14Z

/test ci/centos/mini-e2e-helm/k8s-1.36

Rakshith-R · 2026-08-03T07:45:38Z

/test ci/centos/mini-e2e-operator/k8s-1.36

Rakshith-R · 2026-08-04T09:13:59Z

@Mergifyio rebase

mergify · 2026-08-04T09:14:48Z

rebase

🛑 The pull request rule doesn't match anymore

Details

This action has been cancelled.

Copilot

Pull request overview

This PR adds a lightweight “acceptance” E2E smoke gate that runs on every PR using GitHub Actions with minikube + Rook Ceph, to catch basic provisioning/snapshot/clone regressions across RBD, CephFS, and NFS before the heavier CentOS mini-e2e jobs.

Changes:

Add a new e2e-minikube-acceptance GitHub Actions workflow that boots minikube, deploys Rook + the ceph-csi operator, and runs only Label("acceptance") specs.
Tag 12 core E2E specs with Label("acceptance"), add a --skip-vault flag, and create an empty KMS ConfigMap when Vault is skipped.
Add a GitHub Actions helper script, update minikube/kubectl install logic, and document the acceptance suite (plus centralize KUBE_VERSION in build.env).

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`scripts/rook.sh`	Adjusts Rook pool readiness checks used during minikube-based CI deployments.
`scripts/minikube.sh`	Updates kubectl download URL and handles kubelet verbosity tweaks differently for `VM_DRIVER=none`.
`scripts/github-action-helper.sh`	New helper functions for installing prerequisites, preparing an OSD disk, and collecting logs in Actions.
`e2e/utils.go`	Adds helper to create an empty KMS ConfigMap to support `--skip-vault`.
`e2e/e2e_test.go`	Introduces `--skip-vault` flag wiring for the E2E test binary.
`e2e/rbd.go`	Skips Vault deployment when requested and labels selected RBD specs as `acceptance`.
`e2e/cephfs.go`	Skips Vault deployment when requested and labels selected CephFS specs as `acceptance`.
`e2e/nfs.go`	Labels selected NFS specs as `acceptance`.
`e2e/README.md`	Documents how to run/extend the acceptance E2E suite locally and in forks.
`build.env`	Adds `KUBE_VERSION` as the centralized Kubernetes version for minikube workflows/scripts.
`AGENTS.md`	Documents the new acceptance smoke gate for contributors/agents.
`.github/workflows/e2e-minikube-acceptance.yaml`	New workflow to run the acceptance suite on every PR.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

scripts/github-action-helper.sh:39

After creating/logging into the iSCSI device, the extra-disk re-detection repeats the same (loop|${boot_dev}) pattern. If boot_dev is empty this again becomes (loop|) and filters out everything, leaving extra_dev empty even though the iSCSI disk exists.

    extra_dev="$(sudo lsblk --noheading --list \
      --nodeps --output KNAME \
      | grep -Ev "(loop|${boot_dev})" | head -1)"

scripts/github-action-helper.sh:113

collect_logs runs under set -e, but the kubectl ... -o jsonpath=... inside the $(...) can return non-zero (e.g., namespace missing), which will abort log collection early. Make the pod-listing tolerant to failures (similar to the other kubectl calls) before iterating.

    kubectl -n "${ns}" get pods -o wide > "${LOG_DIR}/${ns}-pods.txt" 2>&1 || true
    kubectl -n "${ns}" get events --sort-by='.lastTimestamp' > "${LOG_DIR}/${ns}-events.txt" 2>&1 || true
    for pod in $(kubectl -n "${ns}" get pods -o jsonpath='{.items[*].metadata.name}' 2>/dev/null); do
      kubectl -n "${ns}" logs "${pod}" --all-containers --tail=200 > "${LOG_DIR}/${ns}-${pod}.log" 2>&1 || true
    done

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (5)

scripts/github-action-helper.sh:111

This curl download can succeed with an HTTP error page (e.g. 404) and only fail later during tar, which makes diagnosing transient or versioning issues harder. Using -f makes curl fail fast on HTTP errors.

  # CNI plugins
  local cniv="v1.6.2"
  local cni_base="https://github.com"
  cni_base+="/containernetworking/plugins"
  local cni_tar="cni-plugins-linux-amd64-${cniv}.tgz"
  sudo mkdir -p /opt/cni/bin
  curl -sLO \
    "${cni_base}/releases/download/${cniv}/${cni_tar}"
  sudo tar -xzf "${cni_tar}" -C /opt/cni/bin
  rm -f "${cni_tar}"

scripts/rook.sh:229

Timeout message is grammatically unclear and does not include the pool name, which makes failures harder to diagnose in CI logs.

	if [ "$retry" -gt "$ROOK_DEPLOY_TIMEOUT" ]; then
		echo "[Timeout] Failed to get RBD pool Ready"
		return 1

scripts/github-action-helper.sh:25

find_extra_block_dev assigns to extra_dev without declaring it local, which leaks a global variable and can cause hard-to-debug interactions if the caller (or environment) uses the same name.

    exclude+="|${root_dev}"
  fi
  extra_dev="$(sudo lsblk --noheading --list --nodeps --output KNAME | grep -Ev "(${exclude})" | head -1)"
  if [ -z "${extra_dev}" ]; then

scripts/github-action-helper.sh:57

prepare_disk proceeds even if find_extra_block_dev returns an empty string, which makes the subsequent /dev/${BLOCK} operations no-ops (because of || true) and can hide a real setup failure. Fail fast when no block device is available.

prepare_disk() {
  : "${BLOCK:=$(find_extra_block_dev)}"
  sudo swapoff --all --verbose || true

scripts/github-action-helper.sh:98

This curl download can succeed with an HTTP error page (e.g. 404) and only fail later during tar, which makes diagnosing transient or versioning issues harder. Using -f makes curl fail fast on HTTP errors.

This issue also appears on line 102 of the same file.

  local crictlv="v1.35.0"
  local crictl_base="https://github.com"
  crictl_base+="/kubernetes-sigs/cri-tools"
  local crictl_tar="crictl-${crictlv}-linux-amd64.tar.gz"
  curl -sLO \
    "${crictl_base}/releases/download/${crictlv}/${crictl_tar}"
  sudo tar -xzf "${crictl_tar}" -C /usr/local/bin
  rm -f "${crictl_tar}"

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Suppressed comments (3)

scripts/github-action-helper.sh:49

After creating the iSCSI backing device, the second probe for extra_dev has the same issue as the first: it does not filter to TYPE=="disk" and may return an unexpected device name. Keep the disk-type filter here too so prepare_disk is deterministic and safe.

    extra_dev="$(sudo lsblk --noheading --list \
      --nodeps --output KNAME \
      | grep -Ev "(${exclude})" | head -1)"

scripts/minikube.sh:275

In the VM_DRIVER="none" path this script runs with #!/bin/bash -e, so systemctl daemon-reload / systemctl restart kubelet will hard-fail the workflow if kubelet is not managed by systemd or if /etc/systemd/system/kubelet.service.d/10-kubeadm.conf does not exist on the runner. Guard these commands so the minikube bring-up remains robust across runner images.

    if [[ "${VM_DRIVER}" == "none" ]]; then
        sudo sed -i 's/\(ExecStart=\/var.*\)/\1 --v=4/' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf || true
        sudo systemctl daemon-reload
        sudo systemctl restart kubelet
    else

e2e/utils.go:2156

createEmptyKMSConfigMap caches success in a package-global kmsConfigMapCreated boolean even though the function takes ns as an argument. If this helper is ever called for a different namespace (or if the suite is executed with parallel ginkgo processes), this cache can incorrectly skip creation and also introduces a potential data race. Consider removing the global cache (the Create call is already idempotent via IsAlreadyExists) or keying it by namespace using sync.Once/atomic primitives.

func createEmptyKMSConfigMap(c kubernetes.Interface, ns string) error {
	if kmsConfigMapCreated {
		return nil
	}

mergify · 2026-08-13T08:30:16Z

Merge Queue Status

✅ Entered queue — 2026-08-13 08:30 UTC · Rule: default · triggered by merge protections
❌ Checks failed · in-place
🚫 Left the queue — 2026-08-13 09:02 UTC · at 258cea1cd1919dd34020252e0a3a98ffe27b3f1c

This pull request spent 31 minutes 55 seconds in the queue, with no time running CI.

Required conditions to merge

github-review-approved [🛡 GitHub branch protection]
- e2e: add acceptance minikube quick e2e smoke gate #6437

Reason

The merge conditions cannot be satisfied due to failing checks

Failing checks:

🟠 multi-arch-build (job log)
🟠 ci/centos/k8s-e2e-external-storage/1.34
🟠 ci/centos/k8s-e2e-external-storage/1.35
🟠 ci/centos/k8s-e2e-external-storage/1.36
🟠 ci/centos/mini-e2e-helm/k8s-1.34
🟠 ci/centos/mini-e2e-helm/k8s-1.35
🟠 ci/centos/mini-e2e-helm/k8s-1.36
🟠 ci/centos/mini-e2e/k8s-1.34
🟠 ci/centos/mini-e2e/k8s-1.35
🟠 ci/centos/mini-e2e/k8s-1.36
🟠 ci/centos/upgrade-tests-rbd

Hint

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

Requeued — the merge queue status continues in this comment ↓.

Add a lightweight acceptance e2e workflow that runs on every PR via GitHub Actions using minikube + Rook Ceph. This gates basic provisioning, snapshot, and clone operations across RBD, CephFS, and NFS drivers (12 specs total) before the heavier CentOS mini-e2e suite. The acceptance suite uses ceph-csi-operator deployment mode, where the operator deploys ceph-csi drivers via CRDs (Driver, OperatorConfig) instead of manual manifests. Key changes: - Add e2e-minikube-acceptance.yaml workflow with operator deployment - Tag 12 core specs with Label("acceptance") across drivers - Add skip-vault flag to skip Vault KMS in smoke runs - Add KUBE_VERSION to build.env as single source of truth - Add github-action-helper.sh with install_minikube_prereqs, prepare_disk, and collect_logs functions - Tolerate pre-existing KMS configmap in operator mode - Document acceptance suite in e2e/README.md and AGENTS.md - Contributors can label new feature specs with acceptance and run the gate in their own fork for quick e2e verification loops Assisted-by: Claude <noreply@anthropic.com> Signed-off-by: Rakshith R <rar@redhat.com>

ceph-csi-bot · 2026-08-13T08:32:36Z

/test ci/centos/k8s-e2e-external-storage/1.34

ceph-csi-bot · 2026-08-13T08:32:36Z

/test ci/centos/k8s-e2e-external-storage/1.35

ceph-csi-bot · 2026-08-13T08:32:36Z

/test ci/centos/mini-e2e-helm/k8s-1.34

ceph-csi-bot · 2026-08-13T08:32:37Z

/test ci/centos/upgrade-tests-cephfs

ceph-csi-bot · 2026-08-13T08:32:37Z

/test ci/centos/k8s-e2e-external-storage/1.36

ceph-csi-bot · 2026-08-13T08:32:37Z

/test ci/centos/mini-e2e-helm/k8s-1.35

ceph-csi-bot · 2026-08-13T08:32:37Z

/test ci/centos/mini-e2e/k8s-1.34

ceph-csi-bot · 2026-08-13T08:32:37Z

/test ci/centos/upgrade-tests-rbd

ceph-csi-bot · 2026-08-13T08:32:38Z

/test ci/centos/mini-e2e-helm/k8s-1.36

ceph-csi-bot · 2026-08-13T08:32:38Z

/test ci/centos/mini-e2e/k8s-1.35

ceph-csi-bot · 2026-08-13T08:32:38Z

/test ci/centos/mini-e2e/k8s-1.36

nixpanic · 2026-08-13T09:25:43Z

/test ci/centos/k8s-e2e-external-storage/1.36

nixpanic · 2026-08-13T09:26:19Z

/test ci/centos/k8s-e2e-external-storage/1.36

Failed during deploying minikube/rook (logs)

nixpanic · 2026-08-13T09:34:49Z

/test ci/centos/mini-e2e/k8s-1.34

nixpanic · 2026-08-13T09:35:05Z

/test ci/centos/mini-e2e/k8s-1.34

nfs Test NFS CSI [It] create a PVC-PVC clone and bind it to an app [acceptance]

mergify · 2026-08-13T12:19:41Z

Merge Queue Status

✅ Entered queue — 2026-08-13 12:19 UTC · Rule: default · triggered by merge protections
✅ Checks skipped · PR is already up-to-date
✅ Merged — 2026-08-13 12:20 UTC · at 258cea1cd1919dd34020252e0a3a98ffe27b3f1c · rebase

This pull request spent 39 seconds in the queue, including 7 seconds running CI.

Required conditions to merge

github-review-approved [🛡 GitHub branch protection]
- e2e: add acceptance minikube quick e2e smoke gate #6437

mergify Bot added the component/testing Additional test cases or CI work label Jul 29, 2026

Rakshith-R force-pushed the e2e-minikube-tier1 branch 2 times, most recently from c95b027 to 916dba7 Compare July 29, 2026 13:27

Rakshith-R marked this pull request as draft July 29, 2026 13:32

Rakshith-R force-pushed the e2e-minikube-tier1 branch 2 times, most recently from 0168d44 to db186af Compare July 30, 2026 08:07

Rakshith-R marked this pull request as ready for review August 4, 2026 09:13

ceph-csi-bot force-pushed the e2e-minikube-tier1 branch from db186af to c3dee6a Compare August 4, 2026 09:14

Rakshith-R requested a lite review from Copilot August 4, 2026 09:27

Copilot started reviewing on behalf of Rakshith-R August 4, 2026 09:28 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Comment thread .github/workflows/e2e-minikube-acceptance.yaml

Comment thread scripts/github-action-helper.sh Outdated

Comment thread scripts/rook.sh

Rakshith-R marked this pull request as draft August 4, 2026 09:40

Rakshith-R force-pushed the e2e-minikube-tier1 branch from c3dee6a to 66033d3 Compare August 4, 2026 09:43

Rakshith-R requested a lite review from Copilot August 4, 2026 09:48

Copilot started reviewing on behalf of Rakshith-R August 4, 2026 09:48 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Comment thread scripts/github-action-helper.sh

Rakshith-R force-pushed the e2e-minikube-tier1 branch from 66033d3 to 55b35a4 Compare August 4, 2026 10:13

Rakshith-R requested a lite review from Copilot August 4, 2026 10:13

Copilot started reviewing on behalf of Rakshith-R August 4, 2026 10:14 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Rakshith-R requested a lite review from Copilot August 4, 2026 11:34

Copilot started reviewing on behalf of Rakshith-R August 4, 2026 11:34 View session

Copilot AI reviewed Aug 4, 2026

View reviewed changes

Comment thread scripts/github-action-helper.sh

Comment thread scripts/github-action-helper.sh

Rakshith-R force-pushed the e2e-minikube-tier1 branch from 55b35a4 to 41f623f Compare August 4, 2026 11:45

nixpanic mentioned this pull request Aug 13, 2026

doc: improve references to the published documentation #6466

Merged

ceph-csi-bot force-pushed the e2e-minikube-tier1 branch from 12999a0 to 258cea1 Compare August 13, 2026 08:30

mergify Bot added the ok-to-test Label to trigger E2E tests label Aug 13, 2026

ceph-csi-bot added ci/in-progress/e2e This label acts like a guard and prevents Mergify from adding the `ok-to-test` label again. and removed ok-to-test Label to trigger E2E tests labels Aug 13, 2026

mergify Bot added queued dequeued and removed dequeued ci/in-progress/e2e This label acts like a guard and prevents Mergify from adding the `ok-to-test` label again. queued labels Aug 13, 2026

mergify Bot merged commit f61d392 into ceph:devel Aug 13, 2026
50 checks passed

mergify Bot removed the dequeued label Aug 13, 2026

Uh oh!

Conversation

Rakshith-R commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Acceptance specs (12 total)

Test plan

Uh oh!

Rakshith-R commented Jul 30, 2026

Uh oh!

Rakshith-R commented Jul 31, 2026

Uh oh!

Rakshith-R commented Aug 3, 2026

Uh oh!

Rakshith-R commented Aug 3, 2026

Uh oh!

Rakshith-R commented Aug 4, 2026

Uh oh!

mergify Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛑 The pull request rule doesn't match anymore

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

mergify Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Queue Status

Reason

Hint

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

ceph-csi-bot commented Aug 13, 2026

Uh oh!

nixpanic commented Aug 13, 2026

Uh oh!

nixpanic commented Aug 13, 2026

Uh oh!

nixpanic commented Aug 13, 2026

Uh oh!

nixpanic commented Aug 13, 2026

Rakshith-R commented Jul 29, 2026 •

edited

Loading

mergify Bot commented Aug 4, 2026 •

edited

Loading

mergify Bot commented Aug 13, 2026 •

edited

Loading

mergify Bot commented Aug 13, 2026 •

edited

Loading