Skip to content

Commit f5cd6f0

Browse files
Rakshith-RRakshith-R
committed
e2e: add acceptance minikube quick e2e smoke gate
Add a lightweight acceptance e2e workflow that runs on every PR via GitHub Actions using minikube + Rook Ceph. This gates basic provisioning, snapshot, and clone operations across RBD, CephFS, and NFS drivers (12 specs total) before the heavier CentOS mini-e2e suite. The acceptance suite uses ceph-csi-operator deployment mode, where the operator deploys ceph-csi drivers via CRDs (Driver, OperatorConfig) instead of manual manifests. Key changes: - Add e2e-minikube-acceptance.yaml workflow with operator deployment - Tag 12 core specs with Label("acceptance") across drivers - Add skip-vault flag to skip Vault KMS in smoke runs - Add KUBE_VERSION to build.env as single source of truth - Add github-action-helper.sh with install_minikube_prereqs, prepare_disk, and collect_logs functions - Tolerate pre-existing KMS configmap in operator mode - Document acceptance suite in e2e/README.md and AGENTS.md - Contributors can label new feature specs with acceptance and run the gate in their own fork for quick e2e verification loops Assisted-by: Claude <noreply@anthropic.com> Signed-off-by: Rakshith R <rar@redhat.com>
1 parent d67d704 commit f5cd6f0

14 files changed

Lines changed: 449 additions & 53 deletions

‎.github/workflows/e2e-minikube-acceptance.yaml‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
---
2+
# Acceptance quick e2e: minikube + Rook Ceph smoke on every PR.
3+
# Complements (does NOT replace) CentOS mini-e2e via ok-to-test.
4+
name: e2e-minikube-acceptance
5+
"on":
6+
pull_request:
7+
branches:
8+
- devel
9+
- "release-v*"
10+
paths-ignore:
11+
- "docs/**"
12+
- "**/*.md"
13+
14+
concurrency:
15+
group: >-
16+
${{ github.workflow }}-${{
17+
github.head_ref || github.ref }}
18+
cancel-in-progress: true
19+
20+
permissions:
21+
contents: read
22+
23+
env:
24+
GO_VERSION_FILE: go.mod
25+
26+
jobs:
27+
e2e-acceptance:
28+
runs-on: ubuntu-24.04
29+
timeout-minutes: 45
30+
steps:
31+
- name: Checkout
32+
uses: actions/checkout@v4
33+
34+
- name: Setup Go
35+
uses: actions/setup-go@v5
36+
with:
37+
go-version-file: ${{ env.GO_VERSION_FILE }}
38+
39+
- name: Source build.env
40+
run: |
41+
set -a
42+
# shellcheck disable=SC1091
43+
source build.env
44+
set +a
45+
{
46+
echo "KUBE_VERSION=${KUBE_VERSION}"
47+
echo "MINIKUBE_VERSION=${MINIKUBE_VERSION}"
48+
echo "ROOK_VERSION=${ROOK_VERSION}"
49+
echo "ROOK_CEPH_CLUSTER_IMAGE=${ROOK_CEPH_CLUSTER_IMAGE}"
50+
echo "SNAPSHOT_VERSION=${SNAPSHOT_VERSION}"
51+
echo "VM_DRIVER=${VM_DRIVER}"
52+
echo "CSI_IMAGE_VERSION=${CSI_IMAGE_VERSION}"
53+
} >> "$GITHUB_ENV"
54+
55+
- name: Install minikube prerequisites
56+
run: >-
57+
scripts/github-action-helper.sh
58+
install_minikube_prereqs
59+
60+
- name: Setup minikube
61+
run: |
62+
sudo sysctl fs.protected_regular=0
63+
scripts/minikube.sh up
64+
env:
65+
MEMORY: "6144"
66+
67+
- name: Prepare disk for OSD
68+
run: >-
69+
scripts/github-action-helper.sh prepare_disk
70+
71+
- name: Pre-pull Ceph image
72+
run: >-
73+
docker pull "$ROOK_CEPH_CLUSTER_IMAGE"
74+
75+
- name: Deploy Rook + Ceph
76+
run: scripts/minikube.sh deploy-rook
77+
env:
78+
ROOK_DEPLOY_TIMEOUT: "600"
79+
KUBECTL_RETRY_DELAY: "5"
80+
81+
- name: Install snapshot controller
82+
run: >-
83+
scripts/minikube.sh install-snapshotter
84+
85+
- name: Build ceph-csi image
86+
run: make image-cephcsi
87+
env:
88+
CONTAINER_CMD: docker
89+
90+
- name: Load ceph-csi image
91+
run: scripts/minikube.sh cephcsi
92+
93+
- name: Load sidecar images
94+
run: scripts/minikube.sh k8s-sidecar
95+
96+
- name: Deploy ceph-csi-operator
97+
run: >-
98+
scripts/deploy-ceph-csi-operator.sh deploy
99+
100+
- name: Build e2e.test
101+
run: make e2e.test
102+
103+
- name: Run acceptance e2e
104+
run: |
105+
cd e2e && ../e2e.test \
106+
-test.v -ginkgo.v \
107+
--ginkgo.label-filter=acceptance \
108+
--ginkgo.timeout=25m \
109+
--deploy-timeout=10 \
110+
--test-rbd=true \
111+
--test-cephfs=true \
112+
--test-nfs=true \
113+
--test-nvmeof=false \
114+
--deploy-rbd=false \
115+
--deploy-cephfs=false \
116+
--operator-deployment \
117+
--skip-vault=true
118+
119+
- name: Collect logs on failure
120+
if: failure()
121+
run: >-
122+
scripts/github-action-helper.sh collect_logs
123+
124+
- name: Upload failure logs
125+
if: failure()
126+
uses: actions/upload-artifact@v4
127+
with:
128+
name: acceptance-e2e-logs
129+
path: /tmp/acceptance-e2e-logs/
130+
retention-days: 7

‎AGENTS.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,21 @@ make run-e2e E2E_ARGS="--test-cephfs=false" # Run with specific args
9292
functional Ceph cluster. Only run these tests when both are available and
9393
properly configured. See `e2e/` directory for test implementations.
9494

95+
### Acceptance E2E (Quick Smoke Gate)
96+
97+
The `e2e-minikube-acceptance` workflow (`.github/workflows/e2e-minikube-acceptance.yaml`)
98+
runs 12 core specs (RBD, CephFS, NFS) on a minikube cluster with Rook Ceph.
99+
It triggers on every `pull_request` to `devel` or `release-v*` branches — no
100+
secrets or `ok-to-test` label needed.
101+
102+
**Fork users:** Add `Label("acceptance")` to new feature specs and open a PR
103+
within your fork for quick e2e verification loops before submitting upstream.
104+
Check your fork's Actions tab for results.
105+
106+
Specs are tagged with `Label("acceptance")` in the e2e Go files. To add a
107+
spec to the acceptance suite, add the label to its `It()` declaration. See
108+
`e2e/README.md` for details.
109+
95110
### Module Checks
96111

97112
```bash

‎build.env‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ HELM_SCRIPT=https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
5656
HELM_VERSION=v3.19.0
5757

5858
# minikube settings
59+
KUBE_VERSION=v1.35.1
5960
MINIKUBE_VERSION=v1.38.1
6061
VM_DRIVER=none
6162
CHANGE_MINIKUBE_NONE_USER=true

‎e2e/README.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,75 @@ follow these steps before running e2e.
126126
./scripts/install-snapshot.sh cleanup
127127
```
128128

129+
## Acceptance E2E (Minikube)
130+
131+
The acceptance suite is a lightweight smoke gate that runs on **every PR** via
132+
GitHub Actions (`.github/workflows/e2e-minikube-acceptance.yaml`). It deploys
133+
ceph-csi on a minikube cluster with Rook Ceph and runs only specs labeled
134+
`Label("acceptance")`.
135+
136+
**Scope (12 specs):**
137+
138+
| Driver | Specs |
139+
|--------|-------|
140+
| RBD | PVC→app, snapshot→clone, PVC-PVC clone, block PVC |
141+
| CephFS | health check, PVC→app, snapshot→clone, PVC-PVC clone |
142+
| NFS | health check, PVC→app, snapshot→clone, PVC-PVC clone |
143+
144+
**Relationship to CentOS mini-e2e:** The acceptance suite does **not** replace
145+
CentOS `ci/centos/mini-e2e/k8s-*`. CentOS jobs run the full suite (~85 RBD +
146+
~48 CephFS + NFS + more) after `ok-to-test`. Acceptance catches "drivers won't
147+
deploy / basic provision broken" before that.
148+
149+
### Running acceptance specs locally
150+
151+
```console
152+
# Run acceptance specs on an existing cluster
153+
cd e2e && ../e2e.test -test.v -ginkgo.v \
154+
--ginkgo.label-filter=acceptance \
155+
--ginkgo.timeout=15m \
156+
--deploy-timeout=10 \
157+
--test-rbd=true --test-cephfs=true \
158+
--test-nfs=true --test-nvmeof=false \
159+
--deploy-rbd=true --deploy-cephfs=true \
160+
--operator-deployment \
161+
--skip-vault=true
162+
```
163+
164+
### Adding specs to acceptance
165+
166+
Add `Label("acceptance")` to the `It()` declaration:
167+
168+
```go
169+
It("my new smoke test", Label("acceptance"), func() {
170+
// ...
171+
})
172+
```
173+
174+
### Running acceptance e2e in your fork
175+
176+
Contributors can run the acceptance gate in their own fork before
177+
opening a PR upstream:
178+
179+
1. Fork `ceph/ceph-csi` on GitHub.
180+
1. Push your branch to **your fork**.
181+
1. Open a PR **within your fork** (base: your fork's `devel`, head: your
182+
branch). The `e2e-minikube-acceptance` workflow triggers automatically on
183+
`pull_request` — no secrets or `ok-to-test` label required.
184+
1. Check the Actions tab for results. Failed runs upload logs as the
185+
`acceptance-e2e-logs` artifact.
186+
187+
This gives you a quick signal that basic provisioning, snapshots, and clones
188+
work before submitting upstream. When adding new features, tag their specs
189+
with `Label("acceptance")` to include them in the gate — this enables fast
190+
e2e verification loops entirely within your fork.
191+
192+
### Test parameters for acceptance
193+
194+
| flag | description |
195+
|------------|---------------------------------------------------|
196+
| skip-vault | Skip Vault KMS deployment for faster runs (default: false) |
197+
129198
## Running E2E
130199

131200
`

‎e2e/cephfs.go‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -241,7 +241,14 @@ var _ = Describe(cephfsType, func() {
241241
if err != nil {
242242
logAndFail("failed to create node secret: %v", err)
243243
}
244-
deployVault(f.ClientSet, deployTimeout)
244+
if !skipVault {
245+
deployVault(f.ClientSet, deployTimeout)
246+
} else {
247+
err = createEmptyKMSConfigMap(f.ClientSet, cephCSINamespace)
248+
if err != nil {
249+
logAndFail("failed to create empty KMS configmap: %v", err)
250+
}
251+
}
245252

246253
err = cephFSDeployment.setClusterName(defaultClusterName)
247254
if err != nil {
@@ -300,7 +307,9 @@ var _ = Describe(cephfsType, func() {
300307
if err != nil {
301308
logAndFail("failed to delete storageclass: %v", err)
302309
}
303-
deleteVault()
310+
if !skipVault {
311+
deleteVault()
312+
}
304313

305314
err = deleteSubvolumegroup(f, fileSystemName, subvolumegroup)
306315
if err != nil {
@@ -336,7 +345,7 @@ var _ = Describe(cephfsType, func() {
336345
appEphemeralPath := cephFSExamplePath + "pod-ephemeral.yaml"
337346
pvcRWOPPath := cephFSExamplePath + "pvc-rwop.yaml"
338347

339-
It("checking provisioner and nodeplugin are running", func() {
348+
It("checking provisioner and nodeplugin are running", Label("acceptance"), func() {
340349
Expect(waitForCSI(
341350
f.ClientSet,
342351
cephFSDeployment.getDeploymentName(),
@@ -940,7 +949,7 @@ var _ = Describe(cephfsType, func() {
940949
}
941950
})
942951

943-
It("create a PVC and bind it to an app", func() {
952+
It("create a PVC and bind it to an app", Label("acceptance"), func() {
944953
err := createCephfsStorageClass(f.ClientSet, f, false, nil)
945954
if err != nil {
946955
logAndFail("failed to create CephFS storageclass: %v", err)
@@ -1502,7 +1511,7 @@ var _ = Describe(cephfsType, func() {
15021511
}
15031512
})
15041513

1505-
It("create a PVC clone and bind it to an app", func() {
1514+
It("create a PVC clone and bind it to an app", Label("acceptance"), func() {
15061515
var wg sync.WaitGroup
15071516
totalCount := 3
15081517
wgErrs := make([]error, totalCount)
@@ -2401,7 +2410,7 @@ var _ = Describe(cephfsType, func() {
24012410
}
24022411
}
24032412

2404-
It("create a PVC-PVC clone and bind it to an app", func() {
2413+
It("create a PVC-PVC clone and bind it to an app", Label("acceptance"), func() {
24052414
var wg sync.WaitGroup
24062415
totalCount := 3
24072416
wgErrs := make([]error, totalCount)

‎e2e/e2e_test.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ func init() {
5353
flag.StringVar(&clusterID, "clusterid", "", "Ceph cluster ID to use (defaults to `ceph fsid` detection)")
5454
flag.StringVar(&nfsDriverName, "nfs-driver", "nfs.csi.ceph.com", "name of the driver for NFS-volumes")
5555
flag.BoolVar(&operatorDeployment, "operator-deployment", false, "test running on deployment via operator")
56+
flag.BoolVar(&skipVault, "skip-vault", false, "skip Vault KMS deployment (e.g. for acceptance smoke tests)")
5657
setDefaultKubeconfig()
5758

5859
// Register framework flags, then handle flags

‎e2e/nfs.go‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -493,7 +493,7 @@ var _ = Describe("nfs", func() {
493493
appClonePath := nfsExamplePath + "pod-restore.yaml"
494494
snapshotPath := nfsExamplePath + "snapshot.yaml"
495495

496-
It("checking provisioner and nodeplugin are running", func() {
496+
It("checking provisioner and nodeplugin are running", Label("acceptance"), func() {
497497
Expect(waitForCSI(
498498
f.ClientSet,
499499
nfsDeployment.getDeploymentName(),
@@ -659,7 +659,7 @@ var _ = Describe("nfs", func() {
659659
}
660660
})
661661

662-
It("create a PVC and bind it to an app", func() {
662+
It("create a PVC and bind it to an app", Label("acceptance"), func() {
663663
err := createNFSStorageClass(f.ClientSet, f, false, nil)
664664
if err != nil {
665665
logAndFail("failed to create NFS storageclass: %v", err)
@@ -813,7 +813,7 @@ var _ = Describe("nfs", func() {
813813
}
814814
})
815815

816-
It("create a PVC clone and bind it to an app", func() {
816+
It("create a PVC clone and bind it to an app", Label("acceptance"), func() {
817817
var wg sync.WaitGroup
818818
totalCount := 3
819819
wgErrs := make([]error, totalCount)
@@ -1103,7 +1103,7 @@ var _ = Describe("nfs", func() {
11031103
validateOmapCount(f, 0, cephfsType, metadataPool, snapsType)
11041104
})
11051105

1106-
It("create a PVC-PVC clone and bind it to an app", func() {
1106+
It("create a PVC-PVC clone and bind it to an app", Label("acceptance"), func() {
11071107
var wg sync.WaitGroup
11081108
totalCount := 3
11091109
wgErrs := make([]error, totalCount)

0 commit comments

Comments
 (0)