Kubevirt-storage-checkup was failing when a virtual machine is already created.

[sakshi-garg1]

Environment:
OCP cluster version: 4.18.0-rc.1

Openshift Virtualization version: 4.17.2

Only 1 driver storage class is present as default.

Case 1: Running this storage checkup without manually creating any Virtual Machines , the check is passing successfully. While running this test also verified manually that the PVCs being created were all in bound state and using my driver storage class.

Case 2: Creating 1 Virtual Machine manually as shown below and running storage checkup test, the checkup fails with error: hotplug volume ready:timed out waiting for the condition.

Seems not a driver issue as case 1 certifies that.
Why it is failing for case 2. Is it an expected behavior?
Is it related to rc builds that this test throws up this issue because when using Openshift virtualization version 4.17.2 with openshift cluster 4.17 case 2 passes as well.

[sakshi-garg1]

Any update on this?

[arnongilboa]

Any update on this?

Note you are using 4.18.0-rc.1 that has a known hotplug bug.
For running the checkup it's better to use a stable release (and not candidates).
If that doesn't solve the issue, try to extend param.vmiTimeout to 10m.

[satyakonduri]

Hi @arnongilboa,

I tried setting the parm.vmiTimeout parameter to a 10-minute timeout but am still encountering the same issue.
Config used:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: storage-checkup-config
  namespace: default
data:
  spec.timeout: 10m
  spec.param.vmiTimeout: 10m
---
apiVersion: batch/v1
kind: Job
metadata:
  name: storage-checkup
  namespace: default
spec:
  backoffLimit: 0
  template:
    spec:
      serviceAccount: storage-checkup-sa
      restartPolicy: Never
      containers:
        - name: storage-checkup
          image: quay.io/kiagnose/kubevirt-storage-checkup:main
          imagePullPolicy: Always
          env:
            - name: CONFIGMAP_NAMESPACE
              value: default
            - name: CONFIGMAP_NAME
              value: storage-checkup-config

Could you please let us know if there are any other configurations we should try to resolve this?

Thank You.!

[arnongilboa]

@satyakonduri can you please share both the checkup vmi and the other vmi get -o yaml here?

[satyakonduri]

Hi @arnongilboa
Please find the config and logs.

hotplug-volume-vm.log
hotplug-volume-vmi.log
storage-checkup-pod-logs.txt

Thank You.!

[arnongilboa]

Note you are using 4.18.0-rc.1 that has a known hotplug bug. For running the checkup it's better to use a stable release (and not candidates).

@satyakonduri ^^^

[satyakonduri]

Thank you @arnongilboa! I will try this once a stable build is released. Just for your information, I executed the above steps in the 4.18-rc3 build.

[prablr79]

Any updates on this defect ?

[arnongilboa]

To verify the issue is not related to kubevirt-storage-checkup, you can simply create 2 VMs, hotplug a volume to one of them and check if it gets to Ready phase or stay in MountedToPod like the one we create in the checkup.