kicbase: fix deterministic machine-id breaking MAC addresses in multi-node Podman rootless clusters

[RobinMcCorkell]

Problem

/var/lib/dbus/machine-id is baked into the kicbase container image at build time. When the entrypoint's fix_machine_id runs systemd-machine-id-setup, it finds that file and derives /etc/machine-id from it — producing the same machine ID in every container.

This breaks anything that depends on the machine ID being unique per node. The most visible symptom is in multi-node minikube clusters using Podman rootless mode: a veth interface is placed into each Podman container, and systemd configures it according to MACAddressPolicy=persistent. That policy selects a MAC address based on the machine ID (via systemd-machine-id-setup, which reads from the D-Bus machine ID: https://www.freedesktop.org/software/systemd/man/latest/systemd-machine-id-setup.html). Because all containers share the same machine ID derived from the baked-in D-Bus ID, all nodes get identical MAC addresses on eth0, causing network failures.

Fix

Per https://systemd.io/CONTAINER_INTERFACE/, add a RUN step to the Dockerfile (before the final squash) that:

• Truncates /etc/machine-id to an empty file — the spec requires the file to be present but uninitialized so systemd can fill it on boot.
• Deletes /var/lib/dbus/machine-id — removes the baked-in D-Bus ID that was the source of the deterministic machine ID.

With these changes, systemd generates a fresh random machine ID on every container boot without any entrypoint assistance. This makes the existing fix_machine_id function in the entrypoint redundant — it has been removed.

Testing

Tested against the real kicbase image (gcr.io/k8s-minikube/kicbase-builds:v0.0.50-1772266598-22719) with Podman rootless.

Reproducing the bug

Both files contain the same baked-in value in the current image:

$ KICBASE=gcr.io/k8s-minikube/kicbase-builds:v0.0.50-1772266598-22719
$ podman run --rm --entrypoint bash $KICBASE -c '
    echo "/etc/machine-id:          $(cat /etc/machine-id)"
    echo "/var/lib/dbus/machine-id: $(cat /var/lib/dbus/machine-id)"
  '
/etc/machine-id:          636135c8833b3f1430bfc1ec69a2a4d1
/var/lib/dbus/machine-id: 636135c8833b3f1430bfc1ec69a2a4d1

Running systemd-machine-id-setup (as fix_machine_id does) produces the same ID every time:

$ for i in 1 2 3; do
    podman run --rm --entrypoint bash $KICBASE -c '
      rm -f /etc/machine-id; systemd-machine-id-setup 2>/dev/null; cat /etc/machine-id'
  done
636135c8833b3f1430bfc1ec69a2a4d1
636135c8833b3f1430bfc1ec69a2a4d1
636135c8833b3f1430bfc1ec69a2a4d1

Verifying the fix

Build a patched image applying the fix on top of the current kicbase:

$ podman build -t kicbase-patched - <<EOF
FROM $KICBASE
RUN truncate -s 0 /etc/machine-id && rm -f /var/lib/dbus/machine-id
EOF

Confirm the image state:

$ podman run --rm --entrypoint bash kicbase-patched -c '
    echo "/etc/machine-id size: $(wc -c < /etc/machine-id) bytes"
    ls /var/lib/dbus/machine-id 2>/dev/null || echo "/var/lib/dbus/machine-id: (removed)"
  '
/etc/machine-id size: 0 bytes
/var/lib/dbus/machine-id: (removed)

Each container now gets a unique machine ID:

$ for i in 1 2 3; do
    podman run --rm --entrypoint bash kicbase-patched -c '
      systemd-machine-id-setup 2>/dev/null; cat /etc/machine-id'
  done
4c971ecb6dc34159ad1da13a867a51c3
2d717cadb4804b1b9863f7b8b2da998d
b25726fead9246f691e97b3a63834de2

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: RobinMcCorkell
Once this PR has been reviewed and has the lgtm label, please assign comradeprogrammer for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[linux-foundation-easycla]

• ✅ login: RobinMcCorkell / name: Robin McCorkell (2251f1e)
• ✅ login: minikube-bot / name: minikube-bot (fb35ed2)
• ❌ - login: @Copilot / name: Copilot. The commit (2251f1e) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[k8s-ci-robot]

Welcome @RobinMcCorkell!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @RobinMcCorkell. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[minikube-bot]

Can one of the admins verify this patch?

[RobinMcCorkell]

The CLA signing page (and the support link) take me to a blank page on https://sso.linuxfoundation.org/login, looks like there's some CORS issues preventing the page from loading. I can't sign the CLA without that. this is now fixed

Also I don't think I can sign the CLA on behalf of Copilot 🙈

[medyagh]

/ok-to-build-image

[medyagh]

@RobinMcCorkell this code is common with Kind, do you know if this affects kind too ? do we have any users complaining about this in the issues?

[Copilot]

Pull request overview

Fixes deterministic machine IDs in the kicbase image that caused identical persistent MAC addresses across nodes (notably in multi-node minikube clusters using Podman rootless), leading to networking failures.

Changes:

• Update the kicbase Dockerfile to ensure /etc/machine-id is present-but-empty and remove /var/lib/dbus/machine-id in the built image.
• Remove the now-redundant fix_machine_id logic and invocation from the kicbase entrypoint.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
deploy/kicbase/Dockerfile	Ensures the image does not ship a pre-initialized machine-id and removes the baked-in D-Bus machine-id to avoid deterministic IDs across containers.
deploy/kicbase/entrypoint	Removes fix_machine_id since systemd will now initialize a fresh machine-id on boot from the corrected image state.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[minikube-bot]

Hi @RobinMcCorkell, we have updated your PR with the reference to newly built kicbase image. Pull the changes locally if you want to test with them or update your PR further.

[RobinMcCorkell]

I believe this is the cause of #22606,

It is also mentioned in #16962 - "I tried the bridge cni in minikube and it wouldn't work regardless of IP forwarding being enabled or disabled"

It's possible these Kind bug reports also refer to this issue: kubernetes-sigs/kind#2996 kubernetes-sigs/kind#3412

[k8s-ci-robot]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.