Skip to content

Add vcsim support for container backed VMs able to run nested containers#3976

Draft
hickeng wants to merge 4 commits intovmware:mainfrom
hickeng:vcsim-nested-containers
Draft

Add vcsim support for container backed VMs able to run nested containers#3976
hickeng wants to merge 4 commits intovmware:mainfrom
hickeng:vcsim-nested-containers

Conversation

@hickeng
Copy link
Contributor

@hickeng hickeng commented Mar 6, 2026

Add RUN.nestedContainers ExtraConfig option to enable running Kubernetes
or other container workloads inside container-backed VMs. When enabled,
the container is created with kind-style flags:

  • --cgroupns=private (instead of host) for proper cgroup v2 delegation
  • --security-opt seccomp=unconfined for systemd/containerd syscalls
  • --security-opt apparmor=unconfined to disable AppArmor restrictions
  • --tmpfs /tmp,/run for systemd compatibility
  • --volume /var for persistent containerd/kubelet data
  • --volume /lib/modules:/lib/modules:ro for iptables/networking
  • --device /dev/fuse for fuse-overlayfs snapshotter

hickeng and others added 4 commits March 5, 2026 16:20
- Add commandError helper for consistent error logging with stderr
- Add RUN.network ExtraConfig option for container network selection
  (needed for rootless podman to get IP assignment via bridge network)
- Add RUN.mountdmi error hint for rootless podman permission issues
- Update syncNetworkConfigToVMGuestProperties to trigger property change
  notifications via ctx.Update for all modified properties
- Retry IP sync after container start to handle delayed IP assignment
- Update watchContainer callback signature to include Context

These changes improve vcsim's compatibility with podman-docker and
enable proper WaitForIP functionality for container-backed VMs.

Made-with: Cursor
…king

Add PropertyDiff and Checkpoint functions to generate PropertyChange arrays
by comparing two states of a managed object. This enables granular property
change notifications when modifying simulator objects.

The typical usage pattern is:
  checkpoint := Checkpoint(vm)
  // ... make changes to vm ...
  changes := PropertyDiff(checkpoint, vm)
  ctx.Update(vm, changes)

PropertyDiff correctly handles:
- Simple field changes (Assign operation)
- Nil to value transitions (Add operation)
- Value to nil transitions (Remove operation)
- Nested struct fields
- Embedded/anonymous fields
- Slice fields

Tests include:
- Whitebox unit tests for PropertyDiff, Checkpoint, and determineChangeOp
- Simulator integration tests verifying changes are visible via PropertyCollector
- Container-backed VM test verifying network property changes from container inspect

Made-with: Cursor
…e Guest.Net

Refactor syncNetworkConfigToVMGuestProperties to use PropertyDiff for
generating granular property changes instead of manually constructing
PropertyChange arrays. This simplifies the code and ensures all modified
properties trigger proper notifications.

Key changes:
- Use Checkpoint/PropertyDiff pattern to track all VM state changes
- Populate Guest.Net from all container networks (not just when pre-configured)
- Include detailed NIC info: Network name, MacAddress, IpAddress, IpConfig
- Populate Guest.IpStack with DNS config and default route
- Update Summary.Guest.HostName in addition to IpAddress

The test now verifies:
- Guest.Net is empty before power on
- Guest.Net is populated with detailed NIC info after power on
- Each NIC has Network, MacAddress, IpAddress, Connected, and IpConfig
- Guest.IpStack contains DNS and routing information
- Summary.Guest includes both IpAddress and HostName

Made-with: Cursor
Add RUN.nestedContainers ExtraConfig option to enable running Kubernetes
or other container workloads inside container-backed VMs. When enabled,
the container is created with kind-style flags:

- --cgroupns=private (instead of host) for proper cgroup v2 delegation
- --security-opt seccomp=unconfined for systemd/containerd syscalls
- --security-opt apparmor=unconfined to disable AppArmor restrictions
- --tmpfs /tmp,/run for systemd compatibility
- --volume /var for persistent containerd/kubelet data
- --volume /lib/modules:/lib/modules:ro for iptables/networking
- --device /dev/fuse for fuse-overlayfs snapshotter

Also adds comprehensive documentation for all RUN.* ExtraConfig options
in container_virtual_machine.go with usage examples.

Reference: https://github.com/kubernetes-sigs/kind/blob/main/pkg/cluster/internal/providers/docker/provision.go

AI-Tool-Used: Cursor
AI-Tool-Use-Level: high
AI-Code-Category: non-production
Co-authored-by: Claude Opus 4 <noreply@anthropic.com>
Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant