Skip to content

Add --gpu: virtio-gpu Venus (Vulkan) device for sandboxes - #1194

Open
maxandrews wants to merge 2 commits into
superradcompany:mainfrom
maxandrews:feat/gpu-flag
Open

Add --gpu: virtio-gpu Venus (Vulkan) device for sandboxes#1194
maxandrews wants to merge 2 commits into
superradcompany:mainfrom
maxandrews:feat/gpu-flag

Conversation

@maxandrews

@maxandrews maxandrews commented Jul 21, 2026

Copy link
Copy Markdown

Closes the ask in #291, structured as the separate-PRs approach you preferred — this is the flag plumbing; the msb_krun_* mapping fixes it depends on are superradcompany/libkrun#91.

What it does: msb run --gpu / msb create --gpu expose a virtio-gpu Venus device to the guest. A guest with mesa's venus userspace enumerates the host GPU and runs real Vulkan workloads.

Plumbing: SandboxRuntimeOptions.gpu (serde-default, so existing configs are untouched) → SandboxBuilder.gpu() → CLI flag → spawn argv → VmConfigmsb_krun ConsoleBuilder GPU flags; enables the msb_krun/gpu cargo feature.

Renderer flag choices (each earned by a debugging session):

  • VENUS | NO_VIRGL — Vulkan encoder only, no legacy GL.
  • THREAD_SYNC | USE_ASYNC_FENCE_CBrequired: the device worker never calls virgl_renderer_poll, so synchronous fences never retire and the first real GPU submission livelocks (host pinned ~120%, looks exactly like a slow shader compile).
  • Linux adds RENDER_SERVER — virglrenderer ≥ 1.x serves venus exclusively through the render-server proxy; without it the venus capset (id 4) is advertised but fills as zeros, and guests silently fall back to llvmpipe.
  • gpu_shm_size(8 GiB) — an address-space window for venus blob mappings, not a RAM reservation.

Also: fixes [linux] build-agentd to target the host arch (was hardcoded x86_64-unknown-linux-musl; arm64 Linux hosts couldn't build agentd — hit while bringing this up on Graviton metal).

Verified end-to-end:

Platform Host GPU Result
macOS 26 / HVF, Apple M4 Pro MoltenVK guest vulkaninfo = Virtio-GPU Venus (Apple M4 Pro); llama.cpp ggml-vulkan generates tokens fully offloaded
Linux/KVM aarch64 (AWS g5g.metal) NVIDIA T4G Qwen2.5-7B Q4_K_M fully offloaded: pp512 647.85 t/s, tg128 29.18 t/s
Linux/KVM x86_64 (AWS g4dn.metal) NVIDIA T4 Qwen2.5-7B Q4_K_M fully offloaded: pp512 833.63 t/s, tg128 37.03 t/s (small-BAR note in comments: guest wants GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 on 256MB-BAR cards)

Build notes for anyone reproducing: macOS links the krunkit-tap virglrenderer (brew install slp/krun/virglrenderer) and the binary needs the hypervisor entitlement; Linux wants upstream virglrenderer ≥ 1.1x built -Dvenus=true (the render-server binary installs alongside), plus on NVIDIA hosts: nvidia_drm with KMS modesetting and render-group access, else the driver hides VK_EXT_external_memory_dma_buf and venus can't map blobs.

You mentioned internal work already underway — feel free to reshape/cherry-pick; happy to rebase onto whatever shape you land, and to test on Apple Silicon and NVIDIA/Linux.

Plumbs a gpu flag end-to-end (SandboxRuntimeOptions, serde-default →
SandboxBuilder.gpu() → CLI --gpu on run/create → spawn argv → VmConfig →
msb_krun ConsoleBuilder GPU flags) and enables the msb_krun gpu feature.

Renderer flags: VENUS selects the Vulkan encoder, NO_VIRGL drops legacy
GL, THREAD_SYNC + USE_ASYNC_FENCE_CB select the async fence model —
required because the device worker never calls virgl_renderer_poll, so
synchronous fences never retire and the first real submission livelocks.
On Linux, RENDER_SERVER is added: virglrenderer >= 1.x serves venus
exclusively through the render-server proxy; without the flag the venus
capset fills as zeros and guests silently fall back to llvmpipe. The
8 GiB gpu_shm_size is an address-space window for venus blob mappings,
not a RAM reservation.

Also fixes the [linux] build-agentd recipe to target the host
architecture (was hardcoded x86_64-unknown-linux-musl, breaking arm64
Linux hosts).

Depends on the msb_krun mapping fixes in superradcompany/libkrun#91.
Verified on Apple M4 Pro (HVF/MoltenVK) and NVIDIA T4G (Linux/KVM,
AWS g5g.metal): guest vulkaninfo enumerates the host GPU via venus and
llama.cpp ggml-vulkan runs fully offloaded (7B Q4_K_M at pp512 648 /
tg128 29.2 t/s on the T4G). Discussion: superradcompany#291
@maxandrews

Copy link
Copy Markdown
Author

x86_64 verification is in as well — same patch set, AWS g4dn.metal (Tesla T4):

Platform GPU pp512 tg128
Linux/KVM aarch64 (g5g.metal) NVIDIA T4G 647.85 t/s 29.18 t/s
Linux/KVM x86_64 (g4dn.metal) NVIDIA T4 833.63 t/s 37.03 t/s
macOS/HVF (M4 Pro) MoltenVK token generation verified

One x86-specific note for GPU docs: on small-BAR discrete cards (T4 BAR1 = 256MB), ggml-vulkan's use of the DEVICE_LOCAL|HOST_VISIBLE heap exhausts under venus and the run dies at context creation (vkMapMemory: ErrorMemoryMapFailed) — GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 in the guest resolves it at full speed. Large-BAR cards (the T4G exposes a 4GB BAR) don't hit it.

@toksdotdev

Copy link
Copy Markdown
Member

@maxandrews yet to dig into the code, but curious if you've given some thought around what this could look like for windows?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants