Add --gpu: virtio-gpu Venus (Vulkan) device for sandboxes - #1194
Open
maxandrews wants to merge 2 commits into
Open
Add --gpu: virtio-gpu Venus (Vulkan) device for sandboxes#1194maxandrews wants to merge 2 commits into
maxandrews wants to merge 2 commits into
Conversation
Plumbs a gpu flag end-to-end (SandboxRuntimeOptions, serde-default → SandboxBuilder.gpu() → CLI --gpu on run/create → spawn argv → VmConfig → msb_krun ConsoleBuilder GPU flags) and enables the msb_krun gpu feature. Renderer flags: VENUS selects the Vulkan encoder, NO_VIRGL drops legacy GL, THREAD_SYNC + USE_ASYNC_FENCE_CB select the async fence model — required because the device worker never calls virgl_renderer_poll, so synchronous fences never retire and the first real submission livelocks. On Linux, RENDER_SERVER is added: virglrenderer >= 1.x serves venus exclusively through the render-server proxy; without the flag the venus capset fills as zeros and guests silently fall back to llvmpipe. The 8 GiB gpu_shm_size is an address-space window for venus blob mappings, not a RAM reservation. Also fixes the [linux] build-agentd recipe to target the host architecture (was hardcoded x86_64-unknown-linux-musl, breaking arm64 Linux hosts). Depends on the msb_krun mapping fixes in superradcompany/libkrun#91. Verified on Apple M4 Pro (HVF/MoltenVK) and NVIDIA T4G (Linux/KVM, AWS g5g.metal): guest vulkaninfo enumerates the host GPU via venus and llama.cpp ggml-vulkan runs fully offloaded (7B Q4_K_M at pp512 648 / tg128 29.2 t/s on the T4G). Discussion: superradcompany#291
Open
Author
|
x86_64 verification is in as well — same patch set, AWS g4dn.metal (Tesla T4):
One x86-specific note for GPU docs: on small-BAR discrete cards (T4 BAR1 = 256MB), ggml-vulkan's use of the DEVICE_LOCAL|HOST_VISIBLE heap exhausts under venus and the run dies at context creation ( |
Member
|
@maxandrews yet to dig into the code, but curious if you've given some thought around what this could look like for windows? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the ask in #291, structured as the separate-PRs approach you preferred — this is the flag plumbing; the
msb_krun_*mapping fixes it depends on are superradcompany/libkrun#91.What it does:
msb run --gpu/msb create --gpuexpose a virtio-gpu Venus device to the guest. A guest with mesa's venus userspace enumerates the host GPU and runs real Vulkan workloads.Plumbing:
SandboxRuntimeOptions.gpu(serde-default, so existing configs are untouched) →SandboxBuilder.gpu()→ CLI flag → spawn argv →VmConfig→msb_krunConsoleBuilder GPU flags; enables themsb_krun/gpucargo feature.Renderer flag choices (each earned by a debugging session):
VENUS | NO_VIRGL— Vulkan encoder only, no legacy GL.THREAD_SYNC | USE_ASYNC_FENCE_CB— required: the device worker never callsvirgl_renderer_poll, so synchronous fences never retire and the first real GPU submission livelocks (host pinned ~120%, looks exactly like a slow shader compile).RENDER_SERVER— virglrenderer ≥ 1.x serves venus exclusively through the render-server proxy; without it the venus capset (id 4) is advertised but fills as zeros, and guests silently fall back to llvmpipe.gpu_shm_size(8 GiB)— an address-space window for venus blob mappings, not a RAM reservation.Also: fixes
[linux] build-agentdto target the host arch (was hardcodedx86_64-unknown-linux-musl; arm64 Linux hosts couldn't build agentd — hit while bringing this up on Graviton metal).Verified end-to-end:
vulkaninfo=Virtio-GPU Venus (Apple M4 Pro); llama.cpp ggml-vulkan generates tokens fully offloadedGGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1on 256MB-BAR cards)Build notes for anyone reproducing: macOS links the krunkit-tap virglrenderer (
brew install slp/krun/virglrenderer) and the binary needs the hypervisor entitlement; Linux wants upstream virglrenderer ≥ 1.1x built-Dvenus=true(the render-server binary installs alongside), plus on NVIDIA hosts:nvidia_drmwith KMS modesetting andrender-group access, else the driver hidesVK_EXT_external_memory_dma_bufand venus can't map blobs.You mentioned internal work already underway — feel free to reshape/cherry-pick; happy to rebase onto whatever shape you land, and to test on Apple Silicon and NVIDIA/Linux.