Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Windows guest

What a Windows 11 guest needs to run NVIDIA's stock driver on kayfabe. Labels: [measured] = a run on named hardware, [inferred] = reasoning.

What was tested

  • Windows 11 Pro, build 26200 (the full retail edition) [measured, 2026-10-14, kayfabe eb9f2aa3, RTX 4070]: the desktop and the application matrix (the benchmark output names "Microsoft Windows 11 Pro", kernel 10.0.26200).
  • Windows 11 Enterprise LTSC 2024, build 26100 (Microsoft's 90-day evaluation) [measured]: the image that scripts/bench/windows/win_vm.sh install builds, and the one used on rented hosts.
  • Driver: NVIDIA's stock Windows driver, version 580.88, installed with NVIDIA's own installer or INF, unmodified. No driver, kernel or Windows binary is patched.
  • Applications run unmodified from the vendors' complete installers; the README's table lists the results (Minecraft, Unigine Heaven, GravityMark, Edge with hardware video decode, CUDA, Blender, OBS, Office, SolidWorks and 3DMark). Several of those rows say their result files are not committed yet.

The settings it needs (and why)

The stock GeForce driver on Windows runs its resource manager on the CPU by default. Kayfabe serves only the GSP firmware path, so the driver has to be told to use GSP. Without the first setting the driver stops with Code 43 [measured on Windows 11 and on LTSC]. All values are DWORDs and take effect after a restart.

#KeyValueWhy
1HKLM\SYSTEM\CurrentControlSet\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\<index> EnableGpuFirmware1forces GSP; <index> is the display adapter's own key (0000 on Windows 11 Pro, 0001 on the LTSC image: read it, never assume)
2HKLM\SYSTEM\CurrentControlSet\Services\nvlddmkm EnableGpuFirmware1the same flag; the service key alone is not enough, set both
3HKLM\SOFTWARE\Microsoft\Windows\Dwm OverlayTestMode5without it, opening the Start menu reset the display driver on Windows 11 Pro (the cause is not known)

From an elevated command prompt in the guest. First read the display adapter's driver key, then substitute it:

powershell -NoProfile -Command "(Get-PnpDeviceProperty -InstanceId (Get-PnpDevice -PresentOnly | Where-Object InstanceId -like 'PCI\VEN_10DE*').InstanceId -KeyName DEVPKEY_Device_Driver).Data"
reg add "HKLM\SYSTEM\CurrentControlSet\Control\Class\<the key printed above>" /v EnableGpuFirmware /t REG_DWORD /d 1 /f
reg add "HKLM\SYSTEM\CurrentControlSet\Services\nvlddmkm" /v EnableGpuFirmware /t REG_DWORD /d 1 /f
reg add "HKLM\SOFTWARE\Microsoft\Windows\Dwm" /v OverlayTestMode /t REG_DWORD /d 5 /f
shutdown /r /t 0

scripts/bench/windows/gsp_on.ps1 does settings 1 and 2 while it installs the driver (it is what win_vm.sh nv-install runs): it disables the device, adds and installs the INF, writes both values, reads them back, then re-enables the device. Setting them before the driver's first start avoids the Code 43.

These are settings for a development guest, not a complete installation guide for a production system.

The Windows harness: win_vm.sh

scripts/bench/windows/win_vm.sh installs a Windows 11 guest once and then boots it on kayfabe as many times as you like. It is one script and it needs a bench tree and a kayfabe QEMU built for the checkout's revision (Building from source, Running the QEMU device). It refuses by name when a command or file it needs is missing.

Install the host tools it names:

apt-get install -y ovmf swtpm swtpm-tools sbsigntool python3-virt-firmware xorriso netpbm p7zip-full

What it sets up

PieceHow
WindowsWindows 11 Enterprise LTSC 2024, the 90-day evaluation (build 26100), from Microsoft's own download link, installed unattended.
Secure BootOn. The VM has its own OVMF variable store; PK and KEK are generated for the VM; db holds Microsoft's keys plus this host's kayfabe db certificate, which signs kayfabe's boot-display driver (gop-efi=).
TPM 2.0swtpm. The state is manufactured once per VM and then reused. The harness refuses to manufacture it again.
BitLockerPrevented from the first boot; check verifies the volume is decrypted.
DriversThe Red Hat virtio drivers (virtio-win 0.1.302), the QEMU guest agent. No balloon, no viomem.
sshWin32-OpenSSH, key only; PowerShell is the default shell.
NetworkQEMU user networking with restrict=on: the guest has no route out, so no Windows Update. Only ssh is forwarded, 127.0.0.1:2224 to the guest by default.

Commands

S=scripts/bench/windows/win_vm.sh
$S install                                  # once; about an hour on a nested box
$S run --detach --tag basic                 # kayfabe + signed boot-display driver, display on
$S ssh 'Get-PnpDevice -Class Display'
$S shot                                     # PNG of the console
$S check --kf3                              # WIN_* lines: Secure Boot, db, TPM, BitLocker, virtio...
$S nv-install                               # NVIDIA 580.88, GSP forced on before nvlddmkm's first start
$S collect nv1                              # Windows-side evidence into logs/evidence/nv1/
$S stop                                     # guest shutdown, then QMP powerdown, then QMP quit
$S status

Other sub-commands in the script's header: scp-to, scp-from, sign (export and sign this checkout's boot-display driver) and fetch (download and verify the pinned media).

run options:

OptionEffect
--overlay NAMEThe disk to boot: disk/NAME.qcow2 over the sealed base.qcow2, created on first use. Default main.
--guest-driver VThe device's guest-driver= (for example 580.65.06 for Windows 580.88).
--kf3-extra PExtra device properties.
--no-kf3Boot on the standard VGA without kayfabe.
--arm A / --arm BWithout (A, the default) or with (B) Hyper-V enlightenments.
--onceDo not restart after a guest reboot.
--ssh-port N, --ssh-bind ADDRThe host port for ssh; binding wider than 127.0.0.1 is explicit only.
--detachRun in the background.
--no-rpc-traceDo not set KF3_RPC_TRACE=1 (on by default for kayfabe runs).
--tag, --max-bootsName the run; limit restarts (default 12).

The harness environment: WINVM_ROOT (default /workspace/winvm), WINVM_NAME, WINVM_RAM_MB (8192), WINVM_SMP (8), WINVM_KF3_FB_MB (8192), WINVM_SSH_PORT (2224), WINVM_QEMU.

The QEMU command it builds uses the pc-q35-10.2 machine with SMM, Secure Boot firmware, the TPM, and the device kf3-gpu,fb-mb=8192,bar1-size=134217728,bar2-size=33554432,display=on,gop=on,gop-efi=<signed driver>,id=kf0. Every guest reboot starts a fresh QEMU with the same firmware variables and the same TPM state; the harness restarts it for you.

Rules the script enforces: every step logs START and EXIT rc= (a log with no WINVM_RUN_END belongs to a dead run); GPU work takes the shared lock /tmp/kayfabe-fastguest.lock; nothing secret is written into the repository. Keep the swtpm state and the ssh key private; a VM is never restored from another VM's state.

Remote access: ssh -J <gpu-host> -p 2224 kf@127.0.0.1 with a key you passed to install --authorized-key. To watch the screen, forward the VNC socket: ssh -N -L 5905:/workspace/winvm/kfwin/run/vnc.sock <gpu-host> and point a VNC viewer at localhost:5905.

Other things a Windows guest needs (not settings)

  • A unique SMBIOS UUID, serial and MAC per VM, and offline setup until after OOBE (Windows Autopilot once matched a default-identity VM to an unrelated company tenant); an explicit gpu-uuid.
  • Do not snapshot an image with a Windows update staged: it crash-loops dwm.exe. Block updates in test images.
  • You supply and license Windows. The scripts download Microsoft's evaluation image, the virtio drivers and NVIDIA's installer from their vendors, each pinned by SHA-256. See Credits and notices.

Known limits

  • Windows sleep/wake is not supported yet.
  • Edge can reset the display driver after 7 to 16 minutes of playback.
  • Other GPU dies need more work under Windows; GB206 under Windows is open (Code 43).
  • nv-install is slow: it copies the 2.6 GB driver package over user networking at about 2.4 MB/s (18 minutes).
  • A failed NVIDIA driver slows everything down. With the driver in Code 43, boot to ssh took about 20 minutes and the guest ignored shutdown /s for 5 minutes.
  • The guest changes its own Secure Boot variables (it adds the 2023 CAs to db and updates dbx). The variables file is per-VM live state; never regenerate it or copy it to another VM.

See also Troubleshooting and Display and the broker.

Not documented yet

  • A Windows install on a machine without the harness (a plain ISO install onto a kayfabe-backed VM by hand).
  • Windows 10, or other Windows 11 builds than the two above.
  • A successful guest run with another NVIDIA driver version. Windows 581.51 was attempted on AD102 and failed with Code 43 [measured in the sweep matrix, revision 24906bf33, P19 profile; historical source 9d6f210ba]. This is recorded on the branch codex/sweep-integrate-20261018, not merged: matrix. Its same-board native Windows control passed, which is not a kayfabe guest pass.

Verified on: Windows 11 Pro 26200 with driver 580.88 on an RTX 4070 (kayfabe eb9f2aa3, 2026-10-14; further results up to master 89fc8a222, 2026-10-21, in the README); LTSC 26100 on rented hosts. Not verified end to end: the reg add snippet is the repository's own text and is not a recorded run of exactly those lines. Settings 1 and 2 are the part that gsp_on.ps1 applies and reads back in the harness.

View and edit source on GitHub (source: docs/howto/windows-guest.md, master branch)