Why can't VirtualBox or VMware run with Hyper-V enabled on Windows 10

VirtualBox and VMware Workstation (and VMware Player) are "level 2 hypervisors." Hyper-V and VMware ESXi are "level 1 hypervisors."

The main difference is that a level 2 hypervisor is an application running inside an existing OS, while a level 1 hypervisor is the OS itself.

This means that when you enable Hyper-V, your Windows 10 "host" becomes a virtual machine. A special one, but nonetheless a virtual machine.

So your question would more aptly be: "Why don't VirtualBox and VMware Workstation work inside a Hyper-V virtual machine?" One can answer because as a VM, the Intel VT-X instruction are no longer accessible from your virtual machine, only the host has access to it.

QEMU works because it does not do virtualization but emulation, which is completely different and explains why QEMU is painfully slow. Virtualization is the process of running a complete isolated machine inside another, but with the help of the processor. This requires the virtual machine and the host to be instruction compatible.

Emulation is the process of running any machine inside a running OS, there is no platform restriction, and is why QEMU can run an ARM machine on an amd64 platform.

Note: QEMU has 2 operating modes:

  • it can work as an emulator, this is the mode explained above
  • it can work as virtualization software with the help of KVM if the guest architecture is compatible with the host's and if the VT instruction is present of course.

Starting from Windows Redstone 4 build, QEMU will be able to run under Hyper-V by employing the Windows Hypervisor Platform API.

The Windows Hypervisor Platform accelerator (WHPX) is being integrated to QEMU patches are submitted for merging.

The Windows Hypervisor Platform API will be included in Redstone 4 build

Experimental support for WHPX is included in QEMU 2.12

Update: Virtualbox 6.0 added support for WHPX Virtualbox 6.0 Changelog.

Update 2: VMWare will add support for WHPX in upcoming version of VMWare Workstation and fusion. VMware Workstation and Hyper-V – Working Together

Update 3: Initial VMWare Workstation release with WHPX support. Software requirements are Windows Insiders Minimum build number 19041. Hardwre requirements are Intel Haswell or newer CPU, AMD Bulldozer or newer CPU. VMware Workstation Tech Preview 20H1 — Hyper-V/Host VBS Support