PCI-STUB vs VFIO-PCI

PCI-STUB is a dummy driver. It was originally developed because the original KVM device assignment didn't actually bind to the assigned device as a host driver, it simply grabbed the device and started using it. As KVM isn't a proper device driver, another host driver could attempt to bind to the device while KVM had it assigned to a user. The pci-stub driver was introduced to occupy the driver slot for the device while KVM made use of it.

As opposed to pci-stub, vfio is a complete interface for userspace drivers. It provides secure, isolated and IOMMU protected access to the device.

Advantages of vfio:

  1. pci-stub is often built into the kernel, this allows it to bind to devices earlier than drivers that are loadable modules. With this setup we can instruct pci-stub to bind to devices before the host driver can get to them so we can keep the device in a pristine state for assignment to a guest.

  2. The other lesser use case with vfio is with IOMMU grouping. If you have a group with multiple devices all the devices in the group must be bound to compatible drivers or else vfio will consider the group non-viable. This is intended to prevent host drivers within the same group as a user controlled device as they would not be isolated from each other. With pci-stub, we know this driver does not initiate any DMA on behalf of the host, so we consider it compatible.

    Thus if you have endpoints in your IOMMU group that you need to do something with to make the group viable, but you want to make sure the user doesn't have access to it, you can bind the device to pci-stub. This adds a little bit of extra protection vs binding the device to vfio-pci as a compromised user could not simply open other devices within the group that are bound to vfio-pci through the vfio interface.


As answered by Ehtesham, pci-stub is a dummy driver to prevent device drivers such as nouvaeu, nvidia, radeon, amdgpu from loading so that your device's BIOS won't be initialized by driver and can be passed through to KVM.

I want to add that, there is a configuration option where you just put a conf file, in which you can list other modules to load after vfio-pci ensuring that vfio-pci loads before device drivers.

For example, in Ubuntu 18.04 you create vfio-pci.conf in /etc/modprobe.d/ with this content:

#options vfio-pci ids=vendid:devid,vendid:devid2,...
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci
#softdep nouveau pre: vfio-pci
#softdep drm pre: vfio-pci
#softdep nvidia pre: vfio-pci
#softdep xhci_hdc pre: vfio-pci

The first line is not necessary if you added vfio-pci id list to grub.

Tags:

Kvm

Pci