Driver for GTX 1080 doesn't work on guest when using KVM PCI Passthrough

I had the same problem, I found the answer at https://www.evonide.com/non-root-gpu-passthrough-setup/. You need to add -cpu host,kvm=off to the qemu command line. I'm using ganeti, so the following fixed the problem:

gnt-instance modify -H cpu_type="host\,kvm=off"

If I understand correctly this flag does not switch off KVM acceleration for the guest, that's switched on with -machine pc,accel=kvm. But it switches off nested KVM acceleration for the guest (so you won't be able to run a KVM virtual machine inside the guest).

I don't know if I have much to contribute, it seems to me that nvidia is actively sabotaging pci passthrough attempts (you are supposed to buy a Quadro card).

Try faking the pci device ids by adding x-pci-vendor-id=0x10DE,x-pci-device-id=0x11BA,x-pci-sub-vendor-id=0x10DE,x-pci-sub-device-id=0x0965 to the -device vfio-pci, argument. This will make it pretend to be a Quadro K5000.