Super late reply here, but I was searching the Linux sub for vfio because I love this topic, lol. I'm happy to report that as others are saying, a vfio setup can be very performant! The GPU is near native, and with the proper CPU configuration, you won't be dealing with much overhead there either. The biggest factor is the overall load on your machine from running a host OS with the guest layered on top. I use my PC host OS primarily as a QEMU/KVM hypervisor, so when I need maximum gaming performance I simply turn off or suspend operation on all other VMs to free the resources for Windows. The only game I've had trouble with recently at 3440x1440 is Starfield, and unfortunately I think that is more a sign that an upgrade is finally due. I've used this setup for all kinds of games with no problem, including demanding virtual reality titles.
The biggest problem I've run into? Some anticheats truly despise any kind of virtualization. I've only run into issues with 2 games I like to play, but that is a real caveat. I consider it 100% an issue with the companies implementing these policies, who as I see it don't deserve my money in the first place. But other than the initial configuration headache and extra hardware required (if you don't have an iGPU anyway), this is the main drawback in my eyes.