That is correct. I even fixed my problems with Nvidia GPU passthrough (Using Either vendor is possible for me now), it just took time to troubleshoot, it was literally down to a bug in the UEFI version I was running for my motherboard causing issues with PCIE passthrough.
That said, Host card wise, AMD GPUs have proven to be a bit more consistent in my configuration when doing native linux or WINE+DXVK gaming, along with tuning CPU governors.
Nvidia is perfectly fine to use as well, though I never really nailed down the frame pacing issues I’d see from time to time, Though it could very well be my particular configuration/
TL:DR; The solution is entirely dependent on what gives you the best compatibility with your given hardware. Sometimes that takes trial and error.