I managed to get it to work. It even shuts down and restarts without crashing. I kind of feel stupid, because Wendell suggested the solution in one of his vfio videos 
Anyway here is the fix: Use a vbios rom
How to: Home · Wiki · Risingprism / Single GPU Passthrough · GitLab → go to 6) Preparation and placing of the ROM file and 8) Attaching the GPU to your VM → Adding your GPU and your ROM
Pre-dumped vbios: VGA Bios Collection | TechPowerUp
