After getting VFIO working the xorg on the main OS crashes [Solved]

I have a vm with gpu,nic,usb passthrough that works fine, but since configuring everything my main os has random xorg crashes. The vm happily goes about its business as nothing happened.

Anyone have suggestions as to how to fix this?

After enabling some debug options i caught this:
Jul 28 21:24:50 drac kernel: show_signal_msg: 7 callbacks suppressed
Jul 28 21:24:50 drac kernel: Xorg[1250]: segfault at 7fffb7682ff8 ip 00007f991a29f379 sp 00007fffb7683000 error 6 in libc-2.27.so[7f991a21f000+1e7000]
Jul 28 21:25:18 drac gnome-terminal-[2150]: gnome-terminal-server: Fatal IO error 104 (Connection reset by peer) on X server :0.
Jul 28 21:25:18 drac systemd[1717]: gnome-terminal-server.service: Main process exited, code=exited, status=1/FAILURE
Jul 28 21:25:18 drac at-spi-bus-launcher[1817]: XIO: fatal IO error 11 (Resource temporarily unavailable) on X server “:0”
Jul 28 21:25:18 drac at-spi-bus-launcher[1817]: after 139043 requests (139043 known processed) with 0 events remaining.

Grub:
GRUB_CMDLINE_LINUX_DEFAULT="intel_idle.max_cstate=1 drm.debug=14 log_buf_len=16M mitigations=on intel_iommu=on vfio-pc$

System:    Host: drac Kernel: 5.1.16-050116-generic x86_64 bits: 64 compiler: gcc v: 8.3.0 Desktop: Cinnamon 4.0.10 
           Distro: Linux Mint 19.1 Tessa base: Ubuntu 18.04 bionic 
Machine:   Type: Server System: Supermicro product: Super Server v: 0123456789 serial: <filter> 
           Mobo: Supermicro model: X10SRL-F v: 1.01B serial: <filter> UEFI: American Megatrends v: 3.1c date: 05/02/2019 
Battery:   Device-1: hidpp_battery_0 model: Logitech Wireless Mouse MX Master charge: 100% status: Discharging 
CPU:       Topology: 6-Core model: Intel Xeon E5-1650 v4 bits: 64 type: MT MCP arch: Broadwell rev: 1 L2 cache: 15.0 MiB 
           flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 86398 
           Speed: 1438 MHz min/max: 1200/4000 MHz Core speeds (MHz): 1: 3541 2: 1902 3: 1345 4: 1537 5: 1953 6: 1960 7: 3341 
           8: 2335 9: 3784 10: 3690 11: 2860 12: 2565 
Graphics:  Device-1: NVIDIA GP104 [GeForce GTX 1070] vendor: Micro-Star MSI driver: nvidia v: 430.26 bus ID: 03:00.0 
           Device-2: NVIDIA GK104 [GeForce GTX 770] vendor: eVga.com. driver: vfio-pci v: 0.2 bus ID: 04:00.0 
           Device-3: ASPEED Graphics Family vendor: Super Micro driver: ast v: kernel bus ID: 0b:00.0 
           Display: server: X.Org 1.19.6 driver: nvidia resolution: 1920x1080~60Hz, 1920x1080~60Hz 
           OpenGL: renderer: GeForce GTX 1070/PCIe/SSE2 v: 4.6.0 NVIDIA 430.26 direct render: Yes 
Audio:     Device-1: NVIDIA GP104 High Definition Audio vendor: Micro-Star MSI driver: N/A bus ID: 03:00.1 
           Device-2: NVIDIA GK104 HDMI Audio vendor: eVga.com. driver: vfio-pci v: 0.2 bus ID: 04:00.1 
           Device-3: C-Media CMI8788 [Oxygen HD Audio] vendor: ASUSTeK CMI8786 driver: snd_oxygen v: kernel bus ID: 07:04.0 
           Sound Server: ALSA v: k5.1.16-050116-generic 
Network:   Device-1: Intel 82571EB/82571GB Gigabit Ethernet D0/D1 vendor: Hewlett-Packard NC360T Dual Port Server 
           driver: e1000e v: 3.2.6-k port: c020 bus ID: 05:00.0 
           IF: ens7f0 state: down mac: <filter> 
           Device-2: Intel 82571EB/82571GB Gigabit Ethernet D0/D1 vendor: Hewlett-Packard NC360T Dual Port Server 
           driver: e1000e v: 3.2.6-k port: c000 bus ID: 05:00.1 
           IF: ens7f1 state: up speed: 1000 Mbps duplex: full mac: <filter> 
           Device-3: Intel I210 Gigabit Network vendor: Super Micro driver: igb v: 5.6.0-k port: a000 bus ID: 08:00.0 
           IF: eno1 state: down mac: <filter> 
           Device-4: Intel I210 Gigabit Network vendor: Super Micro driver: vfio-pci v: 0.2 port: 9000 bus ID: 09:00.0 
Drives:    Local Storage: total: 5.24 TiB used: 1.15 TiB (21.9%) 
           ID-1: /dev/nvme0n1 vendor: Samsung model: SSD 960 EVO 250GB size: 232.89 GiB 
           ID-2: /dev/sda vendor: Seagate model: ST1000LM024 HN-M101MBB size: 931.51 GiB temp: 24 C 
           ID-3: /dev/sdb vendor: SanDisk model: SDSSDH3512G size: 476.94 GiB temp: 26 C 
           ID-4: /dev/sdc vendor: Seagate model: ST4000DM004-2CV104 size: 3.64 TiB 
Partition: ID-1: / size: 238.30 GiB used: 84.06 GiB (35.3%) fs: xfs dev: /dev/dm-0 
           ID-2: /boot size: 961.6 MiB used: 224.7 MiB (23.4%) fs: ext2 dev: /dev/sdb2 
Sensors:   System Temperatures: cpu: 26.0 C mobo: N/A gpu: nvidia temp: 48 C 
           Fan Speeds (RPM): N/A gpu: nvidia fan: 0% 
Info:      Processes: 376 Uptime: 20h 32m Memory: 94.37 GiB used: 20.17 GiB (21.4%) Init: systemd runlevel: 5 Compilers: 
           gcc: 7.4.0 Shell: bash v: 4.4.20 inxi: 3.0.27

I did some more debugging and narrowed it down to this, anyone have any ideas?

Core was generated by `/usr/lib/xorg/Xorg -core :0 -seat seat0 -auth /var/run/lightdm/root/:0 -noliste'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f141917f42b in __GI__IO_default_xsputn (f=0x7fffe6f87600, data=0x7f14192ad030 <zeroes>, n=3) at genops.c:389
389	genops.c: No such file or directory.
[Current thread is 1 (Thread 0x7f141be09600 (LWP 1275))]

Backtrace of thread 1:
#0 0x00007f141917f42b in __GI__IO_default_xsputn (f=0x7fffe6f87600, data=0x7f14192ad030 , n=3) at genops.c:389
#1 0x00007f141917137e in __GI__IO_padn ([email protected]=0x7fffe6f87600, [email protected]=48, [email protected]=3) at iopadn.c:64
#2 0x00007f141914df20 in _IO_vfprintf_internal ([email protected]=0x7fffe6f87600, [email protected]=0x7f141ae5755c “target PCI:%04x:%02x:%02x.%x”, [email protected]=0x7fffe6f87780) at vfprintf.c:1642
#3 0x00007f1419223169 in ___vsnprintf_chk (s=0x7fffe6f87880 “target PCI:\005ie\271\061\060\203U\222”, maxlen=, flags=1, slen=,
format=0x7f141ae5755c “target PCI:%04x:%02x:%02x.%x”, [email protected]=0x7fffe6f87780) at vsnprintf_chk.c:63
#4 0x00007f1419223095 in ___snprintf_chk (s=, maxlen=, flags=, slen=, format=) at snprintf_chk.c:34
#5 0x00007f141ae5568a in pci_device_vgaarb_set_target () from /usr/lib/x86_64-linux-gnu/libpciaccess.so.0
#6 0x0000560090e78038 in VGAGet (pScreen=0x560092578c20) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiterPriv.h:102
#7 VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3543, y=1011) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:948
#8 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3543, y=1011) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949
#9 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3543, y=1011) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949
#10 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3543, y=1011) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949

Backtrace of thread2:
#0 0x00007f14194f32b7 in __libc_write (fd=14, buf=0x7f141063dfd0, nbytes=23) at …/sysdeps/unix/sysv/linux/write.c:27
#1 0x00007f141ae554c0 in ?? () from /usr/lib/x86_64-linux-gnu/libpciaccess.so.0
#2 0x00007f141ae5569f in pci_device_vgaarb_set_target () from /usr/lib/x86_64-linux-gnu/libpciaccess.so.0
#3 0x0000560090e78038 in VGAGet (pScreen=0x560092578c20) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiterPriv.h:102
#4 VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3544, y=1012) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:948
#5 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3544, y=1012) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949
#6 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3544, y=1012) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949
#7 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3544, y=1012) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949
#8 0x0000560090e7804f in VGAarbiterSpriteMoveCursor (pDev=0x560092aa31a0, pScreen=0x560092578c20, x=3544, y=1012) at …/…/…/…/…/…/hw/xfree86/common/xf86VGAarbiter.c:949

I looked at the source code for genops.c at line 389 and it didn’t really help

Known issue with xorg, fixed by installing HWE packages for Linux Mint 19.2

h t t p s ://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/1839174