Looking glass/GPU passthrough/baby's first VM

SgtAwesomesauce · May 24, 2020, 10:02pm

wrap it with backticks for code blocks. “`”

GigaBusterEXE · May 24, 2020, 10:03pm

I’m not actually sure how to get to that grub file, did the command you gave me already edit it or did it just disable read only mode?
sudo grub-mkconfig -o /boot/grub/grub.cfg

SgtAwesomesauce · May 24, 2020, 10:04pm

So the command I gave you regenerated the grub configuration.

That’s done after you edit it.

so you can just sudo nano /etc/default/grub to edit it.

GigaBusterEXE · May 24, 2020, 10:07pm

so I can just add in anywhere
amd_iommu=on
I cannot even triforce

SgtAwesomesauce · May 24, 2020, 10:09pm

Oh, sorry, I should have been more clear.

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

You should see something like snippit above.

Dump it in there, would look like:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=on iommu=1"

Sorry, the code snippits look like this:

Screenshot from 2020-05-24 15-09-49

@Eden needs to update his guide

SgtAwesomesauce · May 24, 2020, 10:12pm

To learn about the forum’s features, please refer to this thread:

New and Existing Forum Features 2017 Rules, Guidelines, FAQ

The forums generally keep up to date with the latest stable or beta build of discourse. Built In Emojis have been enabled. Use responsibly Code Blocks As i've noticed not too many people know how to use them, there's a few ways to do code blocks inline like this inline `like this` as a separate line, use 4 spaces then your code/syntax like this As a block of text with coloured highlighting #!/usr/bin/python print("heres some python") ```python #!/usr/bin/python print("heres some python") ``` Plugins and Stuff A Solutions Plugin In some categories (Windows, and Linux for now) there is a solutions plugin enabled, allowing the topic owner to accept a post as a solution to their topic. Super useful for helpdesk/question-related topics. We might expand this to other areas but think these two are the main places it is useful for. All you need to do as the topic owner is select the check box for the post that had the solution. [image] Details Plugin Updated recently, this allows you to create a compressed details view to hide a bunch of information with a title heading if you want. [details=title to show]hidden details[/details]
title to show
hidden details
Spoiler You should already be familiar, we've had it a while [spoiler]Wilson was a volleyball[/spoiler] Patreon Plugin You may notice a little tick beside some users avatars, this indicates they are patreon peeps. (its done via email, so if you don't see one but should that's why.) Support for Tables You can use tables within your posts using HTML syntax. Example: <table> <tr> <th>header</th> <th>header</th> </tr> <tr> <td>cell</td> <td>cell</td> </tr> </table> Output: header header cell cell

New canned reply ^^

GigaBusterEXE · May 24, 2020, 10:13pm

okay I typed it in, now how do I save changes

SgtAwesomesauce · May 24, 2020, 10:15pm

with nano, it’s ctrl+o and then just hit return.

GigaBusterEXE · May 24, 2020, 10:20pm

okay now how do I get back to terminal

SgtAwesomesauce · May 24, 2020, 10:20pm

ctrl+x

the menu at the bottom, shows your commands. the ^ symbol means ctrl.

GigaBusterEXE · May 24, 2020, 10:23pm

okay now I redid the command
sudo grub-mkconfig -o /boot/grub/grub.cfg

you then said make a script and save it
do I just paste it in the terminal?

SgtAwesomesauce · May 24, 2020, 10:38pm

I was just talking about the IOMMU groups script that you ran above.

Now that you’ve got iommu config up, and you’ve got your IOMMU groups, you need to blacklist nvidia and set up vfio.

blacklist nvidia
blacklist nouveau

needs to go into /etc/modprobe.d/blacklist.conf

Then you need your device ids, lspci -knn, and you will see something like this:

43:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070 Ti] [10de:1b82] (rev a1)
	Subsystem: Micro-Star International Co., Ltd. [MSI] GP104 [GeForce GTX 1070 Ti] [1462:c303]
	Kernel driver in use: nvidia
	Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

you want the part after the GPU name, [10de:1b82] on mine.

GigaBusterEXE · May 24, 2020, 10:40pm

okay so how do I detremine if I need the ACS Patch
the K40 is in group 7 and the Vega 11 is in group 9 and both by themselves in each group as far as I can tell

IOMMU Group 10 0a:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller [1002:15de]
IOMMU Group 10 0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU Group 10 0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e0]
IOMMU Group 10 0a:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e1]
IOMMU Group 10 0a:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
IOMMU Group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU Group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU Group 3 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 4 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A [1022:15db]
IOMMU Group 5 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU Group 5 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 6 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0 [1022:15e8]
IOMMU Group 6 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1 [1022:15e9]
IOMMU Group 6 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2 [1022:15ea]
IOMMU Group 6 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3 [1022:15eb]
IOMMU Group 6 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4 [1022:15ec]
IOMMU Group 6 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5 [1022:15ed]
IOMMU Group 6 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6 [1022:15ee]
IOMMU Group 6 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7 [1022:15ef]
IOMMU Group 7 01:00.0 3D controller [0302]: NVIDIA Corporation GK110BGL [Tesla K40m] [10de:1023] (rev a1)
IOMMU Group 8 02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset USB 3.1 xHCI Controller [1022:43bb] (rev 02)
IOMMU Group 8 02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset SATA Controller [1022:43b7] (rev 02)
IOMMU Group 8 02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b2] (rev 02)
IOMMU Group 8 03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 03:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 03:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 03:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 8 07:00.0 Network controller [0280]: Intel Corporation Wireless 3165 [8086:3165] (rev 81)
IOMMU Group 8 09:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU Group 9 0a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] [1002:15dd] (rev c6)

SgtAwesomesauce · May 24, 2020, 10:42pm

If the Tesla is in a group with anything else, you need the ACS patch.

It’s alone in group 7. Should be all set.

SgtAwesomesauce · May 24, 2020, 10:46pm

Once you’ve got your PCI device ids, you need to throw them in a file, /etc/modprobe.d/vfio.conf, like this:

options vfio-pci ids=10de:1b06

Except replace the id with the one you got from the lspci command.

GigaBusterEXE · May 24, 2020, 10:50pm

when I opened that in nano it was blank? is that right or no

SgtAwesomesauce · May 24, 2020, 10:50pm

Yep, both of the files are new files, so should be blank.

GigaBusterEXE · May 24, 2020, 10:52pm

okay, file made

SgtAwesomesauce · May 24, 2020, 10:53pm

Sweet, now that this is done, you should be able to simply reboot and it’ll take effect.

you can check by lspci -knn when you reboot, and it should say Kernel driver in use: vfio-pci

GigaBusterEXE · May 24, 2020, 10:56pm

hmmm maybe I didn’t do something right

	Subsystem: NVIDIA Corporation 12GB Computational Accelerator [10de:097e]
	Kernel driver in use: nvidia
	Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia