Im trying to activate SR-IOV on a Intel X540 T2 NIC on Fedora 32 and get the following dmesg:
[46381.219112] ixgbe 0000:0c:00.1 enp12s0f1: SR-IOV enabled with 1 VFs
[46381.219238] ixgbe 0000:0c:00.1: removed PHC on enp12s0f1
[46381.546374] ixgbe 0000:0c:00.1: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4 XDP Queue count = 0
[46381.566596] pps pps2: new PPS source ptp2
[46381.566623] ixgbe 0000:0c:00.1: registered PHC device on enp12s0f1
[46381.795282] pci 0000:0c:10.1: [8086:1515] type 7f class 0xffffff
[46381.795803] pci 0000:0c:10.1: unknown header type 7f, ignoring device
[46382.827254] ixgbe 0000:0c:00.1: Failed to enable PCI sriov: -5
[46384.838218] ixgbe 0000:0c:00.1 enp12s0f1: NIC Link is Up 1 Gbps, Flow Control: RX/TX
I try to activate it via:
[root@server ~]# echo 1 > /sys/class/net/enp12s0f1/device/sriov_numvfs
-bash: echo: write error: Input/output error
In Bios IOMMU and SR-IOV are activated.
My grub defaults:
[root@server ~]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet selinux=0 amd_iommu=on iommu=pt pci_pt_e820_access=on pci=assign-busses "
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true
My module settings:
[root@server ~]# cat /etc/modprobe.d/sriov.conf
options ixgbe max_vfs=1
system Infos:
[root@server ~]# neofetch
/:-------------:\ root@server
:-------------------:: -----------
:-----------/shhOHbmp---:\ OS: Fedora 32 (Server Edition) x86_64
/-----------omMMMNNNMMD ---: Kernel: 5.6.15-300.agesa.fc32.x86_64
:-----------sMMMMNMNMP. ---: Uptime: 12 hours, 57 mins
:-----------:MMMdP------- ---\ Packages: 1363 (rpm)
,------------:MMMd-------- ---: Shell: bash 5.0.11
:------------:MMMd------- .---: Terminal: /dev/pts/0
:---- oNMMMMMMMMMNho .----: CPU: AMD Ryzen 7 1700 (16) @ 3.000GHz
:-- .+shhhMMMmhhy++ .------/ GPU: NVIDIA GeForce GT 710
:- -------:MMMd--------------: Memory: 443MiB / 64243MiB
:- --------/MMMd-------------;
:- ------/hMMMy------------:
:-- :dMNdhhdNMMNo------------;
:---:sdNMMMMNds:------------:
:------:://:-------------::
:---------------------://
NIC:
0c:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
0c:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
I’ve also applied the AGESA Patch to my kernel to sort out issues on this side.
Does anybody experienced that error code (-5) yet?
Tnx, Stefan
Hmm the only time I saw that error was when I tried using the ACS patch kernel.
Here’s my efforts with the SFP+ version of that nic, the x520. Maybe something helpful in there?
Does anyone have tips on enabling sr-iov virtual functions for an Intel x520 10 gig network adapter?
OS is Fedora 31, kernel 5.3.13-301.acspatch.fc31.x86_64
Card is an Intel x520-DA1
Debugging info:
# echo 7 > /sys/class/net/enp8s0/device/sriov_numvfs
-bash: echo: write error: Input/output error
# dmesg
[794215.424272] ixgbe 0000:08:00.0 enp8s0: SR-IOV enabled with 7 VFs
[794215.425165] ixgbe 0000:08:00.0: removed PHC on enp8s0
[794215.514577] ixgbe 0000:08:00.0: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4 XDP Queue count = 0
[794215.553253] ixgbe 0000:08:00.0: registered PHC device on enp8s0
[794215.730059] br0: port 1(enp8s0) entered disabled state
[794215.764002] pci 0000:08:10.0: [8086:10ed] type 7f class 0xffffff
[794215.782748] pci 0000:08:10.0: unknown header type 7f, ignoring device
[794215.788066] ixgbe 0000:08:00.0 enp8s0: detected SFP+: 5
[794216.044593] ixgbe 0000:08:00.0 enp8s0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[794216.052254] br0: port 1(enp8s0) entered blocking state
[794216.052881] br0: port 1(enp8s0) entered forwarding state
[794216.827781] ixgbe 0000:08:00.0: Failed to enable PCI sriov: -5
# lspci -s 08:00.0 -vv
08:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-1
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 32
NUMA node: 0
Region 0: Memory at 47e7ff00000 (64-bit, prefetchable) [size=512K]
Region 2: I/O ports at 4000 [size=32]
Region 4: Memory at 47e7ff80000 (64-bit, prefetchable) [size=16K]
Expansion ROM at bad00000 [disabled] [size=512K]
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IO…
Thanks for the quick answer.
I got an Asrock Board and there are options for ACS, IOMMU and SR-IOV. I enabled them all.
I tried the pure Fedora 32 kernel and the AGESA patched version - so ACS patch cant be the cause.
Both with same result.
After some permutation testing in BIOS i was able to sort it out.
The setting in question was:
Advanced -> AMD CBS -> NBIO Common Options -> PCIe ARI Support -> Enabled
It needs to be enabled “Auto” isn’t enough there.
A patched kernel is not needed.
Hope it helps somebody running into the same problem (with an ASRock board).
4 Likes
Another case where “auto” makes no sense … why do the devs even include it, rather than just the basic on/off or yes/no options.
Glad you found the solution!
This is a shameless necro bump, because I wanted to say, thank you @derstef ! This was the last in a series of problems with my Intel X550-T2 that needed solving before it just now started to work!
1 Like
Legit just made a L1T account to say thank you. This was the solution for me as well.