I have a TrueNAS scale box with a connectx-2 card and I’m trying to connect to a PopOS and a windows host. I have both cards set to ethernet mode and both show up in ip addr. I have set an ip on the interface on both hosts on the same subnet. They both say up in ip addr and can ping their own ip but can’t ping the other address. Traceroute shows it’s trying to go through the right interface. The link light is active on both sides but no activity light. I’m honestly pretty confused why it’s not working. I have also set up two other systems with the same generation cards with truenas and ubuntu and I end up with the same outcome. I have tried different cables and that didn’t do anything. Does anybody have any ideas how to fix or troubleshoot this further?
Are they SFP cards or RJ45 cards?
If SFP, are you using a dac or transceivers?
Check the firewall settings? Is the network configured as private on windows, and are iptables/ufw/firewalld configured correctly on linux?
Does the ARP table on each machine show the MAC address of the IP address you’re pinging? Even if ICMP is dropped, or you have any IP-level firewall settings the ARP resolution should still work.
Have any of the options worked before? (such as a NIC in another machine). Are you using Truenas Core or Scale (night make a difference). Any my goto check - the cards are probably x8 lanes on the PCIe interface - is that an option in the slot you are using?
They are QSFP cards with DAC cables.
I have disabled UFW on the Ubuntu side. I didn’t find a firewall running on the TrueNAS side but maybe I just missed something.
Running arp -a doesn’t show any entries for the 10G network, only for the 1G connection.
The NICs were used but some of them might have been used for IB and were used with Cent 6 or older.
I am using TrueNAS Scale.
They are plugged into 16 or 8x slots on both sides and from what I saw on lspci they were running at full 8x speed.