So… I don’t like doing things the easy way. I got myself 2x connectx-3 40gb cards, dual interface. They were cheap, and I pushed the add to cart button on Amazon. I was able to flash them to 40gb Ethernet mode. That was an adventure.
Anyways, I have a 10gig switch. So at minimum my truenas box and main computer connect at 10gig through the switch. I have set up routes on each to allow for the connection directly (on the second port) between the server and workstation to be at 40gig. Sometimes they connect using the direct connection, sometimes they use the switch and go through the ten gig connection.
How do I configure my workstation to prioritize the 40gig connection when asking for files or sending files to the truenas box? The ten gig is on a separate network than the 40 gig (192.168. vs 10.10. something, so there isn’t a confusing of conflict). The 10 gig goes to the switch and has access to the other devices. The 40gig is a direct connection to the workstation from the server.
Also, I may not even be asking the right question from the correct angle. So if there is a better idea to bypass all of this, let me know.