I am 100% new to SFP+, and I realized a server I bought YEARS ago came with what I believe are Connectx-2 cards. I would like to put one in my truenas box and one in my PC to get 10GB direct connection for photo editing and such instead of standard 1gbe LAN.
I have part number 0RT8N1 (MNPA19-XTR I believe) which I believe is a connectx-2, and I am trying to figure out the most affordable way to run ~20 feet of cable from point to point, and determine compatibility.
I think fiber would be best as its thin and can be easily hidden (I know it is more fragile), but I am unsure what ends I would need.
Would these work? Do I want the generic tranciever or the mellanox version?
And yes, I would go with fiber for a 20’ run. I’m not even sure if they make a DAC that long. DACs are mainly intended to connect things within a rack.
Yeah Fiber is the way to go. As a bonus it also helps electrically isolate parts of your network from each other.
For transceivers I save a load of money getting “finisar” transceivers from eBay. They are well under $10 each and work perfectly with my mellanox and Intel NICs, and Ubiquity and Mikrotik switches.
Awesome. Any idea if I need the “generic” or “mellanox” version of the transceivers?
Hmm, I am not seeing finisar transceivers on ebay for good prices, I think the 20 dollar fs.com ones are actually cheaper, at least from what I am seeing.
I thought I was, but, I was putting “transceiver” after it which seems to have limited the results.
Any idea which one I should get? I am 100% new to all of this… I understand optics so I do know what 850nm “mean”, but I don’t know what it means as far as networking infrastructure goes, nor do I understand what the other nomenclature all is…
Yes the transceivers should work fine. The only gotcha is that the mellanox cards should be set to use the same protocol. Some of those cards can use infiniband, but you probably want everything to be Ethernet so you can ignore as much network bullshit as possible.
For the transceiver the magic words are
Finisar FTLX8571D3BCL: The make and model of the transceivers I use. 850nm: The optical wavelength, you can basically ignore this, I’ve not encountered a situation where it’s different given that the following terms are correct for what you want. Multimode: Multiple light signals are sent on the same fiber. This is what you want for a home network. SFP+: The physical connectivity form factor of the transceiver and NIC ports. There are others like QSFP, SFP28. SR: “Short reach”, as opposed to LR “long reach” which is the range the transceiver can push to over fiber. For a home network you want SR. LC: This is referring to the physical connector between the fiber cable and the transceiver. As opposed to SC which is much larger.
Another thing to consider is airflow over the NIC. They don’t seem to need much but they do need some (they were intended for screaming servers) so if it’s in a dead space consider putting a a little fan on it.
I think I am going to go ahead and try and get this working, sounds like a fun little thing to do.
And yes, I usually put 40mm noctua fans on all of my PCIe cards in my servers to give them a little bit of cooling, although now that I am going to be almost fully populating my server with a quad NIC, this, HBA and SAS expander, maybe a 3d printed 80mm fan holder to just blow some air over all of them vs a bunch of little 40mm’s may help to save space…
+1 - I’m currently using some FS (fs.com) generic and Mikrotik 10GBASE-SR SFP+ modules on Mellanox ConnectX-4 cards, used more in the past, no issues.
Sorry to bring up a nearly year old thread. But I have a question regarding heat. Is it the transceiver (SFP+ port) that gets hot, or the NIC chip itself? I finally pulled the trigger on the fiber and transceivers, and while my server has direct airflow across all PCIe lanes, my PC has effectively none (water cooled system with almost no fans in at idle).
I am trying to determine what I need to do that will 1) fit, seeing as this has to go in my bottom most PCIe slot (a 40mm fan would have maybe 2-3mm or clearance with the bottom of my case so it wouldn’t get airflow in that orientation) 2) what is actually required to be cooled.
I have a 3d printer so I can make a bracket and mount a fan off to the side and print a duct for the airflow to go where I want, but I’m trying to figure out what even needs the cooling. Could I get away with getting a small copper heatsink and mounting it to the SFP+ port and let radiation do it’s thing? Can I get a larger heatsink than what comes on the card, drill appropriate mounting holes, and again allow radiation to do its thing?
The transceiver claims to use 1 watt of power, that shouldn’t be an issue… even if I have to use a little copper heatsink that would suffice. But the chip I am less sure about. Do they both produce the same amount of heat while under load as when idle? These will barely ever be moving data around, but if they always consume the same power regardless of load, I need to make sure I have a decent enough cooling solution identified.
The transceivers are very dumb compared to its direct neighbours. I am not saying just a laser and photo diode, but close to it.
The controller though, may do quite a lot depending on the NIC, which may result in considerable heat.
Strapping a fan directly to a heatsink is not necessarily the most efficient mounting spot, so having the fan close and pointing at the heatsink (maybe with some cardboard-engineering?) should at least help.
If you have an SFP Transceiver that converts to RJ45 then they get quite hot, but a basic fiber one does not.
The NIC chip getting hot depends entirely on the chip. Ive had some Intel ones that get so hot they burn my hand to touch. They function without issue at that temp, but it has to be like 100 degrees C and a fan on it would obviously be better. But then Ive had others like a newer 7 series Intel NIC that has more capability but it doesnt get nearly as hot. And some older ConnectX that only get a little more than warm. It really depends on the process node used to make the chip, how well it was designed, the cooling potential of the card and the case its in, how much load is on the card and what kind of work the chip is being asked to process. So many variables, its hard to say whether one will get hot or not.
All good points. It’s a conectX 2, and it will be doing nothing 99% of the time. I’ll be using it for a direct connection to my truenas box, but this link is rarely used. When it is, it’s simply going to be some SMB transfers typically in the single to low double digit GB size range.
Forgot to add, yes I will be using s fiber transceiver not a rj45. Good to know that won’t be an issue at all.
Sorry to necro this thread, but I’ll be upgrading my rig soon, and I currently run connectX-2’s in both my PC and Truenas VM for fast 10 gig network between them.
Reading up on it a bit, it looks like there isn’t a connectX-2 driver for Win 11 and looks like folks have had issues getting them to work.
Does anyone have any feedback, or recommend a simple solution? Looks like I could potentially upgrade to X-4’s as they are supported, but I am not sure if I would also then need different transceivers. I currently am using Finisar FTLX8571D3BCL SFP+SR/SW 10Gb/s 850nm Multimode SFP+ as previously recommended in this thread, and it’s been working fantastically on windows 10.
Thanks for the reply. I have seen this a few places, but I have also seen info state that is not the case. I assume it’s mostly dependent on what card you’re using?
