Infinitiband card

Hardware Hub Networking
1

I had seen a video where they split a 40gb port with two sfp+ to connect two 10gb copper. I am rebuilding an old machine to act as my homelab server.

I saw a used solarflare infinitiband card with two 40gb HCP ports and thought it would be great if I could use as a 4x10GB NIC card.

Is this possible to do?

2

I know a lot of 40g cards are just 4 10g connections can not speak to that one. So you might only need 1 40g port not two
image

3

I was hoping for 2 on one card, but i might be able to do 2 catds.

Currently i have my modem, my terminal and third is my gaming rig. A fourth connection would be for projects.

4

I have seen the 1 to 4 cables. Thank you

5

only one link will work, I use this cable to connect a CX3 to 10Gbe Switch

6

You should check the datasheet for breakout cable support; there aren’t many cards that will support breaking out the 4*10GbE lanes on the NIC end. Breakout cables usually go into switches instead of NICs.

For one I know mellanox cards (CX3) don’t support breaking out the 4 lanes.

2 Likes
7

yes, only one of the four 10Gb lanes will work

8

doh that is not good. Thank you

9

Intel X540-T2 x2 should do the trick. Just thought it would be fun learning if I could split 1 or 2 40s on a used card.

1 Like
10

Yep, All 40gb cards are 4x 10gb links bonded topgether. That is how the 40gb spec works, and why 25gb was actually an upgrade to 40gb. The 25gb while having less bandwidth is a single link at 25gb and so clock rates went up and latency went down.

@tyrecies Remember that Infiniband is not Ethernet. Some IB cards can run in Eth mode, but if you are running Infiniband then you typically are not passing IP packets on it and you are required to run a controller somewhere to manage the network. Infiniband is normally for mounting and transferring storage data, like for SANs.

2 Likes
11

Good to know didnt have enough actual exp on these to be sure of that

12

Thank you for answering my curiosity.
I think I will use a couple Intel X540-T2 cards.

13

Sounds expensive

1 Like
14

also some clones on amazon

15

Refurbished, but here in the states

16

Is there a cheaper option? I am all about getting good tech on the cheap

17

You could get one of the Chinese knockoff Intel EOL NICs if you want. $55 for a two port 10gb RJ45 NIC is really good deal. IDK how much I would trust something like that personally.

Typically the cheapest, high bandwidth NIC options are used Mellanox cards. I wouldnt go under a ConnectX-4 now days though due to driver availability under the new Nvidia ownership.

1 Like
18

Don’t go below a connectx5 if you’re planning to use esxi. The writing is on the wall for Cx4 driver support.

For linux, the mainline kernel still has drivers for cx3 so the most economical option is still the cx3(10g/40g)…

Don’t get the wrong flavour though, a QSA adapter is another $20 and can easily wipe out any savings.

Edit: let back up a bit, are you looking for SFP+ NICs or 10GbE RJ45 NICs?

1 Like
19

What about Broadcom BCM957508-P2100G, a lot cheaper then Mellanox, but already suspiciously cheap for a PCIe4 based card?
I’m looking for new NICs for my homelab myself.

20

I was planning on using 10GbE RJ45 over some CAT 6A. I will do fiber eventually, I just do not know enough yet. I knew I might have to use transceivers to adapt to RJ45 though.