Need the cheapest way to 10G two computers together

Networking Networking Hardware
1

I live on social security so money is really tight. Crypto mining brings in $100 to $150 a month which I can use (some of it) for non-essentials like tech toys.

I have a ‘spare’ box with 3 HDDs in a raid zero config (for now) running on an FX-8350 (8 core, 4Ghz) platform, so plenty of hardware to serve up a 3 disk array. Essentially, I caught a great black Friday sale and just retired my old desktop to serve as a NAS.

I have a 1000 Base-T / WiFi LAN with a router, switch, printer, phone, tablet, and 2 computers hanging off it. It’s all connected and working great. Thus, phase one is completed. The problem is that I got really, really, spoiled with the 400 MB/s of the local raid (which is now no longer local). I’m currently pleased that I get 100 MB/s reading big files, but I feel entitled to all of that juicy 400 MB/s that I know is just waiting to fly over the wire. That terrible, tiny, restrictive, vindictive little 1000 Base-T pipe.

The long-term plan is to upgrade the LAN to 10GBase-T, but it’s pricey right now, and I only need 10G for the desktop and NAS, everything else is fine at 1G for now. So that brings us to phase two, which is to find a short-term ultra-cheap way to connect two boxes together. The NAS is running Windows 10, but will soon be switched over to Linux and some ZFS goodness (phase 3).

I was looking at things like Mellanox ConnectX-3 EN Single-Port 10/40/56GbE Adapter w/ PCI Express 3.0 but I have no experience with anything that doesn’t terminate with an RJ-45. Would something like this work? What sort of cabling is used to connect them? Can they be connected directly without a switch?

The two computers are about 20 feet (or 6 meters) apart, as the bug crawls. A few corners need to be rounded, but not sharply, so optical fiber may not be off the table – if that’s what these things use.

I would need both Windows and Debian Linux driver compatibility. The new desktop is PCI-E 3.0, and the older NAS is PCI-E 2.0 (if it matters). Both motherboards have at least 8 PCI lanes unused. I can’t think of any other specs you need.

Phase two can be short-lived. In 8-12 months, I can buy some new 10GBase-T cards like ASUS XG-C100C 10G Network Adapter . The problem is that I just can’t bear to live with 100 MB/s for 8 months, so I have to get a throw-away solution to calm my frazzled, impatient nerves.

Thanks for any ideas or advice.

2

Get used ConnectX or ConnectX-2 NICs on e-bay. Flash them with the newest firmware.

I think 20ft is too long for a DAC cable, so you’ll need to get short range SFP+ transceiver modules and OM3 fiber cable. Here are my recommends on Amazon, assuming you go with Mellanox cards:

The only thing I’m not 100% sure of is Windows 10 compatibility with the older Mellanox cards, but I would be surprised if there was a show-stopping issue there. That said, you will need to do some tuning to leverage the full 10G of throughput on both the Linux and Windows side.

This should cost you $100 or less for a single point-to-point connection.

You can expand the point-to-point to the broader network later by simply upgrading your switch to one that has 2 SFP+ ports.

3 Likes
3

I’m using a Connect-X card with windows 10 and the drivers were no trouble.

I think the maximum length for DAC is 10m so 20ft should be fine.

3 Likes
4

It’s possible I was confusing feet for meters.

2 Likes
5

I can highly recommend the Connect-x 2s. Usually you can find a pair together on amazon for around 40-ish last I checked.

1 Like
6

@oO.o @Dexter_Kane @Lyynx92

Thanks everyone. I sincerely appreciate everyone taking the time to contribute your expertise. However, could we roll this back a level or two? What is “DAC” and “SFT+” (in this context) and what are the pluses and minuses of those?

What inter-connect media is used? What is it called if I wanted to buy one or what specs do I need to be aware of? I assume it’s optical fiber?

When I said “I have no experience with anything that doesn’t terminate with an RJ-45. Would something like this work? What sort of cabling is used to connect them? Can they be connected directly without a switch?”…

I was being serious, but I never got an answer to those basic questions. I promise to keep up, if you would be kind enough to cover a bit of the big picture so I can put all the pieces in place.

Once again, thank you all. I’m guessing that for 6 meters (20 feet) “DAC” is better or cheaper for some reason. I’d love to know why and what things I’d need to buy to use “DAC” to accomplish this goal.

7

SFP+ Is the connector type that you install either a transceiver (for fiber optic cable) or a DAC cable in to. DAC is direct attached copper, it’s a cable for connecting two SFP devices together without the need for transceivers. For a short run that’s the way to go, especially if you don’t know anything about it. Fiber can get complicated very quickly so stick with DAC if you can.

3 Likes
8

Sweet. Thank you for that.

Edit: @Dexter_Kane
Not sure if the reply button notifies you, or I need to @ you. :slight_smile:

9

So to be nore specific, get a connect-x card with SPF+ ports, make sure you can get windows drivers for it before you get it, but I think the mellanox cards aren’t a problem to get working with windows. Then get a 10g SPF+ DAC cable to link the two together.

As for the configuration just manually configure the network on both machines with IPs in the same subnet and they should be able to talk to each other just fine.

5 Likes
10

@Dexter_Kane
I’m trying to find the forum button to hug you, buy you a favorite beverage, or otherwise express my immense gratitude. This advice was exactly what I needed; at the tech level that I needed (or enough to find some white papers on it).

Thank you so very much.

1 Like
11

No problem.

DAC cables, especially longer ones can be pretty expensive, but probably not that much more than fiber if you have to buy transceivers. fs.com is a good place to get generic fiber, DAC cable, transcievers etc, if you don’t mind buying from china.

1 Like
12

@Dexter_Kane
So something like this item would work for a 6 meter run, yes?

Also, does just hitting ‘reply’ work, or do I need to use the @ notification to get your attention? I love this forum software, it’s really nice, but It’s not obvious by the threading display whether a reply is user-centric or thread-centric.

In any case, thank you so much for your time and kindness.

13

Pretty sure if you reply to a post it notifies whoever made the post and the big reply button at the bottom replies to the thread which only notifies the OP or people who are following the thread.

I’d get the generic version, https://www.fs.com/products/30763.html It shouldn’t matter as it’s usually only switches that get fussy about using the wrong vendor equipment, but just in case.

14

Thanks, got it.

Ahhhh, but the link I chose is the same price and has “EXTREME” in the name so it must be awesome, of course. Like a red racing stripe = 5 HP for sure. :slight_smile:

15

That’s allegedly a switch brand name, it should work but I don’t know. I’ve never heard of a NIC being vendor locked like switches can be but I have used generic transceivers with mellanox cards before and they worked fine, so better safe than sorry considering the only difference is the firmware spoofing what vendor it is.

16

Thanks for all your help. I hope to have paid it forward already, or pay it back in the future (on a different topic).

I built my own “PC” (with hand-made circuit boards and an MC6809), before IBM ever invented the “PC”, as a senior in high school. I’ve written custom multitasking operating systems in many assembly languages. I’ve written programs in dozens of “high” level languages during my career. Surely, I must have something useful to pay-forward. If only it was relevant or anyone cared nowadays. :grinning:

Anyway, my sincere thanks from an old man trying to stay relevant and continue learning. :wink:

2 Likes
17

I have what might be a really dumb question, I have seen lately seen a lot of switches that have 24 1gb connections and have 2 or 4 SFP+ 10gb ports, my question is aren’t those 10gb ports only used to interconnect switch’s or up link’s only.

What I am talking about are the funny look ports on the left side.

18

They can be used to connect to another switch or any other device that uses SFP+, they’re just 10g ports. Although in this case those are SFP ports not SFP+, they are only 1g

1 Like
19

Shadowbane, what forum magic does one use to go from a boring link (like my OP), to a pretty advert box like your post?

TIA, from an old guy trying to come up to speed. :grin:

20

Seems to be automatic, just put in a link (as in, just paste in the URL) and if whatever sorcery is involved decides to make the box then it just happens. I suspect it only happens with certain sites, like ebay or amazon etc