Homelab Upgrade to 10Gb/s+

Hardware Hub Networking
, ,
1

I’m looking to upgrade my homelab with 10Gb networking. I’ve been looking at several switches but I can’t decide which would be best for my lab.

My current lab setup consists of a cable modem with 2.5Gb out (I get about ~1.4Gb/s down right now) that is connected to my Pfsense server (has 6 or 8 10Gbe ports on it) via a qnap 2.5Gbe PCIe card, then I connect my Pfsense machine to my Ubiquity 24 port non-POE switch and from there the connection goes to my desktop (has 2.5Gb) and my 2 HP servers.

I would like to be able to connect my desktop via 2.5Gb and my two servers via 10Gb DACs or RJ45 depending on what switch I get. I’m finding quite a few deals on SFP+ switches like the Dell 4810P and even Ethernet based switches like the Arista 7050T that Wendell did a video on. From what I can tell these options can only do 10/1 and cannot negotiate down to 2.5, so I’m trying to find a switch that can negotiate to lower speeds if possible, I’m guessing a SFP+ based switch and a SFP module that can go down to 2.5 would probably be the best option but I have no experience with this, since most of my work is done with 1/40/100+ switches.

2

I can’t vouch it’ll sync to every sfp+ or sfp module and speed but I get 10 and 5 where I expect it to and didn’t even have to login to specify anything manually (no 2.5 here). I use 10GTek DAC and optical/Rj45 Modules.

3

Yeah I was seeing those around but it’s a bit small and not rack mounted, I’m hoping to do multi 10Gb/s connections to my servers, one of them does AI/ML which will be fine with one 10Gb/s but I’d really like to do 2-4 connections to my NAS/Virtualization server.

4

How many ports would be needed at 10g, and how many at 2.5
Would 40gbit for the Nas be an option?
Do you care about noise?
Do you care about idle power draw?
Will you want layer 3 capabilities?

5

Could definitely do 40 for the NAS just need to grab a card for it. I only need 1 or 2 2.5Gb ports and I’d like 8+ 10Gb ports, ideally 16+. Don’t need layer 3 but it would be nice, don’t really care about power draw too much, and I’d like to not hear it from the room above where my servers are. I do want to avoid the 10+ year old 10Gb enterprise gear with the 800+ latency.

I’ve been looking at some of the MikroTik rack mount switches with 8 or 16 ports but it seems like all of the prices for them have been inflated by quite a bit. The CRS312-4C+8XG-RM was under $500 a few years ago and now the absolute cheapest I can find it for is $555.

6

Thats a pretty good price for that for new norm

7

I did find a Dell S4810P-AC in great shape locally for $225 with a few SFP+ modules included, I’m thinking that may be a good deal, I just don’t want to run into issues with it having latency issues and whatnot. When I was looking into the latency apparently the Dell has .8us of latency and the MikroTik switches have 25us, but I don’t really understand the different measurements of latency and whatnot.

8

Whats the dell idle power?
Max. power consumption: 350 Watts (AC), 300 Watts (DC)
Typ. power consumption: 220 Watts

Mikrotik
Max power consumption 60 W

Not sure on your power costs / how much traffic youll be running through it etc

9

Not too much traffic, it’s just me in my house. Power is only $0.11/kwh and I have about $100/month worth of solar on my roof.

1 Like
10

Then no big deal

1 Like
11

Have you looked into UniFi 10Gb agg switch?

12

Yeah I was taking a look at that, I’m sure it would work well but I feel like I’d use up all of the ports too quickly.

13

If you aren’t in a hurry I see them for sale on Reddit used fairly often for a bit of a discount.

14

Oh? Where can I find said sales? Lol

15

Yes, if you got a 10gb SFP+ switch then you could use modules that support 1/2.5/5/10 and it would auto negotiate the link speed.

If you dont mind spending the extra money, you can always go with the pro model and get 32 ports of 10gb SFP+ and 4 ports of 25gb SFP28
Then stick one of these NICs in your pfsense router for interfacing to the switch:

https://www.ebay.com/itm/133456123522

https://www.ebay.com/itm/364021177256

Or the cheapest option but not sure if pfsense has drivers for it or not:
www.ebay.com/itm/185727604398
16

In that price range a brocade 6610 would be a great option, I think

17

CRS317 is “the same, but much bigger”

18
1 Like
19

Hmm, what advantage would the brocade give me?

20

Less than half the power draw, still meets your current 10g/40g port count, probably less noise

Otherwise a 6650 would be more on par, still less power draw, not sure about the noise

Other than that, it’s a Brocade and not a Dell…
So it would depend of you are already familiar with the cli/naming conventions .