Im wondering as I consider a 10GB network for my two servers and my main PC (I will soon have a 10GB switch with 4 uplinks and 24 regular gb ports) I am wondering what to look for in a networking card that will be compatible with each other, then also what to look for in SPF+ connectors. Should I do fiber? Should I do direct connect SPF’s? Do I need to research compatibility of the SPF+ modules between Cisco and Intel Nics?
I am just looking for advice so I can not spend a lot only to find out it doesn’t play nice together. I do not need to cover more than 15ft (aprox 5 Meters). I think this will affect my choice as I know the length of the run effects the type of tech I should use. Or should I just try to use rj45 ports?
I’d definitely be more inclined to spend the extra on a couple of 10GbE cards such as the Asus XG-C100C, just so you don’t have to deal with fixed-length direct attached copper or fibre. It would make moving things around, should you choose to, a lot easier.
I have no experience with the Intel cards, i was kind of a mellanox schill?
Besides, i recently looked for used cheap SFP(+) Cables, hoped to get a 10/12M DAC for sub 20€ and i was surprised to see that SFP cables are even more expensive then QSFP cables, so i invested into another 12M QSFP DAC again.
Meaning, Base T seems to me to be the sweetspot unless you are going 25GbE or ultra long range or even 40 or 100GbE.
No idea how good the Asus card is. Aren’t some of those using Aquantia chips that aren’t that nice?
Card whise, used Mellanox cx2/3 should go for less then 35$ a piece and if you must go cheaper, the new killer deal in the diy space is an HPE FlexibleLom adapter card (10$) and another HPE FLom Card that goes for 10 to 15$, 10gBaseT is around 15$ QSFP+ and SFP+ should be below 10$.
You might have to solder the adapters yourself though. Might be able to get them done somwhere?
Fiber 100% depends on where its installed and distance. SFP+ DAC’s are cost effective in short runs, once you get over say 3m Fiber + SFP’s can end up costing less.
I would avoid 10Gb T products as the added heat and finicky cables have more downsides than up.
What Switch did you get, because that more than anything will impact your choices. If your switch allows unbranded SFP’s than generic DAC’s at short range will be the most cost effective. If your switch requires branded SFPs and DAC’s than you are to some degree at the mercy of that vendor.
Thanks everyone. @infinitevalence, below is the switch I managed to snag from a buddy for $200… I doubt he would mess me over, but I asked first. Told him it was a pricey switch. It’s also cisco, so should be the same as the SG350-10 managed switch I have been working with but with more ports.
The run length shouldn’t be much longer than 1M (3ft), I could probably do half that for 2 of the connections, then one connection to my gaming PC (IE for faster downloads from steam cache, movement of large files etc) would be roughly 4-5M (12-15ft).
Sweet, That’s awesome thank you for the link! Do I have to worry much about what PCIE card I use in my machines? One is a Dell Poweredge T320, and the other is a home made server Asrockrack x470DU with ryzen 2700. I just want to avoid conflicts with Proxmox, Citrix, Windows Server and VMware ESXi. If you have any experience with that, or anyone else does I would love some input.
generally no, in most cases if a slot is electrically a lower number of lanes the cards will just down clock or run slower. Sometimes you can run into slots that share with a PCIE Sata controller or M.2 slot and you have to tell the motherboard which to use.
I would say plug it in and see if it works, if not (which is possible but not likely) then worry about conflicts.
what kind of construction and style of house? I have a Ranch style 1970’s wood frame with a finished basement. I was able to run two conduits through the wall between closets and into the attic which gives me full access to any location in the house for drops.
Then I simply fish down the wall and install on a nice clean wall plate and box. I use normal electrical boxes and secure to the studs rather than low voltage cut-ins because it always ends up looking better and wont sag over time.
I have a patio home (so on a concrete slab) with standard wood frame and drywall. I can do two runs easy with my cable feed tool, the problem is the vaulted ceilings in the Livingroom. There is a wall I have to reach through out 15 feet, then down 90 degrees about 12 feet. There’s no conduit for the run I can feed through so that is the one I really need to have put in.
No, it laterally is against the frame for the roof in that area. I also need on drop to each side of that wall. One for the bedroom and the other for the livingroom. Those two rooms share the same wall.
even some pics from in the room can help, we may be able to find a path that does not require opening up the drywall.
I ran speaker wire through an enclosed ceiling without attic access by using a long flexible drill bit and pulling the lights out of the ceiling so I could use the space to get the drill bit in.
Some really good youtube vids on standards, cost, ranges, adapters etc that I am in no place to try and sum up, but check them out.
My personal 2 cents from dabbling in this a little bit:
Get cards that are 100++% verified to play nice with your hypervisor. I put “++” because I went with cards that a tech channel uses with Xen to then find out they were not that great (personally and then later that channel verified). There is work, and then working at full protentional IRT the drivers. Add to this the ‘hacks’ to upload drivers to make cards work- I hate this, it always seems hacky, maybe/maybe not works and doesn’t survive updates.
On that note @wendell had a video not too long ago about ESXi making big changes to their drivers and what is supported now, might not be later. So IMO do a deep dive into the card’s driver support on all potential hypervisors (or OS if going that route). I would look into universal’ness, as myself I went from ESXi to Xen, and I might go over to proxmox and then hell, maybe back to ESXi. So to select a card that is known good on all of those would be desired.
Personal peeve, I would like to stick to RJ45 where ever possible. It might come at a cost increase but I like the idea of running cable in the house ONCE that can work with whatever changes I have in the future- meaning maybe the servers move, maybe something that was just a 1 gig line needs to function at 10 gig now. Its nice to not have to worry about a bunch of different lines, connector adapters and standards but make it as PNP as possible.
