How does Highspeed networking work?

Hello world,

I feel really ashamed of not knowing how this works. Could you please enlighten me on the following?

What is SFP and is it just Ethernet with a different connector?
Cat 8 can do 40Gbps, so should I use that instead of fiber?
Can I use fiber between my router and PC/server andor server to server? Would I just have another extra connection between them on another port?

Alvast bedankt.

1 Like

SFP … small format pluggable.

Typically used with optical networks to allow for e.g. 80km laser transceiver based link (e.g. to another city - expensive laser) to share a switch with another link that’s going to a router that might be in the same rack.

Switch will expose the SFP interface, which is closer to serial, and will send very low level formatted power and data to the transceiver module. A transceiver can house:

  • simple logic, e.g. a tiny buffer, some registers, amplifier and laser, and a detector/sensor (different wavelengths/colors).
  • … or simple logic, register, amplifier and ethernet phy,
  • … or could be a straight cable another switch
  • … or could be a whole gpon modem running linux internally pretending to be a dumb transceiver

> Cat 8 can do 40Gbps, so should I use that instead of fiber?

I don’t know, but 40Gbps over fiber is old news and cheap as chips.


Can I use fiber between my router and PC/server and/or server to server?

for 40Gbps, yes. You could also use a DAC aka. twinax cable to connect them directly (you probably have a QSFP+ ports on 40Gbps nics / switches). For short runs it can be cheaper.

Would I just have another extra connection between them on another port?

yes, for example let’s say you have a home server with 2xQSFP+ ports.

and let’s say you have a desktop with a 40Gbps QSFP+ port.

and let’s say you have a switch with a 2 QSFP+ ports, and a couple of SFP+ and a couple of 1Gig ports.

You could do

switch - server (at 40gig)
switch - desktop (at 40gig)
switch - router at 40gig
switch - other desktop at 10 gig
switch - playstation at 1gig

or you could do

server - desktop (40gig)
server - switch (40 gig)
switch - other desktop (10gig)
…

or you could do

desktop - server (40Gig)
switch - desktop (40 Gig)
server - switch (40 gig)
switch other desktop (10gig)
switch - playstation (1 gig)

in this last case, you have a triangle … so you’ll need to pick how you want to connect/contact different things. (e.g. enable spanning tree protocol, or carve out a tiny subnet for the direct server-desktop connection, or run some kind of dynamic routing protocol or … there’s tons of options… sometimes you can just use an entirely separate subnet and things will work fine.


think of SFP transceivers as dongles, and SFP ports as USB-C… depending on length of the link and type of cable, you may want to use different transceivers.

If you’re really looking for fiber … for 40gbps and below at home, look at SMF (duplex LC UPC cabling / transcievers).
If you’re looking for 50-100Gbps and above transceivers, look for MMF cabling (duplex LC UPC cables/transceivers)


AOC (active optical cable) is basically a fiber cable with transceivers permanently attached.


Some switches may end up limiting what brand of transceiver you end up using, mellanox/intel network cards are generally not picky. Lots of folks just buy their stuff from fs.com and they just choose the brand of transceiver they want from the dropdown.

HTH

3 Likes

Thanks for the great answer, opened the package up a little.

So if I understand this correctly, xSFPx just is a universal connector like USB?

Is that why I see 10GBase-T R45J on the end of that adapterconnector or ?

I wanted to try connecting the Fiber from my ISP to the PFsense router I have to see what it can do. Right now it goes into some sort of splitter or something from where it goes as an ethernet connection. At the other place a single fiber cable goes directly into the router. Ofcourse there is some magic box before it but it is still that fiber cable.

1 Like

Most likely you’re getting some form of gpon, which is usually terminated with an SC connector, and goes into an ONT or ONU.

ONT is like a gpon modem/gateway all in one, ONU is more like just a modem

gpon is a fiber network protocol for last mile ISPs, where they use basically prisms to split/merge light beams together for/from customers. When the OLT (ISP unit) transmits it uses e.g. 1500nm wavelength, and packets are sent encrypted to everyone, but only the customer they’re intended to can decrypt them, when consumers are transmitting they’re transmitting at 1300nm in the opposite direction and have to wait for their turn/time slot.

ONT will sometimes do crypto stuff with 802.1x, and sometimes ISP will do VLAN-ing to split off IPTV from generic internet traffic.

Sometimes they’ll require PPPOE, it all really depends.

Sometimes, some ISPs will use entirely different tech, e.g. init7 in Switzerland gave up on various PONs and just does LC-LC/point to point to their customers, each one on their own dedicated fiber.


Who’s your ISP, can you snap a photo of your ONT? … it’s possible you may just be able to take out the SFP/SFP+ transceiver from the ONT (transceiver itself is sometimes the ONU/all you need) and stuff it into a NIC and get it working.

3 Likes
  • SFP as above is a transceiver to convert a generic port into a specific media type; e.g., copper, single mode fibre or multimode fibre depending on requirements
  • fibre can do up to 400 gig or more. fibre also takes up less space than cat8 (which has bend radius requirements, run length limits, etc.)
  • you can run fibre between anything with an SFP port and an appropriate SFP transceiver.

You need to make sure they’re the correct type though.

There’s a few different fibre SFPs

  • Multi-mode: LED driven, cheap, up to a few hundred metres depending on speed
  • single mode: laser driven, come in different variants, good for 1.5km or up to 100km (from memory) in different, increasingly expensive variants
3 Likes

The other advantage/disadvantage of fibre is that it is not electrically conductive.

So…

  • POE is out
  • lightning doesn’t travel down it
3 Likes

This giy is going to do more learning in one throwaway post than he ever expected

2 Likes

Highspeed networking just uses faster electrons /s
Graag gedaan

4 Likes

Here is what I have for one of the networks.


2 Likes

Teleste looks like a Cable TV / IPTV box – and looks like junk left behind given how it’s not connected to anything past that metal distribution block on the top right.

Fiber comes from the outside through that metal “armored” cable into the cabinet, and into the “danger laser box”.

I guess inside, the external fiber is just spliced into an LC or is it SC connector there (SC is bigger… hard to gauge size with the perspective of the photos) , and if you were to unscrew the diagonal screws you’d find a few loops of fiber and a central “holder” for the delicate splice point.

There’s no power going into that danger laser box, that I can see.

From the LC connector, yellow jacketed fiber cable (single mode fiber - yellow is a standard color) goes into CTS HET2006

CTS HET2006 is a 100Mbps switch, with a wall mountable plastic tray that can serve as a fiber spool holder and I’m guessing because single mode fiber is going in, that it’s the HET2006FC variant.

There’s different variants to the switch, but only one can take single mode fiber. If you were to pop it out of the plastic holder (gently please), you might be able to see better and confirm what variant it is - and what fiber protocol it uses.

I’m guessing 100BASE-FX ; you can find SFP transceivers with the connector you need for <$10.


I’m guessing the purpose of the switch is to act both as a media converter and to split out one VLAN for the old TV box, and another VLAN for the data.


This whole setup is relatively old, but that’s dedicated fiber … in theory you could upgrade to 25Gbps with 50-60 euro worth of laser gear on either end. … Going to 100Gbps would cost you about 500 eur in laser gear on either end.

By 25 Gbps “laser gear” I mean SFP28 or similar 25GBASE-LR transceivers, you’d need to plug them into a $200 - $400 NIC, and use e.g. an Alder Lake i3 running pfSense… and you’d need to pay for them to plug on their side into a switch somewhere… I hope this makes sense.

3 Likes