Netgear XS712T

@Logan

First post here, be lenient.

I seen that asked people who are familiar with 10G networking to come on the forum and I happen to be of that breed. I have a question regarding the Netgear XS712T - I do not know what hashing it does to decide which link it is going to use. Other vendors like Cisco, Juniper and the likes of them offer you to specify it or tell you flat out what it is. I couldn’t find it in the documentation, but you telling us that we should have multiple IPs per machine seems to mean that the switch does a XOR on source ip and destination ip and won’t do more than that.

Please confirm.

@DeusQain and @wendell might want to hop in on this as well.

Ah the neat thing about multiple ips with multiple links that no hash algorithm is needed in that case.

It's up to the randomness of DNS . this is a clever way for those with less expensive switches to benefit from multilink with no direct 802.1x / lacp support.

In fact if lacp failover is all that is supported on your switch multiple host ips works best in that case.

For single 10 gig links the hashing protocol is on the nic not the switch. The nic has multiple receive queues that can be pinned to different cpus to help speed things along.

What was your question?

Ok where do I start.

I don't disagree with your theory @wendell but you have to look at it from the switch perspective. When the switch gets a frame it needs to decide which of the ports it is going to use to shoot down the frame to. Given that the mac address is probably already known it will use an algorithm to decide which of the ports to use that are using 802.3ad. I was kindly asking which hashing method the XS712T is using - normally cheap switches use the destination mac address after a XOR operation and it gets hashed and voila, we get a result and the switch will use the same port for the same destination mac address forever no matter how many ports are in the lag.

More expensive switches have the capability to influence which algorithm the switch will use to determine which port to use towards the server.

Some use layer 2 addressing, some use both the source and the destination. Here's a sample of some algorithm that Cisco switches can use to decide which port to use for instance:

dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port

Now given that we are talking to a server and that the mac address of an aggregated port is always the same, if the switch used an algorithm where only the destination mac address is considered (as seen above), the switch would ALWAYS use the same port in the bundle for ALL traffic regardless of how many IP addresses you have as they all have the same mac address.

So to reiterate my question: I looked at the documentation and was not able to see which hash method the switch uses to decide which link it will flood the traffic towards the server. I was wondering if you knew based on your testing.

P.S. 802.1x is authentication related and has nothing to do with this and LACP only provides the control plane for aggregation and does not contribute anything to the aggregation itself besides controlling possible layer 2 loops in misconfigured scenarios. 802.3ad does all the work.

Good read if you are bored: http://cs.uccs.edu/~scold/doc/linkage%20aggregation.pdf

Ok. For link aggregation, the gui does not give you any control over the mode beyond "LACP" and "LAG" options. LACP seems to implement the IEEE standard correctly in that I can cycle through about 4 different modes on an intel NIC that implements LACP correctly. that I'm able to test in a quick test. Both active and passive mode work. With LAG mode only it seems that I can only configure Round Robin or Backup (active/passive) on the Linux side and packets pass as they're meant to. XOR doesn't seem to work in LAG mode.

The other part of my reply I was answering a different question -- for LAG mode you can get the benefits of LAG Round Robin without actually having a LAG group through DNS trickery. You would have a random destination nic where the randomness is down to DNS and not a port selection algorithm, was my point with that part. If you want to use a group of ethernet ports like it is one big fat pipe you need something like LACP.

I think part of your issue with cheap switches is that they support the pre 2008 802.3ad w/o LACP? Because I've run into that too. I should explicitly LACP because now that I think about it the 2x10gb testing that I did would have worked correctly (and I did see 20gb of available bandwidth) whether or not LACP was in the mix. Because different interface addresses (and each card had about 4 receive queues anyway, which doesn't automatically work except on SMB3 right now afaik).

P.S. 802.1x is authentication related and has nothing to do with this and LACP

You are right and wrong. I did mean 802.3ad in this context, so oops, however you have to understand the full network stack here. 802.3ad existed before 802.1x but 802.3ad was moved to the 802.1 layer (802.1ax , 802.1aq) because that is the right spot in the network stack to have link aggregation when you take into account what 802.3ad does. This was a design flaw prior to the change and I believe has been fixed in equipment since about 2009 or so. You'll have to forgive me because I'm so old and these things are super pedantic. ;)