LACP vs SMB Multichannel

Networking
1

Short version:
What is preferable for transferspeeds, LACP or non LACP?
Can SMB Multichannel use LACP or does it just see it as one nic?

Long version: To begin with, there is a 36 tb storage SAS storage server. There is also some iSCSI drives, around 10 tb SATA and 5 SAS. The switches I use are Cisco 2960, they can handle LACP or PAGP no problem. All the switches are connected with 4 nics, 1gigabit each combined with LACP to enable 4gigabit transferspeeds between switches.

With that out of the way, if I want to access the fileserver and get the fastest speeds possible, should I nicteam?
Nicteaming combines the nics from, in this case 4 physical to "1 virtual". This, in theory would create a good thruput of data. Put the speeds I get are pretty much just 1gigabit in file transfers. Mind you, that is 1 gigabit even if there is 4 hosts writing at the same time. But this in theory makes SMB multichannel unavailable.

Now, the test with SMB multichannel, all nics separate revealed that I get pretty much the same speed, when it has used up all the RAM. But the disks are not that slow. I will include the speeds further down.
The iSCSI disks are also alittle odd. Since they are connected with 2 nics each, ALB and LACP are available but non used atm.

So LACP provides the same speed (1gigabit) that non LACP provides, when all the RAM is used.

What am I doing wrong? Why are the speeds so slow?

Sequential Read (Q= 32,T= 1) : 455.796 MB/s
Sequential Write (Q= 32,T= 1) : 48.317 MB/s
Random Read 4KiB (Q= 32,T= 1) : 59.621 MB/s [ 14555.9 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 4.641 MB/s [ 1133.1 IOPS]
Sequential Read (T= 1) : 285.228 MB/s
Sequential Write (T= 1) : 48.656 MB/s
Random Read 4KiB (Q= 1,T= 1) : 4.154 MB/s [ 1014.2 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 0.476 MB/s [ 116.2 IOPS]

2

That's 3+ Gb/s.

Why did you expect SMB multichannel to do anything to iSCSI traffic?

Can you show a net schematic? It's kinda hard to understand where's what just by your description.

3

LACP will not up your data rate unless you do some work and the hardware supports it. LACP is really for failover purposes and only one of the channels will be used.

I know I can get around this on esxi talking to my nas on iscsi if I enable the right options and put each connection on a different subnet. This will not necessarily increase a single transfer though because it is still one thread. It will help on multiple threads going at the same time.

I just got around it by running point to point 10GBit between my esxi and nas.

4

This isn't true, you can use link aggregation to increase bandwidth but not between only two hosts.

1 Like
5

which is what I stated in my second paragraph.

6

You can use LACP and get increased throughput, you don't need to use multiple subnets or any of those other hacked methods. I don't know if or how well it works with SMB multichannel but it is not just for failover.

1 Like
7

level1tech.png1085x538 4.96 KB

The 3gigabit is locally, not over the network

8

Thank you, that was basically what I needed to know!

9

If "switch" on your schematic is either one switch or two stacked switches, I think you should be able to use NIC teaming on server side with Etherchannel/LACP on a switch side and get both SMB multichannel and iSCSI MPIO working on top of that. Not sure how load balancing will work in this configuration.
If "switch" is two redundant (not stacked) switches, you can't use NIC teaming but both SMB multichannel and MPIO should still work.