SMB share slower than expected

My FreeNAS box is running a samba share which seems to be much slower than what I expect.

When I test the link speed using iperf, I get speeds around the theoretical max. of 940 MBit/s. The same happens with WebDAV shares, where I get around 110 MB/s. So I can safely assume that my HDDs and my NIC are fast enough.

However, using the samba share, I get only around 70 MB/s. How can I figure out what is causing this? I tried these parameters I found in a different forum post, but they seem to make no difference:

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536

PS: I am not sure in which forum this post fits, so I thought this one might be fine.

SMB is slower by default since it is a network file browser, not just a network file transfer. Same thing happened to me on my setup as well.

SMB being about half as slow as regular SFTP.

1 Like

But my system doesn't feel overloaded, so where is the bottleneck? CPU load looks fine and RAM shouldn't be an issue. Hmm...

its because there are checks that run on the payload which lowers throughput.

So you're saying that no matter what I do with my current hardware, I won't get more than the ~70 MB/s?

How are professionals solving this issue? I mean they can't be satisfied with that performance. Does this figure improve with 10-GBit-Ethernet?

lol I was getting 60 MB/s.

depends on the situation. If its just used for documents in a SOHO environment, 70 MB/s is more than fast enough (unless they want to use large file transfers). But I believe they speed this up with link aggregation.

Hmmm unsure. Its possible, I just don't have access to that kind of hardware to test it.

Hmm.. So I guess I should be satisfied. ;-)

Me neither. I am just wondering how people like Linus from LinusTechTips do it. They have multiple people working directly on the data off the server via 10-Gbit-Ethernet links. And he was getting multiple hundreds of MB/s if I remember correctly.

SMB on freenas was always way slower for me as well. I could transfer to my Synology more than twice as fast. Same for a Windows machine.

I don't think the cifs on freenas is very optimized.

I get 44MB/s from my Ubuntu server over SMB :( Same with iperf, close to full 1Gb/s. Got the Realtek r8168 NIC though (with the r8168 driver module, not the default r8169 which does not always work properly), don't know if anything would be different with an Intel NIC.

I don't know about freenas but I have no problem (and have never had any problems) getting 100MB/s+ over SMB on linux and windows. Not sure what the problem is but I don't think it's some inherent limitation of the protocol.

2 Likes

Yeah using SMB shares on my freenas box I have no issue achieving 100MBps+ when accessing it from a Windows 10 Machine. I think there must be some kind of issue with the SMB configuration or Network Configuration, because 100MBps should be quite easy to obtain using modern hardware. I don't know specific troubleshooting steps but I think @wendell probably knows more than a couple that could help.

SMB is not multithread; be sure you are looking at top configured by cpu core and be sure that SMB is not pegging one CPU core. a 2.0ghz 8 core cpu, for example, is too slow for gigabit smb because the protocol has so much overhead.

You can also disable encryption, on the client and server, which will help with some of that overhead

4 Likes

In my server there is an 8320E @ 3.4 GHz. Shouldn't that be fast enough?

Yep I have an 8350 and it's fine. 3+GHz should saturate wirespeed
Heck a Xeon x5560 can manage about 500 MByte/sec on 10 gig even

1 Like

hmmm I must've had a different issue then.

I forgot to mention my hardware. The system has a Xeon E3-1220v5 @3.0GHz with 4 cores and 4 threads. When I transfer one big file (which I feel like should be the best scenario for fast transfer speeds), I get around 10-12% CPU load according to top. When I split it up by core, I don't see any core struggling really.

Here is an example screenshot of top on the server:

EDIT:

What kind of system is that? Did you get similar results on FreeNAS / FreeBSD too?

If it is of any help, here is the output of testparm -vvv:

# Global parameters
[global]
    bind interfaces only = No
    config backend = file
    dos charset = CP437
    enable core files = Yes
    interfaces = 
    multicast dns register = Yes
    netbios aliases = 
    netbios name = FREENAS
    netbios scope = 
    realm = 
    server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbindd, ntp_signd, kcc, dnsupdate, dns
    server string = FreeNAS Server
    share backend = classic
    unix charset = UTF-8
    workgroup = WORKGROUP
    browse list = Yes
    domain master = Auto
    enhanced browsing = Yes
    lm announce = Yes
    lm interval = 60
    local master = Yes
    os level = 20
    preferred master = Auto
    ads dns update = Yes
    allow dns updates = secure only
    dns forwarder = 
    dns update command = /usr/local/sbin/samba_dnsupdate
    machine password timeout = 604800
    nsupdate command = /usr/local/bin/samba-nsupdate -g
    rndc command = /usr/sbin/rndc
    spn update command = /usr/local/sbin/samba_spnupdate
    mangle prefix = 1
    mangling method = hash2
    max stat cache size = 256
    stat cache = Yes
    client ldap sasl wrapping = sign
    ldap admin dn = 
    ldap connection timeout = 2
    ldap delete dn = No
    ldap deref = auto
    ldap follow referral = Auto
    ldap group suffix = 
    ldap idmap suffix = 
    ldap machine suffix = 
    ldap page size = 1000
    ldap passwd sync = no
    ldap replication sleep = 1000
    ldap server require strong auth = Yes
    ldap ssl = start tls
    ldap ssl ads = No
    ldap suffix = 
    ldap timeout = 15
    ldap user suffix = 
    lock spin time = 200
    oplock break wait time = 0
    smb2 leases = No
    debug class = No
    debug hires timestamp = Yes
    debug pid = No
    debug prefix timestamp = No
    debug uid = No
    ldap debug level = 0
    ldap debug threshold = 10
    log file = 
    logging = file
    log level = 2
    max log size = 51200
    syslog = 1
    syslog only = No
    timestamp logs = Yes
    abort shutdown script = 
    add group script = 
    add machine script = 
    add user script = 
    add user to group script = 
    allow nt4 crypto = No
    delete group script = 
    delete user from group script = 
    delete user script = 
    domain logons = No
    enable privileges = Yes
    init logon delay = 100
    init logon delayed hosts = 
    logon drive = 
    logon home = \\%N\%U
    logon path = \\%N\%U\profile
    logon script = 
    reject md5 clients = No
    set primary group script = 
    shutdown script = 
    add share command = 
    afs token lifetime = 604800
    afs username map = 
    allow insecure wide links = No
    async smb echo handler = No
    auto services = 
    cache directory = /var/db/samba4
    change notify = Yes
    change share command = 
    cluster addresses = 
    clustering = No
    config file = 
    ctdbd socket = 
    ctdb locktime warn threshold = 0
    ctdb timeout = 0
    default service = 
    delete share command = 
    homedir map = auto.home
    kernel change notify = No
    lock directory = /var/lock
    log writeable files on exit = No
    message command = 
    nbt client socket address = 0.0.0.0
    ncalrpc dir = /var/run/samba4/ncalrpc
    NIS homedir = No
    nmbd bind explicit broadcast = Yes
    panic action = /usr/local/libexec/samba/samba-backtrace
    perfcount module = 
    pid directory = /var/run/samba
    registry shares = No
    remote announce = 
    remote browse sync = 
    reset on zero vc = No
    smbd profiling level = off
    state directory = /var/db/samba4
    usershare allow guests = No
    usershare max shares = 0
    usershare owner only = Yes
    usershare path = /var/db/samba4/usershares
    usershare prefix allow list = 
    usershare prefix deny list = 
    usershare template share = 
    utmp = No
    utmp directory = 
    wtmp directory = 
    addport command = 
    addprinter command = 
    cups connection timeout = 30
    cups encrypt = No
    cups server = 
    deleteprinter command = 
    disable spoolss = Yes
    enumports command = 
    iprint server = 
    load printers = No
    lpq cache time = 30
    os2 driver map = 
    printcap cache time = 750
    printcap name = /dev/null
    show add printer wizard = Yes
    cldap port = 389
    client ipc max protocol = default
    client ipc min protocol = default
    client max protocol = default
    client min protocol = CORE
    client use spnego = Yes
    dcerpc endpoint servers = epmapper, wkssvc, rpcecho, samr, netlogon, lsarpc, spoolss, drsuapi, dssetup, unixinfo, browser, eventlog6, backupkey, dnsserver
    defer sharing violations = Yes
    dgram port = 138
    disable netbios = No
    enable asu support = No
    eventlog list = 
    large readwrite = Yes
    max mux = 50
    max ttl = 259200
    max wins ttl = 518400
    max xmit = 65535
    min receivefile size = 0
    min wins ttl = 21600
    name resolve order = lmhosts wins host bcast
    nbt port = 137
    nt pipe support = Yes
    nt status support = Yes
    read raw = Yes
    rpc big endian = No
    server max protocol = SMB3
    server min protocol = LANMAN1
    server multi channel support = No
    smb2 max credits = 8192
    smb2 max read = 8388608
    smb2 max trans = 8388608
    smb2 max write = 8388608
    smb ports = 445 139
    svcctl list = 
    time server = Yes
    unicode = Yes
    unix extensions = Yes
    use spnego = Yes
    web port = 901
    write raw = Yes
    algorithmic rid base = 1000
    allow dcerpc auth level connect = No
    allow trusted domains = Yes
    auth methods = 
    check password script = 
    client ipc signing = default
    client lanman auth = No
    client NTLMv2 auth = Yes
    client plaintext auth = No
    client schannel = Auto
    client signing = default
    client use spnego principal = No
    dedicated keytab file = 
    encrypt passwords = Yes
    guest account = nobody
    kerberos method = default
    kpasswd port = 464
    krb5 port = 88
    lanman auth = No
    log nt token command = 
    map to guest = Bad User
    map untrusted to domain = No
    ntlm auth = Yes
    ntp signd socket directory = /var/run/samba4/ntp_signd
    null passwords = No
    obey pam restrictions = Yes
    old password allowed period = 60
    pam password change = No
    passdb backend = tdbsam
    passdb expand explicit = No
    passwd chat = *new*password* %n\n *new*password* %n\n *changed*
    passwd chat debug = No
    passwd chat timeout = 2
    passwd program = 
    password server = *
    preload modules = 
    private dir = /var/db/samba4/private
    raw NTLMv2 auth = No
    rename user script = 
    restrict anonymous = 0
    root directory = 
    samba kcc command = /usr/local/sbin/samba_kcc
    security = USER
    server role = standalone server
    server schannel = Auto
    server signing = default
    smb passwd file = /var/db/samba4/private/smbpasswd
    tls cafile = tls/ca.pem
    tls certfile = tls/cert.pem
    tls crlfile = 
    tls dh params file = 
    tls enabled = Yes
    tls keyfile = tls/key.pem
    tls priority = NORMAL:-VERS-SSL3.0
    tls verify peer = as_strict_as_possible
    unix password sync = No
    username level = 0
    username map = 
    username map cache time = 0
    username map script = 
    aio max threads = 100
    deadtime = 15
    getwd cache = Yes
    hostname lookups = Yes
    keepalive = 300
    max disk size = 0
    max open files = 233687
    max smbd processes = 0
    name cache timeout = 660
    socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
    use mmap = Yes
    get quota command = 
    host msdfs = Yes
    set quota command = 
    create krb5 conf = Yes
    idmap backend = tdb
    idmap cache time = 604800
    idmap gid = 
    idmap negative cache time = 120
    idmap uid = 
    neutralize nt4 emulation = No
    reject md5 servers = No
    require strong key = Yes
    template homedir = /home/%D/%U
    template shell = /bin/false
    winbind cache time = 300
    winbindd privileged socket directory = /var/run/samba4/winbindd_privileged
    winbindd socket directory = /var/run/samba4/winbindd
    winbind enum groups = No
    winbind enum users = No
    winbind expand groups = 0
    winbind max clients = 200
    winbind max domain connections = 1
    winbind nested groups = Yes
    winbind normalize names = No
    winbind nss info = template
    winbind offline logon = No
    winbind reconnect delay = 30
    winbind refresh tickets = No
    winbind request timeout = 60
    winbind rpc only = No
    winbind sealed pipes = Yes
    winbind separator = \
    winbind trusted domains only = No
    winbind use default domain = No
    dns proxy = No
    wins hook = 
    wins proxy = No
    wins server = 
    wins support = No
    idmap config *: range = 90000001-100000000
    idmap config * : backend = tdb
    comment = 
    path = 
    administrative share = No
    browseable = Yes
    case sensitive = Auto
    default case = lower
    delete veto files = No
    hide dot files = Yes
    hide files = 
    hide special files = No
    hide unreadable = No
    hide unwriteable files = No
    mangled names = Yes
    mangling char = ~
    map archive = Yes
    map hidden = No
    map readonly = yes
    map system = No
    preserve case = Yes
    short preserve case = Yes
    store dos attributes = Yes
    veto files = 
    veto oplock files = 
    blocking locks = Yes
    csc policy = manual
    fake oplocks = No
    kernel oplocks = No
    kernel share modes = Yes
    level2 oplocks = Yes
    locking = Yes
    oplock contention limit = 2
    oplocks = Yes
    posix locking = Yes
    strict locking = No
    afs share = No
    available = Yes
    copy = 
    delete readonly = No
    dfree cache time = 0
    dfree command = 
    directory name cache size = 0
    dmapi support = No
    dont descend = 
    dos filemode = Yes
    dos filetime resolution = No
    dos filetimes = Yes
    fake directory create times = No
    follow symlinks = Yes
    fstype = NTFS
    include = 
    magic output = 
    magic script = 
    postexec = 
    preexec = 
    preexec close = No
    root postexec = 
    root preexec = 
    root preexec close = No
    spotlight = No
    volume = 
    wide links = No
    cups options = 
    default devmode = Yes
    force printername = No
    lppause command = 
    lpq command = lpq -P'%p'
    lpresume command = 
    lprm command = lprm -P'%p' %j
    max print jobs = 1000
    max reported print jobs = 0
    printable = No
    print command = lpr -r -P'%p' %s
    printer name = 
    printing = bsd
    printjob username = %U
    print notify backchannel = No
    queuepause command = 
    queueresume command = 
    use client driver = No
    acl allow execute always = Yes
    acl check permissions = Yes
    acl map full control = Yes
    durable handles = Yes
    ea support = Yes
    map acl inherit = No
    nt acl support = Yes
    profile acls = No
    access based share enum = No
    acl group control = No
    admin users = 
    create mask = 0666
    directory mask = 0777
    force create mode = 0000
    force directory mode = 0000
    force group = 
    force unknown acl user = No
    force user = 
    guest ok = No
    guest only = No
    hosts allow = 
    hosts deny = 
    inherit acls = No
    inherit owner = No
    inherit permissions = No
    invalid users = 
    only user = No
    read list = 
    read only = Yes
    smb encrypt = default
    username = 
    valid users = 
    write list = 
    aio read size = 0
    aio write behind = 
    aio write size = 0
    allocation roundup size = 1048576
    block size = 1024
    max connections = 0
    min print space = 0
    strict allocate = No
    strict rename = No
    strict sync = No
    sync always = No
    use sendfile = No
    write cache size = 0
    msdfs proxy = 
    msdfs root = No
    msdfs shuffle referrals = No
    ntvfs handler = unixuid, default
    vfs objects = 

[...]

I also tried enabling jumbo frames (set it to 9k) and increased the TCP window size to about 4 times more than what it was :D
But still no change.. :-/

Yes, freenas. Intel x540 + Dell r710
Upgraded to x5690s to saturate 10 gigabit

CPU should be plenty. Enable autotune in system options?

1 Like

Tried that, rebooted and it added quite a few tuneables but it didn't change anything about the situation.


+ some more for ZFS.

The system load seems a bit higher right now, but still nothing major:

Edit: I removed all my tuneables before enabling autotune and rebooting.

1 Like

Remove the samba options also from the samba config ?