I just noticed OpenSSL and GNUTLS performance on FreeBSD is very disappointing on FreeBSD 13 running on a RockPro64.
Yesterday I’ve finished setting a Samba file server. Setting server signing to mandatory yields an atrocious 5MB/s transfer speed over gigabit network. Yikes. Although we are talking about Samba this will definitely affect VPN software like OpenVPN.
SCP which is normally slower than Samba can transfer to the same machine with 20MB/s. Still not good but its SSH. I’ve managed 70MB/s with unencrypted rsync protocol. The armv8crypto seems to be loaded into kernel not as a module but in kernel.
So my guess is that the software cryptographic libraries are not using ARMv8 crypto extensions?
Samba 4.13 uses AES-128-CCM for signing and transport. It also uses GNUTLS instead of OpenSSL.
# smbstatus
Samba version 4.13.17
PID Username Group Machine Protocol Version Encryption Signing
----------------------------------------------------------------------------------------------------------------------------------------
85606 ulzeraj unixadmins REDACTED (ipv4:REDACTED:61683) SMB3_11 AES-128-CCM AES-128-CMAC
Service pid Machine Connected at Encryption Signing
---------------------------------------------------------------------------------------------
IPC$ 85606 REDACTED Fri Mar 11 08:57:38 2022 UTC AES-128-CCM AES-128-CMAC
tank 85606 REDACTED Fri Mar 11 08:57:05 2022 UTC AES-128-CCM AES-128-CMAC
This script should work in every OS with OpenSSL and sh:
#!/bin/sh
for ALG in aes-128-ccm aes-128-gcm; do
openssl speed -evp ${ALG} -bytes 1500 2> /dev/null | grep "^${ALG}"
done
Here are the RockPro64 results (also adding GCM for comparison):
aes-128-gcm 15808.85k
aes-128-ccm 13117.70k
I have a “replica” machine running literally the same software and Operating system versions but instead on amd64. Its an old APU2C2 with a AMD GX-412TC SOC and 2GB of RAM. This machine runs on a 2012 4 core embedded processor that scores worse than a Raspberry Pi 4 in raw power.
aes-128-gcm 319367.06k
aes-128-ccm 190703.62k
GASP is 25-ish times faster!!! OpenSSL runs on FreeBSD like crap! I should mention however that both systems are running encrypted AES256 disks with ZSTD compression and the RockPro64 outperforms the APU in every aspect.
It looks like there is something missing on both OpenSSL and GNUTLS for FreeBSD. I’m looking at the versions:
RockPro:
OpenSSL 1.1.1k-freebsd 24 Aug 2021
gnutls-3.6.16
nettle-3.7.3
APU2C2:
OpenSSL 1.1.1k-freebsd 24 Aug 2021
gnutls-3.6.16
nettle-3.7.3
Here are the results on various machines for comparison:
Pop_OS 21.10 Ryzen 9 5900HX:
aes-128-gcm 5137056.00k
aes-128-ccm 1612059.00k
Ubuntu 20.04 i7-9750H NUC
aes-128-gcm 4880986.00k
aes-128-ccm 1435525.00k
macOS Monterey M1:
This is OpenSSL on ARM. GCM is faster on M1 while CCM is slower than the AMD64 machines. Had to slightly modify the script because Mac OpenSSL sends stdout to stderr.
AES-128-GCM 5901342.64k
AES-128-CCM 1044495.00k
I will try to compile the OpenSSL from ports to check if it makes a difference. For samba however its a hopeless case for now since the latest version on pkg and ports doesn’t seem to support those features.
EDIT: OpenSSL from pkg has KTLS support:
aes-128-ccm 108811.86k
aes-128-gcm 246024.50k
Still kinda left behind on CCM but better on GCM. Considering the M1 results is ARM bad at AES-128-CCM?
EDIT2: Just discovered gnutls-cli --benchmark-tls-ciphers
. There results are consistent with my transfer speeds:
APU2C2:
# gnutls-cli --benchmark-tls-ciphers
Testing throughput in cipher/MAC combinations (payload: 1400 bytes)
AES-128-GCM - TLS1.2 74.29 MB/sec
AES-128-GCM - TLS1.3 67.16 MB/sec
AES-128-CCM - TLS1.2 29.43 MB/sec
AES-128-CCM - TLS1.3 28.55 MB/sec
CHACHA20-POLY1305 - TLS1.2 23.21 MB/sec
CHACHA20-POLY1305 - TLS1.3 21.89 MB/sec
AES-128-CBC - TLS1.0 20.25 MB/sec
CAMELLIA-128-CBC - TLS1.0 8.69 MB/sec
GOST28147-TC26Z-CNT - TLS1.2 3.79 MB/sec
Testing throughput in cipher/MAC combinations (payload: 16384 bytes)
AES-128-GCM - TLS1.2 131.10 MB/sec
AES-128-GCM - TLS1.3 127.85 MB/sec
AES-128-CCM - TLS1.2 36.79 MB/sec
AES-128-CCM - TLS1.3 35.60 MB/sec
CHACHA20-POLY1305 - TLS1.2 27.30 MB/sec
CHACHA20-POLY1305 - TLS1.3 26.94 MB/sec
AES-128-CBC - TLS1.0 31.21 MB/sec
CAMELLIA-128-CBC - TLS1.0 11.26 MB/sec
GOST28147-TC26Z-CNT - TLS1.2 4.01 MB/sec
RockPro64:
# gnutls-cli --benchmark-tls-ciphers
Testing throughput in cipher/MAC combinations (payload: 1400 bytes)
AES-128-GCM - TLS1.2 6.67 MB/sec
AES-128-GCM - TLS1.3 7.91 MB/sec
AES-128-CCM - TLS1.2 6.12 MB/sec
AES-128-CCM - TLS1.3 5.77 MB/sec
CHACHA20-POLY1305 - TLS1.2 14.24 MB/sec
CHACHA20-POLY1305 - TLS1.3 14.29 MB/sec
AES-128-CBC - TLS1.0 7.76 MB/sec
CAMELLIA-128-CBC - TLS1.0 6.59 MB/sec
GOST28147-TC26Z-CNT - TLS1.2 3.18 MB/sec
Testing throughput in cipher/MAC combinations (payload: 16384 bytes)
AES-128-GCM - TLS1.2 7.08 MB/sec
AES-128-GCM - TLS1.3 8.36 MB/sec
AES-128-CCM - TLS1.2 5.64 MB/sec
AES-128-CCM - TLS1.3 5.98 MB/sec
CHACHA20-POLY1305 - TLS1.2 15.79 MB/sec
CHACHA20-POLY1305 - TLS1.3 15.59 MB/sec
AES-128-CBC - TLS1.0 8.30 MB/sec
CAMELLIA-128-CBC - TLS1.0 6.95 MB/sec
GOST28147-TC26Z-CNT - TLS1.2 3.28 MB/sec
Meanwhile, can someone check my benchmark script on a RockPro running Linux or OpenBSD? If Also gnutls-cli --benchmark-tls-ciphers
if you already have it installed. Thanks.