My son and I each have a TruNAS system (I’m on core, he is on Scale).We want to back up the others files for resilience.
Trying the “easy” option of tailscale didn’t turn out so well. The Jail seemed much easier than the container and fiddly differences made it awkward. The final result was also extremely slow - using scp gave less than 10Mbit/sec. In theory we have 40/50Mbit upload and we both have gigabit download. I think we have several levels of indirection between jails, VMs and so on, and possibly a “slow” approach in tailscale.
So my question is what are good combinations on network approach and transfer methods, or anything to avoid?? Is tailscale fast and we have a bad config? Port forwarding of the NAS? OpenVPN site to site? And for transfer, ZFS Send/recv, scp, SMB?
Also if anyone has any practical stats that would help us work out a realistic match that would be great as well - I think
HDD transfer is not an option (should mention between US and the Netherlands), and the amount to transfer is large enough to need the max out of the network!
Wireguard and rsync/zfs send works fine however depending on version of zfs the send/receieve commands and get a bit cranky if connection is unstable. That being said, I’ve never used Wireguard directly on TrueNAS it’s also been running on gateway/firewall.
I would either use a combination of Wireguard and rsync as already mention but do that only if you have a little bit experience with rsync otherwise its easy to wipe the whole data - might be better to work with a test folder first
Otherwise you can also use something like borg or duplicati and sync via wireguard or another VPN solution or to a location in the cloud (google drive, digitalocean/aws/linode S3 and so) and download from there
I have a similar setup with a friend where we exchange huge files via wireguard and rsync (movies usually about 30-50gb per file) with a configured fixed bandwith of 10MBit/s and it works like a charm. You can limit the bandwith with rsync ( -bwlimit=xx)
I mostly exchange large amounts of data these days via sftp.
Wrapping the communication in an additional carrier+cryptographic layer is not necessary for ordinary data transfer, which itself should be independently encrypted and sent via an encrypted protocol (ftps, sftp, https).
If one of you has the ability to open one port you can just use ftps/sftp/syncthing. Concerns about exposing the service to the world of course may arise, but you can always restrict firewall access per IP.
I would generally avoid complexity and extra layers unnecessary complications like tail, vpn…
If you have some budget, I would consider renting a dedicated server as a seed/proxy and use a central data exchange point. If not, start with plain ftps(tls)/syncthing/sftp. Tailscale here is like a tomahawk against a mosquito.
In general, you can argue about the solution depending on the type of data to be sent, their amount, and how often it will be repeated.
If it’s just a one-time exchange of a large amount of data and then only an update of changes, I don’t see any point in complicating it too much.
P.S
Your download will be maximally as fast as your son’s upload and vice versa.
Rolling zfs this way is largely a waste of time… If you have such a brutal need, I’d rather perform a data projection locally and send it traditionally. But I’m apparently not such a nerd as to create a rocket for a home NAS to travel to Mars.
I just shared a different take on the matter, but have fun with zfs send/recv as needed.
How would you go about (in an effective manner) utilizing SFTP, HTTPS etc to do incremental backups that also includes some kind of verification preferably also using some kind of delta-transfer?
I would ask another question first, does the situation really require such an approach?
Art for art’s sake? Is there really a real need to use it?
OP asks, and you answer, and I just throw the wet paper against the wall and see what sticks.
You can, of course, limit each sentence to the narrow point of the question. But sometimes in a discussion one could develop a perspective and ask the question whether there is really a need to do abc and not simply to do xyz. It’s not that xyz is a direct alternative to abc because it’s not, it’s just about moving a broader perspective and asking yourself if I really need it.
The OP mentions that the option to send the HDD is not an option… what is the conclusion?
The OP mentions scp, smb as a potential use… what’s the conclusion?
Where exactly is the center of the problem? It seems to me that the OP mainly complains about speed?
So what theoretically could be the path of discussion… The exact solution to the speed problem? Or suggesting a complete change and rethinking what you use.
What are the OP’s needs? Why exactly does he need such a solution?
Of course, suggesting something different does not solve his problem because it is outside the realm of resources he uses at the moment.
Is the magic solution to the problem suggested to use WireGuard? Maybe and maybe not, although it obviously fits the spectrum of the problem.
My point of view is not to say that abc will replace xyz here, but to put perspective, such a way of forcing you to think creatively. Although I realize that it’s probably not liked among nerd environments where everything has to be 0 or 1.
One cheap if no easy way to test if Tailscale is the cause of the low upload speed would be to set up a free tier VM on OCI with wireguard:
Choose an OCI region that makes sense (us-east or Amsterdam), go for an Ampere VM and test upload speeds from the two locations to the cloud endpoints
If you feel like going the experimental method you could even deploy one vm in each region and test for performance differences …
If you feel like it is too much effort then port forwarding ssh ports and sending/receiving ZFS snapshots will be quicker, but way less secure …
As always, context is king.
In the context of a skilled user that is aware of the security implications of forwarding SSH traffic onto a presumably LAN connected appliance and can maintain proper procedure in setting up said port forward then I agree, it can even be more ‘secure’ than setting up wireguard and then sharing the secrets through an email or posting them to a gist on github …
I we think port forwarding from a TP-link router onto a truenas appliance that has ssh enabled for root with password password1 or welcome1 or 000000 then a standard wireguard setup where secrets are not posted online is going to be inherently more secure
I don’t know how much of all of these ‘boring and obscure’ security implications the OP is willing to dig into, so I just assumed and went with the safe statement
…and how is that different from a cloud VM which you need to access in some way and potentially even causes more security implications?
No TrueNAS comes with root acess enabled by default or is recommened to allow so if your scenario is by misconfiguration software intentionally then I guess we can classify everything that faces the Internet as unsecure right off the bat.
Thanks all for the comments - and @TimHolus thanks for thinking about the issue as well -
is so true!
The HDD method was ruled out for costs reasons (would have to buy some) and similar for cloudy options (but also this could take longer using an intermediary).
The choice of tailscale was to provide a secure connection protecting other machines on each LAN. While I can do some nifty port forwarding (I use Opnsense) and even use dedicated NICs on the router and NAS, my son is limited at the moment to the features of the ISP provided modem. As I am stubborn in using core, I have fewer easy options and tailscale seemed a sane choice (I can configure networks securely, but that doesn’t mean I want the effort all the time!
As we are both on ISPs with DOCCIS, the asymmetry is a pain - but I was expecting more throughput. There are lots of factors that could affect this and so I thought I would reach out to the group and see if the choices were bad or any thoughts as the the realism of the performance
I like the idea of SFTP - I’d kinda missed that because it is under the SSH parts of TrueNAS config (sometimes a UI makes life to easy) and saw only standard FTP It is available by default in a jail and I can attach it to a specific NIC. I can’t limit it to a specific IP address easily but other than that it can be pretty well locked down.
In the longer term, the aim would be to have synchronised filestores, but that needs other changes and won’t happen for a while.
information and network security is not
but I appreciate some of the easier options (like tailscale) for home use as while I do document and try to maintain version control (I get mocked for my OCD on cable labelling for example) my home IT is not perfect and simpler options and pre-built solutions (like tailscale) can help with security and time pressures. I’m not averse to a cloud solution (all data will be enecrypted anyway), but it is yet another system to manage.
And to perhaps prompt more comments - I really like ZFS and I’m fine with ZFS send/recv. I use it on all machines except the unused windows disk (and I did try that too!)
- might be better to work with a test folder first
as to create a rocket for a home NAS to travel to Mars. 