Seeking advice on a NAS build

Like I said it´s been stable for quite a long time for me, proxmox 6 came out 4 years ago and did 2 major version updates. Take that for what it´s worth.

I´m just not doing many changes to the base os. I would not want to install docker on proxmox directly (it would work though). But youd have to install another repo. Maybe docker breaks on updates. I mean it broke on fedora before with cgroups v2 for like half a year or so. Maybe they add their own docker down the line. Maybe you want to pass a nvidia gpu to your container now you gotta install nvidia driver on proxmox. I don´t want to do that either. If something breaks I prefer that to be in a VM that is easy to restore with snapshots.

A lot of this will be more of a pain than useful and I second that a NAS distro isn’t suitable for a all-in-one box which is what you’re trying to do. Just go for a plain distro be it FreeBSD or some Linux-variant, I would personally go for FreeBSD but that’s up to you.

Well the idea was to use the plugins that are provided by TrueNAS (or whatever other software) but as it turns out not everything is available as a plugin :sweat_smile: I’m not sure where I’d even start with a complete DIY setup and honestly I don’t really feel comfortable with that. I’d rather have some software where the majority of the work is already done. If I have to do some legwork for a plugin or just have to do without one that’s fine with me. The primary goal is to have a storage box after all.

No.

It is using their own implementation of kuberneties/docker which is really unwieldy for most use cases. I have docker compose running natively on it using the guide in the forum.

Proxmox would be my choice if you are planning on running VM’s. If you can containerize everything then I would just do Debian with ZFS and docker + portainer.

As harsh as it may sound is it a better idea to have something (hopefully) running and no idea how to fix it when it breaks? That sounds like a good way to loose data. You’re aiming for a NAS oriented distro, you can in theory try to shoehorn X but that’s usually not a great idea at all. It’s like buying a car and trying to use it as a RV.

Any generic application frontend is more or less geared towards a very specific software setup so if you’re deadset on Heimdall (I would recommend against using such software) you should look up what they recommend / require and go with that. I’m quite sure that will rule out any NAS distro available.

Start small and go from there (just Samba and possibly NFS), most of this will be handled by a regular package manager which is by far easier to maintain if you’re not familiar with Unix/Linux in general than Docker, Containers and whatnot.

I’m just going to believe you here, but I will be honest I have no knowledge of Docker or Kubernetes other then they exist so I can’t really follow you here :sweat_smile:

I’m not sure I’m following you here. I’m not trying to shoehorn anything. I was planning on using the functionalities that are provided by the software in whatever way the software is intending them to be set up.

Reading wendell’s article it intentionally changes nothing on the base system and works within the boundaries of what Scale provides. It’s setting up a VM, which is officially supported, and using an NFS share for that VM, which as a NAS appliance is also supported.

I’m not, it was just the first thing I came across when I tried out Core as it was one of the native plugins it provided. It doesn’t seem to be available on Scale but there’s other similar things.
Also are we sure we’re talking about the same thing? I’m talking about the Heimdall Application Dashboard, it’s just a webpage for Bookmarks and Live Stats of linked services. I think you might be talking about the Android Flashing Utility which I can assure you even if I wanted to run, I would not do that on a NAS :stuck_out_tongue:

FWIW I run a lot of similar apps (nextcloud, jellyfin, heimdall, syncthing) and settled on a TrueNAS Core for simple storage and backups as well as a Debian 12 tiny/mini/micro for power efficient QuickSync transcoding and Docker containers where it makes more sense than a FreeBSD jail. Best of both worlds for slightly more money (gen 6 or 7 intel was cheap last I checked, I believe because they don’t officially support windows 11, which doesn’t matter in your use case).

If that drive fails you have two problems, instead of one. A truenas boot device can be replaced and restored from backups in minutes without this complication, add a $20 ssd if you like.

1 Like

Yeah that’s kind of what I was thinking too… if you just backup the TrueNAS config regularly then swapping out a failed boot drive isn’t a big deal to begin with. The wording just made it sound like it would also break replacing data drives which made no sense to me.

But yeah I might just throw in 20€ and buy a second drive so I can mirror them

You want to use a distro that targets using a device as a NAS but then you want to add on a bunch of services and functionality on top that’s not related which will be messy a best.

I’m assuming that Heimdall needs to know where things are located but oh well…

I mean, you’re right in that they are not directly related to being a file storage, but NASes running more then just file storage has been the case for the last 10 years or so. Even when you buy a full built Synology or whatever brand you get the option of adding services by using their plugin system. I’m doing no different here, I’m using the plugins that they provide in the UI, nothing more. I’m not planning on changing anything to the base system.

I don’t mean to be rude but it seems to me that you have either never used TrueNAS to know how the Plugins work or you’re not understanding what I’m wanting to do. As I mentioned before altering the base system was never and won’t be me goal for this. I’m planning to use what is provided by the system.

Yes, but it’s configured right on the webpage. You select the type of service to link, you enter the address and possibly an API key and that’s it. The communication between Heimdall and the services for live stats and whatnot is handled through their respective APIs.

TLDR; by your description it sounds like this box will be your storage nexus where a large majority (if not all) of your data will live. If that is the case, I would definitely say TrueNAS is the right call, specifically TrueNAS Scale.

I was in a similar boat a few years back and settled on Scale as well and it’s been amazing.

On Distros:
Between TrueNAS and Proxmox, my rule of thumb is that TrueNAS is a Storage-First distro, while Proxmox is a Virtualization-First distro. They have heavy overlap, so it’s very common, and super acceptable, for people who have more experience with one to use it for the other usecase.

Scale vs. Core:
Tom Lawrence (Lawrence Systems on YouTube, and a champion of TrueNAS in the SMB space) said it best in a recent livestream: “every time a client hits a problem or limitation with TrueNAS Core, usually the answer is just to switch to Scale”. Because Scale is built on Linux it has a much wider gamut of bolt-ons you can MacGuyver on to the system, better driver support for modern hardware, and easier to troubleshoot online.

On Docker
Since Scale is basically Debian, Docker and Portainer are super simple to get working. Wendell posted a great initial guide here on the forum, and it’s constantly being updated by the community (TrueNAS Scale Native Docker & VM access to host [Guide]). There is currently a minor conflict with TrueNAS Apps, but only if you enable them - however because no one ever uses Apps and it conflicts with normal Docker TrueNAS, iXSystems is actually removing Apps in the next major revision.

On VMs
VM support has come along way, so much so that my gaming rig is a VM and is powered by a 3080 Ti via GPU passthrough. My wife just beat Stray by streaming to the Nvdia Shield in our living room via Sunshine/Moonlight and loved it - she’s working through Dave the Diver now, and I’m playing Starfield (as I’m writing this). She’s a lawyer and super non-technical and has had no issues at all (like I would find her at 4AM still playing).

On Disks
When I was young I was really all about using as much storage space as possible from different drives, but after losing data a couple of time, as well as spending days troubleshooting I’ve realized my time is worth more that the few dollars saved by using an exotic disk setup. ZFS is pretty robust, and has a healthy distrust of any storage medium, so multiple single vdevs/pools mounted to a single folder has worked best. If you’re in the US or Canada, I recommend checking out https://serverpartdeals.com/ - they sell refurbished drives at a great price and are super great at replacing drives that fail a burn-in test. I’m currently a fan of 12TB Helium Hitachi enterprise drives for spinning rust and Intel P4510 NVMe SSDs for primary storage.

On OS Backup
Backups on TrueNAS are just the best. The entire config can be exported to a single file - I have this backup to Backblaze & S3 every day. To recover from a failed OS drive, just reinstall TrueNAS and import the backup config when prompted. Had to do it 3 days after my first install, and was sold instantly. I have a bash script that reinstalls everything off-label (like Portainer) - everytime I add a new off-label component, I just copy the commands I used into the script as I’m entering them.
TrueNAS also has a builtin Cronjob manager and Cloudsync tool in the GUI so it’s pretty simple to manage.

On Data Backup
Having used enterprise software, TrueNAS’s ZFS native is just plain amazing. If you ever have the ability to put another box somewhere else, TrueNAS can perform differential backups to a remote TrueNAS install with a single click - even the disks don’t have to be the same. You can also use the cloud sync tool to backup just subsets of data to a remote site - All my work and personal documents get backed up to multiple cloud services every hour, and it was basically “Select folder, select destination service, select frequency, and DONE”.

On Licenses and Subs
Just not a thing, this is why Tom, Wendell and so many others favour it in the business space as well. (They also love XCP-ng on the virtualization side as well, for good reason, but I digress)

For context some of the components in my home network are:

  • TrueNAS Scale at the center of my env serving SMB shares to the network - running a AMD R3600 on a AsRock Rack X470 w/3080 Ti on passthrough and MCX4121a with Virtual Functions (VFs) and VXLAN for VM & Docker network isolation
  • Performance & Data Sensitive VMs/Containers run on TrueNAS on dedicated ZFS datasets that can then be individually backed up via the WebUI (like the gaming VM, by Seafile (document sharing) docker, and Gitlab docker)
  • Proxmox HA Cluster running on 3x Lenovo M720q Tiny for fun and multi-node development work (they use a MCX354a 40GbE ring with VFs between each other for VM connectivity)
  • TrueNAS/Portainer running on a M720q Tiny for core house services like Home Assistant, AP Controller, Dashy, and Prometheus/Grafana - all are backed up to the core TrueNAS server.

Hope it all works out on the NAS!

1 Like

Hey, just to clarify, it is not improving on Wendell’s guide, it’s a similar solution that has more performance than going the VM Route. If you have any questions, feel free to ask in that topic as well.

Right now we are preparing for Cobia which will remove docker from TrueNAS Scale, so far our tests have concluded that users that use my script should not even feel the effect of that after upgrading.

1 Like

Thanks for your insight. So if I’m reading this right you’re voting for always going Scale and just skipping Core entirely.

I’m not sure I’m following here. The linked guide sets up Docker and Docker Compose natively, however it refers to Wendell’s guide which specifically avoided doing those things to not alter the base system.
I’m not sure I’m really comfortable to doing that right away to be honest. Maybe down the line, but probably not now.

Why would noone use the Apps? From the bit of testing that I did setting them up was super easy and they just worked right away, seems like a no-brainer to just use them? The only downside I see for them (at this time anyway) is that not “everything” is available as an App, but I feel that’s kind of normal.

I also couldn’t find any articles online about Apps being removed, can I read up on that somewhere? I’m wondering what their plan on a replacement is since I’m not sure they’d just leave people that used it in the dust :thinking:

Through googling “truenas scale cobia” I just found the beta announcement:

And there’s no talk on removal, on the contrary it’s (supposedly) improving on the UI :thinking:

I definitely get that, which is why I don’t wanna fiddle with it too much and wanted to avoid altering the base system for less risk of breakage. A 2-way mirror also isn’t something I’d call exotic :stuck_out_tongue:
I don’t have much of a choice on the disk setup right now anyway unless I buy a bunch more drives :slight_smile:

The only thing I still need to figure out is how to use my old drives. But I guess I’ll just come back to that later once everything is set up, there’s still some stuff on there anyway that needs to go over to the new drives and then I’ll know what kinda drives I even have lol.

I’ll be honest I haven’t put much thought into backing up what’s going on it simply because it’s pretty much just my media library and if I were to loose that it’s not that much of a deal since most of the stuff I have physical anyways.

I would take that Youtuber with a bit of salt and your claim about driver support is based on what? Neither Core or Scale uses a “new” software base (stability > new features) and to my knowledge TrueNAS Enterprise is based on FreeBSD.

I agree however that if your goal is to use the box as some kind pseudo multipurpose server but still claim its a “NAS” SCALE is probably your best choice although there has been a lot of complaints about it being buggy and understandbly so because there only so much one company can do trying to shoehorn everything. There’s a reason why you see “applicances” professionally as far as products goes.

Because they are based on a weak implementation of Kubernetes, which in case you only run a single machine makes no sense to run. However, the point of Scale is just that, scaling up, which is why they went that route.

Not apps, docker is being removed, as in the package, they are doubling down on the Kubernetes mentioned.

If you are still debating scale vs. core, just look at the SAN video from LTT. They realize there that every time they start with core, and find something missing or not working, then end up switching to scale.

You’re not altering the system if you follow my guide, you’re enabling features that have been locked away by ixSystems. it will not bork your system if you enable a package that’s already installed.
If you go beyond and start downloading other packages that might conflict with base system, then yeah, there’s a risk.

But, from my personal experience, I went even further than my own guide. I enabled the Debian repos and updates to those packages, and nothing broke. So if you have any experience with Linux, you’ll know it’s quite easy to manage when you know what you’re doing.

As for your concern about needing to always maintain and tinker, I can give you 2 examples where I used both FreeNAS (the old BSD version of TrueNAS) and TrueNAS Scale in my workplace. Both were set and forget. They both cloud backup and have local snapshots. I only connect to the web interface to update the system when needed.

1 Like

For someone who just wants to run the thing and doesn’t really look into the technical aspects too much, does it matter though? I feel like if it works, it works, no?

Ah OK, that makes more sense. Although I don’t really understand how that would work since I thought Kubernetes was a Docker Management thing? Guess I’ll have to take a closer look at some point :sweat_smile:

Not really, I was leaning towards Scale anyway and the comments above pretty much set me on it :slight_smile:

1 Like

It may work, but you get an 1/8th of the performance, and in some apps that’s the difference between hours and days per task.
It’s a very weak implementation as is, and is not targeted at the single computer single user scenario.

Not exactly, kub is a scaleable docker implementation. It basically allows you to run multiple containers from the same config, so if you have a website, and you have a lot of demand you just spin up another pod easily.

Yep, it’s a far more stable implementation, I personally prefer Linux to FreeBSD in my appliances. And I run a pfSense as my router, so you know I have both.

1 Like

Got it, makes sense then.


I’ve been doing some more testing before I put my stuff in there and I’m running into an issue that I ran into before

Basically I’m going through the TrueNAS guide:

And while this version actually acknowledges…

By default, anyone that connects to the NFS share only has read permission.

It doesn’t really help me much in how to set the permissions properly.

To change the default permissions, edit the share, open the Advanced Options , and change the Access settings.

OK, well, so far so good. There is a paragraph about adjusting permissions further up but honestly it doesn’t really help me.

Select Read-Only to prohibit writing to the share.

This is kinda funny when it’s R/O by default, but I get it when you edit it later. Anyway, they mention the maproot/mapall settings but not really how to set it up properly. I’m not sure what I’m supposed to enter where…
For testing I put root into the Mapall User but I feel like that can’t be the right way to do it… right?

And I can’t rsync to the dataset either because root and admin are disabled over SSH and when I use a newly created user it doesn’t have write permissions…
Am I seeing it right that I need to setup an ACL to be able to write via rsync?

I feel like the documentation is missing a lot of how to actually get this to be usable :thonk:

edit:
So this is confusing. I created a new group in case I want multiple users to have the same access to a share. I set the group’s permissions to rwx, added my user to said group. This works as expected for rsync, so then I tried adding that group in the Mapall Group setting of the NFS share. But I’m getting “This field is required when map group is specified” on the Mapall User field… First of all it doesn’t say that in the Documentation or the Help text in the UI, and secondly doesn’t specifying a user defeat the whole point of having a group?

I think maybe you would benefit from reading about NFS more broadly - it does not offer any client authentication the way that SMB does. Instead, NFS assumes that the users and groups match on both the server and the client, which is more typical in a corporate setting. This is not a Truenas specific issue.

If you want to use username and password authentication, you probably want an SMB share.

Best of luck!

3 Likes

I know that much, what I don’t understand though is why an entry in Mapall User is required when providing a Mapall group already. It should be possible to apply the group’s permission (seeing as it works on the actual filesystem), no? I’m not sure where my thinking is going wrong.

Mounting the share works fine, I just can’t write to it because the permissions don’t fit.

But yeah I’m definitely not getting something right with how NFS works.

On my PC my username and group are tarulia, I created a user with the same name (and primary group) on the NAS. I went into the ACLs and added both the group as well as the user with rwx permissions (even though one of them should be enough?). I also removed the Mapall User (that was set to my user previously) because that’s pretty much what I wanted to test.

I mounted the share sudo mount -t nfs truenas.local:/mnt/tank/media ./mount_test and I’m still getting:

z% touch ./mount_test/test
touch: cannot touch './mount_test/test': Permission denied

So I’m assuming permissions on the local filesystem don’t carry over into the NFS share, but then setting up the permissions is kind of pointless unless I do everything through ssh.

Or do the User and Group ID need to match and not the name?