I’ve run a Proxmox server for about a year and have found it to be a generally great experience, however, I’m planning on replacing Proxmox with either RHEL9 with a developer subscription or CentOS 9 Stream.
Specific concerns I’m looking for answers to are in regard to the host OS, data store filesystem and backup strategy:
Host OS:
Do you have any thoughts for or against running RHEL9 with a developer license?
Filesystem for data:
How is ZFS support on RHEL-based systems?
Would it be safe to install a new OS and simply mount the current ZFS pool?
If not ZFS on RHEL, would I be better served using XFS or Btrfs?
Backups:
Do you have any suggestions for cloud backups?
Services:
NextCloud AIO container (reads my data drive and acts as a front-end to those files)
Pi-hole container
Hardware Specs:
4 core CPU
16GB RAM
1x 1TB NVMe for OS
3x 6TB HDD for storage (currently ZFS filesystem)
My desired result:
A RHEL9 server, running NextCloud and Pi-hole via podman in a container, setup as a systemd service.
NextCloud has access to data on the storage drives. The server sends daily backups of my data to a secure(no google-esque snooping!), private cloud for less than $40 a month. Right now I have 2.5 TB of data, so storage costs shouldn’t be close to that amount.
Any advice is welcome and if there’s any more info needed please let me know.
But it’s Linux, so even if not in the repos (didn’t check on my Rocky9 machine), you can get .rpm from Github or elsewhere. And for basically everything refer to the official OpenZFS documentation (which is just great)
Boot drive with BTRFS + ZFS for proper storage would be my recommendation. So you get snapshots on root partition and fancy grub entries and don’t have to deal with ZFS-on-root but still get all of ZFS for data storage.
Backblaze B2 has S3 storage for very reasonable prices per GB and occasional restore doesn’t trigger ginormous egress fees either.
Any particular reason for not running AlmaLinux/Rocky or Oracle? They are even 100% compatible with bugs RHEL has.
Donate the RHEL licence money to OpenZFS project and make the world a better place
edit: Just checked on Rocky9.2. There is a good guide for ZFS here. Looks pretty easy to get running properly. EPEL repo is always your best friend for the good stuff.
That’s an excellent idea and one I hadn’t considered, I will definitely be implementing this.
Backblaze B2 seems very reasonable, I’ll be looking into it along with BorgBase that’s relatively similar in price.
Isn’t there a 5G file size limit in S3? I have some rather large files that exceed that, though I suspect those files would then just be segmented?
I don’t have a lot of faith in Rocky or Oracle Linux and so I avoid them, I don’t have a good technical argument against them so it just boils down to “I don’t like them, I don’t want to use their stuff.”
I know Red Hat has fallen out of favor in the eyes of some in the Linux community but I think their CentOS and source code decisions made business sense so I don’t hold it against them.
AlmaLinux is definitely be a candidate worth considering, from what I’ve seen they seem like they genuinely want to provide what CentOS once offered.
My reasons for considering RHEL are mainly that for my work I use RHEL, CentOS(for now) and CentOS Stream. Also, I believe that with the Red Hat developer subscription, you also get a access to Ansible Tower which may be interesting to tinker with.
So there is a possibility. But S3 put for some TB object…wasn’t really built for that…I’d say try and error and see what connection looks like. I think there is a free trial or so…not sure if that covers CLI and S3 access.
Use what you are comfortable with. Linux is Linux all things considered. If I had RHEL at work, I’d get RHEL too…and I gladly pay for good software I like to work with. I have a Proxmox SLA myself and it’s totally worth my money.
I wouldn’t voluntarily touch anything with “Oracle” on it. But Rocky has convinced me so far, although I’m only running Ceph and Cockpit stuff on it so far. They’re relatively new, basically ever since the CentOS incident. You should spin up a VM and try…RHEL with some Rocky guest VMs…be my guest, so to say
That’s an official recommendation from SuSE I picked up and implemented for my machines. Best of both worlds as far as I’m concerned.
I haven’t heard of this before, truth be told I’m very novice when it comes to zfs.
Any other best practices in regards to zfs? I’ll be reading the documentation thoroughly as @Exard3k pointed out that it’s excellent.
He also has a guide on this forum, sort of an about zfs and best practices for 2023 vs 2009.
Most of the stuff you find online, reference the system administrators admin guide from 2009. The guts are mostly the same but 4gb was a lot of ram, and they didn’t have good ssds.
I made a guide a couple of months ago…and there is a lot of stuff on the internet. At the end of the second posting (scroll down) I got 5-6 links for stuff I used myself when learning about ZFS.
The guide is covering the basics and general stuff. You can’t possibly cover everything without writing a book.
