How do I convert from Ext4 to ZFS (or 'better') filesystem?

First, I’m new: so there’s a very good chance I’m posting in the wrong place/category; I will try to respond to suggestions as to how it might be better placed (albeit on UK time, currently BST).

I’m the proud owner, and builder, of a somewhat bargain-basement Ubuntu desktop, running 21.10; there’s nothing especially remarkable about if (if components matter I’ll respond to requests, but basically there’s a Ryzen 5 2600, an nVidia 970, 1TB NVME, a 500GB ssd, and a four TB HDD).

I’ve come to the conclusion that ZFS seems to be a better filesystem than Ext4 (as it has snapshots, storage pools, and data-integrity), and while I know little enough, so far, about the filesystems available, I imagine there may be even ‘better’ options that exist, that I’m entirely unaware of.

So, my desire is to change from my current Ext 4 filesystem, to ZFS (or, if you make a convincing argument as to the benefits, another option). The question, then, is: how?

I will assume that it’s better to back-up any precious data first, and I’ll be buying a new (external) hard drive for that purpose. But once I’ve backed up, what next? What first?

1 Like

You buy the new HDD, install ZFS package (or use OS that has ZFS capabilities like TrueNAS), create a pool consisting of that new drive and copy the data over to your new ZFS pool.

If you want to change your root filesystem, you better re-install Ubuntu and pick the ZFS “advanced option” during install. I checked the beta branch of 22.04 and Ubuntu now offers disk encryption as well, but otherwise only offers single disk stripe vdev. Backup all valuable things to the external drive before doing so.

4 Likes

So a fresh install is the only / best way?

1 Like

You can probably do it without it, but that’s hours of tweaking and tinkering of rather advanced stuff. ZFS-on-root is no joke on Linux unless you have the Ubuntu Installer to do all the thingsfor you, more less so converting a running system.

My advice: backup everything, fresh install via Ubuntu 22.04 installer. Converting filesystem with a click just isn’t a thing.

5 Likes

Yeah, in all honesty that sounds like many hours of pain I’d much rather not struggle through (only to end up doing a fresh install anyway, if it’s not sufficiently easy or simple).

So, the no joke part is purely the issue of installing on root, or should I anticipate pain with ZFS as a matter of course?

That seems reasonable, really; I’m not entirely sure why I thought there might be an in-place upgrade/change available; presumably just my naïve optimism.

Also, thank you for taking the time to answer and help; my apologies for the ongoing discussion. :slight_smile:

2 Likes

Do you have a NAS for putting stuff in?

1 Like

Sadly not, as yet; that is, however, a future build (and probably the place that the 2600 from this system will be going, once I have the other bits and pieces).

2 Likes

ZFS as a whole, has a couple components, like the actual apps / tools one uses to manipulate and actually use the filesystem, as well as the data format on the disk.

Rather than EXT4, where the tools are already in the kernel / distro, the ZFS tools are kind of separate.

It’s not a bad thing, but something that makes me use a separate, disposable boot drive on extx, and all my data on ZFS arrays (where practicable)

One can do ZFS on root, but any mis-match between kernel version and the ZFS tools needed to interact with the version, might make the system unbootable temporarily. it is rare, but can happen.

I rather just avoid it, but one can for sure recover from it.

3 Likes

ZFS wont really give you its benefits if you are not able to put the backups/snapshots elsewhere, especially if the device you intend to run it on does not have raid1 or any kind of parity.

My suggestion is to keep the EXT4 for now and try to build/learn the NAS so that you have somewhere to put the backups and ZFS snapshots in.

Alternatively there is the cloud, so you can do these kinds of backup online but Backblaze charges dollars for download, as well as the monthly premium for using their services.

3 Likes

ZFS administration is very easy and intuitive. zpool and zfs are the only commands you need. Parameters and options can be learned quickly. Getting it on your root partition on a OS that doesn’t support ZFS natively for legal and “political” reasons, is very hard. Ubuntu Installer is really a blessing in that department for people who just want ZFS-on-root on their PC.

This is a forum. We talk and chat about stuff we’re interested in.

I beg to differ. I use snapshots to rollback my system after messing sth up or if some new update results in bugs. Allows me to tinker with my system at will, because I can rollback time at any moment. I’m new to Linux and I tend to touch stuff I regret doing afterwards :wink:
But data protection…yeah, you need some redundancy to capitalize on that. But I got ZFS on my laptop just for the snapshots and some inherent ZFS features, not because of bit rot or disk failure protection or checksums.

5 Likes

That does not sound like a fun situation, especially since it’s my only machine (though I do have a tablet, and a phone if necessary).

As I read your response, I can’t quite tell if you’re cautioning me against the idea of switching to ZFS or just raising a potential concern that I hadn’t even contemplated? To be absolutely clear, I’m not opposed to votes against the idea, but - as noted - I’m not quite sure how to place your response so far.

2 Likes

I also think ZFS on single drives is handy- it will tell you when a file is corrupt, (it just can’t fix it) and one can replicate elsewhere fairly easily, maintaining snapshots, which themselves allow for incremental backups.

But fundamentally, the method Exard suggested early; get a new drive, format it as a (single) ZFS drive, transfer date (test transferred data) then OP can attach (NOT ADD) the old drive, to the new drive, and have a raid1.

3 Likes

There is that, but it still does not use the 3-2-1 rule of backup, which most people on TrueNAS preach.

@drthomas you can still learn stuff from ZFS on Ubuntu but the whole benefit isn’t there. The only caveat is that most file recovery utilities will prefer EXT4.

(edit: on isnt)

3 Likes

Honestly, I feel I should have considered that myself before raising the question.

I definitely see the wisdom in that, and I am certainly not intent on rushing into anything (it’s been a passing thought for quite a while now, but this is the first time I’ve sought advice from people that quite probably know what they’re doing). :slight_smile:

2 Likes

I would suggest, keep the boot/OS drive separate to the data drives. and go ahead and use ZFS on the data drives.

It is rare to get unbootable, but if you do ZFS on root, one might get into a sticky situation. but lots of people use it fine.

at least, that would be my vote.

You could also look at a MDADM array, perhaps with btrfs on top of it for the data drives as well.
Or an LVM array, but that would not self-heal like a raid1 of ZFS or a btrfs array

3 Likes

It’s entirely possible you overestimate my capabilities, or underestimate my capacity for figuratively starting fires… but with all experience comes some form of wisdom, I hope!

I get that, but I’m also - as a legacy of Stack Overflow - trying to avoid becoming a help-vampire; just cautious is all!

2 Likes

I never got into stack overflow but the idea seems nice. Never thought there is such a thing as a help vampire. Just help on the forum from time to time.

2 Likes

I think I can roll with that quite happily; though I’m very much a beginner with Linux (most of my actual computer use is browser-related, or Libre Office, not so much digging around in the OS itself), so I have to ask the obvious question: are there any caveats to having multiple filesystems on the same machine?

Thank you!

I will use a search engine momentarily, but I will say that I have no initial idea of what most of that means. I know of BTRFS, but that’s mostly from hearing about it years ago having some trouble getting to a stable state (but I assume that state is mostly out-of-date, now).

2 Likes

One can have many differing filesystems in use at the same time.
For huge/multi user environment, the re are tweaks an stuff to reduce latency, but on your system, you have file in ram, files on hard discs, and files on USB sticks, already suing differing formats.

Even Windows, which only supports very few filesystems, can support several at the same time (fat, ntfs,refs,nfs)

But, there are caveats to setting up ZFS arrays’ one should make sure Ashift (size of data chunks) is appropriate (normally, set ashift=12) and don’t enable de-duplication. Compression can be turned on, and the system just won;t compress data that does not compress, while not delaying writes much

1 Like

When you install via Ubuntu, you end up having two ZFS pools in your system: bpool and rpool. Bpool is your “boot partition”, Rpool is your “/”.

If you want to make a Raid0 with a second drive (e.g. /dev/sdb) you do: zpool add rpool /dev/sdb

If you want to make a Mirror out of your existing drive you do: zpool attach rpool /dev/sdb

Get a nice overview over your pool: zpool list -v

That simple. And after an install, everything is just running…so you only have to hit the terminal for changes, maintenance or backups.

4 Likes