Hi guys
A quick question here: i’m setting up a 36 TB volume (based on a hardware RAID5 of 10 drives) for NextCloud. I’m running Ubuntu 22.04 (desktop, not server). Which fs should I use that’s NOT ZFS: Ext4 or XFS?
Thank in advance for the help.
Not to be a giant wet blanket but neither of your options are good and you tossed out the good one. I’m assuming you had good reason so I’ll roll with it
To give you a good non ZFS option I only see BTRFS as viable in my opinion integrated with snapshots would be a really good idea. If your are RAID 10’ing or 5ing this configuration it’s definitely decent at the task
And remember RAID, nor ZFS nor BTRFS snapshots are a backup solution. Make sure you have backups of you care about that nextcloud data. Ideally give the database its own BTRFS partition and take snapshots regularly Incase something happens to it. When I did recovery on the postgres db that sucked alot
2 Likes
There are a surprising variety of ways to provide a logical filesystem to the OS.
Not sure what makes you shy away from ZFS, but there is a good likelihood that the same reasons will have you shy away from BTRFS. These are conceptually very similar.
Both ext4 and xfs are seasoned file systems and either will be a good choice.
What technology have you considered to create a raid5 of 10 drives?
1 Like
Actually crap. Wait. Does BTRFS still have that cow bug for raid 5 and 6? because if so it makes this a non option
2 Likes
Per the recommendation from RedHat:
Another way to characterize this is that the Ext4 file system variants tend to perform better on systems that have limited I/O capability. Ext3 and Ext4 perform better on limited bandwidth (< 200MB/s) and up to ~1,000 IOPS capability. For anything with higher capability, XFS tends to be faster. XFS also consumes about twice the CPU-per-metadata operation compared to Ext3 and Ext4, so if you have a CPU-bound workload with little concurrency, then the Ext3 or Ext4 variants will be faster. In general, Ext3 or Ext4 is better if an application uses a single read/write thread and small files, while XFS shines when an application uses multiple read/write threads and bigger files.
So the recommendation is going to be dependent on the rest of the hardware this will run on and what kind of data you will be serving with Nextcloud.
TBH we could use some more details to give you a proper recommendation.
5 Likes
On that note I’d like to know how the RAID array is being instantiated and what is used to create it so that we have some idea of what’s going to work well
He said its a hardware RAID controller. The storage will be presented as a single huge volume to the OS so yeah ZFS is out of the question.
BTRFS is really only seasoned with mirrors, and EXT4 and XFS are both excellent filesystems for this use case.
But there are a couple niche uses that they are more tailored for hence my request for further details.
For example, if there is a battery for the write cache then XFS won’t shit the bed if there is an unclean shutdown while data is in flight. If OP doesn’t have that on his controller then yeah EXT4 is the clear choice because it doesn’t suffer from superblock corruption like how XFS does.
So we need more deets.
2 Likes
The day I learned this the hard way was the day I stopped using xfs lol but fair point. It’s strange for me to see hardware RAID these days. It’s basically dead. What makes ext4 more suitable than BTRFS? Or rather what does ext4 have as an advantage here?
Ok guys,
So the box has dual Xeon E5-2670s (40 threads in total).
The drives are SAS SSDs on a Dell PERC controller.
It’s got 128 GB of memory.
And the NextCloud workload will be mixed. Presumably large and small files.
I can’t really bring down the system to break the RAID. Hence my aversion to ZFS. But if that’s the best solution I may just have to schedule the down time.
I see some of you are saying Ext4 and XFS are fine though. And I appreciate the Red Hat material. The server won’t have many users so i guess the queue depth will be very low.
I think I still need a definitive tie-breaker though. 
I hope the additional info here can help with consensus.
1 Like
Please read this
3 Likes
Is nextcloud running on this server or will it be just the storage backend, presented to the Nextcloud server via NFS, iSCSI, etc?
1 Like
It’ll be running in Docker on the server.
1 Like
I don’t think that quite answers his question but my take away is that you’ll be pointing docker to this contiguous storage volume that has the file system that wins out here? Is that a valid assumption
Thank you.
Personally, I would advise against that. They ship too fat of an image, you will have difficulty, scaling, perf tweaking, hardening, etc.
But to the topic at hand:
I would go with XFS here. And you have sufficient components in place to be able to withstand failures and redundancy.
But to be perfectly clear, the best way to get you the answer is to fire it up with one solution, do benchmarks, and then compare. Its trivial to create tons of small junk files with fio.
2 Likes
Yep. There will also be other misc apps and data using the volume but NC would take the lions share of storage space.
1 Like
Just chiming in but nextcloud is small files biased. While yes files uploaded could be large all of the little files associated with operating the service + the database and small transactions outweigh the storing of just the large uploaded file.
When I made the ZFS dataset I tweaked it specifically for nextcloud and most pointed this out on various wiki’s. If I had to choose between big file performance and small here I’d bias for small and take a hit to large file performance
1 Like
He has 20c/40t. Dude’s got a ton of cores to throw around which will help the perf with xfs by a lot to make up that delta.
1 Like
True was resource constrained with 12C/24T. After reading red hats exposition (while waiting for windows update on my work laptop 
) he should be fine in XFS. Seems best fit. BTRFS advantages wouldn’t be worth it and it doesn’t handle RAID 5 or 6 that well. I had to look that up. It’s a really sucky failure mode. In that failure mode, each individual block still has contents that match the checksums, so a scrub won’t catch it. It’s a silent data corruption; you would have to check vs. known good copies
So completely disregard HW Raid 5+ btrfs. Out of the two originals I’d go with XFS
Quite some time ago I read an article (on paper, yeah, that long ago 
) with a comparison of various file systems in Linux-land. Their conclusion then was that JFS and XFS are closely matched overall (with a slight advantage for XFS in large file handling) while the toolsets for each FS was clearly in favour of JFS. Ever since I use JFS on all my Linux boxes.
Yup, it does
1 Like
Wasn’t JFS created by IBM?
AFAIK you can still use it, but its been unmaintained so YRMV.
I had to RTFM.
2 Likes