BCacheFS thread

It seems we got a new toy to fiddle with and if its good enough for Linus to accept commits is good enough to me to start playing with it.

This file system has come to take on both ZFS and BTRFS and its written mostly by a lone wolf dude. The best kind of open source software. Seriously the code is quite good.

From their site: https://bcachefs.org

  • Copy on write (COW) - like zfs or btrfs
  • Full data and metadata checksumming
  • Multiple devices
  • Replication
  • Erasure coding (not stable)
  • Caching, data placement
  • Compression
  • Encryption
  • Snapshots
  • Nocow mode
  • Reflink
  • Extended attributes, ACLs, quotas
  • Scalable - has been tested to 100+ TB, expected to scale far higher (testers wanted!)
  • High performance, low tail latency
  • Already working and stable, with a small community of users

Some interesting things:

  • it does tiered operations of the data in mixed disk types. I think it’s a different beast from ZIL/ARC.
  • compression / background_compression?
  • encryption is ChaCha20/Poly1305. Something like Wireguard and very easy on the CPU. Also immune to some known LUKS attacks.

YOU ARE TOTALLY RIGHT! But if you are here at this point in time it means you don’t care about such trivialities like keeping your data.

Start using it:

NixOS: Bcachefs - NixOS Wiki
Arch: Bcachefs - ArchWiki
Gentoo: bcachefs - Gentoo wiki

There has been an ugly bug in kernel 6.7 so Linux 6.8+ is recommended.

5 Likes

For me, personally, the draw is that it aims to solve a few problems not solved by any other file systems (not even ZFS):

  • Read/write cache (promote/foreground targets)
  • Elimination of the RAID write hole problem
  • Tiering compression and RAID levels: recognizing that compression and erasure coding is detrimental to high performance, bcachefs introduces the concept of compressing and erasure coding data written to background devices while metadata, read, and write caches remain uncompressed and mirrored for performance.

There’s also this:

btrfs doesn’t have this problem, but suffers from a few other dealbreakers.

I’m curious as to how this pans out in real workloads.

1 Like

I learned about bcache some years ago, just before the guy started serious dev work on bcachefs. Looks indeed promising and as I’m a strong proponent of RAID6 I’m hoping bcachefs can handle RAID6.

1 Like

If someone could write a shim that lets me zfs send from bcachefs to ZFS that would be amazing. I would love to try it out but don’t want to risk my real data set, and I don’t have any need for these filesystems outside of a NAS.

1 Like

I’ll try to reinstall my fun laptop with bcachefs encrypted root. It uses 2 nvme as system drive. Downloading the steam library is normally a good way to test throughput. Will report disasters.

5 Likes

Installation on Nixos unstable went fine but had to change UUIDs to raw device paths. It seems systemd-boot has an issue finding the UUIDs of multi devices and/or encrypted volumes.

It’s raid1 encrypted. It doesn’t use LUKS though.

1 Like

I guess the following is relevant:

1 Like

I’m on 6.8.1. Other than the weird way the array UUID is set (which makes other user land programs to not be able to see it) everything is alright so far.

I plan to eventually test this out [and might publish benchmarks here] once I have all my “new” hardware set up (see my username). Does anyone know if BCacheFS has an effective alternative to ZFS’s ARC, my reading of the documentation I could find suggests no. Personally I share concerns others (see Proposal: Consider adding warnings against using zfs native encryption along with send/recv in production · Issue #494 · openzfs/openzfs-docs · GitHub ) seem to have about newer ZFS features [ mainly block cloning and native encryption] and instability/corruption and I do see a future where for SSD’s where either https://www.purestorage.com/ or something akin to it is the path forward, there still needs to be a good alternative that is more generic.

Also does anyone know if there is anyone who has done work on making Bcachefs work with QAT as I did pick up a QAT device that I hope gets good use, right now I have a lot of ideas on how to use it mainly for ZFS, but also for other things.