ZFS on Linux 8.0 released

Some very important and much desired features with this release.
I would personally recommend waiting a few months before making the switch.

zfs-0.8.0

@behlendorf|20x20 behlendorf released this 4 hours ago

New Features

  • Native encryption #5769 - The encryption property enables the creation of encrypted filesystems and volumes. The aes-256-ccm algorithm is used by default. Per-dataset keys are managed with zfs load-key and associated subcommands.

  • Raw encrypted ‘zfs send/receive’ #5769 - The zfs send -w option allows an encrypted dataset to be sent and received to another pool without decryption. The received dataset is protected by the original user key from the sending side. This allows datasets to be efficiently backed up to an untrusted system without fear of the data being compromised.

  • Device removal #6900 - This feature allows single and mirrored top-level devices to be removed from the storage pool with zpool remove . All data is copied in the background to the remaining top-level devices and the pool capacity is reduced accordingly.

  • Pool checkpoints #7570 - The zpool checkpoint subcommand allows you to preserve the entire state of a pool and optionally revert back to that exact state. It can be thought of as a pool wide snapshot. This is useful when performing complex administrative actions which are otherwise irreversible (e.g. enabling a new feature flag, destroying a dataset, etc).

  • Pool TRIM #8419 - The zpool trim subcommand provides a way to notify the underlying devices which sectors are no longer allocated. This allows an SSD to more efficiently manage itself and helps prevent performance from degrading. Continuous background trimming can be enabled via the new autotrim pool property.

  • Pool initialization #8230 - The zpool initialize subcommand writes a pattern to all the unallocated space. This eliminates the first access performance penalty, which may exist on some virtualized storage (e.g. VMware VMDKs).

  • Project accounting and quota #6290 - This features adds project based usage accounting and quota enforcement to the existing space accounting and quota functionality. Project quotas add an additional dimension to traditional user/group quotas. The zfs project and zfs projectspace subcommands have been added to manage projects, set quota limits and report on usage.

  • Channel programs #6558 - The zpool program subcommand can be used to perform compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits).

  • Pyzfs #7230 - The new pyzfs library is intended to provide a stable interface for the programmatic administration of ZFS. This wrapper provides a one-to-one mapping for the libzfs_core API functions, but the signatures and types are more natural to Python.

  • Python 3 compatibility #8096 - The arcstat , arcsummary , and dbufstat utilities have been updated to be compatible with Python 3.

  • Direct IO #7823 - Adds support for Linux’s direct IO interface.

Performance

  • Sequential scrub and resilver #6256 - When scrubbing or resilvering a pool the process has been split into two phases. The first phase scans the pool metadata in order to determine where the data blocks are stored on disk. This allows the second phase to issue scrub I/O as sequentially as possible, greatly improving performance.

  • Allocation classes #5182 - Allows a pool to include a small number of high-performance SSD devices that are dedicated to storing specific types of frequently accessed blocks (e.g. metadata, DDT data, or small file blocks). A pool can opt-in to this feature by adding a special or dedup top-level device.

  • Administrative commands #7668 - Improved performance due to targeted caching of the metadata required for administrative commands like zfs list and zfs get .

  • Parallel allocation #7682 - The allocation process has been parallelized by creating multiple “allocators” per-metaslab group. This results in improved allocation performance on high-end systems.

  • Deferred resilvers #7732 - This feature allows new resilvers to be postponed if an existing one is already in progress. By waiting for the running resilver to complete redundancy is restored as quickly as possible.

  • ZFS Intent Log (ZIL) #6566 - New log blocks are created and issued while there are still outstanding blocks being serviced by the storage, effectively reducing the overall latency observed by the application.

  • Volumes #8615 - When a pool contains a large number of volumes they are more promptly registered with the system and made available for use after a zpool import .

  • QAT #7295 #7282 #6767 - Support for accelerated SHA256 checksums, AES-GCM encryption, and the new QAT Intel® C62x Chipset / Atom® C3000 Processor Product Family SoC.

See full list in the github

Also, there seem to be some pretty big “gotcha’s” involved with device removal, see the description here: https://github.com/zfsonlinux/zfs/pull/6900

This is why I always recommend reading the pull requests, as well as the various comments leading to the request. They often contain important information that isn’t documented anywhere else, unfortunately.

8 Likes

0.8 for Linux 5.0?

Okay, I think my body is ready for this…

From what I read, it’s compatible all the way to the 5.1 stable kernels, however do be aware that the removal of native instructions due to license issues (which already happened in the ZFSoL 7.* series) have performance impacts yet to be fully explored or mitigated but which do exist.

NixOS guys apparently were understandably not happy with the big hit to encryption performance, and currently have their own patch. Please note, I am not responsible if Oracle or Stallman send assassin teams.
https://www.phoronix.com/scan.php?page=news_item&px=NixOS-Linux-5.0-ZFS-FPU-Drop

For anyone who wants to argue about licenses, I encourage you to make a separate thread. I intend this one to be purely informative and potentially technical.

WOOOOOOT!

My body is ready!

1 Like

DKMS / packaging let me down … dnf update says it ran dkms and installed the kernel modules, so I reboot and of course, the boot hangs, for whatever reason I put swap on zfs, so I get dumped into maintenance mode.

run DKMS againt, and again it says installed the modules, reboot, and zfs is back.

I think it’s because dnf tells dkms to rebuild modules for the current kernel, but then updates to a new kernel for the next boot?