Zstd output to block devices, feature not bug

Just a heads up, if you use linux to push remote operating system images. Zstd (compression software - facebook’s baby) essentially doubles you throughput on common hardware. A windows 10 raw disk image (including updates, few drivers, apps, and sysprepd) is about 17 GB of actual data. Zstd squeezes this to 6.9GB and because the decompression doesn’t murder the cpu, 1gb ethernet can saturate (~108MBs) While geting something like 210MBs of write speed on a single ssd.

This is all accomplished with custom debian live minbase buiilds and ipxe.

Note. You can’t compress physical block devices tp zstd directly from say /dev/sda with the exception of /dev/loop. My sysprep baselines start as 60bg raw images in kvm-qemu. Once sysprepd they get mounted to loop0 for ntfs-wipe to clean up dirty unused blocks. Then compressed with zstd for excellent speed and storage savings.

Some source info:
https://willhaley.com/blog/custom-debian-live-environment/

One decent laptop in the office took 63 seconds from zero’d to rebooting into w10 oobe.

software in use for deployment: debian, nfs-kernel-server,dnsmasq, apache2, ipxe, zstd.

software for baseline management: zstd, kvm-qemu, guestfs-tools, ntfs-wipe.

FYI it’s fine to host a ova of the debian server that makes this work, it is however not fine to host winpe and sysprepd images due to MS licensing.

2 Likes