I was curious if anyone had implemented encryption of data at rest on their home server. I am aware of options like LUKS, etc, but as far as I understand LUKS does not deal with “data at rest” as I am defining it here. I am thinking about “data at rest” on the live system. I suspect this kind of thing would be a concern for cloud hosting providers, since we would not want cloud hosting employees to have access to government or PII data, for example.
What I am expecting is some kind of process which acts as a middle man when an application requests data. The middle man would provide the key, decrypt data and pass it along to the application in a transparent manner. Thus, the data is encrypted “at rest” and only decrypted on the fly (on request).
I am looking for a high level discussion on two issues:
What Linux software packages exist (if any) to accomplish the task of encrypting data at rest in a transparent manner (from the perspective of applications)? It seems like you could hack together something that would do this with fscrypt[1]…
(Perhaps most important) How to handle the key files to decrypt safely in the event of my home server being stolen (i.e., being physically pwned).
Even if (1) is not possible or inadvisable, it would be interesting to have a discussion about (2) as it would apply to the basic LUKS installation as well.
With respect to (2), I am building a NAS in Fractal’s Node 304 case, so it’s the kind of thing that could easily be carried out of my building under one arm. Therefore the keys should not be stored directly on the system. I am thinking about using a thumbdrive and relying on any potential thief being too technically stupid to realize that the USB unlocks the data on the NAS. This isn’t really a great option, however.
Another idea I had to address (2) is to store the keys on some kind of modified USB that would delete its own contents (i.e., the decryption keys) when plugged into another machine. This essentially involves crafting malware and may also involve hacking on the firmware of the drive. This is well beyond the bounds of my experience, and probably not a topic we’re permitted to discuss here. Anyway, this is kind of tangential to my goal of encrypting data and doesn’t really solve the problem of physical access to the machine.
I would also like to avoid LUKS or similar options in general, as I believe they will complicate my remote access. For example, it’s my belief that the standard LUKS installation prevents reboot from remote location without a KVM (which I don’t currently have). Looking at the Cyberciti tutorial for remote LUKS unlocking leads me to believe this is a somewhat fragile configuration that will require regular testing to ensure functionality.
Is this a wild goose chase or does it seem like an achievable goal?
This is exactly what dm-crypt (which LUKS is built on) is, and what you just described.
It’s transparent, which means the application makes regular file system requests to get/put data which is underlyingly encrypted. The data is decrypted on read and encrypted on write. There is no plaintext data stored anywhere unless whatever your application is doing mishandles the data—but that’s not within the scope of the problem anymore.
You’re probably chasing something more complicated than just using LUKS, though your requirement for remote operation (and my lack of experience with that) means there could legitimately be better solutions.
You can use LUKS and still support remote reboot by enrolling the key with TPM or using something like a YubiKey. I have several headless machines, running Ubuntu 24.04, that boot using TPM-enrolled keys with LUKS and Clevis.
I am not sure how secure Ubuntu’s Secure Boot implementation really is (there is an unencrypted /boot partition after all…), but the threat I care about is having recoverable data on drives I may want to dispose of in the future.
I had a wonky setup before where my disks were all part of a ZFS pool with encrypted datasets, and I booted off of a USB drive that contained a LUKS partition with the ZFS keys. The USB drive key was enrolled with TPM via Clevis, so it would boot unattended and unlock and mount the ZFS datasets (which included /). It worked fine, but my goal was more about not having a non-ZFS partition on disks and wasn’t about having the keys on USB.
Theoretically, if you are using TPM to unlock your drive without any PIN or password, then as long as your OS is secure a thief would not have access to your data. However, if you have a guessable password or there is some system exploit, a thief could login to your booted OS and access your data.
What you are describing does sound like you want the data encrypted end-to-end and you don’t trust your NAS. One solution to what you are describing would be running some blob storage service on your NAS that provides an S3-compatible API and use something like rclone.
An easier solution to protect against a thief would be to manually mount an encrypted file system containing your data when you want to use it, and unmount it when you are finished (maybe this can be triggered automatically by a timer). Then the key is not stored on the NAS when not in-use, and as long as the OS has not been compromised while the file system was unlocked, the data should be unrecoverable.
True, I think we shouldn’t trust anything, within the limits of reasonability. To expand, most people are not auditing the code they self host (who has time?). Plus with the supply chain attacks that have been in the news over the last few years I think it’s not unwarranted to be wary of even your home server. This to say nothing of vibe-coded projects of which there are many!
My only fear with enrolling in the TPM is that the system would boot and decrypt data when plugged in at the thief’s house.
I had briefly considered using cloud storage for PII, access to which could at least be revoked remotely if I discover my stuff has been stolen within a reasonable amount of time. Of course this data would also need to be encrypted before transfer to cloud storage.
I guess that a similar approach could be used to store a keyfile itself and revoke access in the event of a theft. To be extra paranoid I could do remote health checks on the machine and just automatically revoke if the machine failed x number of checks in a row.
The behavior you are describing is the standard mechanism for storing data in services like Amazon S3. There are open-source services that you can run on your own NAS that will have an S3-compatible interface, that will allow you to use a tool like rclone to store data E2E encrypted on your NAS with the keys managed by the client device. I don’t use any of these tools myself so I can’t recommend anything specific, but rclone allows you to mount this storage as a local file system and I believe it is one of the more popular tools for doing what you are wanting. You may have some luck searching for guides on self-hosting rclone.
Think about physical security as a layer that makes theft much less likely, say, site your device in a strong box or locked cupboard that has a disguised door.
You might also use a Yubikey for boot encryption that can be locked in an separate safe or carried with you, so that any rare* reboots only happen when you’re present to confirm that the system has come back up OK.
*: I ran a LUKS-encrypted LVM2 array for 5 years with reboots at most twice a year. I didn’t have PII / GPDR obligations for data on the array but did care to keep my disks cool and my data encrypted.
Checking back in after some research. So, if I setup garage or some other s3 compatible storage option, should I use RClone for presenting this to docker containers that do not natively support s3? Or is it better to restrict this approach to applications that support s3 datastore
Where are you planning to run the Docker containers? If it is on the NAS, then that means you would need to provide your keys to the NAS after all, so there would be no advantage of using E2E over just standard disk encryption using LUKS.
You can handle all of the file system mounting on the host and just map those directories into the container using bind mounts like docker run -d -v /path/on/host:/path/in/container my_image. The container does not need to know anything about the underlying storage system.
I am backing up to a rockpro64 overseas at my parent’s home. I have zero trust in that machine. It’s running minio and has 2x25TB disks attached to it. No encryption. Connection is through tailscale so no https needed.
So first I setup a basic rclone remote targeting the minio. Then I create a second remote of type Crypt on top of the first remote. Make sure to set the option to also encrypt file and directory names.
There might be a possibility of guessing what it is based on file size but other than that if your threat level is not a State backed entity that should do it.
Isn’t MiniIO kind of dead in the water? That seems to be the hot gossip on reddit.
I am building a NAS at the moment, I plan to use podman on that machine.
The services I am more concerned with securing in this manner are hosted on a separate machine entirely. These services are things that I would reasonably expect to use when out and about: personal finance, notes, caldav. This separate machine is running Docker
It does but I’m not changing a physical machine on the other side of the ocean if I can avoid. As explained in my setup the keys are managed locally and the server has no idea whats going in there.
a. SafeBoot security of typical consumer devices (I mostly mean the UEFI firmware itself and settings)
b. and finally cold boot attacks become realistic thanks to unlimited attempts in the attacker’s hands.
You don’t need S3 or rsync. The data can be stored remotely on a server acting as a dumb block device with network access.
Your actual server with services and decryption keys. It has the remote block device mounted (SCSI?) and acts like the storage is local.
Remote server has nothing but storage and a good internet connection to provide raw access to HDDs/SSDs.
Confiscation of either main or storage server leaves the attacker empty-handed. One has the decryption keys, other has the encrypted data. It’s a plus if the other server isn’t in your name or is somewhere really remote.
I made some notes about encrypting data with a nitrokey or yubikey. I think something like this is what you are looking for to protect against physical theft. The security key is PIN protected, so even if you leave it plugged in and the thief steals your PC, it will lock out after too many wrong guesses. It will also allow you to still use a local password or unlock over the network with something like dropbear.
