Return to Level1Techs.com

Encrypted RAID10 with Linux

I’m looking for a storage solution with both good response time and basic security. I want to secure data primarly against low motivated actors with physicall access to the machine. I have backup solution, so I’m not afraid about data loss. I’ve never implemented RAID setup under Linux before, and have basic experience with encryption. I’m following a guide that explaines mdadm tool for creating RAID10, and so far it looks pretty strightforward.

ED: The array will serve as data storage only in a workstation. OS will be on different drive.

My questions are:

  1. What approach is better for performance and security respectively? Creating RAID array and encrypting it after, or encrypting the drives and then pooling them together?
  2. Should I expect performance hit from encryption at all?
  3. Are there any quirks with the following approach that one should know?

performance will be a huge problem with the proposed setup.
encryption will certainly have a huge impact, but the main performance problem comes from the use of mdadm as opposed to hardware RAID.

if your server has plenty of CPU power to spare, then go for it. if not, buy a RAID card.

1 Like

Thanks for the feedback. I didn’t mension, but drives will work in a workstation, not in a server machine. I need good throughput when copying large files from main NVMe drive.

high throughput is not going to be achievable with an encrypted RAID1+0 if its running on common workstation desktop hardware.

I have run many combinations of md raid in Linux and have never run into CPU bottleneck. Even on fast NVMe drives in raid 10 md raid the CPU was idle while the drives were maxed out on iops or throughput. Also I usually am hesitant with RAID controllers because if the controller ever has to be replaced unless you can get the same one I don’t think you will be able to get the information back (have heard of this issue but not experienced it because I have not used very many of these hardware RAID setups). On md raid you will have a lot more flexibility in how you want to set it up.

2 Likes

I may be wrong (hopefully), but I have a bad feeling about encrypting the drives then pooling them together could result in some terrible data loss.

Do you have a method to get a full back up first and then provision the drives like you want to prevent the risk of data loss?

1 Like

I am not sure if it is better to encrypt first then build raid or encrypt after, but my gut says encrypt after raid is the better route.

From the tests that I have seen and run myself unless you are doing a lot of reads and writes on the drives the encryption performance hit will not be noticeable. On newer hardware there will be some overhead, but would be pretty negligible from the benchmarks I have seen.

The only thing you should be aware of on this is booting could be a little more complicated if you have / on the encrypted/raid volume, but should be easily overcome with modern tools if you want that. It would be a little easier if / was on a normal partition and other mount points on the encrypted volume, but that is not required.

1 Like

Most of the data will be either synced with NAS or archived offline. And the recent files I plan to keep on main drive(s) anyway. Array is primarily for fast access without the need to pull files from NAS or archive.

Thanks. OS is on another drive. Array is for data storage only.
I should specify this in first post.

I would luks encrypt every drive and then running every other layer on top of that.
Here is an example script for btrfs but should work also for other filesystems:
http://marc.merlins.org/perso/btrfs/post_2014-04-27_Btrfs-Multi-Device-Dmcrypt.html

I get full performance with aes-xts-plain64:sha256 and spinning disks cpu load is relatively low.

Cough
Cough

There’s no such thing as hardware raid.

Dedicated RAID device for enhanced acceleration.