I got a good deal on a pair of Optane 905P drives and jumped on it out of excitement. I was planning on adding them to my primary pool as a 2-way mirror metadata vdev. After doing some reading I am now unsure. My current setup:
- CPU: Dual E5-2650 v2
- RAM: 96G
- Primary Pool: 1 Vdev Raidz3 w/ 9 5400RPM HDD & 1 spare
- Primary Pool Usage: Plex Video storage, Nextcloud file storage, Proxmox backups, ISO storage, misc file dump
- VM Pool: 1 vdev Mirror w/ 2 SSD & 1 spare
- VM Pool Usage: Docker configs, VM zvols
- Download Pool: Striped w/ 2 SSD
- Download Pool Usage: Fast download and unzip of ISO’s
Relevant Notes:
- HDDs above are WD schucks
- SSDs above are: HSCAC2DA4SUN400G
- Have a single LB1606R SDD not in use
- SSDs & HDDs are connect via SAS2
- 2x Optane 905P drives are PCIE slots
I am looking for suggestions on how best to make use of these 2 Optane drives. I have an offsite backup, but as you might tell by me using Raidz3 I want to avoid restoring from it.
I haven’t setup a metadata special vdev yet, but I have been researching quite a bit here and in other forums.
One thing jumps out to me with your plan: your data is only as safe as your weakest vdev. In this case, this currently is the raidz3 vdev you have in your primary pool. You can loose 3 disks and you will be okay. However, if you introduce a 2 disk special vdev mirror then your data is only safe up to 1 ssd failure. Perhaps adding the LB1606R to make it a 3 disk mirror special vdev?
Alternative uses for them: you could allocate one optane SSD to the HDD pool as L2ARC, though I’m honestly not sold on the benefit of that suggestion myself. It seems like all your high-performance needs are potentially handled by the SSD pools.
The risk of adding the vdev is a good point. As I was typing it out I was considering the idea of making the optane into a pool, migrating download and VM to that. That would leave me 5 SSDs for the special vdev.