Well this is my first post so let me start by saying “Hello all!” The question I have is I recently got 4 new WD Red 4TB drives (Made sure they were CMR drives) to upgrade a pair of mirrors in a zfs pool of 4 mirrors. I’m wondering the best way to go about this.
OS: Debian Buster (kernel 5.10)
CPU: Ryzen 5 3600 CPU
MB: ASRock Rack x470DU
The disks are all in hot-swap bays in a Norco 2U server chassis
This is the current setup:
[email protected]:~# zpool status data
scan: scrub repaired 0B in 00:08:32 with 0 errors on Sun Jun 13 00:32:34 2021
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-ST1000DM003-1ER162_W4Y6NJ9J ONLINE 0 0 0
ata-ST1000DM003-1CH162_Z1DBCS60 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
ata-ST1000DM003-9YN162_S1D1QRM6 ONLINE 0 0 0
ata-ST1000DM003-1ER162_W4Y6NJ18 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
ata-Hitachi_HUA721010KLA330_GTF002PBJ481BF ONLINE 0 0 0
ata-WDC_WD10EZEX-60M2NA0_WD-WCC3FPP9S2TY ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
ata-WDC_WD15EADS-00R6B0_WD-WCAVY0927527 ONLINE 0 0 0
ata-ST31500541AS_5XW0GCK5 ONLINE 0 0 0
mirror-8 ONLINE 0 0 0
ata-LITEON_CX1-JB512-HP_002547100MFL-part2 ONLINE 0 0 0
ata-LITEON_CX1-JB512-HP_002551102VTQ-part2 ONLINE 0 0 0
ata-LITEON_CX1-JB512-HP_002547100MFL-part1 ONLINE 0 0 0
ata-LITEON_CX1-JB512-HP_002551102VTQ-part1 ONLINE 0 0 0
errors: No known data errors
I’m looking to upgrade mirror-0 and mirror-1. Is it as painstaking as failing out a drive and replacing them one at a time? Or is there a means of just removing the mirror, swapping the drives, and adding the new mirror to the pool?
Do you have any spare bays?
Or USB enclosures?
Or, spare headers inside the chassis?
The liteon drives, are they m.2?
Nah all 8 bays are full. The liteon drive are m.2. There are spare sata headers on the board though. The top on the chassis isn’t tool-less so I’d have to unscrew it and open it up and I’d like to avoid that it possible.
Well, to be honest, the Quickest, and safest way, data integrity wise, would be to connect one of the new drives, and use the
zpool attach, then
zpool detach commands.
That way, all 8 data drives can stay connected, a 9th drive be
zpool attach-ed to a drive in mirror. When resilvering complete, one drive can be
zpool detach-ed, and swapped for another of the new drives.
Once the second new drive is resilvered, one could
zpool detach the last old drive from that mirror, and swap the first new drive into a drive bay.
So at all times, there are unbroken mirrors.
there are other ways
Another way, would be to zpool detach one of the drives from one of the mirrors, swap the detached one for a new drive, zpool attach the new drive in to the single drive.
After resilver, you can of course do the same for the second drive in that mirror.
This can be, and is used, but it means the pool might (Might) die is there are some errors from the drive that is being resilvered. and is risky, as the whole pool is less resilient during the procedure
You could also export the entire pool, clone two of the drives to the new drives, and potentially re-import, with the clone in the bay of the cloned drive.
The old drive would still contain the partitioning it originally had, which should probably be cleared after the original pool comes up.
If you went this way, the I would say, do one drive at a time, and ensure the new drive imports fine, before doing the same to the other drive.
@Trooper_ish first solution is the best you can do in this situation. It seems you used to add the drives
by-id which means when you have detached an old drive from a mirror you can just pop the new drive in the hot swap bay and ZFS should do it’s magic and be able to find the drive in the new bay. I wanted to add, that before you attach the drives please run an extended SMART test as well as at least one pass of the
badblockscommand to make sure the drives are in pristine condition.
Well looks like it’s a trip to Best Buy for a USB enclosure. I have a solid back up of the data in the pool, so that’s not the worry. Thanks for the help