Windows 10 StorageSpaces: Possible to use 3 storage tiers? SSds, Fast HDDs, & Slow HDDs

Hardware HDD's & SSD's
,
1

I want to create a storage space with 3 storage tiers which includes a fast HDD tier and slow HDD tier, I think…

Configuration:

  • Windows 10 Education v1909
  • Non-pooled Drives:
    1. Drive C: 2TB XPG SX8200 Pro 3D NAND NVMe Gen3x4 PCIe M.2
      (Currently being shipped)
  • Drives to be pooled and their theoretical tiers:
    1. Slow HDD Tier (12TB, initially 8TB):
      • 4 x 2TB Seagate 2.5" 7mm 5400RPM - ST2000LM007
        Note: Have yet to order any. Will initially order 4, then 2 additional after evaluating for 6 total
    2. Fast HDD Tier (12TB):
      This is currently a simple StoragePool. Migrating to new pool while keeping data intact will be challenging
      • 2 x 4TB HGST Deskstar - HDN724040ALE640
      • 1 x 4TB Seagate BarraCuda - ST4000DM005
        NOTE: Drive may be failing... Click for details. Drive has been kicked for IO Error twice in the past 6 weeks for a total of 3 times this year. I'm still unsure and establishing if it's good or not. It's definitely sketchy but required as the current pool has a 3 column volume. I've tried replacing, but all replacements have been DOA, rant on this later.
    3. SSD Tier (1.5TB):
      • 2 x 500GB Samsung 850 EVO (one of which is currently drive C:\)
      • 1 x 250GB Samsung 840 EVO
      • 1 x 256GB Crucial M4 - M4-CT256M4SSD2 (unfortunately too big for backplane module)
  • 2.5" drives will be stored in 5.25" Bay to 8 x 2.5" SSD / HDD Hot-Swap SAS / SATA Backplane Module
  • Drives in the backplane module will connected via LSI LSI00301 (9207-8i) PCI-Express 3.0 x8 SATA/SAS Host Controller. (I’m aware this will cause my GPU to drop to PCIe x8, but next year I’ll upgrade to zen3 which has more PCIe lanes)
    All other drives will interface with the motherboard directly.

Questions:

  • Is it even possible to have multiple tiers of the same media type?
  • The two classes of HDDs (consumer 5400rpm & enterprise 7200rpm) have very different performance characteristics.
    • If all HDDs were lumped into the same tier, is StorageSpaces smart enough to use each class of HDD’s optimally?
      • I’m not sure it is, but what if I can create 2 HDD storage tiers?
    • Prior experience tells me pool performance is constrained by the slowest drive. So lumping them all into the same tier seems like a bad idea.
      • What if I ensure the number of columns is less than or equal to the number of drives of each type. Will StoragePlaces then optimally place data?
  • Suppose I achieve 2 HDD tiers, what would happen if a tier loses a drive resulting in more columns than there are drives:
    • Will the affected virtual volume go offline? Or will performance just fall to the speed of the lower tier?
    • Do I have to maintain N number of drives in each tier? If so, I’d consider lumping them all together so there’s several drives to fallover to.

Prospective (ideal) Volumes:

Sized so that a failure in any one tier does not result in a ‘lack of capacity’ situation.

  1. ColdStorage/Archive: 2TB - 2 column mirror - 40GB allocated from SSD tier
    • SSD Cost: 80GB from at least 2 SSDs
    • HDD Cost: 4TB from at least 2 HDDs
  2. Media: 8TB - 2 column stripped - 200GB allocated from SSD tier
    • SSD Cost: 200GB from at least 2 SSDs
    • HDD Cost: 8TB from at least 2 HDDs
  3. Documents: 4TB - 2 column stripped + mirror (4 columns total) - 500GB allocated from SSD Tier -
    • SSD Cost: 1TB from at least 4 SSDs
      • Is this right, or does the copy of data on the SSD count as mirroring if it’s on at least 1 HDD?
      • If 1 copy on SSD + 1 copy on HDD counts as a mirror, then SSD Cost is actually 500GB from at least 2 SSDs right?
    • HDD cost: 8TB from at least 4 HDDs
      • Or is it actually 8TB from at least 2 HDDs if stripping from SSDs?

Why?:

The past few weeks I’ve suffered 5 HDD failures. My current storage pool originally had an additional 4TB HGST HDN726040ALE614 (had Western Digital serial number).
Since its failure, every single drive I’ve ordered has failed or been DOA.
I’ve lost my trust in current large HDDs for any HDD made after HGST was acquired by WD.
I want redundancy and performance but can’t afford the capacity required if I added 4 SSDs.

Click to expand rant on 5 drive failures in 6 weeks.

Don’t order HGST Deskstar NAS even if it’s the same exact item you ordered 5 years ago. They always send you something else other than HDN724040ALE640 - that’s how I ended up with the mismatch drive in the first place.

Failures:

  1. Original HDN726040ALE614 that was sent instead of the HDN724040ALE640 failed at 16,014hrs(1.83yrs)
  2. From Amazon, tried orderingHDN724040ALE640 specifically. Still got HDN726040ALE614. Failed several hrs into storage pool rebalancing. A few hours into undoing the prior balance, the ST4000DM005 got knocked out of the pool for IO error. I lost data here. (since then I’ve had no choice but to re-add the Seagate, I need at least 3 drives to recover. It appears to be working again?)
    Here’s what that failure sounds like: can’t link, search youtube for videoId yf7qZIZJBBU
  3. RMA’d #2, they replaced it with HUS724040ALE640. This time I decided to test it before pooling. Drive failed WinfDFT “Short Surface Test”. I requested money back and they had the nerve to charge me shipping.
  4. Next I went with WD Gold 6TB Enterprise Class WD6003FRYZ. At 297hrs it failed catastrophically (sounded like a head crash). Here’s what it sounded like while Windows attempted to rebuild the storage pool. can’t link, search youtube for videoId hP2ec9Nswkk
  5. RMA’d #4 for replacement, got back another dud with bad sectors. Requested full refund.
  6. Now I’m considering several small drives for the price of one of these large enterprise HDDs.

I cannot get a good drive! I need at least 8TB to replace the 2 4TB drives, and I’d like for something like this to never happen again.
I purchased 4 x 2TB Samsung 870 QVO’s, but immediately canceled after reading the reviews.

Good 2TB SSD’s are too pricey right now but in time won’t be. Eventually I’ll swap out the slow HDD Tier. For now I’ve decided to go for 4-6 SSD form factor low capacity drives so that when failure occurs I’ll have fewer eggs in one basket.

Powershell Mockup

During mockup I’ve found:

  • New-StorageTier has -NumberOfColumns and -Interleave params. This kind of throws a wrench I think? I’m not sure how this affects the Virtual Disks.
  • New-StorageTier doesn’t seem to have a way to specify which disks to use for that tier? Does that mean it’s impossible to have multiple HDD tiers?
  • It sure doesn’t seem to be possible… BUT, New-VirtualDisk has a -StorageTiers and -PhysicalDisksToUse params, perhaps maybe something can be worked out?

NOTE: The mockup as is will not accomplish what I want (or run AFAIK). It’s a work-in-progress open to suggestions.

Get-StorageSubSystem | ft # show  names of available subsystems
$subsystemName = 'Windows Storage on Derek-4790k'

Get-PhysicalDisk | ft  # Show all disks to see what serial numbers to use

$serials = @{
  slowDisk  = @('ST2000LM007_A', 'ST2000LM007_B', 'ST2000LM007_C', 'ST2000LM007_D');
  fastDisk  = @( 'HGST_A', 'HGST_B', 'BarraCuda');
  flash = @('850Evo_A', '850Evo_B', '840Evo', 'M4');
}

$drives = @{}
$allDrives = @();
foreach ($key in $serials.Keys) {
  $drives.$key = Get-PhysicalDisk | where SerialNumber -in $serials.$key;
  $allDrives +=  $drives.$key;
}

$pool = New-StoragePool -PhysicalDisks $allDrives -FriendlyName 'MyPool' –StorageSubSystemFriendlyName $subsystemName

# How does -NumberOfColumns and -Interleave affect the virtual disks' own settings for them?
# New-StorageTier doesn't seem to have a way to specify which disks to use for that tier? 
#  - Why don't they just do auto tiering if the drive's being used don't matter!?
$tiers = @{
  # slowHDD will be 4 columns if I use 6 disks.  The remaining two are to account for failures. 
  slowHDD  = $pool | New-StorageTier -NumberOfColumns 2 -Interleave 262144 -FriendlyName 'slowHDDTier' -MediaType HDD; 
  # fastHDD using 2 columns in case one of 3 drives fail
  fastHDD  = $pool | New-StorageTier -NumberOfColumns 2 -Interleave 65536  -FriendlyName 'fastHDDTier' -MediaType HDD; 
  # SSD Using 2 columns in the hope it picks the 2 of the 4 SSD best suited for the data
  SSD = $pool | New-StorageTier -NumberOfColumns 2 -Interleave 65536 -FriendlyName 'SSDTier' -MediaType SSD; 
}

# sized so that a failure in any one tier does not result in a 'lack of capacity' situation
$spaces = @{
  # total remaining capacity: 1500GB, 12_000GB, 12_000GB
  # when accounting for worst case scenario drive failure per tier:
  #   if 500GB Samsung 850 failure: 1500GB --> 1000GB
  #   if 4TB fastHDD fails: 12_000GB --> 8000GB
  #   if 2TB slowHDD fails: 12_000GB ==> 10_000GB
  # remaining worst case capacity: 1000GB, 8000GB, 10_000GB
  ColdStorage = $pool | New-VirtualDisk -FriendlyName 'ColdStorage' -NumberOfColumns 1 -NumberOfDataCopies 2 `
                                        -Interleave 262144`
                                        -StorageTiers $tiers.SSD, $tiers.fastHDD, $tiers.slowHDD `
                                        -StorageTierSizes 40GB, 400GB, 3600GB; # 4040 GB / 2 = 2020 GB volume size?
  # remaining worst case capacity: 960GB, 7600GB, 7400GB                      
  Media = $pool | New-VirtualDisk -FriendlyName 'Media' -NumberOfColumns 2 -NumberOfDataCopies 1 `
                                  -Interleave 65536 `
                                  -StorageTiers $tiers.SSD, $tiers.fastHDD, $tiers.slowHDD `
                                  -StorageTierSizes 200GB, 3TB, 3TB; # 6200GB total?
  # remaining worst case capacity: 760GB, 3600GB, 3400GB       
  Documents = $pool | New-VirtualDisk -FriendlyName 'Documents' -NumberOfColumns 2 -NumberOfDataCopies 2 `
                                      -Interleave 65536 `
                                      -StorageTiers $tiers.SSD, $tiers.fastHDD, $tiers.slowHDD `
                                      -StorageTierSizes 750GB, 3TB, 3TB; # 6750 GB / 2 = 3375 GB volume size?
  # remaining worst case capacity: 10GB, 600GB, 400GB 
  # total remaining capacity: 510GB, 4_600GB, 2_400GB (left on table for insurance + drive formatting overheads)
}

$volumes = @{};
foreach ($key in $spaces.Keys) {
  $volumes.$key = $spaces.$key | Initialize-Disk -PartitionStyle GPT -PassThru;
}

$volumes.ColdStorage | New-Partition -UseMaximumSize -DriveLetter X | Format-Volume
$volumes.Media | New-Partition -UseMaximumSize -DriveLetter M | Format-Volume
$volumes.Documents | New-Partition -UseMaximumSize -DriveLetter A | Format-Volume
2

It’s not possible.
I’m still waiting for parts to do the full setup I outlined and am going to use typical tiering with nothing fancy.

I did receive the 2TB AData then went ahead and imaged my Windows Drive to it and added the two 500GB Samsung 850s to the Storage Pool.
As of right now the pool contains 4 SSDs and 3 HDDs. After setting up regular Tiering, I did some tests, and I’m pretty happy with the performance.

StorageTiering Performance
StorageTiering Performance1368×1948 328 KB

1 Like
3

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.