3x16TB toshiba enterprise MG08 drives in raidZ1 + 1x samsung 860 EVO log drive ( 1 DRIVE currently offlined and rent for RMA)
2x 960 GB samsung DCT sata ssds mirror for ix-app dataset
Seasonic prime 550W (80+ platinum) 2017
=> 28 TB effective storage + 860 GiB ssd scratchpad. HDDs do not spindown. There no addon cards, i.e no HBAs, no additional nics etc.
PSU is overkill, but it was cheap in 2017, despite the reputable OEM and efficiency grade. I dont think there were any platinum psu in sub 550W category available.
Power measuremnt at wall at 240V:
startup peak power use 60W
idle with apps running on ssd zpool 28W
active writes to main dataset 38-42W
So yeah, those helium filled drives are mighty efficient. Keep the ix-app dataset off them though, it constantly wakes them up for little to no reason.
I will be replacing main pool with mirror of 2x 20 TB toshiba MG10 drives when they arrive.
If I’m crunching my number correctly power draw in these tests came out as:
~9,5W for Ultrastar 7K4000
~9W for 20TB WD DC HC560
There may be some overhead from running test in a server chassis with passive backplane.
I cannot get near specsheet 6W idle in any realistic testing scenario on this HC560.
Maybe Toshiba drives are better. I’ll try to get my hands on one of these at some point.
Interesting fun fact, lspci shows me PLX switch chip and 2 SAS3008 controllers. Card is really hot and I probably won’t use for anything other than testing.
It raises my idle power usage by about 29W D:
I’m installed it hl15 and tested performance without any issue.
Spindown works semi reliably. It takes very long times for drives to come back, 2-3 minutes. I once had issue with bringing drives up and
Testing just asrock n100m, RM550x and samsung pm961 nvme:
4TB Ultrastar 7K4000 sata cable NOT connected: 8,95W
4TB Ultrastar 7K4000 sata cable connected: 9,05W
20TB WD DC HC560 sata cable NOT connected: 7,64W
20TB WD DC HC560 sata cable connected: 8.52W
These WD HDDs are just not as efficient as manufacturer claims on specsheets.
Jesus christ on the pogo stick, that fucking nuts.
So note for self, those trimode 9600-8i cards with 7W TDP are looking better and better. I really should have bought that one that fell of the back of a wagon for 100USD.
EDIT: dont buy them, those 9600-8i are no longer as uselful as older versions. L1 fella reported that they (i.e 9500 and 9600 tri-mode-controllers) no longer do simple passthrough.
Yes, those SAS controllers were eight ports at most (SAS3008), hence the kludge+power+heat
In that generation, you’d want the 9305-16i which uses the true 16-port SAS3216 controller, but it isn’t as cheap. May as well get the 9400-16i or 9500-16i series for about the same used (~$200).
Wall power measurements with a DC-DC brick can be quite inaccurate. If you have a DC current clamp meter, you can measure it accurately around the SATA red cable.
Yep, that explains the price. I’ve looked at 9305-16i but didn’t seem justifiable.
For now I’ll stay with adaptec cards. Price is really good for what you get.
I know that AC-DC conversion messes up the measurements.
But at the end of the day I mostly care about numbers on my power bill (and/or UPS capacity)
