Looking for SATA/SAS HBA with working spindown

Hardware Enterprise Gear
, ,
1

I’m searching for a 16 port SAS/SATA HBA with a working sata hdd spindown under linux.

I’ve tested 2 SAS HBAs

  • Adaptec ASR-71605
  • LSI 9400-8i
    Both of them had broken spindown with sata HDDs. (LSI kinda worked but it took over 90s to wake up the drive which is unacceptably bad )
    I’ve used mini sas hd to sata splitter cables but I doubt cables could have caused these issues.

Are there any specific models you had success spinning down HDDs?

Why are enterprise HBAs so bad at it? In my experience motherboard chipset sata ports don’t suffer from these issues.

2

Your tested HBA models are too new and expensive for me :slight_smile:

I had success with SATA HDD spindown with older SAS2308 HBA on latest firmware.

1 Like
3

Thanks.
Sadly 8 ports is not enough for my use.

I could try to buy 9300-16i, it looks similarish but ~27W max power usage is concerning.

Adaptec card is quite reasonably priced imo but has some sata spindown bug.

4

Because enterprise drives are designed to spin 24/7. Cold starts are hard on a drive and constant spin up/spin down in a datacenter would accelerate wear and greatly increase the rate of drive failure.

3 Likes
5

and to add to this hard drives aren’t using up as much power as people think when spinning. An inactive, but fully spun up hdd might use 4 watts while a spun down drive might use ~1 watt; not a big power saving for the huge amount of wear the spin cycles put on the drives.

1 Like
6

The idea is to combine an 8-port card with an expander. e.g.

They’re combined cheaper and more power efficient than the 9300-16i

7

I understand the reasoning for many use cases.

In case of the mostly idle (>90%) home nas use case with >10 drives (see OPs requirements) the savings are quite noticeable.
Plus, most drives are more like 6W idle and <1W spun down.

Depending on the selection of components I can see an argument for both sides.

1 Like
8

You’re not wrong, but I think the power gap has closed more than many realize, at least for some drives. For example here’s the power figures for higher capacity WD red pros:

image
image1527×1021 120 KB

3.4 watts for the drive spun up but not active and 1.2 watts for the drive spun down.
Using a US national average electricity price of 17.3¢ per kWh that’d be a savings of almost exactly $3 per drive per year of runtime if drives are spun down 90% of the time; balance that with the extra likelihood of drives failing due to the additional stress and I think its a loosing proposition monetarily, at least for this specific drive.

On the other end of power comparisons are the lower, presumably non helium filled drives where I can see spin down maybe being useful… but I have separate thoughts on using many low density drives when a fewer number of high density drives could be used:

image
image1525×1026 110 KB

9

You should be able to adjust spindown times outs and schedules using the MSM utility on the adaptec cards.

Spin down time outs are controlled per logical device (not physical):

image
image1265×933 62.8 KB

Spin down schedules are controlled at the controller level:

image
image1263×933 93.6 KB

1 Like
10

“3.4 watts for the drive spun up but not active and 1.2 watts for the drive spun down.” Impressive!

Again (I said it before in my first post) - too new and too expensive for me :slight_smile:

Let’s assume we want to spec a 40TB usable NAS. That would require 3x 20TB WD Red Pro (raid5 or raidz1), which in idle consume 10.2W according to spec sheet. Amazon currently has a discount and sells them at $380 a piece. That’s $1140 for three drives.

For my home lab I don’t need the latest and greatest. A set of refurbished 8TB HGST Ultrastar DC HC510 will do at $66 a piece. I need 6 for a comparable raidz1 setup. I’ll buy 8 to have spares in case one or two unceremoniously meet an early demise. That’s $528 total which leaves some dollars for the additional cables (and HBA/extender - see topic of this thread) needed.
In my experience these drives hold up well for years of frequent spin down and they’ll consume 6W total when spun down. When active, they use more, but they’re about twice as fast as well compared to a 3x raidz1 with modern drives (yes, they have a impressive sequential transfer rate spec but they’re only marginally faster at random read/writes).

These setup decisions obviously doesn’t make sense if the home nas still idles a lot but doesn’t lead to predominant spin-down of HDDs. A small office that’s 8 hours busy and 16 hours idle may benefit from the additional speed, but should probably prioritize reliability and warranty over the janky old stuff.

Niche solution, but it works for me and my budget :slight_smile:

Edit:
I just saw that 16TB HGST Ultrastar DC HC550 drives can be had refurbished for the same $/TB as the above mentioned 8TB drives.
I looked up the specs for these drives and they go for 5.6W (idle_A) or 3.4W (idle_B). They consume 1.1W when spun down (standby_Z). The spec sheets for WD are apparently written by the marketing department because I had to dig quite deeply to find the technical details (p.34. Explanation of idle modes on p.26).

Given that the WD Red Pro are made by the same vendor and I cannot find the same detailed user manual as for the enterprise drives my tin-foiled hat thinks that the specs mentioned are for idle_B power state and therefore identical to the enterprise drives.

1 Like
11

I think you might be right about that. I guess WD isn’t as impressive as I thought, Toshiba’s got them beat with a ~4 watt idle_a on their big drives.

I didn’t see any settings in the adaptec utility to set idle_b, only idle_c and standby_z settings.

1 Like
12

The power of marketing.

Once I found the details documented for enterprise drives I did not want to go back to consumer drives. Even if it meant that I would buy them used.

I have no doubt that these are essentially the same drives hardware wise. I think they tweak the firmware a little (mostly take away enterprise features for consumer drives). Same with Seagate. I never went down the rabbit hole for Toshiba.

13

Also famous for the deceptive marketing of their shingled drives.

I’ve been buying enterprise drives exclusively for the past ~10 years for home use, they’ve been a lot less hassle than I was use to dealing with.

1 Like
14

I don’t know how these datasheets can be trusted. My 15 HDD server idles at about 170W and I estimate power consumption of a single drive at about 8-9W.
Old enterprise 4TB and new 20 TB WD HC560 had almost the same power consumption. Maybe 1W of difference at the wall.

Sure, but it’s not an excuse to dump on the marked half working HBAs. You accidentally enable spindown on your server and all your drives disappear from the machine. This should be unacceptable.

Thanks, did try spindown with through sas expander?

I’ll consider getting an expander, but PCIe slots are a luxury these days and I wouldn’t want to sacrifice another slot. (and powering pcie card via power adapter would be pure gore :joy: )

15

15HDD server, 170W idle vs 66w spindown (measured at the wall)
22 hours * 0.104 KW * 0.28 Euro * 30 * 12 = that would around 230 Eur of savings in a year. This is quite significant amount of money for me.
With that lower power consumption I could justify running my backup server 24/7 instead of turning it on just for backups.

16

Soon-ish I should have real world use power numbers on Toshiba MG08’s. I’m going to have 30 of them in one chassis and I expect the whole system to idle at less than 200 watts; with 30 drives it should drown out much of the other system components power usage uncertainty.

Modern-ish hellium filled drives should have much power usage figures than the old air filled drives.

There are some expanders available that don’t take up any PCIe slots like the adaptec 82885T

17

geez! I understand your question now.
What drives are these out of curiosity?

18

Yes, a separate expander card requires some extra space. Since they don’t use PCIe lanes for data transfer but only draw power through them I have mine in an open PCIe 1x slot.

There are also cards like the Intel RES2SV240 that can operate outside of PCIe slots and receive power via molex adapter.

Hmm. I am not using it now… My always-on system today is running Seagate MACH 2 dual actuator SAS drives connected to a SAS backplane. I don’t have them configured for spin down.

I don’t think I ever used a SAS expander card in an always-on system. Cannot think about a reason why this wouldn’t work with SATA drives, tho. Configure using hdparm -S <timecode> (look up the weird timecode in the hdparm man page).

1 Like
19

Hitachi/HGST Ultrastar 7K4000

They are quite toasty but I didn’t see big difference in idle power consumption between newer enterprise HDDs I’ve tried.

Next month I’ll be building a new small server and I’ll recheck their power usage.

Yea, I know. But it’s also about pcie slots. Consumer boards don’t have a lot of them.

That Intel expander looks really nice. Thanks.

20

Followup about power efficiency of helium filled hdds.

I’ve retested HDD power consumption with dc powered motherboard.

idle n100dc-itx = 8.11W
idle n100dc-itx + Ultrastar 7K4000 old air filled HDD = 18,73W
idle n100dc-itx + WD DC HC560 20TB HDD = 16,45W

Ultrastar 7K4000 old air filled HDD = 10,62W
WD DC HC560 20TB HDD = 8.34W

These results are not amazing.
HC560 datasheet claims idle power usage around 6W. I would expect my results to be lower than recorded.
AC to DC and DC to DC conversions probably don’t account for the difference, it would have to be at 73% efficiency.