recently built my server (7773x) and i got 6x2tb NVMEs in it, 2 on the mobo and 4 on an asus expansion card. configured as ZFS raid10.
I’m considering using the holiday prices to get 4 more so i get a total of 10tb usable space with the speed that comes with it.
what I’m not sure of is the power consumption when idle and when busy while, and if ZFS effects it. since I’m running this server 24/7 at home, I am conscious of the extra wattage.
Does anyone have benchmarks/experience/numbers ?
p.s. if i add 4 more, they will also live on a second asus expansion card.
I’m pretty sure m.2 is limited to like 7w maximum, system fans can use more power than that
also its overall power consumption depends entirely on which specific NVME you have
your CPU getting pinned or pinged by something random will have more impact
be careful about adding too many, they only qualify those for single card solutions and with the mobo it came with, if you have like 7 in your system, the last 3 may fail to post
Are you talking about the various lower power states that NVMe drives can enter? If so ZFS doesn’t do anything specifically to either enable or thwart that, that I’m currently aware of.
By default it’ll coalesce asynchronous writes and write them out every 5 seconds, which likely makes them even more efficient.
NVMe drives have crazy fast power transition latency between states. Note these are microseconds, not milliseconds.
These states do seem to also rely on the PCIe lane power state as well, but overall I would expect the drive to spend most of its time with very low draw.
there was a forum post on here where some of his cards were failing to post when all were in at the same time
to clarify I meant the 4 in one cards not the m.2 themselves
To be fair, only U.2 or other “serious business” drives with a massive heatsink are going to pull that much. The Micron 9100 Max seems to go even higher even
They also tend to idle at a few watts.
Consumer M.2 drives seem to pull roughly ~6-10 watts max depending on the drive from what I’ve seen, though I think some of the recent gen will actually pull past 12.
Iirc most NVMes have idle/sleep power consumption in low triple digit milliwatts.
As for posting, I have a Rome system with a H12SSL-i, with two of those Asus cards plus two motherboard slots for a total of 10 drives, it posts and UEFI shell shows all of them. My pendrive crapped out so I didn’t get to install an OS yet.
If it’s server drives, they often don’t idle. My samsung drive has a power rating of about 8w or so, and this seems fairly typical for NVME under load, and is the same on their consumer drive of a previous generation of product when under load, but the server drive idles in the 0 state, and temps only vary by about 5~10c, compared to 40c+Δ with the consumer drive.
Similar behavior with a SKHynix server drive, except that one doesn’t throttle when it hits dangerous temperatures.
