StoreMI: Why Tiered Storage is better than cache on x470/Zen+ and Threadripper | Level One Techs

@Steinwerks Stop getting my hopes up…

From the AMD StoreMI FAQ:
StoreMI is available as a free download with every AMD socket AM4 motherboard with a 4xx series
chipset, and with every socket sTR4 motherboard with the X399 chipset.

Looks like I’m outta luck.

Watching the video, a question came to my mind: how does tiered storage handle things like backups? What I mean is: imagine you have your data in a StoreMI or similar arrangement, and you back it up regularly. Therefore, every x time you will access all of the data. I guess that would be “neutral” to the algorithm, since it’s equal for all files, and still the ones you access for reasons other than backing up will get priority. But is there any risk of spurious relocation of data due to backups or other type of access for which speed is irrelevant (and therefore, ideally, should not influence the tier assignment)? Or is the algorithm aware of backups or other situations and adjusts accordingly? Since this technology has a longer history on the enterprise side, I expect all such things are covered. But I guess I’m just curious about how the algorithm works in general.

Except this tech is licensed by Enmotus:

Like I said, not included (but you can buy it). Requirements are all AM4 and TR4 socket boards.

Oh SWEET, its available on x399 as well :slight_smile:

Was one of those things i was considering x470 for…

Would this work with a 2400G APU and a x470 motherboard, or just with Ryzen+ chips?

StoreMI is a 400-series chipset feature software, so it should work with any of the Ryzen chips.

FuzeDrive should work with anything Ryzen whatsoever.

1 Like

Not with backups, no. In enterprise, there may be issues when you have different services or systems accessing the same storage with peak activity at different time of day, for example. Then you may not even see any benefit because at the time every new system “spins up”, its data has already been displaced into the slow tier by previously active systems. It depends on implementation, of course: how and how often “heatmap” gets updated and applied. Tiered storage works if storage tiers are relatively stable, but it’s not a silver bullet.

1 Like

I’m still wrapping my head around this stuff, so sorry if this is a dumb question.

Is there a tiered storage solution out there that can access archived data faster than “spinning rust” HDDs?

When I was going to set up a file server, I was really hoping there would be a way for me to get faster read speeds than a HDD would allow. The dream was to be able to install all my roommates’ and my games on HDDs the file server and us just access them from there. Without getting too focused on that use-case, I wanted ~3-5 users to be able to access games from a server on the LAN and have better performance than if you were on a computer with one HDD or ~double that speed thanks to RAID.

I understand that a “cache” is limited to being kept up to date, and it has to “learn” where files are on the drive(s) it’s connected to. I’m having a hard time understanding how this tiered storage is fundamentally different from a cache, besides being configurable by the user.

@wendell said in the video something to the effect that StoreMi ‘won’t access something faster if you run it once’, which is what’s really tripping me up about how this is better than what I understand to be a “cache.”

I’m assuming the answer is no, but is there ever a sensible application where it makes sense to have extremely fast storage tiered over conventional SATA SSD storage, over “spinning rust” HDD storage? Where the SATA SSDs try to help the HDDs respond more quickly to what the user and NVME is trying to do. (I’d assume that even if there could be a performance benefit, the cost of “buffering” with SATA SSDs would outweigh any gains.)

@Steinwerks Ah, when you said “not included” the first time, I translated that as “freely downloadable”. My bad. Still, good to know I have that option, even if it costs money.

Yes, it’s called “either don’t use mechanical drives or use a shitload of them if you need better access speeds”. How do you even imagine it, reading data faster than the media storing said data allows to read it?
The point of read cache is to (a) prefetch sequential data and have next block ready for you before you ask for it and (b) store it for some time in the fast memory in case you’ll decide to read it again shortly afterwards. The point of write cache is to (a) rearrange random writes into series of sequential writes, (b) allow you to edit data before committing it to storage (if you write the same block twice, only one write operation will be performed on media) and © smooth down the write bursts/spikes into something more manageable by media. The point of tiered storage is to virtualize the storage. The data is stored on fast or slow media, depending on access frequency. It’s essentially the same as having system/software/games on SSD and videos/music/backup images on HDD, only you see just one disk of combined capacity and usually you don’t have to manage it, data moves between physical drives automagically.
And no, neither cache nor tiered storage will allow you to read data stored on “spinning rust” faster than it’s physically possible to read data stored on “spinning rust”.

Thanks for the explanation. I understand it better now!

“Free” as in “you pay it with the board price”.

BTW question on the side, is this X470only, or other 400 series chipsets as well?

AMD says all 400-series boards, and X399.

https://www.amd.com/en/technologies/store-mi

From the FAQ:

image

3 Likes

OK, question about that. If I listen to a lot of the same music in repeat, won’t that result in it eventually being moved to the faster tier and essentially waste that space (because the last thing I need is faster reads on music when windows takes an hour to boot)?

Also, anyone care to elaborate what’s so bad about SSHDs? Don’t they do the same thing? Tho I think they copy instead of move the content to the SSD part, it’s essentially the same thing in hardware, no?

I think the main problem with SSHD is the pitifully small size of the SSD “cache”. Most drives I’ve seen will pair an 8GB SSD with 1TB+ platters. That’ll help Windows boot a bit faster, but that’s about it.

It may. I’ve never seen tiered storage having any file-based QoS, it doesn’t care (or even know) about file types, data is data, and the only metric available is frequency of access. At most, I would expect something like that in Vmware VAAI, but haven’t seen it even there. In enterprise storages you can sometimes set up priorities for volumes/vdisks so that data on one LUN had higher chance to end up in higher tier than the other one, or override dynamic tiering completely, but you don’t get per-file granularity with such settings, all I’ve seen is just per-LUN/vdisk/volume/whatever vendor calls it. It’s not entirely block-based, either, as vendors usually use bigger chunk size for this mechanism (which is understandable because keeping heatmap with one block or even stripe size resolution would be kinda stupid), and it may prevent what you’re describing from happening.

1 Like

A less sexy app is the Nand efficiency/durability/heat angle.

AFAIK, MLC nand has an edge in these metrics, but at greater cost.

A hard working MLC fast tier ssd, paired with a cheaper capacious TLC ssd, yields the best of both in a hybrid ~array.

I am confident the files most users access more than once a week, would fit comfortably on 128GB.

1 Like

Can this be used with RAID 0?

I’d like to pair x2 7200RPM 2TB HDDs with x2 2.5" 256GB SSDs. The goal is to have the information redundant between the each tier, and then apply the StoreMI between the 2TB x 256GB configuration for tiered storage.

I strongly prefer a cache to tiered storage like SMI and apple’s fusion drives because if the cache drive dies, you don’t lose any data.