How much Optane secret sauce does one really need?

  • is there a golden ratio of HDD + NVMe + Optane?
  • recommended minimum for Cyber range/home lab?
    -is there any performance gain by using an AIC vs U.2? How much of a bottleneck is there via the chipset [Asus Sage sWRX] anyway?

Caveat:
I’m essentially shooting for the “budget build” version of @wendell sWorkstation

As far as I know the p5800x’s only come in U.2 versions, not AIC.
If I was going for budget version I’d get a smaller p5800x or possibly a 905p, then use primocache or something similar for tiering.

1 Like

keep in mind that the consumer version of primocache (basically primocache) will only allow 2 “disk” storage tiers and a RAM cache.
If you are looking at optane, you would have to chose:

  • SSD + optane
  • HDD + optane
  • HDD + SDD

You would not get any benefit of having more than 2 storage tiers.
since the p5800x (latest and greatest optane drive) is a server oriented product you will only find it in a U.2 2.5 inch form factor as @Jolly said, however you may chose to install it using a u.2 cable that some moboards have or an adapter card as Wendel showed in his initial video, same drive just different installation methods pending slot availability.

If you are going gen 1 optane the p905 (the p4800x will only get you a bit more endurance for a much more expensive drive, not worth it IMO) drive can be found on both a AIC and 2.5 inch form factor, they both use a pcie x4 3.0 connection and have identical specs, the choice would come down to availability, slot availability and cooling consideration

If you are shooting for a budget version maybe consider the older gen 1 optane, you can get them for slightly about 1-1.5 usd per GB on ebay used (also see the p900 version), while the cheapest p5800x is 1200 USD new for a 400gb drive, the newer drive is evidently a lot faster, but the extra speed might not be worth it for the extra price.
(it was not for me, even though I wanted it, and had an available slot, I did not need it)

PS: there are other threads on this forum that go into more details on having the p5800x working if you are interested, hope your build goes well :slight_smile:

1 Like

SPDK has changed my mind, a bit, here.

With LVM it does seem like it would be viable in production as a cache. If what you need lvm can do then… 800gb-1.6tb is actually kind of a lot for the cache tier… easily enough for 8-16tb of flash capacity.

2 Likes

Is your sWorkstation build sheet listed here? I have yet to find it. Also, if you could tell me what RDIMMs you used I would be eternally grateful. I’m not having much success in choosing the correct one despite using the QVL for the Asus sWRX80. Samsung M393A8G40AB2-CWEGQ (64gb) is my latest attempt but still getting code 27. Corsair LPX Vengeance is the only one that has worked but I’m shooting for the 512gb RDIMM for reliability and stability in my home lab. I’m considering an RMA but am reserving that for last resort. If what you used doesn’t work with a similar setup then I would have no choice but to RMA.

Thx for the feedback!

Assuming you’re trying to do things on a budget -

A 100GB p4801x is reasonably priced on newegg at the moment. $67. This is about as fast as you need for a NAS on a 10Gbe network.

This is a pretty good amount if you’re just trying to cache a single drive.

Knowing your use case and budget matters though. If you’re running a small amount (in GB) of VMs or a high performance database… you want to be on flash for the underlying storage. If you’re trying to store a boatload of videos and ISOs… HDD arrays with a bit of cache.

Going off of my experience with a NAS (ZFS based - mostly for photos/videos + STEAM games) 30ish GB of RAM ends up handling 50-95% of my reads for cache. Going to 118GB of L2ARC using an 800p cuts access to the HDD array during the heaviest usage by about a third or so on average. At least in ZFS land, using a large capacity cache drive cuts into RAM available for caching so it’s not “worth it” as much you might think to get a HUGE SSD for caching a big volume of data - optane really is the ideal cache drive in this situation. Most data isn’t read THAT often so diminishing returns kick in quickly with drive size. If you need CONISTENT performance you should probably do a flash based array. Optane as cache can still help flash but it’ll mostly shine when you’re doing things that tax the flash (think flash array is mostly full, they’re low on their internal pseudo-slc cache, etc.)

Correct - but you just pickup a StarTech AIC or similar and mount it on the MOBO if that’s how you like your meatballs. I run my P5800x as an AIC and use it as Level 2 cache in PrimoCache and it’s phenomenal. Can’t recommend this config enough. I edit 6K RAW videos off a Promise RAID array and the cache let’s the system rip at full res. And of course, the OS volume barely gets read cause PrimoCache learns what it needs and grabs it first! Zippy zippy indeed.

Do you by any chance know if this use case works with PrimoCache? @wendell

  • Windows installed on motherboard AMD chipset software SATA RAID volume (don’t like interruption in case of a single failed drive)
  • A single Optane drive added for PrimoCache
  • Encrypt this logical boot volume with BitLocker (the feature configuration you have to enable via group policies where the data is encrypted by the CPU and not by some firmware/TPM/drive combination, this way the data that is written to the PrimoCache cache drive should also be encrypted)

?

My hope is that this way you can work around the shitty performance of AMD’s RAID drivers a bit.

(No free SSDs to test this out myself at the moment)

I’m using the same style of adapter myself. Downside is the lack of pcie slots in most motherboards, im experimenting with adapters to run an optane off the m.2 slot.