Just saw Linus's video on Intel's new "caching" optane product.
It would be great if L1Techs could do a video on how one might be able to take advantage of this on Linux. It would be good to see if one could use it as a standalone storage and fit a tiny MySQL/PostgreSQL database, or use it for caching the database that is kept on an SSD (and see the steps necessary to do this in Linux).
Does that even work on linux?
https://downloadcenter.intel.com/download/26730/Intel-Rapid-Storage-Technology-Intel-RST- dont see linux on the drivers page
I wouldn't be counting on using Intel RST. Linux the drivers are usually already found in the kernel rather than having to be downloaded and compiled in (like zfs and nvidia blobs), and I'd be amazed if there wasn't already something in the later versions for x-point. I think Phoronix is on the case.
For caching rather than as a standalone storage device, there must be some sort of generic caching system that one can set up on linux. E.g I think people did this with SSDs to HDDs back when SSDs were really expensive. At the very least, it would be a fantastic ZFS L2ARC device.
I just wish Optane existed in PCI-e x4 or even x1 format.
Would make it so much more useful for actual older systems than m.2
For those interested in "setting up optane" with zfs, it's as easy as:
clear
format <to see avail. devices>
zpool add tank cache <optane or nvme>
zpool status <check that it worked>and you're good to go.
The reason they're bundling special software on windows is that its filesystem doesn't have something like this built in. As far as I know, the windows implementation is just a smart cache ala L2ARC. Optane isn't a consumer technology to begin with, and people using it on enterprise systems don't really need mollycoddling when it comes to setting up these things outside of windows.
That said, I'd expect to see some sort of dogfooding to that effect anyway for linux down the line, maybe as a part of btrfs or some FUSE abomination. There's also bcache, and nothing's stopping you from setting up zfs in the meantime, either.
There's already people testing Optane on FreeBSD iirc btw
TL;DR: just use zfs (or bcache if you're lazy or need high speed caching for your nokia ngage)
Optane NVME is just fast NVME. the special drivers are pretty much just for the smart cache feature.
You can get a PCIE adapter to use instead of an M.2 socket. By the time the consumer versions are out, the drivers should be in most operating systems.
But I'm not sure if windows will allow it as a stand alone device, or if you will have to use the tiered storage spaces system?
Just saw the last thread about it:
I don't imagine that it wouldn't recognize it as a standalone, the NVME standard doesn't care about the underlying media, you'll most likely be able to use it sans intel driver, and you probably don't even need kabylake outside of marketing compliance.
Expect some smug linux posting about sticking optane into their ryzen systems when the consumer launch hits.
Let's wait and see,
I wouldn't be surprised though if Optane is doing some funky PCI bus <-> chipset negotiation though to hardware limit itself.
The only platform specific support requirement that I know of is for Optane DIMMs, granted I could be way off base.
I remember alan jude talking about testing them with zfs with no special config needed a while back, and I don't really expect that to change on launch with the possible exception of Windows, which will be done software side (error, this won't work because we say so etc.)
That said, they may be disabling windows caching on the budget CPUs for a good reason -- smart caches take a certain amount of resource overhead to run correctly. Someone with a budget CPU and a small amount of ram may actually see a decline in performance from the extra system overhead.
Let's use ZFS L2ARC as an example: it takes about 2gb of ram per 100gb of cache storage to use, and that's on a much more robust filesystem with built in smart-caching in RAM -- so we can expect the windows solution to have a higher "cache tax."
on extreme budget systems, 4 and 8 gb configs are still common, and even half a gig of operational overhead plus however much cpu time is needed could easily make performance suffer under load.
The consumer device is being marketed as a magical "set and forget" drop in upgrade, so they don't want people to use it on systems where it could potentially make things choke up.
You can buy a ssd with 120 gigs and 240 gigs for the same money and just skip the intel retardness 
Throw in limiting their own market and the levels climb even higher.
"Look gais we tuk swapp and mad ssd!!!"
This is so stupid.....
It's much different than swap. Smart-caching is a way to intelligently prioritize the most used files on your system and speed up their access times even when not in memory. Swap is more an emergency measure to account for any failures in memory management on the part of the system or the user.
That said it's only really needed in databases/HA servers and the benefit conveyed to the consumer isn't necessarily the same. A more accurate jab might be "We Took ZFS ARC, locked it down to our devices, and sloppily bolted it onto windows. Now we're selling it expressly to people who don't need it!"
TL;DR you don't understand swap (or Optane) very well, but that's ok, we're all still learning.
Come to think of it though, putting swap on an Xpoint device would make memory saturation way less annoying.
Understand it well enough to realize that this delivery is less than optimal.
So its XFS.
Wooo~?
Question: How does one recover from a broken Optane drive? Is it all still on the HDD because Optane is only caching?
yeah, you'd be fine (if their implementation works like every other production ready one)
The other really nice thing about optane/x-point is that I believe when it reports to the host that the data is written, it is actually written to permanent storage (with the lower latency for that time to take as well). Unlike most consumer SSDs which I believe will write to an on-board cache/buffer and report that data is written, but will need a bit more time for the data to have actually been moved to permanent storage. if you suddenly cut the power to your computer, you can can lose data this way. This is why some "datacenter" level SSDs have on-board capacitors to ensure any buffer has a long enough time to get written through. I believe this product is ideal as a ZFS intent log device.
For anyone actually still interested in this, just a few days after this discussion this article was released:
TL;DR: Optane isn’t really ideal due to low endurability and bandwidth.
(Which also explains why the Optane release was only targeted as only an HDD enhancement.)
But Micron is supposed to release their 3D XPoint products somewhen later this year. (The flaws are not inherent to the technology.)