GPU Accelerated Storage Parity

Anyone know more about this stuff? Is it proprietary? I haven’t heard of it till today.

This is new. I’m not very knowledgeable about it, but it’s very relevant for my company, so I’ll be looking into it.

2 Likes

It looks like they kill half the drives before it goes down, but he doesn’t say how much of the capacity is eaten up by the parity, or if it’s just a performance boost or what…

@SgtAwesomesauce if you guys start playing with this, and you’re allowed to talk about it, it would be great to know the broad strokes of how it works.

A quick search on Google turned up some proposals from last year and an academic paper behind a pay wall, but it doesn’t look like there’s much out there yet.

1 Like

Yeah, I’m going to get the information straight from the horses mouth.

Sounds like they’re more focused on doing analytic data on the storage nodes rather than calculating parity with a GPU. That said with certain large clusters parity calculations offloaded to a GPU could be helpful.

1 Like

To me it sounded like he was saying the GPU was handling some sort data resilience function underneath Lustre but that it was also running whatever “supercomputer” stuff on the data as well… but idk. It’s not the most informative video. Just saw it on the Supermicro twitter.

That’s just buzzwords. He’s basically saying you have the ability to have a compute node tell the storage node to do data set analytics on the compute node so you don’t have to deal with the SAN, just the storage node. Which is a stupid concept because the idea of a SAN is that a single dataset is spread across compute nodes, so to get the whole dataset to one storage node would require utilizing the Network part of the SAN.

When you get a certain number of spindles or NVMe devices, you start to run out of CPU power when it comes to calculating parity. Our Ceph storage nodes usually sit around 28 load. We’ve got 32 cpu on those systems. If we were to offload a bunch of that work to GPU, we would probably sit around 2-3 load.

1 Like

Ah ok, that’s kind of disappointing. I was thinking the GPU was doing some sort of next-gen RAID magic. But it’s more like, they’ve offloaded traditional parity calculations to the GPU. Seems weird that he was so focused on pulling the drives in the demonstration, as if a box of disks surviving some failures was interesting…

That said, it makes me wonder if there are other potential roles for GPUs in storage applications.

Yeah, in the enterprise, we don’t do parity, so it’s a silly assumption that anyone will use this.

Industry standard is to keep 3 copies.

1 Like