Well I know that in general flash storage is not… the best possible solution for long term archival storage, but for my particular use case it’s the most optimal form factor physically since they’re very easy to intentionally destroy physically if one would need to.
I kept encryption keys on microSD card and yesterday this card died (completely, card behaves completely fine but there’s no any data on it, everything’s gone and recovery tools don’t see any filesystem structures on it). Of course (famous last words) I have backup.
It wasn’t actually i/o error but card was btrfs formatted and my dumb ass didn’t balance it with -d dup so I had only single data copy and card experienced bit rot on 4 out of 12 keys so btrfs was throwing error due to csum mismatch. With btrfs restore I managed to recover keys but well… they were obviously damaged and while 3 out of those “survived” (bit rot did not affect actual taken keyslots) one of them is really goner. (no data loss tho since I know what data I had there and it was just redundant copy I had backed up multiple times).
TL;DR - I got bitrot on microSD card that was just sitting on my desk for half a year, doing nothing and I sure did verify 100 times at time of backup that data was in fact consistent (including dropping kernel caches to make sure it’s actually stored correctly). It was also not the first time I experienced bitrot on microSD.
That said all my microSD cards were dollar store, 50% off 3$ cards, first ones sorted by lowest price in store.
So following this experience I decided to look for some better oprions and friend working in data recovery company recommended me taking a look at some industrial / surveilence sd cards since they’re supposedly a bit better quality than cheap ones.
That said I’m not exactly sure what kind of microSD card would be best fit for my needs. I’m looking for something that won’t be so prone to bitrot and preferably won’t die too soon. Idgaf about capacity, performance and TBW ratings because keys are like 50mb tops and I’m changing them once every… never.
So far I bought one Samsung PRO Endurance 32gb and one SanDisk High Endurance 32gb since those were available in local store and I needed something asap and formatted it with -m dup -d dup. But now I’m looking for some better alternatives. So far I found Kingston Industrial 8gb microSD cards with health monitoring, WD Purple surveilence microSD (also with health monitoring) and I’ve seen that some people recommend SanDisk Max Endurance since they’re supposedly pMLC as opposed to TLC used in others (but I really don’t know if MLC cells are in any way correlated with less bit rot over time)
So, generally price goes up significantly the more resilient the microsd. There are micro sd cards made for embedded systems and industrial applications, they generally will retain data for longer, but microsd is probably one of the worst formats to use for long term retention.
If you go to digikey or mouser, you can find SLC or pSLC microsd cards, but they are very pricy and become astronomical over 2gb. The use cases i have seen them for are in use by cold storage wallets, and in industrial display as i said earlier.
Tape or disc. Every other type of media, flash or magnetic, will require you to “refresh” the values inside the sectors or transistors.
Set a reminder to plug the cards into a system every once in a while to do refresh the stored values anyway. Also “high endurance” is more of a writing endurance than an endurance in the sense of “keeping data on them for a long time”.
I guess they are slightly worse than SLC cells due to one cell having to store multiple values. But I don’t think SLC would net you much longer storage time.
Yeah, sorry for the slip up. I’ll make it clear: a tape relies on it’s lattice structure inside due to it containing iron and iron being magnetic. So data is not stored on a magnetic medium but on a medium that’s influenced by something magnetic.
An hard drive is instead based on a platter that’s magnetized, so it’s storing data by magnetizing the surface and alligning all the “unorganized” magnetic fields in a given area.
The difference between these two approaches is that on tape you’re storing data changing the structure of the tape in a given area. On an hard drive you’re orienting magnetic fields in a given area. So on tape you rearranged the structure and you need more energy from the outside to rearrange it and on hard drives you gave energy to a small area on the platter that’s storing it but losing it over time because you’re not freezing into place that state.
If there was a way to magnetize, heat and cool rapidly the sectors on an hard drive I guess it could be possible to orient the magnetized grains of the material the platter is made of in such a manner to make data on it last forever.
That’s something I wanted to ask about - is there anything particular I need to do after connecting such sdcard? I mean how do I know if “it’s done” and it refreshed cells? Do i need to rewrite all data or rebalance btrfs or something like that? Just wait a while? how long? xD
Be careful with this, as far as I’m aware, no microSD exists that will automatically refresh NAND cell charge by simply being plugged in. You’d need to write to all the cells and burn there P/E cycles to refresh them.
MicroSD NAND controllers aren’t very advanced. As an example of this WD touts “read refresh” as a feature in some of there more expensive microSD cards.
Read refresh is one of the most basic things a NAND controller needs to do and the fact that many microSD cards are out there without it is troubling; without a read refresh function reading a microSD card over and over will deplete the charge of the cells of the data you are accessing and eventually corrupt it.
It sounds like you are describing a reluctance based recording medium, I’m unaware of any tapes that use this though. Or perhaps even LIMDOW magneto optical discs? which we did use to use in computing long ago; Plasmon was very active in that area.
LTO tape is using barium ferrite as the magnetic recording medium. It is magnetized just like normal hard drive platters are with a different encoding scheme of course. No physical alteration of the recording medium happens, just polarizing the little magnetic dipoles of the domains.
I believe you are describing magnetic coercivity. The smaller the magnetic bits, the higher the coercivity material they need to use to not be annealed away.
The reason LTO tape is more resistance to being degaussed over hdds is that the magnetic bits stored on it are significantly larger than the magnetic bits hdd use (because tape has significantly more surface area than hdds so it can afford to be generous). LTO tape also uses a very robust error correction scheme compared to hdds.
The reason energy assisted recording has proliferated with hdds is that by heating up the magnetic media, you reduce it’s coercivity so that the recording head can encode the magnetic information onto it before it cools. Smaller pitch magnetic recording heads produce a correspondingly weaker magnetic field so they need the extra help to be able to force the magnetic state into a bit on the hdd. Higher coercivity platters are used for the energy assisted HDDs, also they have finer grain structure too.
The reason energy assisted recording has not and may never proliferate with tape is that tape has vastly more surface area than hdd platters so it is unnecessary to encode domains that small.
Also if this technology path was pursued the a suitable tape substrate would need to be found, the polyester the tape is currently made of would rapidly degrade if blasted with heat.
LTO has many easier pathways to high density, using a higher coercivity media is probably one of the most obvious ones, moving from barium ferrite to strontium ferrite almost doubles the coercivity. I think I’ve read some where thats the development path LTO is going to take for LTO-11 in 2028. ← this is possible because the tape write heads are nowhere near as small as hdd write heads are so they can still produce strong magnetic fields to overcome the high coercivity media; even the older non-energy assisted hdds had higher coercivities than strontium ferrite, ~300kAm versus 250kAms… the new energy assisted hdd platters have even higher coercivities yet.
I’ve found SD cards with very few P/E cycles on them can last a surprisingly long time, multiple years easily. Its the “worn out” cards that seems to loose information in a matter of months, but YMMV.
I had forgot about the SD express cards, I’m also surprised to have never seen one considering how old they are at this point.
This is hardly definitive, but based on flexxon’s offerings, it would appear the SD express cards are a mixed bag on RAS features compared to the nicer “normal” SD cards:
I ended up ordering one Kingston Industrial 8gb microSD and one Western Digital Industrial XI QD334.
WD is (as always) not very elaborate in their spec sheet but Kingston is proudly advertising many nice features such as:
Endurance Up to 1920 TBW
30K P/E cycles
NAND TLC in pSLC mode
Operating & storage temperature -40°C to 85°C
Industrial features:
• Bad block management
• Strong ECC engine
• Power failure protection
• Wear levelling
• Auto-refresh read distribution protection
• Dynamic data refresh
• SiP – System in Package
• Garbage collection
• Health monitoring
Durability:
Waterproof
Temperature proof
Protected from airport x-rays
What has been taught me and wasn’t able to confirm researching online is that the ferromagnetic metals that compose the tape are “suspended” in the tape material so the action of the writing head is not just alligning the magnetic domains but literally polarizing the particles in the tape.
Yes.
I think the much larger magnetic domains make all the difference compared to HDDs.
As soon as I red this part I remembered that that’s already a thing with HDDs. But I was thinking more about a permanent way to write data that way, which I don’t think it’s the case for HDDs yet.
Absolutely! Also the base tape, before it’s coated, is really not resistant to heat and would probably catch fire.
There’s also to say that the magnetic domains on a tape are only in line with each other and once the tape has moved away from the writing head there’s little to no chance to change what’s written on it. Meanwhile HDDs are always interacting with really small adjacent domains so applying a really strong field would surely make a mess with all the data that’s already written to the drive. So no jumps to really high coercitivity for drives yet.
:
