Caching workload on MicroSD in 2021?

So … I got this idea (which is how I always get into trouble),

I’m thinking I’d like something like a steam cache, but for linux distro repos, and I’d also like it to be a time machine for those repos, so you can move forward/backwards/pin the repository, and also a thing where I can ask for a particular version (to bisect issues, when I have time for it).

Debian, for example, already has snapshot.debian.org. - all I’d need to do is cache it, same as steam cache.
ArchLinux has archive.archlinux.org (aka Arch Linux Rollback Machine) - same story.
I don’t think Alpine has this, but I can build it (I kind of did, in my other post it works, there’s some issues I need to solve).

For practical reasons, it’d be nice if I could stick a microSD card into e.g. a NanoPi Neo3-LTS or into a ZeroPI, plug it into a POE switch through a POE splitter, and just have it there as a dongle and just let it run. I want the opposite from a 4U rack supermicro nvme epyc box.

The network port is limiting this to 1Gbps / 100MB/s, and it’s just a bunch of files, average a few megs in size that will be stored - not unlike photos really, probably smaller.

Which microSD card should I use, and what filesystem should I use to minimize the risk of data corruption (really bad and annoying to deal with), or just flat out dying (not as bad)?

In terms of load, I’m thinking maybe between 1GB - 5GB per day of both reads and writes + whatever filesystem metadata stuff.

I’ve had an ok experience running Home Assistant from a Samsung Endurance Pro card- not the fastest, but works.

I also now see Kioxia Exceria High Endurance cards

Are any of the cheaper cards or bigger cards (512G+) good enough to run somewhat of a storage workload on them?

My choice would fall on a Sandisk Max Endurance and use F2FS filesystem to preserve it from immediate death and because has good wear leveling algorithms that you won’t have access too since it’s not an SSD (no IC to do so).

But, a cheap SSD Kingston KC600 is what I would really use because it will outlast a micro SD by a factor in terms of writes before dying and the 512GB costs just as much as a reliable micro SD. Ext4 will work well enough with an SSD.

1 Like

A 1TB SSD costs 100 bucks these days and chances are it comes with an external enclosure that offers USB-C speeds. Probably your best bet, IMO.

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.