But I don’t think there is enough detail to trust this specific implementation, for instance I do intend on copying SD cards with NOOBS installs, and there is no talk about the specific hubs and how their internal trees work nor if these hubs are tested for back current isolation.
Now, I’d want to use a raspberry pi to make it small and cost efficient, but if I have to use an x86 pc I’m fine with that too. Maybe a pcie card with multiple USB outputs?
I’d like to hear some thoughts about this. I’ll probably write my own duplication scripts using native Linux commands, but if there is anything else I should consider please lmk.
Note I’m going after reliability and speed. If I can copy 1 SD card into, say, 7 outputs but it takes several hours then I’m not really at a better place.
You’re way better off doing it through a PC rather than buying a costly device, in my opinion. You can make a bash script that just dd’s one image to multiple SD cards.
I don’t think you’ll need more than a powered USB hub. Just use a USB 3.0 port and you’ll have enough bandwith to write at full speed to many SD cards at the time. Just buy a bunch of these and you’re set.
Maybe doing it with a PC it’s not as “fire and forget” like when you use those SD card duplicators and will surely hit the resources, especially if you’re reading from an HDD and writing to 7 or more SD cards (assuming ~20MB/s writes for each card).
dd if=<input file or SD card> | dd of=<output file or SD card>
One for every copy and you’re done. Though copying to a file and then from a file to SD would be better, because if input is set to the SD card it will directly read the SD, and you don’t want to directly read the SD 6 more times, that will be slow. But that’s it, no coding required. That’s what you would do on RPi itself.
On Windows, I bet you can achieve the same with a simple .NET application. Learn how to get the SD cards as block devices, copy one to another, throw in some WinForms and you’re done. You can make an actually useful application.
Just pick a USB3 hub, that will have enough bandwidth for whatever you need. You won’t flash in full USB3 speeds anyway.
On either hardware platform you can tap udev for more automation. For example when it detects a new flash storage attached, fire the script that copies your source image and then calls eject.
With a Pi you can control some GPIO for status LEDs like run, done, error, don’t even need a monitor.
Maybe if you try to cache the image file somehow in memory, but otherwise nah, dd itself is as simple as it is lightweight. Just be careful about not mixing up inputs/outputs, it’s dubbed Data Destroyer for a reason, it’s as powerful as it is dangerious!
A couple of tips with dd and disks/SDs:
dd if=<input file or SD card> | pv | dd of=<output file or SD card> - pv is a nice utility that measures the data sent and data rate of stuff that’s piped to it, then pipes out exactly the same thing for other applications to consume. So you can get an idea of speed of reading/writing with it. I use it all the time in these circumstances. benchmark your speed with this.
dd of=... bs=1024 - bs=1024 write size, basically the size of the buffer that gets written to file/SD in one go (dd that writes to SD). I’ve found it helpful for performance to tune it to the block size of the SD card. This will ensure optimal writes since with SD, you need to rewrite blocks if they’re incomplete I believe. Best to write whole blocks. Depends on the card, sometimes block length is 512 or 1024. Maybe there is a way to check it for every card individually via udev etc.
You could be able to do it with a Raspberry Pi only if you have the Pi4 4GB in my opinion. Anything less will just eat up hours and hours on end, if it even works at all.
So, I’ve had success so far. I wrote some python over system calls similar to the ones mentioned above. It safely identifies devices in /dev/sd*, unmounts, deletes all partitions (in case you are overwriting an sd), and writes to as many devices as you connect. I want to eventually make a proper gui that is touch friendly to make this device standalone.
The test so far is 6 sd clones simultaneously from my laptop (8gb ram, 8350u, nvme SSD). It takes about 2500-3000 seconds. All SD cards booted properly. The SD cards are the cheapest in microcenter, about $3ea. Byte size is only 64kb I believe this can be further improved.
This isn’t bad considering I couldn’t minify the source images so each card is being completely written (16gb).
This is supposed to ran in a pi, so I’ll run these again in my pi4 4gb. We’ll see how that does.
I spent a fair amount of time trying to properly set visual queues with lights, the problem i couldn’t solve was mapping usb hub ports to SD card adapter to SD card (each adapter has 2 ports to write to simultaneously).
So a single hub has multiple internal hubs, each hub as USB ports, and each port has multiple SD cards (2). The issue is not algorithmic, it’s data. I can’t find key values to map all these together…
I wouldn’t be able to turn a light next to one of the adapters if I don’t know where that sd card is. I need to look more into udev and lsusb outputs but God are they difficult to decipher in detail…
Nope, the wrong value just means you might experience more underlying write commands for the SD card. More write commands = slower. This varies yes, but if does not suffer in performance drastically it’s good enough.
Nah, I always use it by piping though pv so I just automatically use pipes lol.
Interesting indeed. But since you said that you’re using cheap SD cards I’m not surprised. 5MB/s on every card can be managed by a Pi especially the 4.