Mostly due to my work bringing up Pi CM3 on a custom baseboard. Do note I did it in the Pi3 era, and Pi4 might have changed things. Several points:
Bootloader - it’s closed source, there was some work on porting U-Boot, but it died due to unavailability of a compiler (for Pis up to 3, the bootloader was run by the GPU, not sure about Pi4). For the most part, it works, but trying to do something non-standard meant trying to figure it out with the minimal documentation provided by the Foundation.
Device tree overlays - a major part of bringing up SoMs on custom baseboards is writing the device tree. Normally, I would take a device tree from the SoM’s vendor, for their reference baseboard, and edit it to fit our board. For Pi, the official way is to create overlays - small bits which the bootloader patches into the device tree before passing it to the kernel. IMO overlay syntax is absolute crap compared to straight DTs, and, it was harder figuring which node referenced what hardware, iirc also due to lack of docs.
Googling for anything was an absolute hell. I would get overloaded with results from hobbyist with problems I have already passed, or who took the quick and dirty solution I could not. The rare post from a fellow professional was for similar, but unrelated issues, or for stuff which indicated they are at a later stage.
The Foundation itself doing stuff which I found questionable. /dev/gpiomem
is an absolute joke - exposing peripheral registers to userspace? Without the intervening driver? Because people complained their software PWM on a non realtime OS was wonky? I’m guessing their GPIO driver must’ve been mostly stateless for this to even work.
Oooh… and the utter lack of peripherals - our baseboard had, iirc, four peripheral expanders (stuff like SPI to UART converters, USB Ethernet, even a GPIO expander). This mostly annoyed the electrical engineer designing the baseboard, but I worked closely with him.
If you’re asking about NICs, we don’t use dedicated NICs. The MAC is part of the SoC and we just add the PHYs - usually something from Microchip’s KSZ line (the one they got from acquiring Micrel).
If you’re asking why dual ETH? It’s one of two things:
- The Linux device is some sort of gateway between two networks.
- Daisy-chaining - industrial networks often don’t have much traffic, but the cable runs are quite long. Because of this we’d have a switch on the baseboard, to which we would connect the SoC, so that you could daisy chain the network.
Speaking of daisy chaining, Texas Instruments’ Sitara SoCs (like the one in Beaglebone) have a great little 100 Mbps dual MAC, which can be reconfigured in the field between two independent ports and a switch.
Edit:
If the Pi sections reads like a rant, that’s because it is - I still have strong emotions about this, five years later.