Raspberry Pi POE Alternatives for K3s

Hardware Hub Other Hardware
,
1

Long-time no see Forum,

How’s things been going?

TL;DR: I want to set up a Low Power HA K3s cluster that is powered by POE. Noting you can’t get any Raspberry Pi’s right now, what would be my best alternative to the Pi and POE hat for a K3s Cluster?

So I was wanting to see if anyone else has done, or looked into something similar to, a learning project that I want to undertake.

I want to set up a low power High Availability K3s cluster at home that is powered by POE. I currently have a whole bunch of services that I am running on unraid, as well as pi-hole on a RPI4 (bought way before the shortage), and I thought it would be fun/a learning opportunity to set up a HA Cluster on a bunch of Pi’s powered by my POE Switch that I can move all of these unraid services to (after extensive testing of course).

My first problem is that there is a significant shortage of Pi’s, especially the one’s I am after, the 8GB RAM (CM4 or B+, I figure more RAM the better for k3s?). So I’ve been searching for alternatives. I can get the Rock Pi C+ (4GB Ram) for a reasonable price but I’ve got no experience with it and don’t know anyone that has used it before either, so I’m uncertain if it will work with standard linux (I was thinking of putting the cluster on either base debian or arch, an arm specific version of these os’s of course). I also can’t find the official POE hat for it for sale, the documentation for the official one say’s that it also works on the Pi so I’m also wondering if the official Pi POE hat will work on the Rock Pi C+ …?

The Services I want to run under K3s are:

  • Nextcloud
  • Pi-Hole
  • Overseer and associated other programs …
  • Jellyfin
  • Heimdall
  • Ghost
  • SWAG (or something better so that I can safely access my services outside my network)
  • Matrix
  • Mastodon
  • Searx
  • Probably some type of database (MariaDB, PostgreSQL, etc)
  • Probably more that I can’t think of right now …

Does anyone have any recommendations for a low power, POE powered, device that can do K3s and the above services? Will the Rock Pi do the job with only 4GB of RAM? Will the standard Raspberry Pi POE hat work with it?

Thanks Heaps Level 1 Forum, as always you guys Rock my Pi :rofl:

2

RockPis are generally more available and yes I believe they can handle Kubernetes. That said have you considered small form factor systems?

3

I have, but they do use significantly more power than arm, with power prices going to the moon I was wanting to try and save a bit cash (and also cables with Poe). I was also planning on a 7 system cluster.

Any experience with the rock pi amd Poe hats? Is 4GB enough for the services I want to run?

4

I really don’t have experience with POE hats. you might have to wait until someone else who does have that experience can chime in…

If I am being realistic with you… most of the time your system will be idling and when you look at semi modern CPUs at idle… they barely sip. Someone on server builds talked about this in-depth as did craft computing and Wolfgang (linked)

Youd be surprised. Most of the power you pay for is idle time … thus realistically the difference between a 45W or even 65W CPU and a 10.5W ARM board (Total Board Power)… isn’t all that much

People who go after ARM for the power usage are kind of falling for a meme and I don’t mean that facetiously. People who go after ARM for the architectural advantages and security advantages (they don’t deal with as many side channel attacks) have a legit reason.

5

So last night I took the opportunity to have a look around at SFF pc’s like @PhaseLockedLoop suggested. I ended up finding the LattePanda Delta 3 (Intel N5105, 8gb RAM, 64GB eMMC, AU$478.50 each delivered) and the zimaboard (Intel N3450, 8gb RAM, 32GB eMMC, AU$340 each). I also found the cheapest NUC from a local reseller which is bare bones, built up it would have the following specs (Intel N5105, 8gb RAM DDR4-SODIMM, 128gb sata m.2, AU$369 each) (all $'s inc tax’s, local and import).

I was surprised to see that the panda consumes the same power at idle as the Pi, and also only draws about 4 more watt’s at max for more than quadruple the performance. That one really shocked me. (Pi = ~3w idle, ~5.5w under load. Panda=~2w idle, ~10w full cpu load, ~15w full cpu/gpu load). Given how much more powerful the N5105 cpu is (CPU Mark Pi= 834, N5105=4081) the panda’s cpu would likely spend more time at idle than the pi’s? This is honestly something I never considered. So basically twice the power draw for 4 times the performance and more time at idle.

From what I can tell the Panda is more efficient at idle than the nuc, by about a third (uses 6 - 7 w at idle), even though they have the same CPU. I’d love to go with the Panda for the size/form factor (I can rack mount them in a custom case/custom rack if I wanted), usb-c power so I could use an Anker power usb power brick to run multiple from one power point, and bonus Arduino for other projects, but the AU$ is stupid weak right now for some stupid reason … :frowning: . Also the NUC has SODIMM for RAM expandability later, but would I really be needing to expand RAM in a Kube Cluster?

The other option, which is also cheaper, is the Zimaboard. Problem is it’s half as powerful (CPU Mark 1953) and half the storage, but not half the cost, its only AU$30 cheaper than the NUC too. It does have a second Ethernet port, but I would not likely use that unless I was to fully replace my router. Given that I’ve got a gigabit down connection, the router would need to have at least 2.5gig nic to allow for some headroom, which it doesn’t, so I guess it’s out all round …

What would you guys do? Am I barking up the wrong tree here or am I on the right track? (keeping in mind that I’d probably need 5 of these to start my cluster, which is quite the investment for a home lab… at least for me anyway …) is the AU$400 premium for the panda worth it over the nuc???

6

i have been thinking along the same lines. i am not going to build anything right away. if you want some real power for VMs and containers, Wendall is considering purchasing 100 Gigabyte 12400 motherboards to sell around $100, that use DDR4 and the gen 12 i5 12400. this idles at 65W but that is the fault of the MB and can be configured to use less.

i am thinking about the Odroid H3 which idles at 1.9W (?) but Suspends to 0.6W.
max out at 18W
can use m.2 storage, maybe thru PCIe?

7

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.