Phaselockedloopable- PLL's continued exploration of networking, self-hosting and decoupling from big tech

Have you checked out diagrams.net? Saw it on a Lawrence systems vid, it came off as almost too good to be true but with the limited use I have so far (installed it on Windows) its pretty neat.

time to test it out ehh?

What limitations? (number of shapes?

ROFL reached your limit?

I don’t recall any particular limitations- it came off with a libre office vibe, free. I think their business model is the webhosting/cloud service aspect of it if one chooses to have that service.

1 Like

Ahh its Draw.IO on nextcloud

Interesting

1 Like

fuck

If you got the money, why not get a stackable switch and make that redundant, lol. But I doubt the switch is that high risk as a SPOF. I’d rather get 2 more servers and implement HA for the Hypervisor. I’m guessing you’re running XFCE because of RDP / VNC, you could try JWM with PoorMan’sTillingWM if that’s more to your liking, last time I tried, JWM worked with VNC.

The first thing I would do TBH is remove the services from the barebones Pis, make a HA LXD cluster and run the services in containers, so if some Pis go down, your services just get moved on other Pis. It would be the most cost-effective removal of SPOFs, just add a small NAS (2x 500 GB in RAID1 should be more than enough for everything there on the right side, but considering you already have the SSDs, just buy 1 more 120GB SSD and do a RAID-Z2 on a separate box) and make it the storage for the containers. Later on, just add a separate NAS and enable replication of the services. I understand you will be moving the SPOF from the Pis to the NAS for a while, but a NAS should be more reliable (technically) than RPis. And the 2nd NAS doesn’t have to also be RAID-Z2, you can just put a RAID mirror with 500GB SSDs, just enough to replicate what’s on the other one.

If you are running Fedora on the Pis, it should be pretty straight forward to run LXD. I’m doing it on Void with no issue (although I didn’t have the time to set HA on the cluster, I’ll move soon and I have to get more RPis or similar after I move).

already have some raidZ2 arrays… probably will use that if I need more

https://lxd.readthedocs.io/en/latest/clustering/

Interesting. I will look into this…

I prefer the Pis im trying to run ARM as much as possible with existing hardware. The pi4s OC’d to 2.1 GHZ and with SSDs are stupid… stupid fast

This is a good idea. Maybe a 16 RPI4 cluster? 64 ARM64 cores… lol. Thats basically a damn server LOL. Im going to need different power delivery … KEK

They all have large passive heatsinks on them (see somewhere above in the thread)

1 Like

For the main / host OS that would run LXD, SD cards are just fine. But SSDs will definitely make a difference for the service that run inside LXD. OC’ed means even “more better.”

Understandable. I also want to get away from x86. Check out the Wiretrustee SATA and also this article.

At this point, a Gigabyte ARM Server with Marvell-ThunderX2 makes more sense. Maybe 2 for redundancy with 1 Pi for tie-breaking.

2 Likes

well see I already have 8 (some not in diagram) so 8 more would be less than the marvel thunder

2 Likes

Following up Biki … and @Dynamic_Gravity you might as well be included in the loop since we spoke of SPOFs

Im going to attempt instead of multiple NGINX handling the talking of the nodes to each other…

Im going to instead push for this

Which means Im going to have to figure out multiple tenancy on wireguard on OPNSense

The planned centers: (reason is I travel globally alot. and thats kicking back up)

Budget 40 dollars a month max. Im think 5x$5 linodes? and a balancer (Closest Hop not Round Robin)

The question really is how much sense does it make. All of those locations would just be reverse proxies and ultimately would have to route back. I guess making for the closest proxy location is a form of optimization right?

I travel a lot within the US and europe so I was thinking concentrating the server/reverse proxies there… 2 in NA 1 in Europe 1 in AU and 1 in Asia

Also if any one server or data center goes down it doesnt take anything out

But the question is do NodeBalancers also have a location or are they mirrored to all locations. I really dont know if I am doing the right thing here


An alternative route is forgetting that and upgrading the server to Dedicated CPU… but I really dont think that would net much benefit

1 Like

You probably know I love frugality and scaling down to the bare minimum and KISS, so my answer may be a little… unsatisfying.

If the NodeBalancers have to make the trip back to your main server, then this will only make sense if Linode has leased lines between their locations. Not sure if that’s the case, but if it is, you will probably see a difference in speed or latency. And traffic has to travel through their leased lines and I doubt the users are allowed this privilege (that’s usually reserved for in-house traffic, like management or really latency-sensitive communications).

Another case in which it would make sense is if those NodeBalancers are caching webpages and requests and stuff. I have no idea how they work though. Reading a little about it, it appears they are just basic load-balancers. So just like any LB, these things only make sense when you got loads and loads of traffic going through your server (1k+ connection at a time from each NodeBalancer). Are you selling stuff or own a very popular website? I don’t think this makes either technical, nor financial sense for 1 user.

But if security is the issue, well, like any LB or reverse proxy, it is an added security feature. NodeBalancers communicate with the Linode web servers through private IPs (and if you’re on a limited bandwidth in Linode, this will help a little), so the main web server is not directly exposed to the internet.

I read there are some issues with hosting NodeBalancers with SSL on the web servers. You can’t get the real IP of the source, just the IP of the NodeBalancers and some websites don’t show up correctly (the steaming pile of garbage that is WordPress). To avoid this issue, it’s recommended you keep the SSL certificate on the NodeBalancers and have unencrypted traffic from them to your web servers. When it’s your own infrastructure, this should be fine, but I don’t know how it works in Linode, are you the owner of the NodeBalancer, or are they shared among multiple customers? If you want SSL on both the load balancer and on the web servers, HAProxy is the better solution.

And again, it’s certainly a cool factor to add redundancy and load balancing to your services, but if it’s just for you, this doesn’t make much sense. You can play around with VMs / containers in your own home lab and set things up as if it was a large corporate intranet. Paying additional money for 1-100 people doesn’t make much sense.

In the end, most likely you won’t see much of a difference in performance. It might add some privacy if you are connecting via the clearweb to your Linode web server, your traffic will show up as connecting to a regional server, which then goes through Linode’s VPN to your main instance, instead of you going directly to it. But then again, a reverse DNS lookup will show what you were connecting to, so kinda a moot point. If privacy is a concern, you can make a script to spin up Wireguard on new Linode instances and connect to them when you travel (or have them constantly running for ease of use). Bonus points for VPNs is that you can also use the private IP of your Linode instance to connect to it, so you also save on bandwidth costs.

Again not worth the investment. Unless you are serving customers that demand your service not be dog slow and consequently you or both of you losing money, shared resources are better. It’s not often that services are pegging the CPU constantly (and if they do, sometimes VPS providers move the demanding VMs around not really demanding ones).

3 Likes

Just another thing I have been working on. In blog post for index post

1 Like

YAY Series 7 done

Links to Infrastructure Series and Blogs:

Blog: Phaselockedloopable- PLL's continued exploration of networking, self-hosting and decoupling from big tech
Series 1: Infrastructure Series -- Native Dual Stack IP4+IP6
Series 2: Infrastructure Series -- Wireguard Site to Site Tunnel
Series 3: Infrastructure Series -- Recursive DNS and Adblocking DNS over TLS w/NGINX
Series 4: Infrastructure Series -- NGINX Reverse Proxy and Hardening SSL
Series 5: Infrastructure Series -- Taking DNS One Step Further - Full DNS Server infrastructure
Series 6: Infrastructure Series -- HTTP(S) Security Headers! You should use them! [NGINX]
Series 7: Infrastructure Series -- Use NGINX to inject CSS themes

2 Likes

Well, appears I shall need another pi for NGINX… Unless I can run it off my 24/7 PC I use…I can switch over the Linux for everything on it I suppose. (Would use the Ryzen 5 4650G x300 ASRock Mini PC/ 32GB ram… plenty i suppose?)

As a novice I will say, I understand the concept of DNS, reverse DNS sort of, but recursive…not so much. I think I also need to have my E-mail alerts configured properly (I still have yet to learn this), I did get NGINX installed and working properly on a test machine with Fedora as the base OS… I assume I could possibly ru8n a headless distro like debian and install it and get it running as well, but I dont know what kind of resources it will need access to.

For the time being I may have the firewall in a loop to two ports on my switch so I can play with it, see if I can route through it to test its working properly to just this one machine I am on, then move to put it between the ISP modem and the switch to the AP. I do like the idea that @Biky gave me of having 3 VLANs for different trust levels… I myself for my sanity would probably need different IP ranges to keep in straight, then of course see what I allow to cross talk to each other on the switch…Ok head is full now… going to see what I can break. lol

In general I’ll all also have to redo permissions on some of my Samba shares… I could never get them to work how I wanted, or I may wimp out and use TrueNAS in a VM.

But a netmap is forming. yet the wife becons agaion for the 10th time as I write this…lol

2 Likes
1 Like

yeah hes a good resource. Also more currently frequent to the forum. I feel bad that ive taken to an absence but its generally been busy

4 Likes

As I’ve said no worries at all. You do you! Lol I’m having fun bumbling around. :grinning:

2 Likes

I can often be caught on discord more often the level one discord.

My handle is “rear” admiral dox

The issue I have is also that I don’t open the browser that often I don’t live with notifications

2 Likes