Help and Recommendations Needed for "International" Home Network

Hardware Hub Networking
, , , ,
1

I usually lurk because there are plenty of other recommendations that work for me here, but I’ve got a bit of a specific problem, with a time limit (~1 month from now), that I’d like to solve.
Plenty of different possible options though, so if anyone can give me some recommendations, I’d greatly appreciate it.
Oh, and this will probably be pretty long since there’s a bunch of stuff to cover. Apologies in advance.
I’ve split it into sections for that reason, so if you don’t want to/can’t address all the sections, please feel free to pick and choose.

Brief
  • I live in New Zealand, and I’ll be moving to Japan for work in just over a month (maybe permanently, not sure yet).
  • I have NZ$500 (let’s say roughly US$250) to buy any hardware I need before I go.
  • I will be leaving my desktop PC at home (specs below) and want to remotely access it with an IPKVM (so that I can turn the PC on and off remotely).
  • I’ve been trying to set up a home server on a Raspberry Pi “cluster” (not a real cluster, it’s just four Pis next to each other for now) because that’s the hardware I have available to me. Currently, it’s only partially working (more details later).
  • I want some level of network (mini)rack and home server setup to sync my data privately, and to filter my data through my home network (Pihole, etc). Currently, I just use Syncthing between my three main devices but it only syncs on my local network.
  • I also want to host my (business) emails at home. I know all the caveats to this but it’s the most cost-effective method for me right now. I set up Mailcow under Docker on the Pi 5 4GB (more on this later) though it’s not properly configured right now.6.
Current hardware
  • Desktop PC: R9 9950X3D/RX 5700 XT/MSI MAG X870E Tomahawk WiFi/32GB @ 6000 MTs RAM
  • Monitors: 2x 1080p60
  • Laptop: Surface Laptop Studio (Gen 11 i5, 16GB RAM, iGPU only) - will be taking this with me
  • Phone: Samsung S23 Ultra - obviously taking this too, probably not important but thought I’d mention it just in case
  • Raspberry Pis: Pi 4 2GB (w/official POE HAT), Pi 4 4GB (w/official POE HAT), Pi 5 2GB (w/128GB SSD, currently no POE HAT), Pi 5 4GB (w/256GB SSD, currently no POE HAT)
  • Router: * MikroTik RB5009UG+S+IN (the non-POE version)
  • AP: * MikroTik cAP ax (seems to have terrible range compared to my old ASUS DSL-AC68U modem router, can be spotty at times in certain areas of the house)
  • Switch: * MikroTik CSS610-8P-2S+ (1Gb + POE switch, connected to router via 10Gbps SFP+; for powering the Raspberry Pis)
  • Switch: Cisco C2960 (it was $5, it’s good for Cisco revision, and I find IOS easier to configure than RouterOS, but it’s got no POE so it’s probably only good for learning for now)
  • 1Gbps powerline adapters to connect my desktop PC over Ethernet (it’s connected via Ethernet and Wi-Fi), because my house isn’t wired for Ethernet.
  • IPV4 only - my ISP doesn’t support IPV6 (but unlike almost all other ISPs here, they don’t have CGNAT, and I unofficially seem to have a static IP, so I’ll take what I can get)

Here are the areas that need addressing:

  1. Network rack
    • My original idea was to buy a 10” minirack - something like the DeskPi racks.
    • I don’t really have space for a full-size rack right now, though I might be able to make it fit. It can’t be too tall - maybe 12U max, roughly.
    • I need it to house all 4 Raspberry Pis. From my understanding, it’s going to take 2U of space in a 10” rack, or 1U of space in a 19” rack, no matter the bracket/adapter I buy.
    • The rack will also contain my router and switch. Both should take 1U each in a 10” rack, and should fit side-by-side in a 19” rack.
    • Ideally, I want a keystone panel to make wiring easier.
    • I do NOT need a UPS. I know it would be ideal but I don’t have the budget and nothing is mission critical enough to demand it right now.
    • The rack doesn’t need to be particularly portable - I have three printers and the AP wired into my router (might put some of them through the switch in the future), hence the limited space.
    • I see there are Tecmojo racks that are basically the same as DeskPi but cheaper. From what I’ve read, they seem okay. Maybe there are other options.
    • I don’t have a 3D printer but I can probably get someone to 3D-print stuff if that is definitely the best option.
    • The DeskPi T1 and T2 are about $5-$10 difference right now, and cost about half my budget from Amazon Australia either way, so that’s quite steep considering my limited budget.
    • Also relatively pricey, but the [1. * Waveshare Homerack seems like it could be a decent choice too. Or maybe that one would be good for Japan in the future, since I can get caster wheels for it, but don’t really need wheels at home.
  2. IPKVM
    • Needs to have ATX control so I can turn my PC on and off remotely.
    • Doesn’t need a screen (because I won’t be here to see it).
    • Doesn’t need POE because I don’t have it available in the same room as my PC.
    • There is space in my case to sit it internally, though perhaps a PCIe form factor might work better.
    • It needs to be reliable enough that it shouldn’t have any major issues while I’m away, because there won’t be much troubleshooting I can do (my parents can, but they’re tech illiterate, so it has to be very basic such as “turn it off and on again”).
    • Security is a concern, but the price is arguably a bigger deal. The JetKVM, for example, is roughly half my budget, which is probably more than I want to spend, but it seems both more reliable and secure than the NanoKVM, despite the latter being a far better cost.
  3. Remote network access
    • I want to be able to use NZ services from Japan, whether it’s my Pihole setup or NZ-only streaming services (I probably won’t use them, they’re pretty crap, but it’s helpful to have the option to switch geolocations without a commercial VPN).
    • When I was last in Japan, I used a Wireguard VPN set up on RouterOS to keep my data secure on the go. At some point after I got home, I broke it. It’s fixable, but I’m not sure if it’s worth it.
    • Since I need to buy a router for living in Japan, I have considered getting another RB5009, or perhaps a cheaper option like one of MikroTik’s hAP devices, and tying it to my home router with OSPF. I’m not sure if this is a good idea though, since I’ve only set up OSPF in a lab on Cisco routers, and perhaps this could cause issues creating an “international” home network.
    • Tailscale (and similar) is obviously a frequent recommendation. I’ve only used it briefly but I’m not sure if it’s ideal for what I want, and a self-hosted option seems more secure to me (assuming it’s configured properly, of course).
    • Out-of-Band Management for the router would be ideal just in case, but it seems I need USB→rollover then rollover→USB to do it anyway. I have a WoobM device for my router (basically, an OOB device that has its own 2.4GHz WiFi hotspot) but I don’t know of any way I could use that remotely. This probably isn’t a big deal, assuming everything continues to function fine.
    • Maybe something else I haven’t considered?
  4. Home server
    • I have my own domain name, with my website hosted on Netlify. I can add subdomains, which is how I want to access services (for example, I want to use mail.mywebsite.com as the webmail portal and IMAP/SMTP server address).
    • I want to be able to access the router and other network devices even when I am away from the network (without traditional WAN access, since I want to keep as much as possible off the internet).
    • Each Raspberry Pi is currently running the latest Raspberry Pi OS (Debian 13).
    • The two Pi 5s don’t have POE HATs yet, but they both have the official M.2 HAT. I would have got the HackerGadgets ones but they’re back-ordered until December, and would eat half of my budget.
    • All Pis are running Docker, with different services on each.
    • Currently, the Pis are not doing a lot, because they seem to stop responding over SSH when both of the Pi 4s are turned on via POE (turning off one seems to fix this).
    • The Pis have different SSH ports, and their IPs are statically assigned through DHCP leases on the router, so this shouldn’t be an issue. I haven’t looked into it further yet.
    • I’m wondering if it would be better to group the 4 Pis together with Kubernetes (I’m still very much a beginner with Kubernetes though) to make them easier to manage.
    • The 4GB Pi 4 is to be the reverse proxy.
    • The 2GB Pi 4 is to be a monitoring device (SNMP, Promotheus, Grafana, maybe more). I have a Pi Pico with a Waveshare v4 e-ink display that I’d like to use to display some of this data (or just act as a networked clock GUI for the rack), but this seems a bit complicated, and my programming skills are… not great.
    • The 4GB Pi 5 is my mail server. It currently has Mailcow installed but it doesn’t send and receive mail correctly at the moment (I think it’s to do with port conflicts and mapping).
    • The Mailcow web portal works, but it takes over ports 80 and 443, meaning I can’t access the reverse proxy (and therefore can’t access my router) from the web. It also means I can’t access webmail from my home network, as my router uses ports 80 and 443 for its web interface (which I would prefer to keep enabled). I have attached the Mailcow config just in case.
    • The 2GB Pi 5 is currently pegged for a Syncthing Discovery server, so I can use Syncthing between my devices even when I’m not at home. Eventually, I’d like to combine this with Nextcloud, but that might be too much for any of these Pis right now.
    • I have attached the docker-compose.yml files from all four Pis (changed the extensions so I could upload them). They’re obviously not all finalised.
    • SNMP doesn’t seem to work between the Pis and the MikroTik equipment. The switch in particular doesn’t support a lot, and seems not to actually communicate anything, as it’s limited to HTTP-only, SNMP v2 (no auth, no priv), and uses SwOS Lite.
    • I’m configuring SNMP with snmp_exporter so I’ve just included the generator.yml file for simplicity. It did generate “correctly”, but doesn’t seem to do much, as above.
    • I’ve also included my promotheus.yml config, though Grafana doesn’t seem to be receiving anything at the moment.
    • Finally, I plan to set up VLANs on my network, but it’s not a priority - I can do it easily enough on Cisco IOS, but the few times I’ve tried it on RouterOS, I’ve broken everything. RouterOS VLANs are simply… not simple, it seems.
    • A lot of this stuff is fiddling with subnets and ports, so I mainly just need to be pointed in the right direction for now: is this a good approach, or should I change my approach?
  5. New Computer
    • Simultaneously the easiest and hardest: I need a computer that is somewhat powerful for living overseas, but not a big ATX machine (as much as I’d like another). This is NOT part of my $500 network budget - it’s something I’ll buy when I get to Japan.
    • Pricing is obviously a disaster right now, but I want more than a laptop. I’m not expecting to be able to match my 9950X3D, but I want something that can at least handle some video editing (Premiere or DaVinci, and I own Final Cut Pro X on macOS, so that’s an option if I go back to a Mac), as well as Photoshop, and preferably something that can handle gaming (doesn’t need to be high-end, since I’m used to the relatively underpowered 5700 XT - I don’t play many brand new games anyway).
    • I can use any OS, though I’d prefer to get away from Windows. A Mac Mini seems like the best bang-for-the-buck right now, but I’m hesitant to give my money to Apple again, for hopefully obvious reasons.
    • My salary is pretty crap, so I don’t want to spend more than about NZ$2000, maybe $2500 - roughly US$1000-$1800, let’s say.
    • Will likely buy a separate monitor (doesn’t need to be anything fancy), as I want to use it for game consoles too (Switch and possibly some other stuff).

Phew, that’s a lot.

I’ve been spending many, many hours doing my own research, but having a limited budget and trying to stretch it to fit everything (except a new computer, of course) is quite difficult to figure out, especially when I can’t realistically test anything before I buy it, so I’ll be stuck with whatever I buy.

If anyone can help me on any of these points (especially the first four), I’d greatly appreciate it. I’ve got a lot to sort out in a month, so I could really use some help please!

prometheus.yml.txt (2.1 KB)

mailcow.conf.txt (10.8 KB)

generator.yml.txt (916 Bytes)

docker-compose_pi5_4gb.yml.txt (4.1 KB)

docker-compose_pi5_2gb.yml.txt (1.2 KB)

docker-compose_pi4_4gb.yml.txt (839 Bytes)

docker-compose_pi4_2gb.yml.txt (2.1 KB)

1 Like
2

If you have a home mail server i presume you have a public ip so you can just open a port to run a vpn.

Thats all there is to do, and wireguard is fine, but you can also make an openvpn, so choose the right article from here Virtual Private Networks | RouterOS Manual

As for ipkvm, your best bet imho is to cannibalize the pi4 2gb and use it for a pikvm or a blikvm. Blikvm v2 pcie and blikvm v3 hat (the blue box) are basically cheaper hardware clones of pikvm v3 box and you can use the pikvm image on them. Check their docs to be sure but i have the pcie version and it runs pikvm image with no modifications. Both need a raspi 4, they are basically raspi hats

1 Like
3

I want to suggest Netbird/Tailscale/Zerotier. These are three options that mostly do the same thing, an encrypted overlay network that lets you access computer joined to the network or you might even use one computer in a given network as an exit node to access the whole network.

I personally have only experience with Netbird that offers a fully self-hosted option but all three of these have SaaS offerings that are mostly cheap for home users if you do not want to self-host.

Netbird also gives you the option to expose services in your network to the public internet.

This software does handle the routing, traversing NAT and carrier grade NAT on their own. The only computer in the entire network that needs a static IP address is the control server if you self-host. If you pay a subscription to use these services the companies take care of the public IP for you.

If you have an exit node, like a RPI or something in your network you can configure it in a way that as long as it is online you can access the network.

1 Like
4

This is what I do.

I have CGNAT, so no public IP for me. So I have a $5 month VPS in Linode that runs Wireguard. The Mikrotik routers have Wireguard support built in, so they make a connection to this, along with my phone, laptop, everything else. So everything is on the VPN 24/7. It’s been really stable.

I can also port forward from this VPS down the VPN to my home server to provide a public services hosted on my home setup, if I wanted to.

For an KVM, if all that is needed is rebooting, a smart plug with a web interface (ESPHome) will do that. Set the computer to auto turn on after power failure, then in the Mikrotik, port forward a port to the HTTP interface of the smart plug. So if your home computer stops responding, from your phone, goto 10.8.0.1:8880 and that goes down the VPN to the smart plug and power cycles the computer power. You can also run HA on a Pi and all the other good stuff, but this is the easy way to do it with the least dependencies.

I run my own email server, but even when I had a public IP, hosting on a home IP caused issues with everyone spam detection and emails got rejected a lot. So I host it on a VPS in linode.

Any web based internal services work like this.

I access https://jellyfin.mydomain.com on my laptop → 10.8.0.1:443 (home Mikrotik) → Home server (could be a Pi) with HAProxy that looks at host name, and forwards on the traffic to the related back end service. HAProxy also handles the LetsEncrypt.

I could use the same VPN to RDP into my Windows PC if I wanted for remote control. It would just be a port forward down the VPN tunnel.

1 Like
5

Thanks for the suggestions, everyone. Definitely some helpful ideas here. I’ll try to address them one-by-one.

Sorry for the late reply. Was busy last week and now sick this week!

@starshipeleven Yes, I have a public IP, and I tied it to my domain name (via a subdomain - currently mail.mywebsite.com). It’s not officially static but I’ve had it for many years and it seems to have never changed. No CGNAT either - one of the very last NZ ISPs that doesn’t use it.

I did consider the BliKVM but the current PCIe version is out of my price range right now since it requires a Compute Module, and my spare CM4 is literally falling apart. Otherwise, the BliKVM PCIe would be a no-brainer.

The HAT version might be a viable idea but then I’d be short a Pi 4 for my fake cluster. I did consider getting a cheap mini PC and throwing Proxmox in it but I’m not sure if it’s worth trying to squeeze that into my budget at this point.

What about the NanoKVM Pro PCIe? It works out about the same price as the BliKVM HAT V3 right now, a little more than I wanted to spend but cheaper thanks to the AliExpress sale that started today. I considered the regular version since it’s much cheaper but it seems the lack of RAM really starves Tailscale, even when manually adjusted. Apparently you can run the PiKVM image in that too - whether or not it’s worth it over the NanoKVM image, I’m not sure.

@H-i-v-e That’s what I’m leaning towards, yeah. I’d normally go for self-hoated but since I have four weeks left and plenty of other stuff to do, Tailscale might be the de facto choice right now just to eliminate issues short-term.

RouterOS can use Tailscale under Docker now. It’s how I run Pihole. My USB-SSD enclosure failed though (after about three years, and by NZ law, the retailer should replace it because that’s really not very long for such a product, but that company is notorious for breaking that law, to the point they’ve been fined before), so I’ll have to get another one before I can set up Tailscale on it.

I’d normally avoid that on the router itself but it’s just easier right now. Else I could probably use the reverse proxy Pi as an exit node.

I definitely want to be able to access my router while overseas, just in case.

@Porch I did consider using a VPS. The ongoing cost is something I’d prefer to avoid but there’s no denying it’s more convenient. I used to use Google Cloud for my mail server, until they wiped it after charging me some exorbitant fee I didn’t pay for the supposedly free tier, which they claim never existed (even though it did) and instead was just free credits (it wasn’t, none of my credits were used).

I’ve used Linide before but it seems like it’s been acquired by Akamai and changed significantly so I’m not sure if it’s still the best option or if DigitalOcean, Vultr, etc. might be better. How do you handle SMTP? Do you need a third-party relay, or are you able to use port 21 on Linode? If it’s for my mail server, the small ongoing cost is probably worth it. I’m just not sure that I want to deal with relay server headaches again.

Somewhat unrelated but I wanted to set up InvoiceNinja too, so clients can pay me via my website (basically a separate instance pointed to something like invoice.mywetsite.com), but I guess it would make more sense to run this in a separate instance to the mail server.

You’re right that Mikrotii has Wire guard built in, though I admit I had trouble configuring it initially, and never got it working after I broke it when I got home from Japan previously. I need to look at it again but I think I was getting confused with what IPs to allow, what DNS to use (wanting to use my Pihole when connecting to the VPN, but then that was on its own subnet, and that was causing issues too), etc. Maybe it would be easier to go with Tailscale for now, else I can try to hash out the specifics of Wireguard configuration again.

I did consider your idea of a smart plug, but ultimately, I want to be able to use the BIOS in case of any issues (Linux secure boot, for example) and any other boot-time operations (i.e. anything I might want to do before applications are started to connect with RustDesk).

I’ve set up Home Assistant on a Pi CM4 for my brother, but I probably won’t do it myself for now. I don’t have much in the way of IOT devices outside of the mountain of Raspberry Pis and a few Arduinos, but it might be something worth setting up when I get to Japan. Who knows, it might even be able to control the fancy toilet! :laughing:

I’m ditching Windows (again) and will probably run Tiny11 or some such under a hypervisor for the things I still require, as my Windows installation continues to break itself apart (long story), so I’ll likely be moving back to Arch Linux full-time. What do you use for RDP? I’m currently using RustDesk but it’s a bit slow and seems to use some Chinese servers, so I’d like to move away from it. TeamViewer seems untrustworthy, and traditional VNC seems perhaps a bit insecure, but maybe they is exactly the way to go.

I can always do VPN>SSH, since I have SSH on LAN (not WAN, of course), and it’s locked down fairly tightly so it should be okay.

Rack-wise, I think a 10” 8u rack would be perfect, but it’s about $5 difference between the DeskPi 8u and 12u at the moment, and expensive either way. I’m considering just buying a Tecmojo one, getting the supposedly superior patch panel from DeskPi, and maybe taking some solvents to the stupid Tecmojo logo. Anyone got experience with any of these racks?

6

I maintained a server + network at an overseas home that sometimes went unattended for more than a year at a time. One bit of advice I have is:

  • Configure your devices to power on after power loss. That way if power is lost, the devices will restart once power is restored.
  • Get a “smart power” switch so you can remotely cut the power to your PC to force a hard reset.
  • Configure your router to automatically reboot on a schedule, hopefully if you run into some bug that disables your router this will eventually reset it for you.
  • Do not plug your router into the smart switch!
  • Having a camera that looks at your hardware can be helpful. If you already will be having some sort of surveillance camera setup, just have your PC and router in visible range so you can confirm power/activity LEDs.

For syncing, I was using Synology at the time and just shipped snapshots from my active location to the backup location (and I switched active/backup when I changed physical locations). There are numerous ways to handle synchronizing data effectively, it is having a plan to handle problems that proved more important to get right. If you have someone that you can call and have them go and reboot or diagnose issues on the computer, then you may not have to worry too much.

1 Like
7

My mistake, not a blikvm, I got a couple of these https://it.aliexpress.com/item/1005004975293077.html

1 Like
8

Thanks, that’s quite interesting. Definitely cheaper than the other options. I can’t find much info on it other than from Geekworm themselves. Any idea how it compares to the NanoKVM Pro? Aside from having more RAM on the Pi (not sure if 1GB vs 2GB really matters), I’m not sure. True Gigabit vs. “Gigabit" tied to the USB bus on the Pi, I suppose, but I’m not sure if that matters.

Still not sure about pulling out the Pi 4 2GB vs. using it in the cluster though.

@brandon.c That’s useful info, thanks. I’m actually still living at home until the end of the month, so my parents could always hard reset it for me if necessary, though the smart plug would definitely be more convenient. Not sure if it’ll play nice with a powerline adapter though, especially when I already have an 8-plug powerstrip on top (not ideal but don’t have much choice).

I’d probably leave the PC off after power loss like I do now, but all the other devices are set up to restore after power loss already, so that helps. I used to do auto-reboots of my old router, so I guess I can configure this one to do it too.

Don’t have any cameras set up, unfortunately. If I had more time, I might be able to set up a Pi Zero with a camera via Motioneye but it’s just not really practical right now.

I don’t have a NAS, haven’t managed to budget for one yet (especially with all the price increases lately), so I’m thinking that using one of the Pis to run a SyncThing discovery server to just sync between devices might be the easiest. Not sure if there’s a better approach I’m missing that I can take with the limited hardware I have. I have a spare 1TB 3.5” HDD, but that’s pretty much it right now.

9

I have had no issues with Linode after Akamai acquired them. The website changed, but that is it. No issues with opening up port 21 and running SMTP. I have been a happy customer for over 10 year now. For the $10 bucks a month I am paying for that VPS, it’s worth it not having to host that internally and deal with all the networking and security issues of having a public service inside my network.

As for DigitalOcean, when I was an email admin at my last job, DigitalOcean was the top single sender of phishing and spam emails. It got to the point we setup custom filters to drop emails from them into a special spam bucket by default. I don’t know if anyone else was doing the same, but their subnets are on most of the email blacklist, so it was not just me dealing with them.

1 Like
10

Really useful information, thanks. Sounds like it’s worth moving my mail server to Linode then. Makes it easier too since I can set that up anywhere, so I don’t need to rush to do it before I leave.

I might go for the Geekworm PiKVM option in that case, but I’m curious if there’s a Pi 5 version. That might be overkill but also a little better for what I want. Haven’t managed to find a PCIe version for Pi 5 though. Still not sure if the hardware is better vs. the NanoKVM Pro. Seems like a bit of a wash.

EDIT: Turns out there’s no Pi 5 support for PiKVM (yet). I have two screens at 1080p60 at home, though I guess I only need one to be output to the KVM. Hopefully the PiKVM option is enough for that.