Return to Level1Techs.com

Trying to selfhost everything

OK so I’m moving into a new apartment very soon, and has a hobby project I’ve given myself the goal of self hosting as much has humanly possible. (Plex/Jellyfin, DNS,Router,NAS,HTTP,GitLab, Security Camera’s, etc etc)

Though what I’m wondering is, should I go for physical hardware for most of the services or should I invest in one massive virtualization server?

Kinda curious what any of you think so I can come to a better conclusion

1 Like

I would suggest keeping your router as a separate box. That way, internet stays up even over reboots of your main server, and it also stays up if something (hardware or software) breaks on your main server.

I would also suggest keeping your NAS as bare metal. You could run other things (in containers, VMs, or also bare metal) on the same box, but having your NAS os as bare metal is probably best. Not to say that it is impossible to have a decent virtualized NAS, but to do that well you are getting into passing through a HBA or similar so the OS has direct access to the disks.

6 Likes

What I’d do is trying to give physical hardware to the critical components in the chain so the router and NAS should have their own machine. Then you can use one with multiple VMs/containers for proxy, gitlab, Plex and so on.

2 Likes

I started off with one 24 core 96GB ram machine thinking that I would just virtualize everything. I’ve slowly added more and more machines because it’s really hard to find a balanced machine that works for everything.

The biggest issue has been drives. I wanted SSDs for services and software, but HDDs for storage. Enterprise grade SSDs are expensive, so I opted to go with many small ones, but that are up drive capacity. 3.5" drives are cheap, but take up more space as well.

I am now at the point where I have one machine with a lot of RAM, 32c/64t that I both host services on in one VM, and use as a workstation in another VM. I’ve got another machine with 12x2TB drives that’s acting as a NAS, and I’ve got a 3rd machine with mirrored SSDs as a router, because I decided router redundancy was important.

2 Likes

Thanks for the response!

I will take this into consideration, the router I can see should be standalone, though the NAS would be interesting to do via a HBA card

1 Like

+1 for storage and router on separate hardware. For services, you can:

A) virtualize/containerize services on the NAS
B) virtualize/containerize services on a 3rd piece of hardware
C) run services on small, dedicated hardware (raspberry pis or similar)

For option C, you’d still probably want to run Plex/Jellyfin on the NAS and maybe the surveillance NVR depending on what you use for that.

2 Likes

Really hard to gauge how much storage or compute you’ll need. Everything on your list, could theoretically run good enough for you on a raspberry pi; or you could find your self happier filling up a box full of disks.

If latter and you’re going for 6-8 drives and more, lookup LSI controllers in IT mode for ZFS on Linux… that way you get both docker and native container support without awkward VM shenanigans.

My setup has a dedicated hardware router, then a R720xd for Truenas and another R720 for VSphere. The 10GB nics for those are like $20 on ebay so you can have really fast storage access between the two, as well as local storage if needed on the R720.

I had a hard time getting the included raid controller flashed into IT mode so I bought preflashed ones on Ebay.

I also highly recommend Nextcloud for hosting your own stuff. It checked boxes for a lot of things for me. Bookmark sync, File sync, Password Manager. Plenty of other good ones too.