Server OS Questions

Hi there!

New to the forum. Not much of a forum posters myself (though I do read regularly), but I specifically felt like this would be a good spot to look for thoughts on something I’m sure a lot of people post on here. But, I often find a lot of recommendations aren’t exactly what I’m looking for, so I’m hoping someone might have some more specific ideas on what I can do. I tend to be a little long winded, but hopefully it will be detailed enough to really give quality recommendations.

Background: A few things are beginning to line up that brings me to the point I’m at. I own a photography/videography business, graphic design, some very light web development and some moderate programming. I enjoy playing around with networking stuff and general PC tinkering. I have been preparing my systems to move houses. When I move, I’m going to set up a studio in the house for my business, as well as my own personal network. I’ll be using my main rig for editing (Threadripper 2920X) It contains drives which hold filesfrom now-past few years. It also runs Backblaze to backup my files offsite, my previous editing rig as my assistants’ rig (Intel 4930K) as my assistant’s editing PC, I’ll have a few other PCs (my wife’s and mine) for non-work use, 2 IP cameras, a couple raspberry Pis, etc. I say this because at the center of this, I’m running two servers. 1) A 2012 Mac mini with 12GB ram. This server’s main purpose is really just to run some MineOS VMs for my minecraft servers as well as a VM for OpenVPN. 2) A Dual-Xeon 5650 system with 64GB of ram. This was the main server for my network. It was running Windows Server 2016 which was managing Active Directory, DNS and DHCP for my network. On the Windows server install, I had a Plex server running with a couple TB worth of movies I’ve accumulated over the years as well as backup software that would make daily backups of all the drives on my main editing rig, as well as doubling up the movie backups so I’d have two in-house copies of all files. It also ran a couple VMs for VPN services, and a VPN for offloading rendering from my editing rig via Adobe Media Encoder. All of this has been pieced together over time as things have slowly snowballed.

The ask: Recently, the boot SSD has had a few issues which I traced back to a hardware issue, so with the setup of a new SSD, along with the move of homes, I figured it would be a good opportunity to be re-configuring this server from the ground up. The big issue I’m having trouble deciding on is what I should end up going with as my base OS and how it should interoperate with the other roles of my server.

ZFS
I know a lot of people are using either Proxmox, ESXi, or FreeNAS. I really don’t want to go the ZFS route because of the way my server has evolved over time, the collection of drives in the PC is very mixed really won’t work out well into building pools, including the challenge of trying to move around existing data on the drives long enough to wipe them to make the pool. Additionally, I like having the flexibility of being able to pull a drive and plug it into any other PC and be able to read files directly off the drive, which you can’t do when you use pooling/array schemes like ZFS and RAID. Instead, you have to move over the whole pool. I also typically install drives in pairs, one into my editing rig, one into the server, so the server only gains 1 drive at a time which won’t work for a ZFS pool. For these reasons, I don’t think a ZFS based option is great for me.

Proxmox/ESXi
I’d like to do something like Proxmox or ESXi, but I would really like to ability to directly pass through HDDs through these into the VMs that will end up using the files, instead of creating pools. I don’t have much experience with these, so I’m not sure if that’s possible in a reliable way. If it is, I could spin up two VMs for storage, one for plex and one for my backup and use something like snapRAID to manage parity for those drives. I could spin up another VM to run my Windows Server.

Windows Server
Another option is to go back to Windows Server as my base OS. The benefit here being that I’m super comfortable with it. The major downside for me is that for the storage side of the server, I was looking into using MergerFS to create a single mounting point so that it would have the benefit of drives appearing pooled to the network, but I could still keep them unpooled and have a snapRAID instance in the background for parity. However, it doesn’t seem like there’s any windows versions of MergerFS available. Maybe I could use a Linux-based VM and do hard-drive passthrough in Hyper-V for the storage side of the server?

Ubuntu Server
I could use an instance of Ubuntu server as the base OS (or another reliable linux distro) and then use that base OS to run my Plex server and storage solutions (using MergerFS and snapRAID, or something similar) and then VMs for Active Directory and my other tasks. This could be kind of the reverse version of Windows Server above. Downside to this is just a logical one, where my Active Directory Server would be a VM actively managing the bare metal OS underneath it.

I’m still very open to ideas and if you have other software that can do similar tasks to the things I’m considering that I may not have though of. While I like to play around, I definitely don’t consider myself very experienced with any of these other than your average linux distro and Windows Server. Look forward to hearing peoples’ thoughts. Here’s the base requirements:

Active Directory
DNS
DHCP
Plex
Storage Backup
VPN Virtual Machine
Media Encoder Rendering
(trying to avoid pools/arrays)

I think almost always my recommendation for a small business / self-employed type scenario is the virtualize everything. Yes there’s a little bit more overhead, but being able to move a business critical server to new hardware with minimal downtime is worth a lot.

You already own Windows Server 2016 and have AD set up, so in my opinion it’d be worth keeping as one of your VMs. You could have Windows Server plus whatever else you want including your Minecraft servers all on the dual xeon server, mac mini can go live somewhere else.

For the hypervisor it really depends on your requirements. ESXi and Hyper-V are both excellent, but they have limitations at the free tier. Here’s a good blog post about ESXi free tier vs paid tier:

Hyper-V has some more complex licensing options, but to summarize it probably is not worth it in your situation, I’d recommend VMWare or Proxmox. The downside of Proxmox is that it is considered a little more on the prosumer side than stable side, but I’ve heard great things and I don’t know too many people who have ever run into an issue.

Hopefully that at least starts to lead you down the path I’m recommending. Basically install ESXi on your server and start spinning up VMs until you have what you need, don’t need to get too complicated with it.

Please for the sake of your sanity don’t use Ubuntu. If you want an APT based OS, use Debian, it is hardened, high performance, ultra-stable and supports live migrations to new versions. It’s also what Ubuntu fork from and screw with.

In the professional industry, I have only ever seen perhaps 2 Ubuntu servers in the wild and only because the person that set them up didn’t know better.

Even as a desktop OS I avoid Ubuntu like the plague, I have zero trust for Canonical especially after they embedded spyware into the desktop environment (https://www.gnu.org/philosophy/ubuntu-spyware.en.html).

Canonical also make wild accusations and jump to push change when there is no real logical reason to do so, or because they think it’s better. Ie, forcing everyone off Gnome2 and to Unity with a fallback to the very unfinished and broken Gnome3 (Both Unity and Gnome3 were not even close to production ready at the time and in part it spawned the Cinnamon fork to fix the mess that was Gnome3, and MATE which is a Gnome2 fork). There is no way in hell I would trust a production machine to Ubuntu, who knows what they will break on you tomorrow.

Can I start by suggesting you edit your post and remove the point about your brother’s use of your business server. Unless you are based in international waters I suspect that is not something you want to publish on the internet.

Given this is a business expense I am not sure I follow your logic. Your data, especially historic photo archives, must be infinitely more valuable to you than a handful of hard disks. If you price your array for reasonable data growth over the medium term and invest in a scheme like ZFS you can expand pools by adding individually resilient vdevs, or just segment your data better (especially non-critical data). Another reason for doing this is resilience against cryptolocker attacks. Not having both production and archive on the same share is sensible.

You could consider having a “working area” on all SSD arrays then cold storage on multiple disk arrays. This would allow you to reuse old disks if you want to but not impact workflow. Check our Linus tech tips “Wanick” server (I may have spelt that wrong, someone will correct me).

I agree with @GoSuBrasky that a hyperconverged setup would suit you for your work data, dropping the personal stuff onto another box. Either Esxi, HyperV given you already have a windows server license and use a domain.

If you want to keep your data on separate hardware from your security and domain stack, a simple Freenas server for your data on bare metal would be my recommendation. You can add expand pools with a bit of planning, and you clearly are not a novice.

Final point, I assume this is all in a domestic property? Have you done a cost benefit assessment of migrating to a managed service (with separate backup service in case they go bust) and rather than spending your money building and managing hardware, spend a bit more on internet and a lot more time in the pub?

I’m sure Wendel will share an affiliate link for Linode…

2 Likes