As the title suggests, I’m seeking advice for something I could either build or purchase to replace my aging T7500 for my home server. Specs:
192GB DDR3 ECC
2x Xeon X5650
GTX 1070 added in.
10GB X520 card added in.
I got the base unit for free some 8 years ago from a company that was upgrading, couldn’t complain, piecemeal added in the 1070 and X520, some hard drive docks, an LSI-9211-4i and made some very poor decisions along the way due to being a budding sysadmin back then; namely installing ESXi 6.7 as the host OS of choice, then virtualizing truenas under it and passing the card through to it, and have further mucked things up by placing all my drives attached to the VM in mirrored vdevs because I am hyper paranoid and gave in to older advice on the then FreeNAS forums. Using iSCSI to access the TrueNAS array via a Windows Server VM under ESXi (dumb but at the time I was also using it as a steam drive and couldn’t figure out a better way to do it.)
I don’t have a budget in mind for the job yet, my biggest concern is not losing performance while also trying to have the new unit consume less power. I primarily now just use it for my massive Plex library, hosting a couple of game servers for friends (Minecraft, L4D2, Stardew, Terraria, Conan Exiles, Ark, etc.) I have it running my Unifi controller, a forbidden router (before I knew it was forbidden,) and had started to work on a homeassistant VM before realizing how stuck I was with the unit going forward.
I live in the states, am in an apartment with a less than stellar breaker situation but could fudge something if need be. Any insight/advice would be appreciated.
(And in case it needs to be said, zero intention of continuing to use ESXi going forward, primary consideration is Proxmox with TrueNAS virtualized on it (I know TrueNAS itself can run VMs but it’s primarily to teach myself Proxmox.))
Sounds like you could do everything you want with a Mini PC running docker or a truenas in a small tower.
I’ve done a similar migration myself recently from an old Dell R610, and while I initially did want to upgrade to a like for like solution all in one box, I came to the conclusion that it’s just not the best way to do it neither efficiently nor practically.
Firstly, let’s talk about the systems that you should keep independent. Your router and home assistant I would highly recommend keeping on their own systems. Generally speaking, you don’t want to be touching your router too often, and you’ll need to reboot it everytime you update your hypervisor if it’s a VM, taking down the entire network in the process. I too was fine with this for a while, but given how cheap you can get a fanless box to run pfsense on, it’s worth it to never have to think about it again.
For home assistant, again I’ve just moved from a dedicated vm to just having it a Pi. The performance is basically the same, and like the router I now can still monitor my home, even while production systems are rebooting or have gone offline.
Now for the production systems, it sounds like a truenas scale system could do pretty much everything you need. I’m not sure the state of all the game servers, I know minecraft is pretty solid. However with the upcoming release of TrueNAS CE, you could easily have dedicated LXC containers to run the games servers. This will give you some separation from your internal apps. You could use a tool like ansible to manage the config for them.
The issue you’re going to run into with running some of this virtualised under proxmox is stability. One thing I came to realise is that having everything on the one system significantly limited my ability to learn and play with the system, because I didn’t want to risk breaking the production apps and services.
With the T7500 being a relatively big tower, you could replace it with a small truenas, pi, and router that would take up a similar footprint and use less power. Then in addition, I’d suggest a cheap mini pc, or even a used laptop to use as a proxmox machine for learning. It can use ISCI or NFS to connect to the truenas for it’s storage so you keep the redundancy of having a zfs pool, while gaining the benefits of a compact compute machine, and the separation of your production apps and learning environment
My sincerest apologies for the delay in a reply, a poor decision to agree to fast food Thursday evening left me with what I beleive was food poisoning… do not recommend, heh.
I am inclined to agree.
Do you have a suggestion for one of said boxes? I am indeed running pfSense CE, though am potentially looking to make the hop to OPNSense (meaning preferrably not one of the Netgate ones.) Regardless… I currently have a 10 gig home network and 2.5 gig fiber ONT, so the box would need to account for that.
Humorously and/or sadly, I do not own a Pi for my homelab, but that is easily rectified, and they do tend to have Zigbee/Z-Wave hats for just such an occasion… or would you suggest swapping to a Home Assistant Yellow device like Jeff Geerling showcased?
To give a shameful admission; the gaming server configs, Plex, etc, all run atop a Windows Server installation. I don’t really mind it and have grown marginally accustomed to it because it strangely allows me to utilize the GTX 1070 for transcoding in Plex, or letting the girlfriend play Warframe via Parsec because she is a mac user and I try not to hold that against her.
That said, I am not opposed to the idea, but I only have the single GPU for this purpose, and correct if mistaken but TrueNAS in any flavor cannot segment said GPU up between multiple jails/containers?
The TrueNAS device in question would be where I lack the most direction. Do I try something like the new 45 drives homelab unit? Do I go smaller and try to get an asustor I can put truenas on and go all SSD? Etc. If you have a suggestion for a specific device, or multiple possibilities, I would apprecite it.
Humorously, just learned that my company has won a UK bid in which we’ll be building a new stack in a London colo, so I might have a work excuse to learn Proxmox, heh.
The ones I’m running are a few years old but they run an AMD G-T40E. For 10/2.5G you’ll want to make sure it’s something more modern. A NUC/SFF PC from a couple generations ago would likely to do the job, but i’m a bit out of the loop on options available. In terms of Netgate offerings, the 2100 would do the job, but I’m not a huge fan of using them for home use due to the premium.
For HA, that’s up to you, I considered the HA yellow but since I already had a Pi it was obviously cheaper to just grab an SD card. If you just want the dedicated box to set and forget, go for the yellow. I’d only recommend the Pi as it’ll be easier to repurpose the hardware should HA every grow beyond what the yellow can handle. But as someone who been using HA for over 5 years, I don’t see that happening.
Wanting to use the GPU as a spare gaming rig definitely complicates things. I don’t have a ton of experience using GPUs on truenas but my understanding with the new docker backend on scale you should be able to make use of the nvidia card on multiple containers. Of course this is mostly useful for plex as you’re unlikely to be running warframe that way. There are 2 ways to approach it, if you want to keep things as close together as possible, having the GPU and accelerated apps on one system will be necessary, and a VM will be far better suited to that. However if you’re looking at getting new hardware anyway, either something with a decent iGPU or a cheap old GPU would be more than enough for Plex transcoding and would let you separate the systems. The GF’s gaming system can become a VM on whatever hypervisor you choose with the 1070 passed through.
The 45Drives stuff is very nice and great quality, but you do pay a premium for it. The deciding factor is going to be how appliance-like you want the system to be. A TrueNAS Mini from Ix-systems is a nice compact unit, but it loses some of the benefits of a custom solution. Once the hardware is no longer suitable, you’ll need to swap the disks into something newer. This is isn’t something you’ll need to worry about for a few years, but it will come up eventually.
The other option is to build custom, there are many sff cases that are designed to be used as a NAS and come with extra HDD slots or even hot swap bays, something from Silverstone or Jonsbo should do nicely.
Depending on your scale, proxmox might not be the best platform here, if you have any say in the decision it might be worth exploring options like xcp-ng too. Coming from the enterprise world myself and we much prefer xcp-ng, but that’s not to say proxmos isn’t good or suitable for the job, just worth exploring your options