Minimum Viable General-Purpose VM Server

I’m looking for the smallest viable VM host for a lab/home environment (“an everything server”).

The plan is to build one now, and add new ones as I need them (if I need them).

This machine will run a minimal Linux on the metal with KVM/QEMU/libvirt stack for Linux VMs.

This box will host:

  • Gitea
  • bind 9 DNS (maybe not immediately)
  • Plex for 2 concurrent 1080p streams
  • Postgres for a tiny app
  • CI system + workers (pretty much for me only)
  • Kubernetes on KVM (CI stuff might leverage this)

Criteria

  1. The parts have to be available to Canadians.
  2. amd64 is a must.
  3. Must consume very little power (<$120.00/yr @ $0.09/kWh).
  4. Disk IO has to not suck (>= SATA 3 SSD; more is a nice-to-have).
  5. Has to be quiet enough to have a conversation around it.
  6. Out-of-band management is a nice-to-have.
  7. Must fit inside a 1U or 2U case that can be installed in a 2-post network rack.
  8. ECC RAM is a must.
  9. Has to handle 24/7 operation.
  10. Highest number of CPU threads for the money.

Parts

If someone has any experience with this kind of system/use-case, I’d love any comments or suggestions you might have.

Probably would skip asrock rack board its overpriced and your targeting minimum viable.

http://www.plinkusa.net/2u.htm
Some more good price chassis options (not sure with Canada shipping and conversion prices).

Well, a consumer B450 board wouldn’t have onboard video and I’m not sure how to deal with that without adding a costlier video card (in terms of power). Then there’s ECC support on consumer boards that I haven’t looked into much, is it there?

Cant run it headless?
Also would probably not go B450 would probably go x470 to give you the extra lane options since you can split the GPU lanes

AMD pretty much supports ECC on all boards only APUs dont support it and even then the Pro APUs do

The only thing I have to add is that for a VM host RAM is always an issue. I run a few VMs on my home built NAS and I often wish I’d built it with 64 instead of 32 GB.

Whenever you want to add a new VM you’ll be tweaking all of the others to see how far you can squeeze them.

1 Like

The plan is to add another server if it comes to that. I’m looking at a minimal box so I can scale horizontally.

EDIT: Thinking about it now, it would probably be a lot less painful to just spec 64 GB instead since 8 cores can stretch a lot further than 32 GB. You make a good point.

I was thinking about installing an OS or dropping to an emergency shell-type use-cases when I mentioned the video card.

Probably not enough horse power, likely unavailable and I wouldn’t trust it if it has to be available from outside… but Intel haswell 1150 stuff might be an option. Boards and CPUs should be cheap, ECC DDR3 is still cheaper than modern RAM, IPMI is available, …

Did you mean 2011

1150 that accepts ECC is 4 core chips

That’s why I said probably not enough horse power. :wink:
Socket 2011 haswell also only uses DDR4, I think. So you wouldn’t get around that expense.

No, I mean socket 1150 Haswell, like a Xeon E3 1240 on something like the Asus P9D-M with 32GB ECC DDR3.

Thought you were referring to the 2700, I was like its possible then was like wait 1150 haswell is not any faster an was confused. The 2011-3 stuff is a bit compelling on the used side tho, really surprised how much it actually fell in price, guessing the market is flooded due to all the issues with Specture/Meltdown and the like.

@claude if you need more cores and are ok with Treadripper 1st gen wendell has one for $250 usd OBO for sale. ([WTS]Wendell's junk for sale thread)

Also DDR4 had dropped decently in price for even the ECC unless its back on the way up. (not sure about canada)

1 Like

Threadripper is a little too much for me, I don’t think there’s a much better value than Ryzen 2700 for me right now.

I’d just like to know how you’d deal with not having a GPU at all on a B450 consumer board. I’ve read briefly about Linux installers that bring up sshd so you can do a text-based install, but I’ve also seen conflicting reports that a Ryzen system won’t POST if no GPU is present.

I never did headless on b450 but i did on x570 (ended up with gpu anyway) Wendells used 1950x is probably a bit better value for you then the 2700, not sure on x399 mobo prices but probably not that hard to find some used ones as well. Not sure if you are interested in used at all tho.

Usually boot up installer with a gpu installed then pull it after it has an ip and basic config then you can access via ssh.

Honestly I would stay away from b450 tho as you lose the ability to split the 16x gpu lates to 8x / 8x allowing more expanion cards in the future (HBA 10gbe etc)

Usually boot up installer with a gpu installed then pull it after it has an ip and basic config then you can access via ssh.

That’s the thing though, I have a strong preference for having a tiny GPU in the box and not juggling cards (in large part because I don’t have video cards that will fit inside a half-height slot). I suppose a tiny video card is still much cheaper than the premium for AsRock Rack. But, the tiny GPU on the AsRock Rack board is gonna be more efficient.

Honestly I would stay away from b450 tho as you lose the ability to split the 16x gpu lates to 8x / 8x allowing more expanion cards in the future (HBA 10gbe etc)

AsRock Rack has x470, and two NICs, and I get to play w/ IPMI (never have, kinda wonna try), and it’s purpose-built for uptime (whatever that’s worth).

I said “minimum viable”, but I’d rather drop an extra 2-3 hundred bucks than have the system be more complicated/less-straightforward to administer.

My router would like a word with those “reporters”. :stuck_out_tongue:

It might be a board thing but in general it should boot. Under protest … by beeping … but it should boot.

lol, I’d still prefer just to have the GPU in there so I can hook up peripherals and get to a shell like I’m used to doing

1 Like

I highly recommend you read the thread on that board. While Asrock Rack has done well in the past this board has had some mixed issues.

There are a ton of x470 boards with 2 nics on it so really all you get is the IPMI and I guess the igpu. You could throw a low power gpu in if you want on any x470 and still have similar experience. If you are set on it, I dont blame you but I was in a similar decision but ended up on an x570 to give me pcie 4.0 for the future (only 2700 in it atm) and the lane spit i wanted and x470 was more expensive.

1 Like

@mutation666 I didn’t get half-way down the thread you shared about the AsRack board and out of 8-10 boards, 1 died and only a single one had no issues.

After my experience with a 1700 on an ASUS PRIME X370 PRO (idle lockup bug on two CPUs), I think that even now that we’re on Zen 2, this platform still requires a significant amount of caution. More than if one were to say, purchase a gently-used Dell R630.

Thanks for the insights everyone.

EDIT: Reworded.

I have two of those boards, one as a desktop with a Ryzen 7 1700 @3.8GHz and the other one with a Ryzen 3 1200 stock running my SSD NAS. You disable C-states and it’s rock solid.

Most issues were power supply related or ipmi