[Noobie] Questions about building a virtualisation server for 20 concurrent users

High-Performance Computing Workstations & Servers
1

Heya everyone,

This question is in many ways probably going to resemble the classic “Hey, never build a PC before, what parts do I buy?”, except this time, it’s in the realm of servers. I’ve got a half-decent amount of PC builds under my belt, though perhaps more importantly, I know my “resources” when it comes to building a PC. That is, simply put, I know where and how to pick the right components, what videos to watch for confirmation, or on which Discord to get further help.

You probably see where this is going… Here I am, trying to plan out a server build. Suddenly, questions like “What CPU do I pick?” or “Do I pick this or that part?” are relevant again, and I’ve got no idea as to where to look for answers. Obviously, I don’t expect there to be as much as if I’d be building a 500 USD gaming PC, just given the price difference between that and a profesh server alone.

So, to be more specific, here is what I’ll be trying to build:

A virtualisation rack-mounted server for a small company, which should be able to run about 20 concurrent users performing your run of the mill office tasks (email, Word, ERP…). CPU budget is around 5500 USD, with a budget of roughly 4500 USD for other components, though bear in mind that I’m from Europe, and prices here run slightly higher than in the US. If there is already a build, or a video of a build that meets these criteria, I’d love to copy it so I don’t have to worry about compatibility hick-ups, however, if not, here are some of my questions:

  1. Intel is dead to me, right?

  2. Does it ever make sense to choose a Threadripper over an Epyc for a server workload like this?

  3. When building a (new) gaming PC, it rarely makes sense to go for an older generation of CPU, rather than the newest thing available. I suspect this won’t exactly be the case here, right? In other words, should I be considering Zen 4, 3, 2…?

  4. How do I go about picking out all the other components? What ECC RAM? What motherboard? Rack mounting “case”, or however that’s called? Power supply? Obviously I know that the parts need to be compatible, both in terms of socket, power output/requirement, physical size and so on, but I assume that’ll still leave me with a lot to choose from. So, are there any “go-to”/“you can’t go wrong with this one” parts?

  5. Cooling… how? Is water even a thing here? Again, any “go-tos”, like say the infamous Noctua NH-D15 in higher-end gaming pcs?

  6. Uh… any other questions I should be asking / know the answer to?

I’ll probably come up with a lot more questions eventually, but I can’t think of anything else right now, so… any help/input on this is very much appreciated, thanks in advance and a great day to everyone!

2

are you talking about virtual desktop? Are users using thin clients?

Do you know what OS you’ll be using or what other softwares you’ll have to license will be?

2 Likes
3

Yes, I’m talking about virtual desktops. Users will use their existing office PCs.

We’ll be using Windows 10 for the desktops… I’m not too concearned about all the other software as someone else will be handling that, they just don’t know much about building PCs/servers.

4

usually you know what software you need first, and that will have a list of supported hardware and operating systems.

3 Likes
5

Perhaps the first question you should answer is why you are doing this - they already have desktops, why are you wanting to give them virtual ones?

The answer to that will inform many of the decisions subsequently made.

The follow up and very important question is just what software platform you are intending to use - as nx2l notes many of these will have supported hardware or other requirements, and the way that they work will also inform hardware decisions made.

One of the main challenges with what you are doing is providing graphics acceleration for your end users - CPU virtualization is basically a solved problem, whereas sharing graphics resources can get very expensive and complicated.

While your end users will probably be fine without it if they are strictly using email, word, ERP, the reality nowadays is that any website they hit with rich content, or a youtube video, or a teams meeting can rapidly bring a powerful cpu to it’s knees if it is having to do the work of a video card.

I’d suggest looking at a remote desktop session host, instead of a virtual desktop host - it tends to be less demanding on hardware, often cheaper to license and easier to manage.

2 Likes
6

Mostly just questions here.

  • Given the relatively low overhead of normal desktop usage; do you intend to overprovision? That may heavily inform your choice of CPU due to memory capacity/address space limitations.
  • Are there any particular graphics APIs you must have acces to? Is graphics-virtualisation (“multiplexing”) required here (for security or other…,), or are you looking for a dedicated-graphics (passthrough) setup, or would API-redirection’s (sharing) performance penalty be acceptable to your userbase?
  • Why is intel dead to you?
  • Rackmount servers tend to be quite loud, so is there a server room with a cabinet waiting? Can you immediatly exclude any tower-server or micro-server chassis on that basis? Also, how many units of space do you have to work with (1U-5U, or more)? What is the existing power budget for this space (e.g. Wh/year, voltage, UPS, etc.)?
  • Is the storage for this co-located in the same rack/cabinet (JBOD), or is lane allocation for networking going to be a major consideration (NFS, NBD, iSCSI, Samba)?
  • Can you say how “On-The-Clock” you are with this?
2 Likes
7

Hey, thanks for detailed reply. I’ve been trying to get more info for you guys during the past few days, so here are your answers:

  • Yes, I do intend to overposition; bear in mind I have a budget to work with though. After additional discussion, it looks like we’ll only have to account for 14 concurrent users maximum at the moment. Hence, we thought we’d go for the new 9554P. Is that sensible?

  • I was assured the users will only be running Word and Outlook, hence we won’t be buying graphics cards. What solution out of the ones you mentioned should I be going with?

  • I thought AMD is steamrolling Intel, especially when it comes to server CPUs?

  • There is a server room with a rack/cabinet and about 10U of space in there, we thought we’d go for a 4U to leave some space for additional things. The power budget is essentially considered unlimited.

  • The storage will be in the same rack.

  • We’d like to have it done within a few months.

8

  • I don’t want to give them virtual desktops, they came up with that themselves. Apprently, they are having a lot of trouble with PCs getting bricked by viruses, and they want more control of their system / easier restoration.

  • TrueNAS & VMWare… if that’s what you’re asking about? Sorry, but I’m still somewhat lost when it comes to these things…

  • I was assured they will be strictly using email, Word, ERP; there’ll be 1 or 2 users at most browsing the internet/Youtube.

9

Not sure if virtual desktop infrastructure will put you in a better position regarding this.
Your “viruses” will still “enter” your computers in the same way(e.g. uneducated user click on bad links in outlook, people downloading personal software on work computers, bringing infected devices, etc.).
You’ll still need to re-image the VMs, make sure no important data has ever touched the VM, your hypervisor has not been compromised, etc). An infected VM is in a lot of ways just as bad as an infected PC. Sure, you can make secure VDI, but then you also can make secure non-V DI.

What you probably want to do is this(some schools in my area use a system like this; nothing like testing infrastructure with students; they’re evil, I’d know :wink: ):

  • Setup your clients computers to boot exclusively via PXE
  • Setup some kind of central HDD imaging solution to distribute images over the network on boot
  • Setup Windows group policies
    • Disable outlook’s most evil features(e.g. external links, loading images, …)
    • Disable random .exe’s from working
    • Disable any unexpected external storage or USB devices(e.g. USB tethering on phones often gets ignored, but bypasses any network-level restrictions you put in place).
    • More stuff I don’t know, I don’t administer Windows on that level.
  • Setup “private”(non-public) storage for your users files.

The HDD imaging server can be any old machine, it’s basically a file server with PXE.
Your users then shouldn’t be able to make the machine un-bootable, or permanently infected.
And if needed, you can just install a GPU or other upgrade in a client machine, when needed and where they are used.
As a bonus, you get always updated windows/drivers on all your clients.

Your users still probably can infect the installations for a short time, so keep that in mind.
Professional IT security is a ongoing process - There isn’t a single technical measure you can do to solve it. Educate your users, make sure they know that a “work computer” is not the same as a “personal computer”, and that people are trying to get to their stuff, and ruin their lives(or at least their companies, which will then inevitably blame it on the user).
If you’ve had problems with this and don’t have any experts on this “in-house”, might I suggest getting your best people together, and hiring an external consultant for those things? Maybe just to get you started in the right direction, or to have on call when something happens?
I don’t know how important the IT part of your business is, but if some ransomware takes out your IT it can get expensive quick.

I like virtual desktop infrastructure as well, but from what you’ve described it might not solve your problems, is expensive, difficult to maintain, and has some drawbacks for your users, like input latency, requiring network connection, not being able to watch YouTube videos or do any 3D acceleration, etc.

Maybe you can tell us more why VDI is actually necessary/wanted in your case?
Otherwise, maybe just go with a more conventional approach?

Kind regards from Germany.

10

There are a lot of things to consider, that others before me mentioned. I will try to add some details:

If you are going with Windows Server / RDP, in order to have a unified OS for all users (and arguably use less resources used, as opposed to virtualizing everything), then you need to consider the software licenses for the multiple RDP sessions.

In addition, if you have certain programs you want to run, it might have a per CPU core license and / or a per socket license, which will impact what server CPU and motherboard you buy. This can sometimes be avoided with virtualization. But if you want all users to access a single piece of software without additional licenses (unless it’s a per user license, instead of per installation), windows server is the way to go.

Since we are dealing with workstations getting infected, we are just shifting the affected place in a central system if we go with virtualization. While unlikely, it’s still possible for malware to break out of the VM and affect all users. For a single VM getting affected, it’s pretty easy to revert to a previous snapshot and act like nothing happened, but this does have drawbacks.

Max said well, fixing the root cause should be the priority. PXE Boot and diskless installs are cool and all, but they also require that users stay constantly connected too.

I would say streamlining the backup and restores should take priority. Shifting work to a VM doesn’t fix the problem, but only gets a bit more manageable in case something happens (because of things like snapshots and linked clones and such).

11

  • I should mention; I have no experience w/ zen architecture, so I don’t think I can usefully comment on Epyc processor choice, other to say the 9554p seems to be positioned comfortably above the requirements you’ve outlined four times over. If you’re using Software graphics emulation, I would expect the CPU workload to maybe about double, but your mileage may vary, especially w/ the newer architectures. Was not aware of the ‘Secure Processor’ tech., but it’s certainly intruiging! Also, any quantity of $300-500 SAS drives will gobble up your budget pretty quickly, & SSDs are even hungrier. If these must be covered by the budget, you may consider scaling back on the processor.

  • Not being overly familiar w/ Windows in general, Windows (-Server?) w/o a graphics card would also be an area I can’t comment on. What I can say is this:

    1. For me, graphics-sharing in qemu-kvm (but called vSGA in vMware docs) has been mostly positive, because it’s essentially a push-button solution requiring very little of a gpu’s on-die resources. However, VRAM usage is consistently a major limiting factor w/ 500MB/per-head/per-seat being, about the minimum at 1080p. So, This one in particular will almost certainly call for one or several multi-thousand dollar accellerator cards. Also, in vMware, this requires a proprietary driver for each guest, & API support is limited to Directx & OpenGL acceleration. This tech. can make use of basically any GPU.
    2. My very limited experience w/ graphics-virtualisation (for vMware, vPGA(?): virtual shared passthrough graphics accelleration) tells me, it’s way more trouble than it’s worth, but I can’t really speak to it’s effectiveness in terms of security, which is always the main point brought up in it’s favour. AMD’s MxGPU extensions can supposedly allow for dynamic rescource allocation, w/o the need for Nvidia’s GRID profiles. No special drivers or liscenseing needed either, but actual support seems to be a little unclear.
    3. Gpu passthrough (vMware’s vDGA(?): virtual dedicated graphics passthrough), is the one I’ve had the least experience w/, but no real complaints about the setup process (again, this is using the libvirt API for qemu-kvm. The situation on Windows may be very different). Can use ‘entry level’ workstation cards, if there are enough slots/mezzanine.

  • Sorry, no idea. The one rack I have to babysit will be old enough to drive soon. Not the snappiest experience, but still very usable (Ubuntu-20.04-desktop).

  • Always a plus! I believe there are servers using water cooling successfully, but would imagine it adds considerably to thier upfront cost, & are likely the exclusive province of enterprise OEMs. There are barebones chassis advertising water cooling compatibility, but how well the’re able to support that claim, I don’t know.

  • No notes!

  • Impressive timeline. No other thoughts at the moment.

12

Assorted vMware compatibility docs & other resources. vGPU-Smackdown is a ‘bit’ outdated, but still a decent primer I think. Hope it helps!

13

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.