Hardware Recommendations for Low Power, Low Cost Home Lab for Virtualization Practice

Hardware Enterprise Gear
, ,
1

I know I might be asking for a lot here, and I have researched the forum and the web for the past few days , but I am looking to create a home lab on which I can learn about virtualization in general, and VMWare, ESXi and clustering more specifically. I would also want to learn Hyper-V as well, although I am not sure if the information would transfer between the two. I also hope to use some sort of shared storage solution for the VM’s which can double as media storage for movies, music, as well as my hobby photography work.

Just for background I have worked in IT Support for about 7 years now, and I would like to gain some hand-on admin experience with some of these cool virtualization technologies.

Now below are the constraints I have:

  1. I live in a studio with my wife, so anything rack-mounted or loud is out of the question.
  • My space available is at most two (13 in. x 13 in. x 15 in.) compartments out of a Kallax shelf + a space that is about 1.5 feet wide, 3 feet tall and 18 inches deep where I can put some shelves.
  1. I need something that consumes low power.
  2. I have gigabit internet service at home, but I do not have a hard wired connection to the router as the owner is not in favor of running a cable to our place.
  3. My budget is about $600 for the cluster PC’s.$300-$400 for the NAS / shared storage solution. $200? for networking equipment, although this is where I am the least sure.

The idea I have in my head for the VMware cluster would be 3 USFF pc’s from either Dell, Lenovo, or HP. Ideally, specs would be at least quad-core i5, SSD for boot drive and 32 GB RAM (not sure if this would be overkill for virtualization or not). Also, I don’t know if I would be able to get away with using only 2 USFF PC’s in lieu of my budgetary restraints.

For shared storage, I was thinking of using some sort of Synology or QNAP NAS, although considering my budget, I am not sure what models and specs I should be looking at. Also, I saw Wendell’s video on making an HA NAS cluster and I thought that was super cool, but I am not sure if I could do that with the lower end 1 or 2 bay NAS’.

For networking, I am probably the most lost here lol. I have an unused 6 year old NetGear combo router/AP. I believe the router ports are 1 gigabit. I was thinking if it was possible to use this router/AP as a wireless bridge from the landlord’s wifi AP (ATT gigabit fiber service) that we connect to. Then, I would connect a switch to my router/AP and run the lab equipment on the switch. Also, from what I gather, for VMotion and shared storage to work well, I should shoot for 10 gigabit links, right?

I’ve attached a crude MSPaint diagram of what I am envisioning the setup would be like. Am I being unrealistic with my budget, what should I adjust, is there anything I am missing? Any help would be greatly appreciated :slight_smile:

homelabplan
homelabplan1060×820 187 KB

2

You may need to look at the Raspberry Pi (RPi) environment. Sadly, due to enormous demand there’s a serious shortage, but they tick most (but perhaps not all) of your boxes. If you go that route, I’ll refer you to the excellent YT channel of Jeff Geerling (also present on the L1T forums!) who has a ton of stuff on pretty much anything RPi and then some.

Alternatively, find the Serve the Home channel, they have a project called ‘tiny-mini-micro’ (or in similar order) in which they review very small business and pro-sumer workstations. Unfortunately, most are probably out of your budget when new, perhaps find used models instead.

Other YT channels to investigate, if you aren’t familiar with them already, are Networkchuck and Craft Computing. I’m sure others will suggest more channels for investigation.

As for networking, I doubt you’ll be able to use the old router/AP as you intended, as it probably doesn’t support being an AP itself. Find the manual, it should be noted there if it does and otherwise, you must assume it can’t!

HTH!

1 Like
3

You can go elegant or cheap. pre-built NAS from QNAP/Synology/various other small companies. But you can’t learn much there, because you pay half the price for that all-in-one with custom OS and so. But getting 10GBe out of it requires quite expensive models as most <1000$ come with 1-2.5 GBit. 10GBe in general is rather expensive…you have to buy NICs for each device as most devices don’t have that by default. But getting more than 1GBit access to your storage (and 1GBit is WAN speed for both of us, you really want more than 115MB/s from your local storage).

DIY…with 400$ budget you’re basically down to “old PC +10GBit NIC”. And that’s without any drives. But any quad core with 16+G memory will do, more is always better. And drives are expensive, a single 16TB mirror is 500$ alone.

2 Likes
4

Thank you for the informative response. So I can use the Raspberry Pi’s as a VMWare cluster?

1 Like
5

Thanks for the reply! It seems in this case the best option for a NAS would be a cheap pre-built with a 10 gig NIC, add some drives to that, and run something like TrueNAS. Does that sound about right?

1 Like
6

PCIe slot for 10GBit NIC, enough SATA and/or M.2 slots for your needs and the case should be able to accommodate the drives (+some room for storage upgrade). There are specialized NAS cases available, but they are more expensive than standard cases but offer really what you want from a NAS. I went with the CS381 but there are versions with less bays too. In the end it’s about budget and convenience. And I bought basically the only AM4 board with on-board 10GBit Ethernet. But that board was 400$ alone. There are a lot of nice things for NAS and storage out there that can drain your purse quite rapidly.

But a basic NAS is a low-end PC with extra NIC, extra drives, above average memory and TrueNAS ISO on a stick.

About power…I calculate around 5 Watts each for SSD/HDD, but SSDs are more economic when not in use. CPU will be the largest contributor in terms of power. Ryzen CPUs can be tweaked down to <50W while still having plenty of horsepower left for a NAS, also reducing heat/noise which is nice for sth that just sits in some corner and does its job.

Using second-hand or your old PC is common, but power consumption on those old CPUs may be detriment to your goals, because a NAS usually runs 24/7 over years where every watt less is more important than on a device that only runs for 2 hours a day.

1 Like
8

I think some of the SFF Lenovo/Dells (the small form factor, not the ultra small form factor) would be ideal for this. From what I’ve seen you can find those with low TDP CPU’s. I will see if I can find one of those that I can plop a half-height NIC in. And I’ll make sure to research the amount of SATA slots in each model too.

There is actually a lot on eBay to take a look at, so I will definitely be scouring that plus craigslist plus OfferUp to find a deal.

I am factoring in drive costs separately from what I budgeted above, because I want to start out with flash storage, even if it’s initial higher cost, I gain back speed and low consumption.

1 Like
9

Sounds good. Just be aware that systems by major OEMs often have proprietary parts/boards that might limit expansion and customization or force you to buy special OEM parts as replacement.

TrueNAS and especially ZFS is excellent at combining flash with traditional HDDs, really masking the poor characteristics of a HDD in a well-cached pool. Be sure to hit up the forums if you have questions on how to set up disks and pool. But All-flash is much easier than a tiered setup for sure. Be sure to use compression to get maximum value for your limited space.

1 Like
10

This depends on how much DIY you want to make yourself, but, I’m currently considering making a DIY Kallax case for 2x systems per 13 in compartment. This should be more than doable but would require some crafting knowledge. Should be possible to handle cooling with 2x140 fans in the front and 1x140 fan in the back, plus allow expansion cards. Make sure to leave some room in the back of the shelf to let cables and air escape! This, however, will cost you both more time and money than you are willing to pay right now.

As for low power, Raspberry Pi clones are your best friend. The Helios 4 SBC is a nice 4-port NAS SBC that could prove quite useful, for instance. I would take one of those + a few Odroids. That is around $400-$500 total, drawing a total of 100W or so.

One step up, you have the intel NUCs. Those are still small and low power but costs a small penny, and not as power efficient as the RPis. You can get those for maybe $200 per machine, power draw is around 30-50W per machine.

Another step up, you could go with ITX. You could theoretically fit seven 1u height ITX units in a 13x13 inch enclosure, though six are more realistic. That leaves a 13 by 5 inch space for power delivery and drive mounts. Four 140 fans at the front, Four at the rear, done. The bad part is now we are starting to reach expensive land, with around 300+ for each board and the power draw is between 100-150W, the good part is now it is a lot easier to run an x86 cluster.

Anyway, my point is that cheap, low power, small form factor - pick any two.

1 Like
11

Hope you post the results of your Kallax project here if you decide to undertake it! Would definitely be interested in how that would turn out

1 Like
12

I just built my home lab not too long ago. The suggestion you have in your idea seems fine with the smaller Lenovo or whatever PC’s as nodes. Its best to have all matching machines if you want to try HA (High Availability) Nodes like you were eluding to I think.

Then a centralized Media Storage NAS would also be useful here. You can get one that can integrate services like Plex etc.

Then there’s another option of using one machine and Nested Virtualization. So running a hypervisor with Nodes as VM’s all interconnected on one machine.

I built my all in one machine, then decided to dedicate different devices for different purposes (IE DNS, DHCP, Firewall, VM server/File Server and so on) for diagnostic purposes if something failed I didn’t loose Everything because this is still my home network. And if wifey loses her services its bad juju.

Like @Dutch_Master recommended Pi’s would have been great for this application. I dunno if Orange Pi or other SBC (Single Board Computers) would be another viable option as nodes for a cheap setup.

Have my machine here which is WAY more than is needed for a single machine for VM’s etc. Least for my needs messing around, sadly prices have increased significantly. The ASRock Board and ECC ram were the big purchases. It’s nice to IPMI for remote monitoring and control of the server completely remotely.

As someone who spent the money on 10G nic’s etc…its nice, but not at all necessary for home tinkering. 2.5G is cheaper but even 1G is plenty for most cases.

1 Like
13

Then there’s another option of using one machine and Nested Virtualization. So running a hypervisor with Nodes as VM’s all interconnected on one machine.

You know I was thinking about this option. To actually centralize my build and just virtualize the nodes. I was just not sure how spec’d out I would have to make the build.

1 Like
14

Really depends on how much load you put on the nodes (VMs). IE Ram needs and processor needs. Can usually get away with processor over provisioning (unless vm is running 100% useage all the time), but rams another story. If Im wrong anyone please correct me here.

Nested virtualization can be difficult on some platforms as well. Windows hyper v can be difficult if I remember what I read a while ago (dont trust my memory). Worth a google depending on what you plan on doing with everything.

I have yet to move much past a glorified NAS and plex. Well besides pihole and OPNsense firewall. Still learning and definitely a novice still.

1 Like
15

Hey thought you might be interested but I finished the first phase of my build. Details here:

3 Likes
16

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.