My homelab project 01: Project brief

[frnkquito]

Table of contents

1. Homelab idea
   1. Vps emulation
   2. Modular service system
   3. Hardware
   4. Main arch os
   5. "Clean" Windows installation
2. Storage setup
3. Work distribution
4. Problems I had
   1. X server on linux
      1. Problem
      2. Resolution
      3. Fix
   2. AMD graphics on linux
      1. Problem
      2. Resolution
      3. Fix
5. Resolution / next steps
6. Questioning points
   1. Why?
   2. Efficiency
7. Future adaptations
8. Conclusion
9. Sources of information

1.Homelab idea

1.1 Vps emulation

The main idea of this project is to emulate a real case VPS (Virtual Private Server) that handles all the services that I use daily in my computer, let that be: a web server, a file server, media server and so on. My first approach for this was to build somehow a server with PC hardware, but because I also didn't have a good standing computer, I decided to merge both setups: a good, modern PC for general use and development and a server that would run all kinds of services.

1.2 Modular service system

I wanted to make every service independent of the others as I would do if I had a physical server. This services (specifically systemd services, mostly) will run in virtual machines, interconnected by the Host OS, which will be an Arch installation.

For now, I have the idea of having the following emulated servers:

• Git
• Media
• LAMP
• Logging
• Database
• Rat labs
• Torrenting
• File consumption
• Backup management

I saw that it's pretty common to associate some services, like the media server and the torrenting one. I don't want to do that because I want to be able to choose explicitly to choose which server to run. Also, debugging will be easier as I plan to do it; I plan to create snapshots and backups of this VMs, so if any problem happens, I can just refer to a stable state of the VM.

1.3 Hardware

• Sentey A20 (Case)
• ASUS B350M-A (Motherboard)
• Ryzen 5 2400G (APU)
• LNZ SX650-FS (Power supply)
• 2x HyperX Fury Black DDR4 2400 4GB (RAM)
• 2x 1TB WD Blue 7200RPM (HDD)
• 500GB Hitachi 7200RPM (HDD)
• Kingston A400 240gbSSD (SSD)
• HyperX Gaming Keyboard Alloy (Keyboard)
• PulseFireFPS (Mouse)

No graphics card: I didn't bought a dedicated Video Card because I was kind of on a budget, and the most graphics intensive operation I'll probably do is playing games that match this setup, without trying things like overclocking or running at low FPS. Even though, I saw some benchmark / gameplays of this setup (Ryzen 5 2400G + 2xDDR4 2400 4gb) and GTAV could run without issues.

Ram is enough: It could be more (RAM always can be more) and I would’ve liked that, but again, I was kind of on a budget. I could’ve be more cheap on the keyboard/mouse but I would just end up saving not enough for buying no ram. Of course it will be updated as soon as possible, even before graphics card.

1.4 Main Arch OS

As this project is going to be my main computer where I'll do some programming, internet browsing, media consumption, and general use, I want a stable system.

Up until now I’ve been using Antergos, but this time I wanted to install raw Arch. It was extremely easy. I thought it was going to be harder, at least find some difficulties but just following the guide was more than enough. I don’t know why I didn’t even tried before.

In this Arch installation will reside the virtualization platform that I’m going to use (I have not decided which one I’ll use yet, I’m thinking libvirt), but not the VMs themselves. The VM’s (their virtual disk images and configuration files) will go in a separate drive, to which the Arch installation will have access, among with the documents disk drive and the isolated Windows installation. Arch as superuser, as it should be.

My plan B was using Proxmox and having my “main” os installation as a VM itself, but I discarded this because I didn’t have the hardware to support a good Proxmox installation (raid, more ram and a graphics card if I were to have also a Windows installation with the purpose of running games).

1.5 "Clean" Windows installation

Of course I was going to have Windows installation, because games. I wanted to give a try to looking glass a the beginning but I decided that for now I'm good with dual booting. Maybe I'll try that later.

I put clean in quotation marks because it’s been long ago since I don’t own a Windows installation. I think I’ve never owned a Windows 10 installation, so yeah, it’s been a long ago. And because of this, I lost the track of good configuration, “tips and tricks” for making a Windows installation run smooth and performs good, making a good advantage of the hardware.

2.Storage setup

This was what I had to think the most, because I didn’t know how to spare the storage I had in the form of two 1TB drives, one 240gb SSD and one 500GB hdd.

At first I didn’t had that 500GB drive where I’m going to put Windows now, so I decided to put it in a 500GB partition of some 1TB disk drive. Fortunately I found that 500GB drive where I’ll install a Windows system, having it as isolated as possible.

I also knew for sure that the 240GB SSD was going to be used to the main Arch OS installation. I think it would be more efficient to install it on Windows, since a clean Arch installation doesn’t really need a SSD (everything needs an SSD though), and because I’m also experiencing slow performance on disk usage on Windows even having a 7200RPM drive. But for sure I didn’t’ wanted to give Windows the pedestal of handling the SSD. Maybe I’ll buy another SSD for Windows.

As for the 1TB drives, one will be fully dedicated to handle files (media files, documents, programming code, configuration files, programs, ISOS, etc.). And the other one will be partitioned in two, a 500GB partition for all the VMs (virtual disk images and vms configuration files) and a 500gb partition for snapshotting/handle the logging and backing up the vms (among with some documents backup). Writing this I’m thinking that maybe 500 / 500 is not as appropiated, since the VDIs (virtual disk images) will need more space, and snapshotting/logging is not that space intensive. Maybe backing up is. I have a lot of old drives which I plan to use as some kind of NAS (appart from this machine).

I plan to set up all the partitions as EXT4, a very stable filesystem. In the graphic, the ‘files’ drive has tow filesystems labeled because I have not yet decided if that drive will use NTFS for a more universal access, or if it is also going to be EXT4, having it guarded by a Linux installation and not expose it to more unsecure platforms or having it exposed, because if I format it to NTFS, it’s for the sole purpose of using it outside Linux installations.

3.Work distribution

That is a small diagram that displays what I mentioned before. Having a main Arch installation handling the virtualization platform, that itself handles the VM’s that will emulate servers. I mention some of the emulated servers I want to be able to run with the added titem “Lab rats”, refering to virtual machines that exists with some OS installation just for messing around (debugging, testing environment, configurations setups and so on).

And appart from that work path, resides an Isolated Windows machine, that will be accessible at booting times, available through dual booting. Of course Arch will be able to access this since Arch is the superuser here, but this path will only be used probably for logging or fixing Windows when it brakes, or maybe passing some downloads there if I have not set up a shared folder.

4.Problems I had

3.1 X Server on Linux

3.1.1 Problem

 

The X server was unavailable to start from both the installation USB stick among an installed system. Same case where everything would work fine until the X server tried to start has been replicated with 4 linux distributions: Manjaro, Antergos, Arch and Ubuntu. The problem was with the AMDGPU graphics module not being load properly.

3.1.2 Resolution

Mandatory to have at least Kernel 4.18.9. As I looked up this issue on the internet, I found out that in this driver there was a proper graphics module for the Ryzen 5 2400G graphics (vega 11). So keep that in mind.

All the possible solutions I found were executed by passing kernel parameters. Some of the ones I found were:

• nomodeset This would give me bad resolution and buggy graphics
• amdgpu.dc=1 I tried this and did nothing at all
• modprobe.blacklist=”something” I don't remember the value of "something" but what this parameter meant to do was to blacklist the AMDGPU and RADEON modules
• iommu=pt

3.1.3 Fix

The last parameter (iommu=pt) solved the issue for me and let me install Arch linux without issues.

What this parameter does: disable IOMMU bypass, using IOMMU for PCI devices.

3.2 AMD graphics driver on Windows

 

3.1.1 Problem

The AMD Graphics driver for the Ryzen 5 2400G was not being properly recognized by Windows. This was by using the latest AMD driver, downloaded from the official page, and also using the drivers included with the motherboard ASUS B350 (the chipset ones)

3.1.2 Resolution

I found the following possible solutions:

• First install the chipset driver, then the graphics driver (did work)
• You must have the latest update of Windows (1803) (this did work).

3.1.3 Fix

Updating Windows was enough for solving this issue. Even I didn't have to re-install the drivers after updating. Just the update made this work.

Worth to mention: Windows Update application sucks, I had to try to update like 5 times because the updates were failing by unkown reasons. Not explicit messages does not help at all.

I finally succeded updating using the Windows Update standalone program, and not the method available at the control panel (“Check for updates”).

4.Resolutions / next steps

Resolutions

• I did not think I was going to have problems with windows

  But you definitely can’t blame outdated software, and that’s why updates happen, to make new hardware/software work. And the issue I had w/ Windows was fixed by updating, so techincally there was no trouble at all, except for my horrible experience with the Windows update platform.
  Also, I would have liked AMD notifying that you must have the latest version of Windows installed, or have a proper error message other than “AMD Graphics driver is not installed”, just after “Your AMD Graphics driver has been installed succesfully”



• Windows on the 7200RPM HDD is slow
  
  I thought it wasn’t going to be that slow. But I think this is after being used to SSD daily. I won’t be investing on an SSD for now, so I’ll just have to deal with it. I could’ve used the SSD to Windows, but Arch deserved it better.


• Hardware too new
  Not having the needed software for running the Ryzen 5 2400G correctly on the LTS kernel says much (or at least something) about how I should’ve read more before buying. But it’s working with no problems for now, so I won’t mind too much about this. Hopefully I won’t find any bug.

Next steps

• Configuring / personalizing Arch

  I had only installed Arch and got it up and running, but with no configuration at all. I have horrible rendering of fonts in rxvt and i3 is only installed, that means blue background, i3 modules and the status bar at the bottom.
  I wanted to wait to have Windows working correctly (which I have done by now) before proceding to customization.

• Adding Windows to the grub entry
• Setting up the VPS emulation environment After personalizing Arch I must configure my hole emulation environment. I have to decide which virtualization platform will I use. After that, I'll have to create all the needed (or wanted) vms, set up the services I want, configure some auto-start vm's, interconnect them, set up shared folders....a lot and a lot of configuring has to be done.
• Partitioning the drives and dumping files

  I have to create sort of an inverted backup by moving (or pasting) my files that I have spreaded all over several hdd’s, pendrives, online services, cellphone and so on. I want to make an easy to access, understandable and name-ruled directory system. That will take large amount of time and deffinitely I’m not pleased to do this.



• Install some games
  

  I also want to play some games on Windows (that’s the main purpose of the standalone Windows installation), so I’ll be installing games, hopefully I don’t get tempted to do this before having the vm environment configured.

6.Questioning points

5.1. Why?

Because.

Just kidding. Probably more than one people that reads this posts or watches the video will ask “Why?” “Why in the hell would someone go through that much process to create a more obstructed path between the OS and a program?” Because.

Just kidding again. But mostly, that is the only and truly response. I want to experiment and try out stuff. I want to learn. And I want to be prepared when the time of building a real life homelab server comes.

5.2. Efficiency and low ram

I haven't run anything yet. Hell, I haven't even installed a virtualization platform yet. So, I can not provide a detail on how this will flow in real life performance. I know what I can expect. I expect overall good performance. Maybe I'll be running tops 3 vm's with as little as possible resources. Of course that the resources will depend on the services running.

I want to somehow manage the startup of vms and choose what should be running at which times. More on this on the following section.

7.Future adaptations

Once I have all of the software installed, configured and tested (test the performance too). I want to build some kind of automation system for emulating even more a VPS.

• VM's timetable

  For now, just an idea. Create a timetable of which VM’s should be up at which times, and have them autostarted



• VM’s interconnectivity
  
  Grant VM’s the hability to start up other VM’s if needed (like apache starting up DB). Make modular scripts for running more than one VM at once (./web.sh starts up apache and db e.g).


• Shouting IP’s
  
  I’d really really, really like to implement the following in the future: have an arduino with a SIM card that recieves SMS messages with parameters, starts the main OS, starts the VM’s that were recieved as parameters in the text message and shout back the Main OS public IP as a response to that message.

8. Conclusion

This hole project was made for testing only. So I don't expect anything to go as good as perfect, comfortable or high performance (low ram is the key in this con). I'll be pleased with having it to work and manage to have the media server up at all times, or get as low as possible when talking about VM's boot times (including the services startup).

I hope this post inspires and help someone that is facing a similar idea. I was excited about the idea of having a home-made server in my room. And as I saw more and more youtube videos of people showing theirs, I only wanted more to have that. But I knew I couldn’t achieve that. Currently in my country electricity has a high price as well as computer hardware, so this project was totally limited by my economy, even though, I think it went very well. Only thing I regret, and probably you already read this more than 5 times in this post, is low RAM. But I’ll be upgrading that in the future.

EDIT: I edited the Youtube link for it to be embeded.
I also want to mention that this is my first post, I registered just now just because I want to share the hole project in a “live” matter with this forum, where I gained the motivation to do it.
So mainly I have to thank this forum for that!

EDIT2: Formatting

[Goalkeeper]

Oh my god bro.

This is a lot. Thanks for posting.

[frnkquito]

Thank you very much! This is my first post and I didn’t know if this was going to fit here.
I used this forums to gain a lot of knowledge and a lot of stuff I found here helped me to get the push of doing this project.

[frnkquito]

Sorry for the wrong formatting of the first link, I was meaning to embed that Youtube video. I think I can’t edit this topic so I’ll leave it as is.

[Goalkeeper]

Usually videos should embed themselves by just posting the link. But this does not always happen.

To edit you post, you can click on the pencil at the bottom of your posts.

[frnkquito]

Thanks @Goalkeeper!