So, I'm going full headless on a server which hosts my VMs. It's pretty basic, half tb HDD, 4gb ram, host OS = Ubuntu Mate 15.04, uses virtualbox to create the vms. It hosts several separate VMs like a file server, VPN server, local webserver, and a print server. Lastly, all the VMs are CLI centos7 instances.
Since I need to go to headless on this box I set it up with VNC so I can manage the host OS to create and manage the instances. I use SSH for the actual admining of the VMs.
Ok two things -> should I get a real hypervisor like proxmox for this? And, if so, upgrade the hardware for this machine?
Please keep in mind, this is at my house, and I use this server for some local services for my house and as a lab environment. Last thing, my next lab is going to be a cluster of web servers behind a load balancer. Then I want to manage it with Ancible. I kind of just want to get my lab set up properly before moving forward.
Absolutely. Virtual box is for use on the desktop, not a server. Get proxmox, VMware, KVM etc. Remove the GUI from the server and manage it from the web UI or over SSH. You can probably also have Ansible manage the VM host to handle bringing up and taking down guests, and possibly also building guest VMs.
You could also consider managing some tasks in Docker to reduce load of having full VMs. If you decide to go full Docker, it may be easier if you replace your host OS with something like CoreOS.
Both Proxmox and VMware ESXi can be managed through a web-app, Proxmox supports containers and VM's so also might be a better fit for your limited resources.
Some extra RAM definitely would not hurt and would allow your workload to grow. I would also add a second HDD and run them mirrored just in case. I always find a HDD failure happens at the worst possible time - even if it is just labs.