[Build Log] Home lab in a box (first time trying)-x470d4u Proxmox/ Tech Blog

@TheCakeIsNaOH Got it working, it says not secure so I’ll have to look into that but I was able to get a message through to my email successfully. Now I need to figure out how to do the same for ZFS and figure out how to set up some automated processes for zfs for my database. Thank you for the help!

1 Like

This is the next one I need to figure out…lol AND how to get it to send a test email lol

1 Like

Well after building a now 4th pc in the collection to monitor my Proxmox server, as seen here…


I have now discovered I can install X overtop of it! I dont know why I didnt think of that… I’m holding off at the moment as I WANT to learn the command line.

For now I have this build to mess around with to learn linux a little better so I dont have to make detours to google as much for small commands I should be familiar with if I’m taking on a project like this.

I have decided to use Fedora which is a different base from Debian from what I understand. With a different philosophy on software. It uses different packages too so good to learn on something different to expand on my knowledge.

I haven’t been able to get some things automated on the server yet for maintenance or any security besides whats base at the moment, so I’m hoping working with linux through some of my books I have here will help and I will be able to come back to this project strong and have a better understanding of what I’m doing and how to ask more intelligent questions of this group let alone know what to google before I ask a stupid question I cant phrase correctly lol.

All in all I am proud of the progress so far just 2 months in tinkering around and messing with linux. I appreciate everyone’s patients with me and my incessant posts lol.

2 Likes

I have managed to get proxmox mail to work. VM’s are humming along for playing with different linux distros as I work through my Linux books to learn. Plex is great in its lxc container with gpu pass through.
I recently added pi-hole which has been interesting to say the least. Didnt realize how may pop ups and windows I used to have to navigate at sites searching the web. I also didn’t realize how easy it was to implement. I still haven’t gotten it to work network wide yet, but I have individual machines using it so that works for now.
I belive I have implemented automation for my zfs scrubs and reslivers. I don’t know if those need to be done :confused: :thinking: I’ll look it over some more.
I think the next goal is a second node and maybe a pi or mini pc with attached storage as a backup node. From what I’ve seen that is a different install? I don’t know but I’m going to try it here soon just to see. As we’re having some major work done electrically in our area I have invested in a upset I need to integrate to the server somehow for a clean shutdown if powers out for more than 20 min or so. Thinks that’s my next step. Been really fun learning all the ins and outs and being able to actually see what your pc is doing. I still haven’t got the second gpu to pass through to a vm without killing the lxc pass through, but its not needed right now so I’m holding off.

1 Like

Well I am running into my first issue with my server after a side track down playing with a new Raspberry Pi 4B. THAT was fun for sure!

I am having high I/O useage spikes in Proxmox that is Installed On ZFS on my system listed above in this thread.

The process I have found that is eating up I/O but NOT doing any read or writes is [txg_sync] and [z_trim_int]… I have found some information from searching but not definitive answers on fixes.

The problem seems to occur when plex is recording to my DVR drive which is a SSHD (I used this because it was cheap) that also has a 1TB samsung SSD as cache and SLOG. Yet plex is installed on the rpool. Did I make a mistake using this configuration? Should I have installed directly onto the DVR drive? I dont know if plex is DVR recording to the rpool first then to the DVR drive and that is slowing things down… I could really use some help though if anyone has some ideas.

Well, back to the home lab. Stepped away from some other projects. I’m still working on the Silverstone RVN03 with custom waterloop and plexi side window.

I’ve been having some issues attempting to set up access with another user so I dont use root. I’ve been reading best practices and other pages. I’d like to make sure I always log out (I hear theres a script or reminder of how to do that), and make sure theres no keystroke logs, and any other good practices. I attempted to set up SSH keys but I dont know if I need to do that with sudo or root. Does anyone know of any suggested reading for basic security steps for a debian based system. (Proxmox)

I also wanted to see what basic cron jobs I should have. I believe I got ZFS modified to do a scrub every other week. Is there anything else I should have going besides updates?

I appreciate any resources you all can contribute.

This should do it for your SSH keys. Try it without sudo first.

ssh-copy-id [email protected]

Thanks for that, Im going to give it a try.

I seem to have a bigger issue, I may have used the wrong command to create a user as the user I have created receives this message all the time unless I “sudo” the command

-bash: *command here- IE iotp*: command not found

So I cant run any commands, I belive it has to do with access to a file??? I also had to manually create a home folder, and now when I tried to install SSH keys they just installed to my default folder for the user

[email protected]:~$ ls

jrhaa_key jrhaa_key.pub

Any ideas on a fix for this? Should I use a different command than "adduser "?

update: Nevermind I have a book and I will attempt it with the Linux Bible as a guide :slight_smile:

Well I’m back. I’ve ironed out some issues with the server. I belive I have permissions taken care of, accounts set, basic security in place, plex humming, zfs drives set and on a schedule, using promox to be my samba share for plex as well as home serving of files and a combined backup location for pc’s, got pie-hole working…now I’m searching for more I can do at home. I’m considering a steam cache as I have many rigs and it would save my data cap, but I wondering if there is more. Any tips on starting a web page? Hosting it myself maybe…I need to create one as well…and double check my security. I have a domain I am paying on…mail server? I don’t know… are there any fun projects people can think of? I’d appreciate any input.

I am back again. This time with a second server to mess around on with consumer parts. This is presenting a challenge with the 2.5G built in nic not being supported by either Citrix or VMware. I can’t get past the install. I had this issue before, its a MSI B550 Mortar (Non-wifi) with the 2.5G realtek lan I believe.

I had attempted to recompile the install for VMware using their tool and supplied 3 different drivers each one at a time individually, but to no avail. I’m continuing to pursue solving the issue via googling, reading tech doc provided by VMware. and Xen Server.
I had also attempted to use the VMware windows tools to recompile with the divers added but keept getting errors that it could not execute any of the applets to redo VMware.
I continue down this path because I have a working server. I do not want to mess that up, but I do want to play with another hypervisor. Installing one on a VM wouldnt give me the experiance I want of figuring out pass throughs and setting up data sets, shares, and program integration. I was also spraked by @wendell s video on dockers to want to give that platform a try on this secondary server.
I have fedora 33 installed, and am looking at maybe just using Boxes to try out docker.
I also, created a full network map with “Draw.io” which I found on Linux as well as a windows version both available in software repositories and am very pleased with the results.

I imported pictures of all my items, have logical and physical paths as well as physical hardware and virtual environments. As I learn more I’m going to try to set up subnets. I’m looking for a way to make my drives with personal data only available locally. I’m sure I’ll figure it out as I go.
I also need to figure out how to utilize the second NIC on the ASRockRack X470. It seems to be inactive according to Proxmox, but my dashboard for the board itself shows as activated.
I am also going to try to play with windows server because I belive that is still popular for use in enterprise enviornments? Maybe not?
Either way I’m continuing to learn. I have changed the SSH port from the default. I originally had a very high number till I was told that wasnt much better because it can be scanned or monitored. So i stuck with a good low number I could remember.
I also plan on looking into/resetting my SSH certs, possibly setting up a HTTPS for home. I dont know what the best security is besides my very long alphanumeric caps and lowers with special caracters passwords.
Still learning and still having fun.

Well new PC I built is worthless to me with a MSI B550 Mortar non-wifi edition. The NIC on it is incompatible with VMware ESXI 6.7(uses Realtek Nic RL8125). There are plety of guides for other NICs and pletey of files for the other NICs, just not this one. There are also guides stating that people have the code but cannot post it. So that leaves me to compile it on my own but the very few guides I found on that were a bit to beyond me. Citrix as well is a bust as the same issue is involved with that install. Windows 2019 server works…if I use the install cd for the windows driver. Not my top pick for possible new Hypervisors to try…Next I may use the PC for target practice, or as a wheel chuck and forget about it when backing up the truck lol. Not really but is frustrating. I guess I can try to find a older USB nic to maybe try getting it working. I dunno. Been 3 days straight of trying all day to get it to work, pissed off the wife Im spending so much time, but once I get an idea in my head I have to make it work. Either way, dont want to take down the proxmox server thats working great at the moment so I guess I will revisit this maybe monday and see where I can go from here to try to install a diffrent hypervisor to play with…

I have now attempted to install just debian then install proxmox packages on top of it, yet again the 2.5 realtek nic impedes the install and stops it dead in its tracks. I managed to find the drivers for the linux kernel but it is in a tar.bz2, and I dont know what to do with that to put it on a usb so debian can read it and install the driver for use. Im going to see if there is a way to load it before I start the download maybe with just the bash? I’m kind of at a loss at the moment. I may just have to scrap the board and try and use another one…

Small success. I had managed to get the pc to boot AND have a network connection using a version of Debian 11 (Bullseye). I was able to use the USB to run the program but when I try to install it creates one partition then says it failed to make the partition and exits installing. I am getting closer… Once I can get soemthing installed, I will try to install proxmox over it.

Debian 11 is now working on my MSI B550 mortar with a 2.5G Nic! It was super slow and unresposive like when you use a KVM with the wrong video driver and CPU useage was very high on one core… After Installing the nvidia driver using the repository “deb Index of /debian bullseye main contrib non-free” added to the /etc/apt/sources.list and doing # apt install nvidia-driver firmware-misc-nonfree.

Now time to attempt adding proxmox overtop of debian 11!

Proxmox over debian was a flop. Debian 11 was the only thing to work with the b550 motherboard and 2.5gb nic. The changes in its structure from what I understand made it incompatable with the current version of proxmox and I can’t find a development branch.
I decided to move on to possibly trying nested virtualization with proxmox in hyper-v on windows 10 pro. I chased my tail through roughly a dozen tutorials trying to get proxmox to connect to the internet. I tried internal, external, a second nic, gen 1 and gen 2 VMs, I tried legacy connections all kinds of things. I can not seems to get it to work properly. I know there’s more steps to enable nested virtualization but I can’t even get that far. Esxi was equally as unsuccessful on the same machine. I was able to get a usb ethernet connector to work on esxi 6.7 but thay would only be a trial for 60 days. I may go back to that. My other option is wipe my daily driver Asrock mini x300 with the ryzen 4650g and try that platform. Over all not much success recently…
The $299 Dell poweredge r520 is looking better and better lol.
Or… tearing apart the build and replacing the b550 with a asus rog x470i…which I know has issues with the chipset drivers for monitoring that can be overcome with asus-wmi files…decisions decisions…

My Proxmox machine has an Anker USB ethernet dongle because of the same driver issues with the 2.5G controller. Haven’t had any issues with reliability using it.

You can install the 2.5G controller’s driver once you’ve got a connection with the USB one, but I’ve not bothered since everything in my house it gigabit, and I’ve had some stability issues with this controller on my Windows desktop.

Success has been achieved finally. I can not belive how powerful notepad or a pen and paper are… I discovered that the process of installing Proxmox as a nested VM in Hyper-V all came down to the ORDER of the steps.
1.) Create the VM like normal as a type 1 vm on hyper V with the proxmox iso image mounted as a cd. Finish this process. DO NOT START THE VM.
2.) Delete the network adapter of the created vm.
3.) Create a new network connection that is external and check box to share with management os (ie windows). This will drop your windows connection momentarily to the internet.
4.) Add new network adapter to VM as a legacy adapter and select the external network connector you created.
5.) Start the vm and install proxmox and it should read and create a proper ip congruent with the range on the network adapter you selected!
6.) Dismount or removed iso DVD and start up. Proxmox should be running and able to update and be logged into by another pc or host pc on the network.

-----------------------------------------------HUGE UPDATE------------------------------------------------------
Soon I will be moving up to real and dedicated test server. The above build using the Asrockrack X470DU and ryzen 2700 is complete. Once I have the new server I can attempt some more creative server programs and home services without fear of interruption of the services it provides.

I have decided on the Dell Poweredge T320 server as a first toe to dip into the enterprise server pool. I got it with the Xeon E5-2450 Eight Core 2.1Ghz (2.9 boost), 64GB ECC ddr3 ram, dual 750w PSU’s (the dual 450w is not recommended if you plan to use a GPU), and the front is setup for 8 Hot swap 3.5in HHDs and H310 sas pcie card. I’m happy with the purchase (Haven’t received the unit yet). This form factor and platform I chose because of availability of parts, ease of upgrade with interoperability of boards and the like (IE- if I ever need dual CPU’s and more ram for my needs).

Some of my preliminary research has involved the Nvidia Grid CPU ecosystem. I had seen a few videos from Craft Computing involving making a “Game Server” with virtual GPU’s out to thin clients like a Raspberry Pi. It seems from his findings that it is possible, but overall performance is limited, and set up is particularly limited to certain hypervisors and versions. As well, the type of card you can use is limited due to needing an enterprise licence to use some of them, and needing to modify some cards for use as a graphics card vs a compute card. Very interesting series if you have any interest in it.

I did a little more research on Nvidias website and found this …

On page 16, section 1.4 it outlines that GRID gpus support all windows VM’s (Provided you meet all the criteria), yet the GRID K1 and K2 seem to not support linux VM’s… only the Tesla series seem to. There is one ray of hope. It is mentioned in the document that drivers may have updated information.

I mention all of this because this is one ave of investigation I plan to do with the new machine.

I also researched the iDRAC unit that provides the IPMI service. It appears that the basic IPMI services are provided through the unit. I hope this is sufficient and I wont need to get a licence for the software to communicate with the server.

I also plan to utilize a server management software possibly like co-pilot or something to manage both servers.

Thats the plans so far. Along with setting up a rsync possibly or other automated services. I originally planned to utilize cron jobs, but it was suggested to me to use systemd timers, so I plan to educate myself on both.

1 Like

Well, so far t320 has not worked out as hoped. I am at a cross roads. So far attempting to use the h310 sas card in IT mode has broken the ability to use any drives attached to it to boot from, as well as making them inaccessible in ESXi and possibly citrix. I can install windows server or proxmox to one of the drives, but the Dell system won’t boot from it.

I may try reverse breakouts from a sata card to sas backplane, or forward breakouts from h310 sas card to the drives directly bypassing and removing the backplane, or something else I haven’t thought of yet. I am ordering a h310 thats flashed to IT mode to see if maybe I messed up because I have seen a video of it populating in the Dell Life cycle software. I may also boot up into windows server and see if I can update Dell software and drivers. Either way I have more parts coming to play with to see if I can solve this.

Or I can just give in, re-flash to original raid card configuration to set up hardware raids, then lose ability to use zfs…then maybe able to use ESXi finally… or leave it the way it is and continue to just use proxmox, or mess with citrix some more to see if I can get that to work.

Either way, it’s keeping me busy.

Are there any m.2 slots or additional sata ports on the board which you could use for boot media? I could be wrong, but booting directly from the hba may be an atypical use case.

This is my step right now as I install windows server 2019 for updates… I hope

I added a pcie card with a pcie and sata slot… the pcie slot isn’t recognized by dell either at the momemt.