[Build Log] Home lab in a box (first time trying)-x470d4u Proxmox/ Tech Blog

[HaaStyleCat]

Let’s start at the beginning. I’m a disabled veteran. Medically retired and at 38 I need something to keep busy. I love to learn, as long as its a topic I like, and well technology is something I like. I grew up around it as my father worked for Burroughs/Unisys most of his work life. He also worked at Oracle and Sun Microsystems a while too doing… you guessed it managing Lvl 1 tech support groups at the end. He used to deal with design and matinance of servers, he kinda did it all from the ground up learning electronics. Me working on this has brought alot back to my father who is getting up there, and gave us something to bond over.

My general Idea was to create a VM box to learn in and apply some of my readings such as “Linux Bible” 10th edition, “How Linux Works”, and “The Linux Command Line.” I’m trying to create a near enterprise level enviornment to learn in. The goal being to possibly get some certs maybe do some IT stuff from home or maybe a small computer shop to keep busy. (1-its hard to get out, 2-I do have a brain injury so learning is a little tough, 3-have some physical limitations)

I have already gotten to a point with windows where everything is easy or plug and play then mess with settings or regedit etc. I started with gaming, moved on to doing over clocking with intel and nvidia GPU’s, then applying my skills as a ASE certified mechanic for 10 years to apply water cooling to my PC’s and messed around with close to 15 builds from soft tube to hardline. First ATX sized builds in mid towers to applying it itx builds.

Now I look forward to a new challenge as I have finished my degree in College care of my GI Bill (it was psychology-thought it might be useful…sadly…no) and as I now have 4 computers (I dont need to build any more right now my wallet hurts lol) of which I need to sell a few, I’ve decided Linux, proxmox, and working with “some” server garde parts would be a fun project.

So here we go… (I will add as I go)

Also on a side not I blame @wendell for his video with GN for this …

Building the Ultimate DIY NAS on AMD, Ft. Level1Techs, Unraid, & ZFS (Part 1/2)

Yes, I changed the annoying super bright BLUE light Fractal loves to a RED led. The white disk led indicator that for some reason shines out the bottom I left untouched for now. The hope is to integrate a server type led panel that can hook up to my SAS controller and motherboard to display data like dive indicators etc. I have not mound a module like that yet as Im not sure what its called. I may have to buy a cheap server I can see locally to see if its panel may work then do a custom cutout and mount on the front of the case.

[reavessm]

I was looking at buying an x470D4U2 or the X470D4U2-2T to expand my homelab. Hope it works out for you!

[HaaStyleCat]

So far so good. Having the dedicated graphics and IPMI is a real boon and running ecc unregistered ram is supposed to be good. Im starting with 32 gb, and plan a move up to 64 gb when I need it. Its already built and done so trying to catch up here. Been busy but I have a complete log of what I have done so far. Just need to post it all.

[reavessm]

Do you mind sharing your full parts list? I’ve heard about some ram incompatibilities with the stock wraith coolers

[HaaStyleCat]

This is true, I got a large air cooler and had to modify it using the noctua parts base, that was fun to figure out lol ill get all that posted if not today then tomorrow for sure. She’s up and running with plex gpu pass through unlimited encodes etc and a few vms for me to mess around in so far. Its been a journey.

[reavessm]

well congrats on taking the plunge! Next thing you know you’ll have several more nodes clustered together and you’ll be looking at racking it up with dedicated storage networks and … and … and …

[HaaStyleCat]

I know look forward to it but have to learn linux and networking lol

[reavessm]

Well this forum is a good place to ask questions, as well as r/homelab.

[HaaStyleCat]

Roger on there too. Like the slightly more professional atmosphere here reddits hit or miss lol

[HaaStyleCat]

So, lets start with the build and a few pictures.

Heres what I have-

-cpu- Ryzen 7 2700 (non-x)- No OC (may want to change later but bios is very diffrent on mobo I’m using.

-cooler- modified MSI Frozr L (modified with noctua mount-post later?)

-ram- 32GB Kingston KSM26ED8/16ME Server Premier unbuffered ECC 2666 -this was on there QVL list and was one of the few UNBUFFERED EEC memories I could find. ECC function seems to work great.

-mobo- ASRock Rack X470D4U

• SAS9211-8I 8PORT Int 6GB Sata+SAS Pcie 2.0 (which was flashed to IT mode for use) with 4x 6TB Ironwolf Drives in RAIDZ1 (main data archive writen to seldomly and accessed for use more- ie movies music etc) and a Samsung 883 DCT Series 480 GB SSD for cache - the SSD has power failure protection so it appears there is a capacitor that allows it to finish any writes needed to safe guard it as a cache option and it was only $120.

-Drives- Samsung 970 Evo 500GB M.2 pcie and two SanDisk Ultra 3D NAND 500GB sata SSD’s all in RAID1 (the hope being I can unmount one to have as a roll back.
Seagate Firecuda 2.5in 2TB SHHD with a Samsung 850 1TB sata SSD for cache- this is dedicated as a DVR drive as i can easily replace it when it wears out from recoding etc.

-GPU1 - MSI GTX 1650 Super Gaming X -this is passed through to Plex LXC for transcodes and has patched software to remove limits to transcodes.

-GPU2 - EVGA GT 710 2GB DDR3 64bit Single Slot, Low Profile - this I hope to use as a GPU for a VM I can access and use via KVM (I have not set this up yet, and I have a few ideas about it I’ll have to ask about.

-PSU - I always go over powered so I have a CORSAIR AX Series, AX850, 850 Watt, 80+ Titanium PSU. I wanted Titanium for the low lower consumption. So far its been amazing going between 90 to 99% efficiency at about 80w with this setup.

-Case - Fractal Node 804 with 6 Noctua NF-P12 redux-1700 PWM (I have never used these and always went for the NF-a12’s…I am plesantly surprised at how well these work and are very quiet for my needs) 4 in front for intake on both “sides” of the case, and I used 2 more on the top of the motherboard side. The front fan before the CPU cooler is intake, and rear one after CPU cooler is exhaust. The in the rear of the case on the cpu side it only takes a 120 thats not ver restrictive so I put a Noctua NF-S12A PWM chromax and on the disk drive PSU side I put a Noctua NF-A14 PWM chromax becuase it allowed up to 140mm.

Here are some pictures

That all for today on hardware. Next Ill get into Proxmox and some setup stuff I did like custom CPU cooler mounting and a few other things I ran into or changed.

I dont know whats the best etiquette here on two fronts. Should I just have Imgur links instead of whole photos taking up space? Second, should I just keep one long log or reply to it as I add?

[Zavar]

Interesting fan config in the top. What was the thought process behind that, I found airflow in the Node 804 to be quite good?

Also, it might be worth throwing a fan pointing at the HBA heatsink. A shame the GPU’s so close.

[HaaStyleCat]

Well here was one of the issues I had. I thought about going full watercooled …

…but I decided I’d go air cooling. This led to a few issues.
The Noctua NH-D9L, Premium CPU Cooler with NF-A9 92mm Fan I had from another build was my go to for this. If I kept it facing up or down that probably would have been fine, but I wanted a front to back flow to go with cases flow. This wouldn’t be possible with the mounting suppled by Noctua, as seen here.

Not only did it hit there but it also hit the heat sink on the opposed side.

I also in my “parts closet” a a slightly used MSI Core Frozr L. I chose that cooler because it was “offset” to one side. After I unmounted the top plate and flipped it 180 degrees, it was just how I wanted and would clear the ram and face front to back, but then I ended up with a mounting problem as well being unable to mount it the direction I wanted.

This is the problem people run into with stock ryzen coolers as well.

I needed a solution so I started to look at the two mounting kits. I discovered if I used the Noctua mounting kit on the board, and the two short ryzen adapters with the grey risers for ryzen. I had a good base that matched up with the cross-bar holder for the MSI cooler. I then removed the spring loaded screws from from the noctua cooler by removing the c-clips and attaching them up to the MSI cooler cross bar. Heres a picture of the hardware and it mounted.

This worked out great I had good contact with the CPU and TIM was spread evenly when I removed it to check. I was using the bare sticks of Kingston ECC ram, and as long as they are installed BEFORE the cooler the clearance works fine.

@reavessm You had asked so here’s some info for ya. Hope it helps.

[HaaStyleCat]

@Zavar

Interesting fan config in the top. What was the thought process behind that, I found airflow in the Node 804 to be quite good?

It is but I wanted to add a little more positive flow. I used to use a LSI 9260-8i and I mounted a fan… I may do that again. I may remove the second GPU the EVGA 710, The M-ATX layout is kinda tight making me wish it was a E-ATX or ATX server board.

If I can make it work I might add that. It is on my mind thanks

[reavessm]

@HaaStyleCat

You had asked

That’s exactly what I wanted to know. The CPU cooler was the only thing that worried me. I might end up sticking with LP DDR4 anyway just to be safe.

But thanks for the follow up!

[HaaStyleCat]

Of course, happy to help with anything I’ve had hands on with personally.

[TinyGoliath]

nice lil project you have going ,

I been doing something a similar but with a 3900x instead of 2700
but if you need any help setting up software i try can help,
though I mostly run centOS as i trying to emulate a more industrial environment as I’m trying to learn the environments for dev/production servers.

[HaaStyleCat]

Fantastic, thank you. I will most likely take you up on that at some point. I’ve been debating how far down the proxmox hole to go down because I want to emulate that type of environment. I have had fun setting it up, I do need to make a choice soon. I currently have a Grid K2 I’d like to utilize and it seems I’ll need either ESXi or Xen I think. I forget. This is all new to me so I’m learning linux on the way too. LOL The only thing I have to run is Plex for sure or the wife gets upset. lol I don’t know, just messing around at this point and learning some good lessons along the way. Thanks for the support.

[TinyGoliath]

@HaaStyleCat

good lessons along the way. Thanks for the support.

dude I’m more than happy to help, as for plex I’m planning on setting it up in a container such as docker or podman most probably podman as its a bit more secure than docker and native to cent os.

doing it in a container means if you exposing it to the rest of the world you limating how much of the os you are opening up to though who might want to steal the data/resources on the rig.
P.S I’m still leaning too

[HaaStyleCat]

Nice, Yeah I set my mine up in a LXC container and did a GPU passthrough to get it up to snuff so far. I have my other Distro’s of Linux set up. I used native Samba and ZFS to set up my NAS, there’s a lot done, but I need to learn a lot more bout safe networking and get my mail notifications set up but there’s a extra layer here I’m not sure how to deal with or set up properly. I have rights to a domain, don’t even know what I can do with that LOL.

[TinyGoliath]

@HaaStyleCat

ail notifications set up but there’s a extra layer here I’m not sure how to deal with or set up properly. I have rights to a domain, don’t even know what I can do with that LOL.

For security I would look in to fail2ban its a tool that will help stop brute force attacks.
as for a domain it depends what you want to do with it I mostly use mine for web hosting.

my current set up is zfs but centos and Zfs require minor work arounds to get it all working