This is true, I got a large air cooler and had to modify it using the noctua parts base, that was fun to figure out lol ill get all that posted if not today then tomorrow for sure. She’s up and running with plex gpu pass through unlimited encodes etc and a few vms for me to mess around in so far. Its been a journey.
1 Like
well congrats on taking the plunge! Next thing you know you’ll have several more nodes clustered together and you’ll be looking at racking it up with dedicated storage networks and … and … and …
1 Like
I know look forward to it but have to learn linux and networking lol
1 Like
Roger on there too. Like the slightly more professional atmosphere here reddits hit or miss lol
1 Like
So, lets start with the build and a few pictures.
Heres what I have-
-cpu- Ryzen 7 2700 (non-x)- No OC (may want to change later but bios is very diffrent on mobo I’m using.
-cooler- modified MSI Frozr L (modified with noctua mount-post later?)
-ram- 32GB Kingston KSM26ED8/16ME Server Premier unbuffered ECC 2666 -this was on there QVL list and was one of the few UNBUFFERED EEC memories I could find. ECC function seems to work great.
-mobo- ASRock Rack X470D4U
- SAS9211-8I 8PORT Int 6GB Sata+SAS Pcie 2.0 (which was flashed to IT mode for use) with 4x 6TB Ironwolf Drives in RAIDZ1 (main data archive writen to seldomly and accessed for use more- ie movies music etc) and a Samsung 883 DCT Series 480 GB SSD for cache - the SSD has power failure protection so it appears there is a capacitor that allows it to finish any writes needed to safe guard it as a cache option and it was only $120.
-Drives- Samsung 970 Evo 500GB M.2 pcie and two SanDisk Ultra 3D NAND 500GB sata SSD’s all in RAID1 (the hope being I can unmount one to have as a roll back.
Seagate Firecuda 2.5in 2TB SHHD with a Samsung 850 1TB sata SSD for cache- this is dedicated as a DVR drive as i can easily replace it when it wears out from recoding etc.
-GPU1 - MSI GTX 1650 Super Gaming X -this is passed through to Plex LXC for transcodes and has patched software to remove limits to transcodes.
-GPU2 - EVGA GT 710 2GB DDR3 64bit Single Slot, Low Profile - this I hope to use as a GPU for a VM I can access and use via KVM (I have not set this up yet, and I have a few ideas about it I’ll have to ask about.
-PSU - I always go over powered so I have a CORSAIR AX Series, AX850, 850 Watt, 80+ Titanium PSU. I wanted Titanium for the low lower consumption. So far its been amazing going between 90 to 99% efficiency at about 80w with this setup.
-Case - Fractal Node 804 with 6 Noctua NF-P12 redux-1700 PWM (I have never used these and always went for the NF-a12’s…I am plesantly surprised at how well these work and are very quiet for my needs) 4 in front for intake on both “sides” of the case, and I used 2 more on the top of the motherboard side. The front fan before the CPU cooler is intake, and rear one after CPU cooler is exhaust. The in the rear of the case on the cpu side it only takes a 120 thats not ver restrictive so I put a Noctua NF-S12A PWM chromax and on the disk drive PSU side I put a Noctua NF-A14 PWM chromax becuase it allowed up to 140mm.
Here are some pictures
That all for today on hardware. Next Ill get into Proxmox and some setup stuff I did like custom CPU cooler mounting and a few other things I ran into or changed.
I dont know whats the best etiquette here on two fronts. Should I just have Imgur links instead of whole photos taking up space? Second, should I just keep one long log or reply to it as I add?
2 Likes
Interesting fan config in the top. What was the thought process behind that, I found airflow in the Node 804 to be quite good?
Also, it might be worth throwing a fan pointing at the HBA heatsink. A shame the GPU’s so close.
1 Like
Well here was one of the issues I had. I thought about going full watercooled …
…but I decided I’d go air cooling. This led to a few issues.
The Noctua NH-D9L, Premium CPU Cooler with NF-A9 92mm Fan I had from another build was my go to for this. If I kept it facing up or down that probably would have been fine, but I wanted a front to back flow to go with cases flow. This wouldn’t be possible with the mounting suppled by Noctua, as seen here.
Not only did it hit there but it also hit the heat sink on the opposed side.
I also in my “parts closet” a a slightly used MSI Core Frozr L. I chose that cooler because it was “offset” to one side. After I unmounted the top plate and flipped it 180 degrees, it was just how I wanted and would clear the ram and face front to back, but then I ended up with a mounting problem as well being unable to mount it the direction I wanted.
This is the problem people run into with stock ryzen coolers as well.
I needed a solution so I started to look at the two mounting kits. I discovered if I used the Noctua mounting kit on the board, and the two short ryzen adapters with the grey risers for ryzen. I had a good base that matched up with the cross-bar holder for the MSI cooler. I then removed the spring loaded screws from from the noctua cooler by removing the c-clips and attaching them up to the MSI cooler cross bar. Heres a picture of the hardware and it mounted.
This worked out great I had good contact with the CPU and TIM was spread evenly when I removed it to check. I was using the bare sticks of Kingston ECC ram, and as long as they are installed BEFORE the cooler the clearance works fine.
@reavessm You had asked so here’s some info for ya. Hope it helps.
3 Likes
It is but I wanted to add a little more positive flow. I used to use a LSI 9260-8i and I mounted a fan… I may do that again. I may remove the second GPU the EVGA 710, The M-ATX layout is kinda tight making me wish it was a E-ATX or ATX server board.
If I can make it work I might add that. It is on my mind thanks 
2 Likes
That’s exactly what I wanted to know. The CPU cooler was the only thing that worried me. I might end up sticking with LP DDR4 anyway just to be safe.
But thanks for the follow up!
1 Like
Of course, happy to help with anything I’ve had hands on with personally.
1 Like
nice lil project you have going 
,
I been doing something a similar but with a 3900x instead of 2700
but if you need any help setting up software i try can help,
though I mostly run centOS as i trying to emulate a more industrial environment as I’m trying to learn the environments for dev/production servers.
1 Like
Fantastic, thank you. I will most likely take you up on that at some point. I’ve been debating how far down the proxmox hole to go down because I want to emulate that type of environment. I have had fun setting it up, I do need to make a choice soon. I currently have a Grid K2 I’d like to utilize and it seems I’ll need either ESXi or Xen I think. I forget. This is all new to me so I’m learning linux on the way too. LOL The only thing I have to run is Plex for sure or the wife gets upset. lol I don’t know, just messing around at this point and learning some good lessons along the way. Thanks for the support.
dude I’m more than happy to help, as for plex I’m planning on setting it up in a container such as docker or podman most probably podman as its a bit more secure than docker and native to cent os.
doing it in a container means if you exposing it to the rest of the world you limating how much of the os you are opening up to though who might want to steal the data/resources on the rig.
P.S I’m still leaning too 
Nice, Yeah I set my mine up in a LXC container and did a GPU passthrough to get it up to snuff so far. I have my other Distro’s of Linux set up. I used native Samba and ZFS to set up my NAS, there’s a lot done, but I need to learn a lot more bout safe networking and get my mail notifications set up but there’s a extra layer here I’m not sure how to deal with or set up properly. I have rights to a domain, don’t even know what I can do with that LOL.
For security I would look in to fail2ban its a tool that will help stop brute force attacks.
as for a domain it depends what you want to do with it I mostly use mine for web hosting.
my current set up is zfs but centos and Zfs require minor work arounds to get it all working
1 Like
Nice, that’s awesome. Yeah before I host anything like a web page or something I’ll probably learn some security first lol. First I have to master the mailing system as I’m a little confused there because the mobo seems to have a firewall and proxmkx does as well…but its also deeper in my net structure so protected by asus’s AI security. I really have to figure all that out first I think. The goal i think would be to filter everything through piehole on my server maybe, but that may be an issue with mixed network at the moment. Im not sure yet.
Looks like I now I have a few issues I am trying to solve.
1.) I’m trying to setup the mail system for proxmox, zfs and the x470d4u motherboard. On this one im not sure if I need to do some fancy network changes because the board appears to have its own firewall and im not sure if that’s just for the IPMI or not. All in all im confused. Maybe I can host my own mail server on proxmox and have it forwarded to my Gmail? I’m not sure I’ve messed around a bit but no luck so far.
2.) I’m trying to set up a vm that uses a second gpu to output video and that uses USB devices connected to the server directly as a sort of monitor for proxmox using Ubuntu or some other GUI based linux distribution. I’d like to be able to pass through things like the super IO to monitor temps and cpu usage etc. Is something like this possible?
3.) For the above number 2 I can’t seem to pass through a gpu to a vm without breaking the current pass through to a LXC container. As soon as I add the modules for vpio or whatever they are called and reboot it kills my ability to use my 2nd gpu with plex on the lxc. Nvidia-smi command returns nothing on both host and lcx. I don’t know a way around this as all searches have to do with setting up a gpu passthrough. Any advice on these issues would be appreciated.
It is possible that you are trying to do something more that might require a server, but normally you should be fine to just send email via SMTP.
SMTP is basically the standard way to log in to a server and send an email.
Not via PCIe passthrough. If you managed to pass them through via that, they would not be able to read the sensors on the host.
You could look into either some sort of remote monitoring in a web browser, or connect to the host from the vm via a emulated serial or over ssh. Then just access it like a normal terminal.
1 Like
Thank you for the help and the link. I’m going to try that link. I may be worrying the motherboard is interfering for nothing. I have some time today and am going to put the work in today some more now that the man caves been cleaned. 