New workstation build

[timbeauhk]

You seem to have 4 NVMe on board and two on the expansion card, yet you have a 4 drive zpool and a 2 drive zpool.

2 of those on-board NVMe slots are on chipset lanes. Personally, I would not mix them. I would leave the two chipset slots empty for now.

[ahowell]

I was going this route as it would be easier to add 2 more NVMe to the AIC than pulling the GPUs back out after draining the loop. What are the dangers of having a zpool with 2 of the drives in the CPU lanes, and the other 2 in the chipset lanes?

[timbeauhk]

Not sure of “danger”, but more performance and latency.

ZFS will want to complete all IO across your striped devices, so everything risks being as slow as the chipset lanes if stripes cross them, and if you are using other high bandwidth devices, this will get worse even if your entire zpool is on the chipset.

Such high bandwidth devices are your wifi, dual 10GbE and USB 3.2 which can be non-trivial. These all run from the chipset.

Note also 4 of the SATA devices contend with one of your M.2 slots, so make sure you connect your Reds outside ports 4-7.

HTH

EDIT: maybe flexible tubing?

[ahowell]

For some reason this issue hadn’t crossed my mind, but I believe you are correct. I guess I got too distracted with other details. Thank you for pointing this out!

[timbeauhk]

No problem.

Anytime.

Hope you really enjoy your rig.

[Ethernet_Warrior]

Excellent quality post. I hope you enjoy the new build. Looks amazing!

[AirMAn23]

Hi, did you finish this build ? I am interested to see what temps are you getting on the 5700xt with the waterblock. I have this card with a default fan and a custom loop with a 3900x.

[ahowell]

I haven’t been able to focus on this project as much as I anticipated recently due to scope creep on another project.

Project mini-me

I’ve been putting extra time into finishing some house projects before our little guy joins us in early march. Working to finish a basement remodel and get the little guys room ready (little more to it than just painting).

But I have been slowly making progress on the wiring. I’ve got the 24 pin, 2x 6pin (CPU), 4x 8pin (GPU), and 1x 6pin (mobo aux). This was the first time I’ve done custom wiring, so I wasn’t necessarily quick/efficient at what I have done. But I have decided to delay the rest of the custom cabling just so i can move forward and get this thing running.

My next steps:

• Setup a small (120gb) windows SSD purely for controlling all the unicorn vomit (turning it all off, or at least leave some on as white)
• Pull out the 5700 XTs and put in another GPU so I can get Proxmox setup on the 2x 240 GB SSDs (mirror). For some reason I cannot get through the Proxmox installation when using the 5700 XTs
• then finally put the 5700 XTs back in, setup the water cooling

[vljio]

The EKWB stuff is real nice.

I’ll be interested to hear how you make out on doing passthrough on Proxmox.

Right now I’m having a pretty rough time getting a Windows VM, particularly USB passthrough working on this same mobo/cpu in Manjaro.

GPU is able to get passed through no problem though.

[ahowell]

Are you trying to pass through the controller or direct devices? I haven’t really had any issues with USB passthrough under Proxmox (have been using Proxmox before this build)

[reavessm]

FS will want to complete all IO across your striped devices

Isn’t that only true for synchronous writes? Normally you would add a SLOG to fix that, but you can’t really get any faster than those NVMEs, unless you add Optane and move some of the NVMEs to the AIC to make room?

Also, it’s still going to be dummy fast, so personally, I don’t think this is something to worry about… Especially with PCIe 4.0 to the chip, although I don’t know exactly how much bandwith will be an issue

[ahowell]

Finally got the machine mostly together. This was my first custom water loop and it was a hard line at that, but it was super fun! I’m now moving on to getting software installed and configured, but will be updating with some completed build pics soon!

Here are a couple I’ve managed to get so far…

I spent quite a bit of time planning the loop to be as least obstructive as possible, especially around CPU and memory area, and am pretty please with how it turned out. When I get the rest of the pics up, you’ll be able to see the complete loop.

[noenken]

Damn that first shot screams flux capacitor.
Generally when it comes to fancy looks on PCs…

But that looks awesome!

[MazeFrame]

That is looking very sexy

[FeeNiX]

Did you have any trouble fitting the TRX40 Aorus Xtreme (XL-ATX size and ‘armor’ backing included) into the O11D-XL? Any surprises or gotchas?

[ahowell]

@FeeNiX No troubles at all.

Just keep in mind that because almost all of the connectors are right-angle on the right side of the board, if you choose to use a 360m rad on the side, it will be a tight fit for some connectors. I ended up using 2 of these for each of the front-panel USB connections and just used a sharpie to color the plastics black. This made a ‘U’ shape connector where I could then plug the USB in behind the motherboard.

But everything else is perfectly doable. Nothing really out of the ordinary!

[FeeNiX]

Thanks!

[darkenergy2020]

Hey bro, do you have some way of contacting you? I’d quite like to speak to you to try and get some help, I’ve got similar components and I’m about to set up hackintosh also.