Building a general-purpose home server

Some SM boards dont have specific cpu fan header so it should boot up but its very risky! i mean super risky! its not a normal pc board that you get ram check after 5sec. its gonna be at least 2min until you can check the ram.

All CPUs can run for a little bit without a cooler, I think as long as you do not leave BIOS it should be fine. Be exceedingly careful though, and do be aware you are running out-of-spec.

Adding a fan blowing right at it would be possible, it’s just the radiator itself i’d be missing for tests (not anymore).

The question was, considering EPYC’s requirements for specific mounting pressure, would it even start with no radiator applying pressure on it?

The most we would be able to add to the cooling - is unbolted LGA3647 radiator put on top of the CPU.

I think that could be the case because they probably designed around it. There are so many LGA pins on it that if you look at the force you need per pin. and multiply that with the number of pins you get a really high force.

That force will create a slight bend in the heat spreader and PCB because the bracket is only on the outside edges of the cpu. So you have a chance that the pins in the middle don’t make good contact. I wouldn’t chance it!

Added the cooler:

It's gonna be a tight fit indeed...

To nobody's surprise - the space between the ATX power connectors and the 120mm coolers is barely there:

IO shield i ordered from half the country away turned out to fit perfectly:

Overwiew of the build so far

Tests from the day i went to buy the RAM:


Right now i’m interested in variants of how to fit the ATX power cables there.

  1. Use 90 degree adapter, and just redirect it upwards again
  2. Use lower-profile coolers, like 10mm or 15mm tall. Maybe these will have enough clearance for the cable to fitover them.
2 Likes
  1. put the fan on stand-offs, route the cable between fan and case, close the gaps between fan and case using duct tape

For the power cables - these are gonna be quite tall. And then it’s gonna interfere with front panel cables (they already block me from installing a grill on the fan without cutting it), or with the HDD cage. The easiest way would be to get 80mm coolers, which are 15mm tall, but at this point i really don’t wanna spend extra.

Radical mod would be to replace the motherboard connectors.

1 Like

Also, the problem is with fitting the solid connector part inbetween the motherboard and the fan, not the cable itself. There’s like 1-1.5cm gap, which has to fit the connector and some room to bend the thick cable.

Now we’re talkin’! Got meself Deepcool PQ750M and now server has power! Alas, no storage yet…

Cable management is really tough...



something like https://www.amazon.com/Cable-Mounts-Self-Adhesive-Clips/dp/B07954LJQY/ can help greatly. i am not going to endorse any sellers/brands as i have had my 1000 pack for a while, even though they get used in basically everything i build.

Next steps are:

  1. Get some sort of GPU, just for video output. i suppose even crap like GT730 will do for now.
  2. Get a KVM switch, so i won’t have to replug my keyboard every time i want to use the server (already in the works)
That one

image

  1. New network setup:
    Currently, all the devices are expected to be connected through a 100Mbit router (Keenetic City). Since getting anything faster than a gigabit router is gonna be too expensive, i want to get a pair of mellanox cards, specifically 40Gbit MCX354A-FCBT ones. The question is - will they work between Win 10 and Win Server 2019/2022? Which cabling and transievers will i need to use? Length of cable won’t be more than 2 meters. Also, it’s gonna be connected like this:
    Server connects with 40Gbit to PC, and with 1Gbit to the router.
    PC connects with 40Gbit to Server, and with 1Gbit to the router.
    Router then goes to the internet.
    So it’s like a triangle? Will this work, or it’s gonna disrupt the network?

P.S. Will that cable work with the aforementioned Mellanox cards?

Yes this works, but you will have to setup some routing rules to ensure traffic between the two systems go over the 40gbit nics instgead of taking the slower path. That or just give them fixed IPs on a different subnet.

I also have a H12SSL, you should note that there is no REBAR support for this board, but I am working with Supermicro to get support built in. The mellanox cards greatly benefit from resizable bar.

2 Likes

I’m using TYAN S8030 though. That triangle connection is there to ensure both devices will be able to connect to the internet regardless which one is currently turned on.

Yes, this will work.

Lets assume this topology

If you want to talk to the server over the 40gbit link, use the 192.168.2.0/24 network. All other network traffic that goes off net (ie, anything except for 192.168.0.0/16) will go to the default gateway at 192.168.1.1 over the 1gbit link.

There are ways to keep them on the same subnet and assign the same IP to both network interfaces, but this is more complex and in your case, likely not worth the effort.

2 Likes

What about the cabling? Will the one i linked before work?

As QSFP+ is a standard, provided the seller is honest about the quality of the cable and the rating of 40gbit, it should just work. I have no experience with that specific card though and the only way you are going to find out is to try.

It’s ALIVE! Turns out, that ASPEED graphics adapter is just fine for WinServer2019 desktop, so i don’t hav to buy some basic stuff just to get GUI.

2 Likes

Got both Mellanox cards today. One is gonna be running at Pci-e 3.0 x4 though. All got detected with the now-Nvidia drivers.

Normal one, FCBT
Second, Pro one, FCCT
1 Like

Aaaaand i’m rekt. Apparently, ConnectX-3 don’t relly love running in P+E-cored systems, so i get a BSOD of mlx4eth63.sys every time i try to change the port protocol.

Checked the firmware on both cards:
Pro one had it alright

Regular one was vendored
image

After a succesful reflashing - it changed to this

Since one of the issues with BSODs could be me having P+E-cored CPU - is there any way to lock the card to specific cores? Or, at least, the driver?

P.S. Will at least ConnectX-4 cards work with the Alder Lake system? I don’t really wanna spend a bunch of money on another BSODing card.

1 Like

Why do you think it is to do with P+E cores? I have 2 of the same cards, but on linux. One in a alder lake i3 12100 (so no e-cores). But I’d still suspect a driver or Windows issue. The drivers for these cards are not really maintained any more.

You could try on linux, I believe the port protocol should ‘stick’ after a reboot and be OS independent.

1 Like