AMD Epyc Milan Workstation Questions

My WinVista machine (Code name “ZombiePC”) is actively used for games the don’t run well on Win7 and above. The Win98 box kinda just lives because I want to get the vast collection of 2.5" floppys saved to disk.
I also have a random collection of GPUs, just for the hell of it.

1 Like

Not flexing or anything, the oldest PC I have an authentic IBM PS/2 486SX25, complete with floppy drives, a 40 MB HDD and “Windows 3.11 for Workgroups”. I’m always amazed at the build quality and thickness of the steel. You don’t need a Kensington lock cable on that beast : no one is going to steal it unless they eat steroids for breakfast :grinning:

This old gear reminds me that we used to do a lot with a lot less. I mean, Bill Gates once did say he couldn’t imagine anyone would ever need more than 1 MB of RAM and we’re here talking about EPYC workstations with potentially terabytes of RAM. It also reminds me that we used to build stuff to last. I sincerely doubt that sexy case I’ve been showing off is going to last a quarter of a century, or that it will look “as new” if it does.

1 Like

Thanks for continuing to post info on your build. I do have a couple of questions based on what you have said:

  1. Is it possible to disable the Intel X550? Does it still consumer power in that state? Since you say that it easily throttles, I might want to just use my Mellanox Connectx-3 instead and could then turn off the X550 to save a bit of power and heat.

  2. How quick is the POST time of the motherboard? Some Supermicro boards can can a few minutes to start booting the OS.

The BIOS has the option to disable each of the X550’s ports separately. I’d expect that disabling both also disables the whole chip but I don’t know for sure, and the watts reading on my UPS isn’t nearly accurate enough to verify it that way.

POST times are comparable to a consumer PC boot time. It’s initially longer (i.e. 2-3 minutes) but there are a few ways to make it faster. The first thing you’d need to do is disable the BIOS option “wait for BMC”, as the manual says the BMC takes 90 seconds to boot.

There’s a 2-digit display on the board that gives you boot status in real time (and turns off once the machine has fully booted). Indeed when it waits for the BMC at some point it’ll show a countdown in seconds. It starts at 90 but doesn’t go all the way to zero.

Right now it takes mine about a minute to boot to Windows 10.

1 Like

Anyone seen any ETA’s for 1P Milan chips yet? Scan seem to think they’ll have the 7113P at the end of May, but no word on the 7443P.

1 Like

Do you mean 7313p rather than 7113p? I don’t think the latter exists…

I learned from my server retailer here in Norway that there are two release waves, and that 7313p and 7443p are both in the second wave, available in 2-3 months counting from March 20 (that was when I was in contact with them, I have no newer information). This seems to be compatible with availability at the end of May.

Of course this does not preclude differences in actual availability between different models…

2 Likes

Been busy with gathering the hardware for my own server, replacing my 10 year old Intel based ‘scrapyard’ server.

Also choose for the AsrockRack RomeD8-2T, topped with an EPYC 7313p and 4*KSM32RD8/32MER memory.

Came along this thread and thought I could hopefully contribute a little bit.

Have been in contact with AsrockRack about the missing Fan-setting option and it appears to be in the IPMI. They send a word-doc to me, describing it step by step:
https://www.dropbox.com/s/cgdwdihcdpwqzjv/How%20to%20modify%20fan%20in%20IPMI.docx?dl=0
Can’t test it out yet, as I’m still waiting for parts.

Furthermore, I understood from my supplier that the 7313P should arive in 2-3 weeks from now, fingers crossed!

1 Like

You might be right. Here’s a Norwegian store that lists both 7313p AND 7443p as available May 10! (@jtredux )

7313p
7443p

If the estimate is real I expect global availability by then, Scandinavia is hardly early in any supply chain.

A little point about 7443p vs. 7313p. When comparing clocks it may be worth keeping in mind that cTDP limits are different. 7313p has a default TDP of 155w and a cTDP range of 155-180w, whereas 7443p has a default TDP of 200w and a cTDP range of 165-200w. So the 7443p can only be adjusted down compared to default, and 7313p can only be adjusted up, suggesting the possible clock-range for the latter will be pushed up relative to spec.

That said, the max configurable TDP for 7443p is still 20w higher, so with fewer cores loaded we might still see higher clocks on this one. Tests will show how they compare, but I just wanted to highlight cTDP for those who are weighing between these two models, and don’t want to wait for tests before ordering.

2 Likes

Noticed that too, @oegat , but coming from a 8 core, 2,9 Ghz Intel CPU, everything is faster anyway. I don’t need high clock-speeds for the VMs I’m going to run.

Did get my RomeD8-2T in yesterday. I was able to update the Bios thru IPMI, but not the BCM. In the BCM version 1.00, the Fan-speed settings are not present. Trying to update the BCM firmware using IPMI and the Server Management utility doesn’t work unfortunately. The update does start, but doesn’t finish. Strangely, when logged into IPMI, I did see a lot of extra items appear in the ‘Settings’ tab, but just disappear after 10 minutes or so and the update proces stops.

Edit: After mail contact with Asrock Rack CS, I learned that Fan speed settings are only available with BCM firmware 1.10.1 and up. Can’t update BCM without processor.

I’ve finally pulled the trigger - just ordered a SM H12SSL-NT motherboard and a very cheap EPYC 7262 to use for bring-up while I wait for 7443p availability.

ETA on MB isn’t until mid May though :frowning:

BTW - just spotted this : AMD Ryzen Threadripper Pro 3955WX Review the Lower-end of the High-end

I found it interesting as I have been on the fence between 16-core TR-Pro and 24-core Milan 7443P.

Nice, I’m currently very close to doing something similar with a 7252! Backstory is that my NAS (Atom 2550-based) died on me the other day, and this led to rearrangements among my hardware that suddenly makes a Rome-based 8c server useful in a few months - in addition to the already planned workstation :slight_smile:

Interesting read. I still find it a bit funny that they talk about the specific niche of 3955wx in the EPYC Rome ecosystem, without discussing how its position will be relative to Milan chips. Esp. this passage from the article’s final remarks:

(from the last page of the StH article, emphasis mine)

True, the 7313p has lower clocks, but not much lower, and IPC will offset most if not all of the performance difference. It’s also supposed to be cheaper, at least judging from the released Milan chips staying close to msrp.

In contrast to how StH puts it, 7313p, 3955wx, and 7443p all seem to compete within a narrow price range, where both cache size and price/core is a win on the Milan side. Also they all have a similar topology, with cores divided in 4 clusters each sharing LLC, so that’s no differentiator.

In the end I see only platform-related reasons (such as validation, IO, cooling…) to choose 3955wx over 7313p for a 16c part. Unless your specific workload is such that you end up far below the average IPC gain of 15-19%.

Or am I missing something? :slight_smile:

Edit: OK I realize that they are right in 3955wx not upsetting EPYC sales - rather it’s the new EPYCs outcompeting 3955wx on several points. Either way the niche of 3955wx is not as clear anymore.

I suspect that some of the choice between TR Pro and EPYC at the low core-count end really does come down to how many ‘workstation’ niceties you really need. I think I can live without Thunderbolt/USB-C and audio (PC onboard sound is IMO terrible, get a proper audio-interface if you really care about audio).

Also, I know people obsess about memory bandwidth and the magic of having 8 channels on EPYC/TR-Pro but you do actually have to populate all of those channels to see the bandwidth. I probably want 8GB/core so 256GB would probably be ideal, but my wallet and the lack of expansion beyond 8 DIMMs (what a hard life!) will probably lead me to start off with a mere 64-128GB in 2-4 DIMMs.

I agree, that and platform validation for certain tasks, is what differentiates.

Onboard audio is among the first things I disable on a motherboard, haven’t had use for it in 10+ years - I use USB DACs when needed. For my gaming VM I use a dedicated USB controller, which connects keyboard, mouse, and audio via a KVM switch. So I basically plug-unplug audio when switching between host and guest.

However, I believe you technically can get USB-c out of the SM mobo, via the MB USB header? Provided the front panel has the port.

The limitation of 8 memory slots on these boards is an issue, I belive I will settle with 128Gb in 8 channels, but maybe start with 64 in 4 channels until I tested out the rig. Esp. if I start out with a 7252, which is a 4-channel “optimized” (meaning “more than 4 channels gains nothing”) part.

Btw, just a double check since you ordered a SM mobo - you are not planning to run Windows 10, right? Compatibility is so-so from what I gathered when searching around (mostly based on reports from the H11 series, but H12 should be similar in terms of almost everything). At least a year or so ago that Broadcom NIC did not have Win10 drivers, for example.

Personally I don’t plan to run Windows on baremetal, and I usually don’t passthrough NICs, that’s why I consider the SM boards.

1 Like

Thanks for the heads-up on Windows, I am planning to run ProxmoxVE on this machine, and CentOS on top of that. If I can get a current-gen GPU at a semi-sensible price any tiime soon (TSMC sound busy all year) then it may eventually run Win 10 in a VM.

If the onboard NIC’s don’t play nicely, there’s an ebay full of reasonably priced Mellanox stuff, and I’m going to have PCIe slots aplenty!

Hello guys, just dropping by with an update. As you can probably guess from my silence lately, I’ve been enjoying my EPYC. If by “enjoying” you mean “installing tons of software and working my rear-end off”.

I’ve noticed a quirk with the ROMED8-2T that I failed to notice before because I’m using a KVM switch : for some reason, the board doesn’t always boot when you power it on. I’m investigating why that is, but sometimes it takes a power cycle (as in : actually switching off the PC’s mains input and then turning it back on) before the machine will boot. It will stop with the code “64” on the LED display.

Unfortunately, I’m running the “lab” BIOS for Milan and that code is not documented in the board’s manual. I initially thought that the board was waiting for a keyboard (maybe an issue with the KVM) but that’s not it.

Other than that, everything’s peachy. I’ve installed an Asus Aquantia-based 10 Gb NIC in one of the PCIe slots because I needed a third Ethernet port and it’s perfectly compatible. I haven’t met with any software that misbehaves on this system.

I was about to pull the trigger on an external Sound Blaster when I noticed that my RTX 3090 acts as an audio output device. All my PC’s until this one had on-board audio so I hadn’t realized that HDMI-capable GPU’s also implement audio. As a result, I’ve changed my audio setup a bit : now all my PC’s output through their GPU, and I connect my audio amplifier to my monitor’s headphone jack.

This means I switch audio source along with video when I use the KVM.

In terms of quality, well, I have very good loudspeakers (esoteric ones, you might say). Philips units with aluminum cone drivers. Very crisp sound, very powerful. I’ve made my own class D amplifier for them. My monitor is a 43’’ LG with a very low noise audio output. I’m not an audiophile but I know good sound when I hear it.

To put it simply : GeForce audio (whether it comes from a 970 or a 3090) is more than good enough for YouTube, Netflix, listening to FLAC files and playing video games. It has very neutral (possibly non-existant) equalization which I find refreshing. That is to say, if you like bass you’ll need to make sure your speakers can produce them.

I suppose if you need an equalizer you should be able to run VoiceMeeter on this machine.

Obviously there’s no audio input but it just so happens that I have a Rode USB mike so I’m covered.

3 Likes

Hey all,

I too pulled the trigger last week. I ended up following the path of @Nefastor and @jtredux and ordered a rig with a Rome CPU, while waiting for better availability of Milan.

I went with Rome 7252, Supermicro H12SSL-I and 4x 16Gb 2R DIMMs (for now). Most if not all of the stuff should show up during next week.

Previously I hesitated to get a Rome “placeholder”, because I did not know where to put it after the likely upgrade to Milan. But then last week the mainboard in my NAS/homeserver died (Supermicro A1SAi 2550F), that had been in service since 2014. Even though I’m replacing it for now with a cheap Celeron SoC, just to get the storage back up quickly, I got to realize it is about time for a more thorough upgrade… that’s where the 7252 is likely to end up.

Logic behind choices:

Mainboard
I was initially looking at H12SSL-NT, but I went with H12SSL-I, differing in that it has one of the two SlimSAS ports replaced with 8 regular SATA ports, and a 1GbE NIC instead of 10GbE. The breaking point was the NIC, considering this:

The Broadcom 10GbE on H12-boards sits in the same relative location as the X550 on Asrock, and is likely to produce heat in a place where I don’t really want it. Also I don’t have plans for 10GbE infrastructure at home currently. There is also this point:

I agree with this thinking, and from thermal considerations an addon card would be better ventilated, should I need 10GbE later. This will really be like back in the 90s, when almost all peripherals were addon cards :slight_smile:

CPU
7252 (non-P) is almost as cheap as 7232p, but the latter has half the cache. For the server I’ll move it to later, 8c @ 3.1-3.2GHz will be more than enough.

Heatsink
Refusing to divert from front-to-back airflow, I went with Supermicro SNK-P0064AP4. I’ll try both the stock fan and a 2.2krpm 92mm Noctua.

Memory
4 DIMMs - Since 7252 is a 2-CCD (half-bandwidth) part it did not make sense to populate all channels just yet.

Chassis
I went with beQuiet Silent Base 802, following the logic in my previous post about it. With this one I can try both normal and inverted mounting, and see what works best.

Upgrade path
My plan is to use this setup to evaluate whether the platform covers my needs. If it does, the upgrade path will look something like this:

  1. replace the CPU with a 7313p (or possibly a 7443p), and get 4 additional 16Gb DIMMs. I count on satisfying with 128Mb ram for this machine.
  2. build a fairly compact server tower, likely based on ASrock RomeD6U 212t (mATX!), populating 4 of the 6 memory channels
  3. Put the replaced 7252 in (2) :slight_smile:

Should it turn out that my workstation/game-VM-host plan does not work for some reason, I’d just make a server out of this machine instead. And re-think my workstation plans :slight_smile:

2 Likes

I think that’s a sound plan. Like you, I really like the idea of having my peripherals on PCI cards instead of being stuck forever with whatever is on the motherboard. I’ve got an X58 board with integrated FireWire : awesome… except I’ve never owned anything that has a FireWire interface. But I still had to pay for that feature :sweat_smile:

I guess only time will tell whether the 10G NIC on my H12SSL-NT gets too hot, but IMHO, if you’re not putting these boards somewhere where they will experience server-level airflow, you are making problems for yourself…

I suspect I will start off using those ports at 1G anyway as I need a new 10G switch as my current one has no spare ports and no 10G-baseT.

BTW - can anyone recommend a torque-screwdriver for CPU installation, I didn’t get one with my retail 7281 nor used Rome. I was thinking about one of these : https://www.amazon.co.uk/gp/product/B000PAR61Q/ref=ox_sc_act_title_3?smid=AQPVMNNS97CC1&psc=1

Hey Nefastor, I’m looking at building an EPYC rig myself as the next gen workstation (currently on a Skylake-E 7900X) but one diehard requirement I have is to be able to game on it after the workloads have finished.

I noticed you booted up Cyberpunk and that it ran nice. Did you ever run 3DMark or a similar benchmark on it? I am curious how an EPYC would fare in “consumer software” that uses few threads which it is clearly not optimized for. I am expecting the high-frequency Milan chips, such as the 7443P, to produce fairly good results in this area.

And yes, I know gaming on EPYC is a terrible idea, but I am curious nontheless. And while I don’t expect Ryzen 9 levels of performance, I do hope to know if it’s working satisfactorily before making the plunge from my current intel setup.

Thanks!