Hi Guys, in this build log, I’ll post the progress of my server and workstation builds for my small business. This will take some time.
First up server. It will be a headless, Ubuntu and ZFS affair. Going all out Intel.
Just placed the order for a Gigabyte MS03-CE0.
Just placed an order for an Intel Xeon Silver 4410Y.
This whole setup is way overkill for my use case for right now but I found a potential service that I’ll need that is somewhat demanding so having that extra breathing room is something I was looking for. Plus it has the added bonus of multiple power settings so for the short term I’ll be able to save a tiny bit of power:
|Config | Active Cores | Base Frequency | TDP |
|4410Y(0) | 12 | 2.0 | 150 |
|4410Y(1) | 8 | 2.0 | 130 |
|4410Y(2) | 6 | 2.2 | 125 |
Will you be posting any pics of your progress?
Yep :)!
Looks like Ubuntu is probably not the best choice of OS?
https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2022.04%20Root%20on%20ZFS.html
Because they depreciated zsys?
Probably a good thing
I don’t run root for ZFS on Ubuntu/other linuxes myself, but for those who want to… Boot environments might be the way to go.
Native in BSD, but for Linux, Overview — ZFSBootMenu 2.2.2 documentation bootmenu might do the trick
Problem I had, was the kernel and the suite of zfs/spool tools once fell out of sync.
I was able to boot to a live session and transfer data and such, but was an inconvenience.
So I just keep / on a ext system, and use Zpools for /home and whatever data.
But that’s just me… Despite my enthusiasm for zsys and root on Ubuntu
So the CPU has arrived, but I won’t be cracking it open until the motherboard arrives. Either way you can see how massive it is! Got a pencil for size reference. Once the mobo is here we’ll take a closer look at it!
Fun fact:
Just placed an order for the Noctua - NH-D9 DX-4677 4U CPU cooler.
https://noctua.at/en/nh-d9-dx-4677-4u
No idea what case I’ll be using at this point, but I want to keep my 4U server rack chassis options open. Even if I don’t get one right off the bat which is looking likely. I might get a used server chassis off ebay at some point and just go with a Fractal 7 XL for now. Not sure at this point. Really not liking the bottom of the range 4U chassis on offer for the considerable prices these are all going for!
It comes with x2 fans so I’ll have to manually set the fan profile for the second fan in the motherboard management software, which I’ve never done before, so will be something new.
The heat sink is here. 92mm fans hmm… didn’t read what size fans it came with. Didn’t realise they were going to be so small. Oh well, the price of flexibility.
If you will house the equipment in an air conditioned space then the air cooler with decent thermal paste is fine. If out in ambient air, in a warmer clime, you may need something beefier, like an AIO to keep performance up.
Also, keep the gear isolated from dust, dirt, carpet and pets. Dog / cat hair clogs things up quickly. No use putting together a killer system, just to kill it. 
If you’re going to run ZFS on Sapphire Rapids, be aware of this issue:
The fix has landed in ZFS 2.2.0-rc4, so until 2.2 is released and packaged, you may want to disable AMX for the time being (clearcpuid=600).
I have an air con unit in the room that it will be in luckily. So I’m not too worried about cooling unless the air con takes a dump. It’s an old unit so it is a possibility. Anyway, high availability isn’t something I care about. So on those hot days when the air con is off, I don’t mind turning the thing off.
As for dust, I usually clean out my computers every couple of years anyway. I just use the exhaust from a vacuum cleaner, cleans things nicely. My current workstation has a filter so it’s not too bad inside. This server won’t have a filter so will need cleaning out periodically.
Thanks for the heads up. I have no problems turning it off for now as I haven’t touched machine/deep learning/AI for my purposes yet, that’s still some ways off and even then, I might just use a Quadro for that stuff anyway. Overall I’m holding out for Ubuntu 24.04, it’s oh so close. So I might not even have the thing online before then. Hopefully it’s all fixed by then.
When you get the system running, it’d be amazing if you could please share some power consumption numbers.
I’m looking into a similar system myself but there’s almost no information out there on these lower-TDP Sapphire Rapids SKUs
I plan on tracking wattages. Not sure how I’ll go about doing that at this stage though. The Kill-A-Watt meters are a bit too inflexible. The best video I’ve seen to date is:
I’m open to others’ ideas on this. My goal is to know how much energy my computers consume per year. Mostly for fun and curiosity as I suspect the cost will be in the order of $100 - $200 a year, which really doesn’t significantly affect anything.
I’ve realised for single digit numbers of computers, you’re really not going to break the bank. For server farms and data centres, electricity starts to play a bigger role.
I got some smart outlets for that that can also measure wattage. Dont think it would help you much If I share the specifics as its an eu outlet. Some random brand I have never heard off (though its Not really an issue as it runs tasmota). Found it on Amazon.
Comparing numbers is kind of hard as that includes efficiency of your power supply and every drive connected.
I need to research Tasmota, no idea what it is.
an open source IoT OS. think of it as the Andriod OS of internet connected devices. it comes preinstalled on a handfull of lightbulbs and things, and can be flashed to a LOT of other smart pieces of kit.
y tho? well it can make some devices more compatible with other smart home controllers, like homeassistant, smartthings, etc. and also open up new features that the manufacturer software locked.
SOHO setups seem to be increasingly adding ‘smart home’ features. i went down this road about 4 years ago and ended up in a pretty unmanageable spot, i might try again when ‘matter’ becomes mainstream. (also, i don’t get why wifi became the primary smart home connection, Z-Wave and Zigbee did it better )
ALmost all the ‘smart’ devices you’ll find on amazon/ebay will phone home to some server or another in china. If you don’t dig that then Tasmota gives you the possibility of running a firmware where you control where the smart appliance is sending data, typically losing the ability to use the vendor app. That means you need to deploy locally a smart home hub (Home Assistant, openHab) and an mqtt server if not using the one included in home assistant. You can then ‘plug’ in any number of tasmota decices in you smart home network and monitor/use them depending on their capabilities
Im’ Eu based, bought a bunch of these:
that cost a smidget more than the chinese based ones but come with Tasmota preinstalled. I don’t think thsy have an US version though …
Once you link them to your local infra, they will send data like this to your server (homeassistant transparently - cough- should let you do that) …
And from there you can plug them into whatever system you use to collect/catalog/display data, I use EmonCms for energy and power data:
