Built a Plex/Web/Email/Cloud Server... perhaps I went a little overboard?

Hey everyone,
first post on Level1Techs.
First Linux PC build, as well.
I leaned heavily on Wendell videos, ARSTechnica articles, and DigitalOcean articles.

@wendell mentioned in a PGP stream on 2/17/17 that I should make a thread about this build.
So, here we go.... !!

Ebay links to my purchases... hope they last a while:

SuperMicro X8DTL Dual CPU Motherboard - $97

Intel L5640 - (2x) - $27 each

Cooler - Intel Extreme Cooler - (2x) - $24 each

24GB ECC Reg RAM - (6x4GB) - $36 for set

Total: $235 + free shipping!!!!

  • Used an EVGA 1050W power supply (bought several at once, so it was on the shelf, I'd recommend the EVGA 750BQ for a new buyer - its also compatible with dual CPU for much less money - like $70)
  • I also used an old Fractal R5 because it was also on a shelf, and its very quiet.
  • Finally, I loaded up a bunch (5) of refurb 3tb drives from amazon (HGST refurbs.)
  • I installed Ubuntu Server 16.04 LTS, made a LAMP for hosting all my sites, and started setting up the ZFS pools.
  • I went with (2) 3tb drives for a cloud pool, and the other (3) as a start for a media server named PMSpool -because jokes. (And PlexMediaServer...of course)

ZFS was the perfect file system for this.
I got imaptient on shipping for coolers and ended up buying a pair of Noctua nh-u9dx-1366 coolers from amazon prime... this proved to be wild overkill for these chips. Not necessary. Intel ones are quiet enough.

I added Plex media server and began the LONG process of transferring over media. I connected drives directly via SATA to transfer files.
Everything works perfect!! Web server is up, NextCloud is excellent, and Plex Media Server is BEAST MODE. I went to my day job (financial planner, nothing IT - so sorry if I've made huge errors) and tested the media server. I have a 300mbps down connection at work and a 20mbps upload at home. I played a dvd rip and there was next to no load time and smooth as it could be, played "original" quality, so no transcode. I started opening additional tabs and was able to get to 5 or 6 dvd and/or bluray rips beforre stuttering.. (I suspect upload speed was limitation.)

I wanted to stress the CPU's, so I ssh'd into server and ran TOP command to see CPU utilization, and started crunching the video quality down to 320kb/s and start adding tabs. I ran out into issues on work-workstation after 15 streams... no stuttering up to that point - TOP showed CPU at 59%... so that's awesome.

I may have overbuilt this thing....

I wanted to share this experience so others could take advantage of these great deals on eBay. I took a bit of a chance, but man- it worked out in a big way. I have dead quiet, very powerful server now! ...and a place to keep experimenting with Linux!
Happy to answer question, if any of you have any!

_pig.

PS - thoughts if I were to do this again. I'd do it just the same. The deal is too good. 10/10 recommend. I think I want to add a SATA controller because the onboard SATA and SAS, while plentiful, are not 3gb or 6gb connections... But, as you see from post above, this isnt an issue in real world implementation. It works great! Good luck!!

PPS - I'd be happy to receive feedback about build!

10 Likes

Nice build, I have a similar setup except its on a dual-core J1800 SoC with 4GBs of memory, I kind of went the under-powered route but it works fine for the most part unless I'm transcoding on plex or running sidekiq tasks on GitLab. My plan is to eventually upgrade to a similar system one day so I can do some extra stuff. Also if you do any sort of programming/web development I'd also recommend running GitLab on this server, it'll make things so much easier when managing projects and you definitely have the capacity for it on your server.

1 Like

Since I have this server in service, and I am so new to linux, Im a little hesitant to pile on packages. I've got a separate laptop that I am running a VM of Ubuntu on that I am testing around with. I'm working on Ruby/Rails now. Using web tutorials and "courses" to learn it. I also havent implemented an email server yet... Im really interested in waiting for ARSTechnica to update their Taking Back email series. I was really impressed with the first version, so Im interested in seeing what they have to say and potentially copy their method... unless Wendell beats them to the punch.

You may want to look into running your services in VMs/containers as it makes managing your server much easier, by keeping them separate they can't conflict with each other and aids vastly in management and maintenance of your system, also makes it easier to get back up and running if a service breaks as you can create snapshots and roll the VM/container back to a previous version. I am currently using LXC containers on my server, mostly because it has less overhead than running VMs and is better for my poor SoC but you could also run them in VMs for better separation or even by using docker containers.

@MichaelLindman Thanks for the tip.
I have looked into containers as a potential option, but I have no experience beyond basic testing in Windows. But this gives me a bit more confidence that I'm looking in the right direction. It's still something I may implement on this server to play around with various containers in the future. For now, I'm just happy to have the thing running so well... The setup has surpassed my expectations.

I've looked at those J1800 rigs in the past. Seems like a neat passive solution. I actually considered velcro'ing one to the back of a tv before- kind of like a ghetto NUC. Would one of those work as a HTPC client?

Sounds like a great platform to run Lucida.ai! How much power does it draw? Can you tune it in the bios to slow down and draw basically nothing when idling?
I might have room in my computer desk for a large motherboard if I re-arrange some stuff... If i cant get some second hand stuff from work I might consider ebaying similar stuff.

4 Likes

@anon54210716 It is a full ATX, so... hope that desk is a big one if you're planning on multiple complete systems running in it.

Regarding power draw and bios: to be honest, I haven't checked. But it is plugged into a nice UPS/regulator and it's easy enough to check. I'll do that when I'm home and update you. An additional admission, I didn't do a thing in BIOS but check to make sure they were there and functioning. I've not been too concerned about power draw because I knew I'd be using the system regardless.
It does throw a bit of heat off of the cooler stacks, so I'm not expecting anything spectacular.

I'd really like to know how much power it draws at idle and with 1 stream going. If you have the equipment, that would be very useful information to me. Nice guide!

@anon54210716 @Worldinc So, looks like my server is going to cost a crisp $15-20 per month to power at my current rate ... (electricity pun not intended.)

Looking at 198W on idle with absolutely no optimization, a graphics card installed that I don't need because I was too lazy to find a VGA cable, and some fans that are probably spinning too fast- or not necessary at all... I can probably shave 15W off this number.

About what should be expected from dual cpu server class equipment of this age bracket, methinks.

I started one plex stream and saw a momentary spike to 272W, and then a drop after about (10) seconds to 207W.

I added (9) additional streams (all transcoded real time down to 320kbps to stress the system - these are all blue ray or dvd rips) and I got about 6-8W per stream up to 225W. Odd that it didnt keep adding after that. I guess transcoding was buffering enough and I wasnt clicking fast enough. So, streaming seems to have little effect on this system's power consumption. CPU utilization would tend to agree.
The (10) streams are still playing as I type this on my laptop, and Im looking at CPU usage of around 20% currently.... amazing...
I also started a large file transfer and Im doing random searches across the file structure on a windows share.
spiked is up to around 35%. power consumption went down to 198W... but jumps to 250--ish watts back and forth. Looks like transcodes are caught up.

Hope that better informs you... its rock solid, but it certainly is not a high efficiency low-powered rig.

PS - its super wierd hearing teenage mutant ninja turtles cutting up and making jokes while 2fast2furious cars go buzzing about all the while te dude from 12 years a slave is just getting the hell beat out of him... very disconcerting....
PPS - wattage is back down to 203W with all (10) streams going.

1 Like

That seems a bit of power. What have you got set in the BIOS as far as:
* C1E Support
* Intel C-State Tech
* C-State Package Limit Setting
* C1 Auto Demotion
* C3 Auto Demotion?

Wondering if it is possible to get the CPUs to idle down since you have a heap of cores. Shouldn't be any issue doing this, as any delays spinning the cores up won't be that noticeable on the server.

These are all valid thoughts. In the next day or two, I'll take a peek and see what I have going on. While, ultimately, Im unconcerned about the from a cost standpoint, I'd definitely benefit from reducing any amount of dissipated wattage from my home office.

Thanks!