HPE DL385 G10 general questions from a noob

[Kilping]

Hi guys, I’ve watched Level1Techs for a while now and lurked in the forums when I built my FreeNAS box using the Asrack x470du-2t. This place has been incredibly helpful and a great resource.

A little context for the situation. I would say I have a intermediate amount of technical ability and my current profession opens up a lot of doors in many different technical spaces and I have had a blast exploring them. Recently I went to a course that was an introduction to Linux. I am by no means a linux guru but I’m looking to change that as well. However, I knew enough to breeze by and had a lot of time to talk with the instructor who has become a friend and great mentor. He introduced me to the idea of self hosting and level 1 hypervisors, so much so I went in during my days off to learn more and bounce ideas off of him. I was hooked and I knew I was in for long nights of banging my head against the wall and explaining to my significant other what a 15u network rack and 42u server rack were doing in the bonus room. Also why the internet was intermittent recently. (messing with pfsense and surricata, lol)

I wanted a virtualization server so I could start learning as it translates to my current profession (kind of…lol). I started off looking to build my own which seemed like a good option as I enjoy the building process and finding the right hardware, but soon realized I was going to be making sacrifices on certain things. For example; If i went with consumer hardware I IPMI with pcie space to grow was incredibly limited. I should mention here I wanted an AMD machine. My first build was with AMD and I like a good underdog story. Anyway, I decided to go the full enterprise route. I know I will catch some flak for not buying a used server on the cheap, more on that later. I wanted something new that was today’s technology to learn on and to hopefully grow with as one day I hope I can take this machine from a homelab setup to a production one. The use case for this machine is vast as it is more than capable for almost anything I can think of throwing at it. I was also given a free server after ordering this one from a friend I recently met through work. It is a Dell C2100 with dual x5560s. I now have the opportunity to work with old and new which I couldn’t be more excited about. The questions I have are the following.

What are the nuances the uninitiated might face with new hardware specifically HPE? (I found out recently about the premium for memory, storage, and what I assume is any other hardware going into this thing.)

I’ve looked around at compatibility charts for memory and such, is it necessary to use the SMART hardware from HPE?

Opinions on hypervisors? I have a feeling the HPE may not like proxmox since it’s not supported(I have the opportunity to double the fun as well as the headache.)

I have the two friends I mentioned before to help me along the way but, they are newer friends with priorities of their own and I hate to be intrusive so I rarely ask for help outside of being in the same room. So where can I go for support and to generally learn how to use this newer machine?

I know I sound a bit naive and I admit I vastly underestimated the difference in enterprise systems vs consumer, but I like a challenge and am eager to learn about something that a few months ago I knew almost nothing about. I’m sorry for the lengthy post, I have trouble putting thoughts into text sometimes. I hope I offered enough detail to those who read it, and if I didn’t just let me know. I’ve got thick skin and am open to constructive criticism, I mean this is the internet right? Thanks for everyone’s time and input, cheers!

[rcxb]

What are you trying to accomplish? For virtualization in general, you’ll find the first bottleneck is disk IO unless you go to proper SSDs (not just slow consumer junk), and the second is memory. You can get R720s with H710 RAID controllers and 128GB of RAM under $500 (minus hard drives). You won’t max out all the 12-18 CPU cores, or memory, bus or network speed on even and old server.

You’re going to find it an incredibly expensive hobby that doesn’t gain you anything. There’s very little that you can learn on today’s hardware versus a generation or two older… particularly if you aren’t planning to spend truckloads of money on ultra-fast NVME drives and a bunch of GPUs. Now… shopping around for decent enterprise SAS SSDs (check IOPS specs carefully!) for your used server will be a serious learning experience. Ditto for figuring out how to bypass the IPMI license requirement, setting up the hardware sensor monitoring, etc., etc.

Getting to play with a shiny new high-end server your company bought is a bit of fun, but not for long, and certainly not worth the price of admission just to play around. What’s more, without the heavy usage and problems constantly cropping up in big company with lots of servers all getting constant, heavy usage and people noticing them, you won’t learn all that much… You’ll think you’ve got a handle on things when you’ve barely scratched the surface.

For hypervisors, free ESXi or proxmox seem the likely candidates. Which one you choose again depends on what you want to do.

[gordonthree]

Since you have a new machine, you should learn about what it takes to set one up proper, in a supported fashion, rather than treating it like discarded junk.

Register the SN with HPE, and download the firmware update package called SPP. Learn how to apply the SPP, even new servers arrive with outdated firmware.

Next, register with Microsoft or VMware for a free account. There’s a free version of ESXi available for HPE, Windows HyperV Core is also free. Learn how to install one or both of these using the official method, Intelligent Provisioning.

You don’t have to buy Smart memory, but the generic stuff isn’t much cheaper so no huge savings for the headache. Drives, well yeah those are expensive. You can buy used oem drives if you’re just messing around, however if this is for a production environment that might not be the best idea.

[Kilping]

Thank you both for your replies! I may have the option to do what rcxb stated being that I got an email today saying the server may not be in stock. I’m still waiting for a reply to confirm.
Not sure if it’s a blessing in disguise as rcxb did bring up some good points. I do disagree a bit with the part of not gaining anything. For the price of a high end graphics card I get a system although limited by RAM and drives allows me to learn a ton of new things. A prime example is what gordonthree presented. The working through the way you’d set up a server in a production environment. What’s the icing on the cake is 64 gbs of ram and 2 900gb 10k sas drives arrived at my doorstep today, lol.

[Windforce]

I am no expert but I have prepped a few DL 380 G10 servers before. Never worked on the new AMD variants before but I don’t believe there would be much differences.

Something to keep in mind hardware wise with the G10 is that depending on the base model you purchase, expansion may have some extra costs…for example expanding past the base amount of HDDs will require you to purchase an express bay drive cage, wanting to add internal tape or DVD drive will require you to purchase a media bay kit, may need to purchase an additional PCIe riser card depending on your card amount and population.

Another gotcha is KVM/remote access from within the ILO interface. There is a specific license to purchase for the ILO otherwise the remote access interface times out after a 1 minute interval.

Once you create an HP account, you should be prompted with a request for contract and warranty information which I believe will then open up access to SPP and individual firmware downloads.

Because your OS is not supported on the Intelligent Provisioning does not mean it will not work. I can’t say for Proxmox but we have used other virtual environments, installing them without Intelligent provisioning such as Oracle VM. Most of our customers usually buy these for VMware and Windows. Of course if it is supported, go ahead and install through intelligent provisioning.

As for resilience we have not experienced hardware failures on the G10…as of yet. To be honest outside of HDD, Fans and power supply failures we have barely had failures from the G8 - G10 line. The generations before…well just about anything fails…well there is of course the age to consider.

[Kilping]

Thanks for the response, sort of sucks about the ILO. I’m going to see what the cost is and if it’s not ridiculously high I’ll buy it. The expansion I definitely realized is where the cost is going to stack up over time. I think this is okay since I plan on this being a platform I can grow with, and since I have the c2100 that was free I have room to tinker. I was thinking about using esxi on the HPE since and go the intelligent provisioning route just to see what it’s like. All the firsts are exciting, lol.

Also I almost forgot to mention, I got a hold of the reseller and they do have them in stock and it was shipped today. If I remember correctly they come with rails so I should be mostly good to go once it gets here. Unfortunately I’m starting another course next week so I’m not sure how busy I will be this week. To set everything up. Who am I kidding I’ll be up all night as soon as it gets here.

[gordonthree]

ILO advanced isn’t very expensive, couple hundred bucks maybe. Well worth it if you’re managing a rack full of these machines.

[Windforce]

It should come with the rails standard. I believe it also comes with the management arm but don’t quote me on that as I wouldn’t be able to say if it is an additional cost or not.

[Kilping]

Cool, I’ll give it a go!

[Kilping]

Nice to know it comes with rails, and the management arm isn’t a huge deal nothing some velco straps can’t control.

[Kilping]

I have 2 300gb sas hard drives that are from HPE. Through some googling I also found some hard drives that say they are compatible with HPE that are from Waterpanther.com. Has anyone had any experience with them? I might grab some 200GB ones to test.

[Kilping]

Quick update for anyone who might find this in the future.

I ditched the overpriced drives and bought some mx500 crucial ssd’s popped them in the caddies and just passed them to Proxmox. They work like a charm. I have a total of 80Gb of ram the 64gb I purchased from amazon and then the 16 that came with the system. There has been 0 issues with anything as far as fans or the system getting angry. So far I’m super happy with everything and plan on adding more SSD storage since it’s cheap (relatively). I understand that it isn’t enterprise grade but the local storage on the server is used for ISO storage and vm installs. The bulk of data is on my Freenas so once everything is set up I will be doing backups to that.

Also I bought the proxmox license since it was only 1 cpu and it has been great. I’m a little hesitant to buy 2 more for the old Dell c2100 since, one it has no drives right now and two it is really old lol. I’m thinking of trying to create a cluster but I need to do a bunch of research before I commit it to anything. Proxmox has been pretty straight forward and I’m enjoying it, however the Vlan setup is a little confusing to me as I have 4 usable nics and 1 nic for IPMI. Out of the 4 usable 1 gets bridged by default and I decided to LACP all of them together and try to separate the VM’s on a vlan with minimal success. I’m sure this is a configuration problem and not a problem with proxmox. Anyway sorry for rambling, first time in a while I’ve had time to sit down and post. If anyone has any questions or comments feel free, but understand I’m very new to this I just jumped head first lol.

Here is a link from Reddit of my setup…I’m proud of it and the RGB is the true way you download more ram, lol!!!

Reddit post