Success has been achieved finally. I can not belive how powerful notepad or a pen and paper are… I discovered that the process of installing Proxmox as a nested VM in Hyper-V all came down to the ORDER of the steps.
1.) Create the VM like normal as a type 1 vm on hyper V with the proxmox iso image mounted as a cd. Finish this process. DO NOT START THE VM.
2.) Delete the network adapter of the created vm.
3.) Create a new network connection that is external and check box to share with management os (ie windows). This will drop your windows connection momentarily to the internet.
4.) Add new network adapter to VM as a legacy adapter and select the external network connector you created.
5.) Start the vm and install proxmox and it should read and create a proper ip congruent with the range on the network adapter you selected!
6.) Dismount or removed iso DVD and start up. Proxmox should be running and able to update and be logged into by another pc or host pc on the network.
-----------------------------------------------HUGE UPDATE------------------------------------------------------
Soon I will be moving up to real and dedicated test server. The above build using the Asrockrack X470DU and ryzen 2700 is complete. Once I have the new server I can attempt some more creative server programs and home services without fear of interruption of the services it provides.
I have decided on the Dell Poweredge T320 server as a first toe to dip into the enterprise server pool. I got it with the Xeon E5-2450 Eight Core 2.1Ghz (2.9 boost), 64GB ECC ddr3 ram, dual 750w PSU’s (the dual 450w is not recommended if you plan to use a GPU), and the front is setup for 8 Hot swap 3.5in HHDs and H310 sas pcie card. I’m happy with the purchase (Haven’t received the unit yet). This form factor and platform I chose because of availability of parts, ease of upgrade with interoperability of boards and the like (IE- if I ever need dual CPU’s and more ram for my needs).
Some of my preliminary research has involved the Nvidia Grid CPU ecosystem. I had seen a few videos from Craft Computing involving making a “Game Server” with virtual GPU’s out to thin clients like a Raspberry Pi. It seems from his findings that it is possible, but overall performance is limited, and set up is particularly limited to certain hypervisors and versions. As well, the type of card you can use is limited due to needing an enterprise licence to use some of them, and needing to modify some cards for use as a graphics card vs a compute card. Very interesting series if you have any interest in it.
I did a little more research on Nvidias website and found this …
On page 16, section 1.4 it outlines that GRID gpus support all windows VM’s (Provided you meet all the criteria), yet the GRID K1 and K2 seem to not support linux VM’s… only the Tesla series seem to. There is one ray of hope. It is mentioned in the document that drivers may have updated information.
I mention all of this because this is one ave of investigation I plan to do with the new machine.
I also researched the iDRAC unit that provides the IPMI service. It appears that the basic IPMI services are provided through the unit. I hope this is sufficient and I wont need to get a licence for the software to communicate with the server.
I also plan to utilize a server management software possibly like co-pilot or something to manage both servers.
Thats the plans so far. Along with setting up a rsync possibly or other automated services. I originally planned to utilize cron jobs, but it was suggested to me to use systemd timers, so I plan to educate myself on both.
Well, so far t320 has not worked out as hoped. I am at a cross roads. So far attempting to use the h310 sas card in IT mode has broken the ability to use any drives attached to it to boot from, as well as making them inaccessible in ESXi and possibly citrix. I can install windows server or proxmox to one of the drives, but the Dell system won’t boot from it.
I may try reverse breakouts from a sata card to sas backplane, or forward breakouts from h310 sas card to the drives directly bypassing and removing the backplane, or something else I haven’t thought of yet. I am ordering a h310 thats flashed to IT mode to see if maybe I messed up because I have seen a video of it populating in the Dell Life cycle software. I may also boot up into windows server and see if I can update Dell software and drivers. Either way I have more parts coming to play with to see if I can solve this.
Or I can just give in, re-flash to original raid card configuration to set up hardware raids, then lose ability to use zfs…then maybe able to use ESXi finally… or leave it the way it is and continue to just use proxmox, or mess with citrix some more to see if I can get that to work.
Either way, it’s keeping me busy.
Are there any m.2 slots or additional sata ports on the board which you could use for boot media? I could be wrong, but booting directly from the hba may be an atypical use case.
This is my step right now as I install windows server 2019 for updates… I hope
I added a pcie card with a pcie and sata slot… the pcie slot isn’t recognized by dell either at the momemt.
Progress has been made. A little slow and painful at the moment. I’m relying on windows server 2019 to get updates completed. I’m also attempting to integrate windows server and iDrac…the t320 is only compatible with up to server 2016 though so I’m not sure how that will pan out. I’ve also been buried in pdfs reading up on iDRAC, Life Cycle controller and other Dell proprietary software and hardware such as dual SD card reader for the t320. The new cooler, front bezel, fans and few other items have arrived or are en route to me. I have the 10 core 2.4/3.2 Ghz E5-2470 v2 on the way for a little more processor power.
I have learned a few lessons the hard way.
1.) It appears the dual SD card slot requires Dell specific SD cards to operate in raid 1 or cloned mode. I attempted to sync two identical make model SD cards and each time there was 0% progress and sync would fail.
2.) The PERC H310 raid controller can be flashed with LSI firmware for IT mode to allow direct passthrough for zfs implementation. This however makes drives inaccessible to vmware for use in VM’s or the main software.
My main reason for buying the new hardware was to try esxi, so that was a little sad. I may try flashing back factory settings to H310 to use the hardware. The backplane wasn’t a issue.
3.) Dell software doesn’t recognize any drives attached to the H310 in IT mode to boot from so another sata drive or pcie adapted sata drive was needed to install the hypervisor to. I have not tried a pcie drive via a pcie adapter. This also did change the fan speed supposedly because it can’t see the drive temperatures, but can see the backplane is populated with drives. It’s still fairly quiet relatively especially compared to a comparable rack u2 server.
That’s about all I have discovered so far. Once everything is updated I may load proxmox via internal SD for kicks. I’ll also have to try citrix. I started to set it up, but was a trial version… I have to find out where exactly the free version is. I sent them a email to see if I can get a license just for home use, but not sure how that will turn out.
Once I have everything changed around, silenced, set up etc, I will post some pictures of the modifications I made.
Yeah, I mean you could always set up a truenas box using cheap ebay/spare parts some time if you want to dig into ZFS more. My home office NAS was a franken-machine of spare parts for a couple years before I finally upgraded it.
It’s still odd the flashing to IT mode caused you such problems, but if I had to guess I’d say that smells more like vendor lock-in shenanigans than a real compatibility issue. Maybe flashing the controller changed something that was used to identify the device.
From what I read it removed the Dell drivers built into the h310. It keeps you locked into using the PERC line of sas controllers. I can’t confirm for sure, but its what I have read more than three different places so might be some truth too it. Makes sense with enterprise gear to make you need their products.
Progress is being made with my new Dell Poweredge T320. I had attempted to do some individual updates to the server… I knew there had to be a better way.
I tried to research it a little after the built in tools failed to guide me to what I needed. I had tried to update via iDRAC and Life Cycle Controller, but using ftp and http were both unsuccessful. I then discovered “Platform Specific Bootable ISO” images.
Platform Specific Bootable ISO images have all the current update for a server all in one place in a bootable ISO that updates the server all at once to the current firmware and onboard software all at once. I am currently running it. I know it will take a long time but I’m hoping it is successful. Will update when its completed.
Well I just spent the better part of a day attempting to upgrade my Dell Poweredge T320 to a T420. It uses the same socket, ram, psus etc… For whatever reason when I powered it on with one CPU I had no video output whatsoever and no post codes according to the display, but the idrac system didn’t function at all. With no way to give commands or anything, I was left with the reset jumpers. I attempted this but nothing changed. All in all it was a very disappointing experience.
What I dont get is that the cases are identical. The wiring and connectors all appear to be identical. I’m not sure what the issue was, but I am abandoning the project for now.
I had done some googling prior and no one had posted anything saying it did or did not work. I can say for sure it appears to NOT work for whatever reason. I tried a number of different fixes but no joy.
If anyone else has some advice, or has done it in the past I’d love to hear from you.
you did try a cpu in the other socket/ check the block diagram for what is ran from each socket in case you had the “wrong” socket populated for a 1p system?
Presuming the system was a dual socket system…
face palm Im going to review the PDF manual for the t420…
presumably the socket closest to the backplate would be the main one, but I could only see “CPU1” and “XDP CPU” and never ran a server board.
I just heard that Some systems only activate certain ram slots/pcie slots with a second CPU installed
That’s what I "ass u me"d… I may have had to populate the rear cpu socket closest to the backplate. I did the opposite. A quick look at the pdf didn’t yield any results. I may have to review some images from the interweb…lol. I’m going to keep at it, but for now so I can play I just reinstalled the old t320 mobo and may revisit the project at a later date.
HOWEVER… I am going to attempt having the Grid k2 installed and see what hypervisors it may work with now with driver changes from NVIDIA.
Added note: I still have all the ram, second cpu and cooler either here or its already en route. I may next week after a little more research attempt the swap again to the t420 board.
In reality it’s kind of a over reach as this is more than enough machine for testing and messing around. I already have the 8 core 16 thread AMD server, this 10 core 20 thread beast is a testing platform. The extra 20 threads has no real use till I discover more things I can do with the server.
I also should note- using the Dell server on their web page are specialized images with built in support for the dell server and integration with its monitoring systems. Windows was not one of those surprise… BUT VMWARE, Red Hat, Citrix, all had specialized image iso’s for anyone interested who hasent worked with enterprise gear.
Yeah I don’t think you’re going to have a problem with that. 
Eventually, but I’m not that well versed yet…YET
@Trooper_ish
UPDATE on Dell Poweredge T320 to T420 motherboard swap. I got a little deeper into the manual for the T420 motherboard. If you run one CPU your supposed to populate the CPU 1 slot (which is where I had my CPU installed) so I did have it in the right place. CPU 1 is clearly marked on the board and is the one closer to the drive bay/backplane and further from the rear I/O ports.
Can see it here on the left, board is oriented how it would be mounted in the unit.
Next step is a email to Dell Support… See what happens and they say. With the video output not working and the idrac Locked and not resetting with the motherboard jumper I may have a bad board… 
Well I’m getting a refund on the board from the seller so thats a plus.
BUT…
I now have a dimm in the t320 server that has failed to pass testing and is disableing ECC functions. Yeaaaahhhhh. Never ends and this is with one home server. I cant imagine what all you guys do that have this job for a living. Thankfully, this is the backup/testing server.
I bought new… maybe this is a less reliable company? Ether way replacement will be on order soon.
how to build a HACKING lab (to become a hacker) - YouTube might not be what your after on the whole but theres some good easy to follow instructions for setting up your own virtual lab and internal network using vbox.
forgive chuck he’s always having one to many coffee’s
@HEXiT Sweet. Thanks for this. Always happy to find some good info and other things to try on the server. I appreciate it.
Update on Dell Poweredge T320:
I was wrong on the new A-Tech ram. It was fine. I had read the dimm’s in the wrong order (backwards).
One of the ranks of the ram that came with a refurbished server I bought from TekBoost. I contacted them and told them about the issue and sent a screen shot of the error for the failed dimm and they are sending a replacement free of charge. I’ve been really happy with this company.
I also ran into issues installing windows on the server on one of the drives. I had to pull all the drives with all the different OS’s out from the machine to install it. Weird because it had worked earlier.
So now I have Windows server 2016 datacenter, Proxmox and ESXi all installed. The next one I may try if I can find a good guide to help me is QEMU with libvirt that @SgtAwesomesauce suggested. I do want something opensource and linuxed based so I can use the CLI as much as possible to learn.