New AMD 7352 Server : Looking for feedback on nested virtualization

[kuzko]

Heyo chaps!

I finally got the opportunity to use those new CPUs from AMDs rome nicely bundled in a supermicro server with 256 GB of RAM.

The performance is as planned really mind fanning, and my first tests on this platform are good, but I’m right now stuck on trying to setup a small nested virtualization lab :

I have been using XCP-NG for my virtualisation needs as L0, helped by their “Vcenter like” solution Xen Orchestra , it’s been a breeze.

But whether i’m installing ESXi or Proxmox as L1, the L2 guest VMs don’t do anything, and I get this line.

Unexpected nested vmexit: reason 0x7b

So, does anyone of you have an idea of what I should be looking into, or simply if I should even bother and it’s a known issue with EPYC CPUs?

Virtualization options in BIOS are enabled, as well as IOMMU (tested both on and OFF), and C-States are now OFF (tested both ways).

Also, if people want me to run a few commands go ahead just ask.

[2bitmarksman]

https://github.com/xcp-ng/xcp/wiki/Testing-XCP-ng-in-Virtual-Machine-(Nested-Virtualization) - This may help. Particularly on your nested VMs, you’ll need this:

• enable nested virtualisation with the special command on CLI: xe vm-param-set uuid=<UUID> platform:exp-nested-hvm=true

[kuzko]

Thank you, but it seems that the ESXI vm already has the flag among others.
Here’s the output of xe vm-param-list uuid=654d2153-fdWWWWWWWWWtherest

platform (MRW): timeoffset: -3; hap: 1; cpuid: [0x1:ecx=0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]; videoram: 8; vga: std; exp-nested-hvm: true; nic_type: e1000; device-model: qemu-upstream-compat; hpet: true; nx: true; secureboot: false; pae: true; apic: true; viridian: true; acpi: 1; cores-per-socket: 8