First off, I tried this process with IOMMU passing-through of NICs but had a LOT of kernel-panics/issues… So resigned to getting this working with normal Proxmox linux bridging of NICs first… And TBH - the speeds I am seeing are more than acceptable - I may just stay with this and not bother passing-through - we’ll see.
Products
The 9955HX/64GB/1TB model of the MS-A2
4-Port Realtek based 2.5GbE SFF NIC
2x 2TB PCIe 4.0 Crucial (single-sided) NVMe m.2
External m.2 Enclosure (keep the pre-installed 1TB m.2 for other uses)
2x MikroTik SFP+ 10GbE copper (use fibre if you have it installed - I don’t)
Hardware to avoid
6Com SFP+ do not seem to work despite being sold as compatible with Intel. Had some spare MikroTik ones and these work great with the x710 built into the device.
I had issues with the first 4-port PCIe NIC - the IOcrest Intel I225-V based one - avoid. The Realtek one I replaced it with works out-of the-box.
Hardware Setup
Really simple - I won’t go into too much detail, swap the m.2 drives over into the currently populated slot, and the one next to it. Keep the pre-installed 1TB and install it into the external enclosure for other uses.
Install the NIC - ensure it is fully seated into the PCIe slot - can be a bit tricky but nothing outrageous.
MS-A2 BIOS settings
Firstly, you may want to check the BIOS version - I was up to date with the versions provided by Minisforum.
Set the second M.2 slot to gen 4.0 speed - I hear they’ve downgraded them to gen 3.0 for heating concerns but during testing I’ve noticed no real difference, a delta of maybe 2c. The second M.2 has no heatsink but is directly air cooled. It being single-sided likely contributes to this outcome also.
Enable the usual stuff - IOMMU, etc, if you intend on using it.
Disable anything you don’t plan on using - pointless having the WiFi card on if it’s not part of your build.
Install Proxmox
For this, I hooked it up to my PiKVM, monitor/keyboard will work fine as well.
- Connect port 8 (counting book-wise, so the bottom-right port) to a live switch you can reach from another machine and set this as Proxmox mgmt port during install
- Install Proxmox using PiKVM
- Select ZFS RAID-1
- Reset your Proxmox subscription if you need to and then register subscription (this note is here because I had to do multiple installs while I figured out some of the issues)
- Wait for Enterprise repo to become available
- Update
- Install lm-sensors net-tools iperf3 unzip
- Run
sensors-detect - it shouldn’t find anything new but is good practice to do this
- Download Ubuntu 24.04 from CT Templates
- Set up Ubuntu LXC, update it and install net-tools nmap iperf3
- For each network interface, one at a time, set up a temporary bridge and enter each interface (only that one) as the slave/bridge port
- To test, have this temporary bridge connected to the LXC and do your ‘ip a’ / ping / iperf3 tests where appropriate
- Check/note each port’s details in a table for your reference later.
- Reboot
N.b. PCIe card appears recognised in x8 mode but interfaces don’t come up, use 4x4 mode and they work…
You should end up with something like this:
Proxmox Networking
Network Ports (from top-left, read like text)
| Port |
Chipset |
Interface |
Mac Addr. |
Speed |
Comment |
| 1 |
r8169 |
ens1 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
WLAN-1 Port |
| 2 |
r8169 |
ens9 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
WLAN-2 Port |
| 3 |
r8169 |
enp10s0 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
|
| 4 |
r8169 |
enp9s0 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
|
| Next Row |
|
|
|
|
|
| 5 |
X710 |
enp5s0f0np0 |
xx:xx:xx:xx:xx:xx |
10Gb |
|
| 6 |
X710 |
enp5s0f1np1 |
xx:xx:xx:xx:xx:xx |
10Gb |
|
| 7 |
I226-V |
enp4s0 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
|
| 8 |
RTL8125 |
enp3s0 |
xx:xx:xx:xx:xx:xx |
2.5Gb |
Proxmox Mgmt. Port |
Update Intel x710 Firmware
From Intel, search “Non-Volatile Memory (NVM) Update Utility for Intel® Ethernet Network Adapter 700 Series” (no links allowed)
You may be able to update the x710 firmware to the latest. The others seemed up to date for me.
Install OPNsense
- Upload the DVD install ISO to Proxmox
- Set up a VM with the following settings:
- Machine: q35
- Qemu Agent: yes
- BIOS: OVMF (UEFI)
- Disk: 100GB, SSD Emulation, Discard
- CPU: 8 cores, host type
- Memory: 32768, non-balooning (balooning seems to give an incorrect qemu-guest-agent reading for RAM)
- NIC: testing bridge from earlier, VirtIO, no firewall, multiqueue 8
- Upon starting new VM, catch BIOS with DEL and disable Secure Boot
- Login as installer / opnsense
- Install on UFS (as using ZFS at Proxmox level, no benefit)
- N.b. it will try and take over 192.168.1.1 / routing duties if left plugged in for the install
- When OPNsense comes up, log in on the console via Proxmox and set the single connected interface as WAN/DHCP
- Can now log in via the Web UI and do initial install wizard and updates
- Install iperf in System->Firmware->Plugins
- Install os-qemu-guest-agent in System->Firmware->Plugins, and set it to run on startup
*** Now you can set up your bridges and bonds and assign them to the OPNsense as appropriate - N.b. no need to give these an IP address and invite Proxmox to bind a WebUI to these interfaces.**