OPNsense uses up all the RAM its allocated in Proxmox

I made sure that qemu-guest-agent is enabled in Proxmox, and os-qemu-guest-agent is installed & running in OPNsense.

I have the memory set to 16GiB with 12GiB as the minimum, ballooning is also enabled.

According to Proxmox, OPNsense is using up about 14GiB/16GiB, but Proxmox reports that the RAM usage is just 2.6GB/16GB.

What else needs to be enabled? I looked up this issue, and most of the ones reported were very old

Proxmox version: 8.2.2
OPNsense 24.1.8