Inconsistent NIC Speeds

I am running a Proxmox hypervisor with various VM’s and services running on it. I have two NIC ports on the board, one is an Intel GB Lan port and the other is a Qualcomm Atheros Killer GB NIC. The issue I am running into is that the Qualcomm NIC is very inconsistent with it’s throughput. I am running speedtest-cli and different speedtests in the browser but not getting anything consistently from the Atheros NIC.

Right now the network flow is:

Router–> Dumby switch–> Proxmos Host (in two both Nics) --> Pfsense VM (Atheros Nic is the Wan and a virtio port is the lan) --> Win10 VM, other containers.

I’ve ran speedtest-cli through the Proxmox host and I’ve gotten constantly 100Mbps down, and 10Mbps upload. On the Atheros NIC, I would see these speeds but then after a day of a reboot it would shoot down to ~80Mbps down and ~2-3Mbps upload.
I’ve tried running the tests without the traffic passing through the Pfsense VM, turning off all traffic shaping on the network, running the Pfsense VM through the Intel NIC and then comparing back. Also changing the ethernet cord attached to the NIC as well. Nothing changes. The speeds are always inconsistent and never predictable.

Any hints on where to look next or if there is another driver for the Atheros Nic that I should look to trying?