Return to Level1Techs.com

pfSense MASSIVE CPU utilization, need help troubleshooting


#1

I have currently 3GB of RAM allocated to a pfSense VM. Even with 3GB it never fills more than 15% with /var and /tmp both residing in Ram. It is a near freshly installed pfSense in a VM on my unRaid server. I migrated from a separate machine since it didn’t support AESNI and I would like to save on power anyway. So I have been noticing there is some HUGE CPU usage going on. Sometimes it literally maxes out the two cores I gave pfSense entirely and causes up to a 60% drop in internet speed for as long as it is happening. usually, after a few minutes, it settles down to normal CPU usage, idling at a max of 1%-2% and barely kissing 5% with heavy usage.

Firstly, could it be related to running pfSense in a VM? I know a few guys that are very successfully running pfSense on unRaid through a VM for everyday use and it works great for them.

Secondly, I did actually make a complete screw up and I borked my pfSense a few days after I installed it on this VM. Thankfully, I had a backup of the configuration on hand so i was very quickly able to reinstall onto a fresh instance and restore. I honestly don’t know if this issue was present before but it may have been and I didn’t notice it. I say this because I only realized it was an issue when it was eating all the CPU cycles from my Plex docker making transcoding not work at all.

I went and did some digging and found that I am not the only one to have had this issue and so I turned on SSH and ran a few commands outlined there and this is what I have found. Seems that every time this happens it is this specific process. /usr/local/bin/php-cgi -f /usr/local/sbin/prefixes.php It can easily hit 99% CPU usage. From what I can tell, some posts were saying it has to do with DHCP6. I can’t disable IPv6 entirely since my wife works from home and some of the services her work uses are IPv6 only. Though, frankly, I shouldn’t have to disable IPv6 to fix this. I have never had this problem before and just need a hand figuring out what the heck I can do to stop this insane CPU thrashing. Maybe some script is stuck in a loop? I simply am not skilled enough here to have any idea.

My motherboard is fully capable of virtualization. I passed through a physical 4 port Intel gigabit NIC. WAN, LAN, ROOMMATELAN, and the fourth is a spare I currently don’t have hooked up anywhere. For storage, I am not using the virtio drivers and am fully emulating the storage. I did this because even though I am fully aware pfSense has virtio support in the kernel, on unRaid you still have to do the initial install emulating a standard SATA interface and since I had to reinstall it due to my own screw up, it has remained in SATA mode.

Since it is virtualized, would the VMtools package be useful for me? I thought that was only for VMware products but honestly, I have no idea. Could it have something to do with my virtualization setup or settings? It is entirely possible I am not doing it the ‘best’ way.

Sorry for the wall but I figured too much info is likely better than not enough. I just don’t know what is going to be useful and what is not.

Thanks for any assistance you can provide in advance!


#2

Since your router is already virtualized, if your setup isn’t too complicated, try booting
OpenWRT, specifically the combined ext4 image from here: https://downloads.openwrt.org/releases/18.06.0/targets/x86/64/

If I look at that particular .php script , it loops over the lease file and the log file (without any interprocess coordination, that’s one bug right there, but not what you’re running into) and emits routes. Can you check the sizes of those files?


#3

So, that file right now is only about 5KB but I have been running a few days with IPv6 turned off.

I wish it were as easy as just booting another OS right now. At the moment, I have a remote access VPN I need and several subnets that need full isolation to work properly. I might be able to try this come Monday or Tuesday since most of the house is at work and my wife isn’t working. I just can’t afford any down time right now.

I can turn IPv6 back on and report back though. See if those files end up swelling.