I was thinking of replacing my old setup with pfsense, but i would like to run it on a vm with 2 dedicated cores(fx 6300) (or an i7 3700 that i have laying about) and on the 4 other cores i would like to run a freebsd server or freenas(also freebsd) server alongside it is this a good idea? as the 2 systems will be seperated the pfsense part should be safe from the freebsd part and vice versa. but is this even an optimal configuration? it would be recommended to have them seperate physical machines but that is for non-adventurous people.
So my question is, any problems with the idea? Because i can't really find any problems myself in it nor can i find posts of people doing something similair online I ask for advice :p.
Quick word of warning. I've found that FreeNAS does not like to be virtualized on libvirtd+KVM. Work fine in Proxmox, though. I've seen a number of setups that use a pfSense VM as the firewall for a network, and that's actually how I've had my network setup for a number of years now.
In general I'm not a huge fan of virtualizing FreeNAS. I feel like ZFS is badass enough, and what I want it to protect is precious enough that it warrants having its own hardware. But I have done it before. One crazy setup I did was when a company I was working for got tired of hardware RAID controllers (seriously, they're a PITA), we just added disks to this Proxmox build as individual disks. We then created a FreeNAS VM, passed the disks to it, it then created a RAIDZwhatever and presented the storage back to Proxmox and on that storage we created the rest of the VMs. Wonky as shit, but it worked.
I see, proxmox is based on KVM so wierd it acts like that. but pfsense will gladly accept passed through nics? and you seem to have some experience, is ecc memory a must, or will i be just fine running it 24/7 without it(because it would be a shame where my data to become corrupt)? Thanks for the quick response, appreciated :p
I've actually not used passthrough with pfSense, I've always just used a normal bridged NIC. So let's say that the 2nd physical NIC is hooked up directly to your cable modem. We would create a bridge in Proxmox, call it vmbr1, and Proxmox will have an IP of 192.168.100.5 netmask 255.255.255.0. This puts it on the same network as the cable modem (if you didn't know, try hitting http://192.168.100.1 on your network right now). So Proxmox should be able to ping you cable modem at 100.1. Then one of the virtual NICs that is given to your pfSense box is pointed to vmbr1, pfSense is set to DHCP on that interface, and boom, away you go. You might need to reset the cable modem once pfSense is up, but that should be the extent of it.
ECC memory in FreeNAS? Honestly I've been running without it for gosh like 8 years now. I hate saying that because it's a lot like saying, "I've been riding a motorcycle without a helmet for X number of years now, and I'm fine." I guess I would say if I needed to rebuild my FreeNAS box tomorrow, I would build a system with ECC. The price for ECC is pretty much on par with regular memory, so might as well just go for it. My situation, like so many situations in IT, I had a proof of concept deployed and it immediately became production. SMDH.
Like I wouldnt even run Pfsense in a FreeBSD jail dude... let alone a non jailed one. Pfsense is your edge of your network, its a router and firewall, putting your freenas at edge is how you get data theft.
i will be running pfsense next to freenas not on top of and they will be using seperate nics and drives in turn having freenas go through pfsense regardless :p, let something else handle the separation like proxmox. but thanks for replying :P.
I see that is probably better than using passthrough, i think i will give proxmox a try and test it out on main pc before ordering drives, ram and a nic :P. thanks for the great response :D
Go Ahead an do it and just please post you Ip Publicly. I'm done. You have seem to made up your mind, so i am not putting any effort and I advise others to do the same, this is what we call a lost cause folks. That and you contradicted yourself when answering me.
The reason you cant find information on running an VM of PFsense IS BECAUSE THATS A TERRIBLE IDEA, even a Jailed one. We have explained that, and I wouldnt even use promox, your better off running PFsense of bare dedicated secure hardware. This is your edge router..... Your first line of defense, it just doesnt seem like good practice to instill what you want to do. Unless you trying to make a completely powerless resilient Router system, I just guess Im failing to grasp why one would want to attempt it. Even then I wouldnt do it though promox I would do it thought something like ESXI.
Dont take my wit as an attack, its not, remeber poe's law.
I've lost count of how many threads on this forum from people wanting to run both pfSense/FreeNAS on the same machine or run pfSense in the VM and the answer to these threads is always "its a very bad idea from a security standpoint" It is literally your first line of defense, it is a firewall and is meant to be as secure and reliable as possible.
Also go to the pfSense forums and ask this question and you'll get exactly the same answer from everyone. It is a bad idea and people don't do it because it is a bad idea.
I'm not getting into the security aspect of this. Just here to discuss functionality.
If you're using something qemu/KVM based, I don't recommend pfsense. It generally worked so I could sometimes go months without issues, but every once in a while I would lose connectivity on one or more interfaces. I tried virtio and e1000 for a long time without luck. Every time I lost connection, I needed to reboot the VM to fix it. I seemed to not be the only one having these issues. Later I switched to a Linux based VM for the task and all problems went away.
I haven't tried freeNAS, but I do run ZFS on Linux on the same KVM hardware. Having a HBA for passthrough is highly recommend for running ZFS on a VM. I use the LSI 9300-8i
Running pfsense on Proxmox can be done pretty easily. I am doing it right now for my home setup, it has been solid for six months now. I switched from bare-metal because I wanted to lower my power consumption and save a little space while I was at it.
As others have vigorously stated in this thread. Running your router as a vm wouldn't be called "best practice" because it is the edge protection for a network. However in a home environment compared to some cheap router from your ISP, I would take pfsense virtualized any day. Again I wouldn't recommend this for a business but for home use to learn and experiment, go for it.
If you are going to do it, then a dual nic card will have to be purchased purely for the pfsense vm connection. Then your motherboard nic, can be used for the hyper-visor.
Here is my network layout for proxmox
vmbr0 is the node, Vmbr1 and 2 (Wan and Lan) are directly connected to the pfsense vm, like so
As for Freenas, I have no Idea. Good luck with that
pfSense in a VM can work. I run it in ESXi 6.5 and seems to be good so far. Running it on bare metal yields more performance obviously, but that doesn't seem to be big of an issue most of the time.
The unspoken "security issue" (all I see is meme pictures) of running a firewall virtualized is that any network glitch/bug/0-day of the host (ESXi) could mean the complete box is compromised before traffic is routed to the pfSense VM, completely defeating it's purpose. If you can route the WAN NIC via VT-d directly to pfSense, the VM's host will not load the driver and network stack for that NIC, theoratically reducing the risk of this issue cropping up. Of course pfSense also runs FreeBSD which has a network stack; but if that is compromized than any pfSense box can be compromised.
How big of a risk is this? Not sure, alternatives like ISP routers or crappy off the shelf router boxes don't sound great neither.
I have been running pfsense on virtual box with 2 bridged nics for over 6 months now. I have it connected between my cable modem and PCs.
I got fed up of buying routers as I always encountered the same symptoms after the first couple months of purchase. I would end up with my internet speed slowing down to a crawl and then eventually requiring a restart of the router. I would guess these problems were related to the hardware probably overheating but these problems completely went away since switching to my current pfsense setup.
I don't know if this has changed on recent versions or anything, but if you're going to run pfsense on a VM you should use vmware or esxi as pfsense lacks the drivers for virtio (used by KVM) and paravirtualisation (used by xen) but does include the vmware tools (as a package) for vmware.
The security risks are exadurated, it's mostly an issue of increasing the attack surface and complexity so an attack has a higher probability but isn't guaranteed. I'd say that a larger issue for home use is that having all your stuff running off one machine can be a real headache when things go wrong.
You need a cpu that has amd-vi (fx-6300 has it) or intel vt-d to do this. But I run pfsense in a vm at the moment. I simply use kvm/qemu with libvirt and virt-manager on a ubuntu server. To watch tv we need a isp modemrouter that needs fucking internet to access the configuration panels no 192.168.0.0. So I'm not really concerned about the security. I would however be carefull with the freenas part. Zfs and virtual machines don't go together. (Has something to do with the paging if i recall correctly). What you can do however is setting up a zfs storage pool with ubuntu on your host and install samba to make it available over the network. But if you have sensitive information on your nas I would not risk it.
We use them in our production servers as back up firewalls In case a hardware firewall goes down(EG ASA, Watchgaurd, Sonicwall).
Esxi 5.5 runs them well from my expereince from work. I recommend that you have a CPU and motherboard that can handle virtualization before doing any of this of course.
On thing that can be odd though is that vlaning might not work and you will have to reboot the VM to fix it, seems very hardware dependent. We were able to replicate this with the dell r210 servers that we use but i am not able to replicate it with the testing machine i had which is an INTEL NUC.
Tried ESXI 6.0 and 6.5 with no issues that i can personally report on.
