@mathew2214 what OS are you running on the host, and what virtualization software? Also, do you have a need for any two VMs to talk to each other directly or should this work through the router?
Usually, it’s technically possible to give each VM it’s own VLAN, and your host can usually somehow be isolated from the VMs.
normally you’d isolate other hosts on your network via the switch, which wouldn’t forward tagged frames of VM vlans to rest of your network.
If your host is connected using WiFi, you can maybe to tunneling of some kind (e.g. each VM gets a GRE tunnel to the router or do VXLANs which is basically ethernet + vxlan header in udp)
debian 10 buster
Linux 4.19.67
Qemu KVM/Libvirt
i want every VM besides ones i specify to be able to directly communicate with each other.
i want the firewall in my external router to apply to my VMs.
i think maybe it might be worth the trouble to set-up SR-IOV?
SR-IOV - you’ll need hardware that supports it, and you’re relying on your router, etc. to do the segregation. You’ll need VLANs for each virtual network adapter.
As above, like i described, just put all your VMs on isolated host only networks, then use some sort of firewall software (e.g., pfsense) to route for them off the isolated network, treating your physical network as it’s gateway.
VMs that are on the same “isolated network” can talk to each other within the isolated hostonly network.
The.30 is how I tag which VLAN to put the VM onto, the below script uses this to setup the port.
cat /opt/VM/bin/ovs-ifup
#!/bin/bash
SELF=$(readlink -f $0)
if [ $UID -ne 0 ]; then
sudo $SELF $1
exit $?
fi
switch='br0'
vlan=$(echo $1 | cut -d'.' -f2)
ip link set $1 up
for PORT in $(ovs-vsctl list-ports $switch); do
if [ "$PORT" = "$1" ]; then
exit 0
fi
done
ovs-vsctl add-port ${switch} $1 tag=$vlan
/opt/VM/bin/ovs-ifdown
#!/bin/bash
SELF=$(readlink -f $0)
if [ $UID -ne 0 ]; then
sudo $SELF $1
exit $?
fi
switch='br0'
ip addr flush dev $1
ip link set $1 down
ovs-vsctl del-port ${switch} $
This will give you a managed switch on your host. The pfSense VM can just bridge with the root bridge device (br0 in this example) and then configure each VLAN in pfSense so it can filter and forward to the public network. Obviously the pfSense VM needs a 2nd network device (WAN) also that it can forward public traffic over.
This gives 100% isolation using 802.11Q. All VMs on the same vlan will have unrestricted communcation with each other without involving the pfSense VM, only routed traffic to the other subnets or the internet would go via pfSense where it can be filtered.
Your VM guest OS does not have to be configured with tagging, etc… it’s completely transparent to them as the tagging happens on the port. You can even run DHCP services on each vlan from pfsense to make things easy.
Assuming your VM host is connected directly to the openwrt switch, you’re all set. You can configure VLANs on openwrt, assign interfaces, assign them to firewall zones, and give those zones the ability to talk to whatever (or not) as needed.
However, if you have a dumb switch in between your host and openwrt router, be aware that some network stacks on some devices will blindly strip the VLAN tags from ethernet frames on packets and let them into the rest of stack… while this communication is unidirectional and byitself somewhat limits an attacker in what they could do - it’s rarely used by itself ; and still should be considered insecure.
I actually was aware of this but completely forgot about it and defaulted to OVS. I work a ton with OVS in enterprise environments where we use the IPsec features. I will have to have a play. Thanks!
It very likely is the simpler and better solution in this instance.
