InterVlan routing - cisco cbs350 and opnsense - problem with internet access

[Adures]

Hello,

So I am banging my head and can’t really figure out what is preventing me from having internet connection on vlan 20. Maybe someone with fresh look at the problem could help me.

I am using cisco cbs350 intervlan routing capibilities for intervlan routing and opnsense as firewall to the internet. Additionally cisco switch acts as a dhcp server and default gateway address for devices in vlan 10 and vlan 20.

My opnsense IP lan address is 10.28.10.1

Cisco CBS config

interface vlan 10
name Server
ip address 10.28.10.10 255.255.255.0
!
interface vlan 20
name Trusted
ip address 10.28.20.10 255.255.255.0
!

ip default-gateway 10.28.10.1
ip route 10.28.20.0 /24 10.28.10.1
router rip

I have Opensuse vm with ip 10.28.20.20 on vlan 20 and default gateway 10.28.20.10

From devices in Vlan 10 I can ping opensuse in vlan 20
From devices in Vlan 10 I can ping 8.8.8.8 and have internet access
From opensuse vm in vlan 20 I can ping devices in vlan 10
Traceroute from Opensuse VM

shows that it is routing correctly to Opnsense.
I can ping Opensuse VM on 10.28.20.20 from Opnsense VM

Opnsense Gateways:

and I added static route:

It even is visible in Firewall log and I allowed the traffic to pass:

So I have no idea why it does not work. All ideas or suggestions are welcome.

[FunnyPossum]

Out of interest, why not just do intervlan routing on the firewall?

[cowphrase]

@Adures

router rip

If you want to use a dynamic routing protocol, use OSPF if possible. RIP is ancient and annoying. Though you’re using static routes so you don’t need dynamic routing at all - best to disable it if its not being used.

@Adures

ip route 10.28.20.0 /24 10.28.10.1

You don’t want this route on your switch. The fact that you have a 10.28.20.10/24 IP means there will be an implicit route for local traffic, so that route does nothing.

Your static routes look correct. Can you ping 10.28.10.1 from a device in vlan 20? If so the firewall knows how to route packets back to vlan20. If not then you still have a routing issue.

My guess is that Opnsense is not doing NAT to outgoing packets from vlan 20. Have a look at tcpdump on your WAN interface, and on your inside interface - the source address should change (aka get NAT’d) when it goes out the outside interface.

[Adures]

This if for my trusted and server vlans. It allows me to offload some traffic that will be happening there.
Opnsense is virutalized on xcp-ng host so with current setup, in case my host or firewall vm is down I still have access to everything on my LAN, which I find really useful.

[Adures]

@cowphrase

If you want to use a dynamic routing protocol, use OSPF if possible. RIP is ancient and annoying. Though you’re using static routes so you don’t need dynamic routing at all - best to disable it if its not being used.

Thanks, will do that. I think this switch only supports RIP and static routes.

@cowphrase

You don’t want this route on your switch. The fact that you have a 10.28.20.10/24 IP means there will be an implicit route for local traffic, so that route does nothing.

I am not networking expert, so when it was not working correcly I tried few things Thanks for suggestion.

@cowphrase

Your static routes look correct. Can you ping 10.28.10.1 from a device in vlan 20? If so the firewall knows how to route packets back to vlan20. If not then you still have a routing issue.

Yes I can ping it.

@cowphrase

My guess is that Opnsense is not doing NAT to outgoing packets from vlan 20. Have a look at tcpdump on your WAN interface, and on your inside interface - the source address should change (aka get NAT’d) when it goes out the outside interface.

You are indeed right. They are not NAT’d

[Adures]

Adding this rule in Firewall/NAT/Outbound fixed my issue:

Thank you very much for your help!