Making a bi-directional host <-> vm network

Hardware Hub Networking
1

I tried using the default nat config for this, but that only allows guest → vm, not host → vm or at least i could not get it to work

at this time i have them connected via a physical network (macvtap on the vm), but a internal connection would allow full loopback throughput and if i needed to move a lot of data from the host to the vm i could without going though what will be the top level switch in my network

i am very new to using kvm/qemu and i am using the cli controls

2

Use a bridge?

1 Like
3

I recall you ended up using macvtap (from another thread) … to get your host<->guest to work, you need a macvlan interface on the host. …

physical host<->macvtap don’t work
macvlan<->macvlan , and macvlan<->macvtap do work.

If you want to keep things simple, you can leave the physical interface up but not configured (not even DHCP), and do all your host networking over a macvlan interface.

(Alternatively, you can play various complicated games in the kernel with static routes and static arp/neighbor entries, I wouldn’t recommend it, it’s hard to get settings to stick).

4

I have 4 physical NICs,
onboard reltek (full duplex) (host connection to lan)
2x PCIe intel (full duplex) (mac vtaps on both for the pfsense VM’s LAN and WAN )
1x PCI intel (half duplex) (spare)

going over the physical network from host to vm or vise versa can be brutal on the cpu

put this in netplan

  bridges:
    br0:
      addresses: [ 172.16.69.1/32 ]
      dhcp4: no
      dhcp6: no
      optional: true

and used this xml

<network>
  <name>kvm-bridge</name>
  <forward mode="bridge" />
  <bridge name="br0"/>
</network>

set the static ip in the guest and there is no connection between host/guest

here are my host interfaces

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 50:e5:49:d9:87:6b brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.126/24 metric 100 brd 10.0.0.255 scope global dynamic enp2s0
       valid_lft 84500sec preferred_lft 84500sec
    inet6 fe80::52e5:49ff:fed9:876b/64 scope link 
       valid_lft forever preferred_lft forever
3: enp1s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 00:15:17:be:13:e4 brd ff:ff:ff:ff:ff:ff
4: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:15:17:be:13:e5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::215:17ff:febe:13e5/64 scope link 
       valid_lft forever preferred_lft forever
5: enp3s6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:1b:21:c4:fa:08 brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fe:e7:a1:b5:c2:ea brd ff:ff:ff:ff:ff:ff
    inet 172.16.69.1/32 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::fce7:a1ff:feb5:c2ea/64 scope link 
       valid_lft forever preferred_lft forever
7: macvtap0@enp1s0f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 500
    link/ether 52:54:00:95:04:64 brd ff:ff:ff:ff:ff:ff
8: macvtap1@enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 500
    link/ether 52:54:00:08:57:83 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe08:5783/64 scope link 
       valid_lft forever preferred_lft forever
9: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:db:29:27 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fedb:2927/64 scope link 
       valid_lft forever preferred_lft forever

and this is the nic on the guest

vtnet2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: Host_Direct
        options=c07b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE>
        ether 52:54:00:db:29:27
        inet6 fe80::5054:ff:fedb:2927%vtnet2 prefixlen 64 scopeid 0x3
        inet 172.16.69.2 netmask 0xffffffff broadcast 172.16.69.2
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

on a side note from my last thread rick mentioned, it seems iperf on pfsense is just brutal on the cpu compared to on ubuntu bare metal or vm it does not matter, this CPU will pull 0.6A on ubuntu and 2.6A on pfsense when using iperf in full duplex

5

Make another macvtap <-> macvlan forwarding is just a host kernel / generic networking driver construct. Physical nic won’t be affected by that at all, unless you have the macvtap in one of the strange non default modes.

6

no idea what i am doing… i just know what i have tried does not work

<network>
  <name>macvtap2</name>
  <uuid>b3e8a4a4-0b6e-49b0-9252-4944c2a855b6</uuid>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:26:65:ab'/>
</network>

tried setting host ips on vnet0 and virbr0 and have no connection with the guest

how hard can it be to have a virtual crossover cable?