KVM: Getting Bonding, Bridges, VLANs and Macvtap Playing Nice Together

I finally made the jump from using a Physical Ethernet -> VLAN -> Bridge stack to Physical Ethernet Devices -> Bond -> VLANs ->MacvTap. Notice bridges are not included, that’s because they don’t work with Linux bonding, you need to use Macvtap interfaces.

Here’s a sample configuration (you’ll need to install ifenslave & vlan if you haven’t already):

# The loopback network interface
auto lo
iface lo inet loopback

#Intel Interface #1
auto p4p1
iface p4p1 inet manual
    bond-master bond0

#Intel Interface #2
auto p4p2
iface p4p2 inet manual
    bond-master bond0

#Onboard Intel NIC
auto eth0
iface eth0 inet manual
    bond-master bond0

#Primary Bond Interface
auto bond0
iface bond0 inet manual
bond-miimon 100 # Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is inspected for link failures.
bond-downdelay 200 # Specifies the time, in milliseconds, to wait before disabling a slave after a link failure has been detected.
bond-updelay 200 # Specifies the time, in milliseconds, to wait before enabling a slave after a link recovery has been detected.
bond-mode 0 # round robin (think of it as network raid 0)
bond-slaves none # defined in the interfaces above with bond-master

#DMZ VLAN
auto bond0.11 # this sets a vlan tag of 11 for all traffic on this interface
iface bond0.11 inet manual
    vlan-raw-device bond0

So, the above will aggregate three NICs and round robin packets across the three interfaces to get the combined output of the three. Once you’ve setup the above, you can go into virt-manager and add a Macvtap interface paired to either the bond0 interface or one of you VLAN bonded interfaces.

I set mine as virtio and mode of bridge. This allows other guests in the same vlan to communicate within the host.

High CPU Usage in QEMU/KVM for Windows 7/8/2008 R2 Guest

It may be QXL. If you have SPICE graphics enabled (probably affects VNC too), check to see if you’re using QXL. Changing it from QXL to VMware’s VMVGA took idle CPU usage of 10%-30% to 3-7%.

Virt-Manager has a bug in it that won’t let you change it from QXL, so go to the command line and type in:

virsh edit domainname

Go down to to the video section and update the model type to be:

 

Shutdown and start the Windows guest again and you should be good to go.

KVM: Configure Mirrored Port’s Traffic to Be Visible in Guest (SNORT)

This one was tricky. If you want SNORT to run as a guest, getting the port mirrored traffic to it wasn’t exactly trivial.

My Host Setup:

NIC 1: Handles all bridges and VLANs

NIC2: Dedicated to SNORT, plugged directly into mirrored port on switch.

 

Before we start, make sure your SNORT guest is off and the bridge interface that you want to use for packet sniffing is down. Getting the packets to flow properly requires a very specific sequence and failure to do that will only result in frustration and confusion.

First Step (on KVM Host):

Edit /etc/network/interfaces to reflect for NIC2 (eth3 in my case):

auto eth3
iface eth3 inet manual

#SNORT Interface
auto br3
iface br3 inet manual
bridge_ports eth3
bridge_stp off

Bring up br3

ifup br3

I’ll need to test it, but you may ensure the host doesn’t get an address on that interface by adding in a line: address 0.0.0.0

Now, the tricky part is to get the traffic to show in the guest. On the host, just to make sure your port mirroring is working properly, do:

sudo tcpdump -i br3

You should see a ton of traffic if configured properly.

Now, to get the traffic not destined for the host to go to the guest (SNORT), type in:

brctl setageing br3 0
brctl setfd br3 0

Now, if you boot up your guest (it should have two interfaces), you should be able to tcpdump -i eth# and it will show the same large amount of traffic. The above is manual and will not persist across reboots.

To make it persistent in Ubuntu (replace br3 with your bridge interface name):

cd /etc/network/if-up.d

touch br3-mirror

chmod +x br3-mirror

nano br3-mirror

#!/bin/bash
if [ "$IFACE" = br3 ]; then
brctl setageing br3 0
brctl setfd br3 0
fi