KVM Bridged Network Not Working
Preliminaries
This following worked for me for Ubuntu 12.04. You should disable your computer's firewall as you test this so it won't interfere.
The /etc/default/qemu-kvm file should be as originally installed.
You will need to have bridge-utils qemu-kvm
and libvirt-bin
installed. Any users using virtual machines should be added to the libvirtd group.
There no longer seems to be any need to add CAP_NET_ADMIN capability.
Network Setup
The default network mode is the User mode, also called SLIRP. It uses a predefined virbr0 bridge which is NAT routed to the guest computer. The NAT routing uses the kernel's ip_forwarding feature and iptables. Bridge mode uses a virtual bridge in the guest to which the (unnumbered) Ethernet interface connects, and on which both the host and the guest have their network interfaces.
The following diagrams may make the differences clearer:
You can see how the default User network is defined with:
virsh net-dumpxml default
I can set up the bridged mode with the following approaches:
In /etc/network/interfaces (from the bridging part of the post you mention in your question):
auto lo
iface lo inet loopback
#auto eth0
#iface eth0 inet dhcp
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0
Reboot; and make sure that wireless networking isn't active. Check the default IP route with ip route. It must be using the br0 interface.
N.B. If your Ethernet isn't hooked up when this change is made you need to have your Ethernet cable plugged in and getting a carrier or the boot will hang for two minutes and you won't have network capability That's because the eth0 interface, by being in this file, must come up before the boot can proceed normally.
N.B. Generally you can't use a wireless network instead of eth0 because of their inability to use multiple MAC addresses (I infer that they need a second one for the bridge).
As an alternative you can disable the use of Ethernet and make sure that it does not have an IP address, and that there isn't a default route set up with ip route. Then:
sudo ifconfig eth0 0.0.0.0 up
sudo brctl addbr br0
sudo brctl addif br0 eth0
sudo ifconfig br0 up
sudo dhclient br0 &
You could also supply a static IP address here, as well as defining the default route and DNS address. For this example dhclient does this.
Here's my route table:
$ip route list default via 192.168.1.1 dev br0 metric 100 169.254.0.0/16 dev br0 scope link metric 1000 192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.45 192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
Using kvm
I can then boot a bridged kvm machine with:
$ sudo kvm -name Quantal -m 1024 -hda foo.qcow2 --soundhw ac97 -device virtio-net,netdev=tunnel -netdev tap,id=tunnel,ifname=vnet0
The -netdev tap parameter makes sudo a requirement. As the VM is started qemu-kvm runs the following commands:
ifconfig vnet0 0.0.0.0 up brctl addif brctl addif br0 vnet0
This is done by /etc/qemu-ifup
The VM's vnet0 interface is added to the br0 bridge because the default route above uses that bridge interface. If it weren't there the tap interface instead would be added to the virbr0 interface. Since that's not connected to the Internet, NAT would be used to connect the guest to the host and the Internet, in my experiments. You can direct the vnet0 to a particular bridge in /etc/default/qemu-kvm. Using virt-manager below you can explicitly direct which bridge to connect to.
Because of the above commands issued by qemu-kvm, and the -netdev tap,id=tunnel,ifname=vnet0 parameter, the vm virtual machine is connected to the vnet0 tunnel, and the tunnel is connected to the br0 bridge.
I can now directly ssh into this guest VM from another computer on my network.
My host ifconfig (note the vnet0 interface that appears on my network when the VM is running):
$ifconfig
br0 Link encap:Ethernet HWaddr 00:1e:33:88:07:e5
inet addr:192.168.1.45 Bcast:255.255.255.255 Mask:255.255.255.0
inet6 addr: fe80::21e:33ff:fe88:7e5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6526 errors:0 dropped:0 overruns:0 frame:0
TX packets:7543 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2712940 (2.7 MB) TX bytes:1071835 (1.0 MB)
eth0 Link encap:Ethernet HWaddr 00:1e:33:88:07:e5
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7181 errors:0 dropped:0 overruns:0 frame:0
TX packets:7740 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2974585 (2.9 MB) TX bytes:1096580 (1.0 MB)
Interrupt:43 Base address:0x6000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:664 (664.0 B) TX bytes:664 (664.0 B)
vnet0 Link encap:Ethernet HWaddr ca:0c:73:c3:bc:45
inet6 addr: fe80::c80c:73ff:fec3:bc45/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:226 errors:0 dropped:0 overruns:0 frame:0
TX packets:429 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:26919 (26.9 KB) TX bytes:58929 (58.9 KB)
virbr0 Link encap:Ethernet HWaddr d6:18:22:db:ff:93
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
My bridge configuration while running the VM:
$brctl show
bridge name bridge id STP enabled interfaces
br0 8000.001e338807e5 no eth0
vnet0
virbr0 8000.000000000000 yes
Note that both the virtual machine's vnet0 interface and the eth0 interface are connected to the br0 bridge.
And the MAC's on the br0 interface:
$brctl showmacs br0 port no mac addr is local? ageing timer 1 00:05:5d:cf:64:61 no 2.54 1 00:19:d2:42:5d:3f no 36.76 1 00:19:df:da:af:7c no 2.86 1 00:1e:33:88:07:e5 yes 0.00 1 00:60:0f:e4:17:d6 no 0.79 2 52:54:00:12:34:56 no 0.80 1 58:6d:8f:17:5b:c0 no 5.91 1 c8:aa:21:be:8d:16 no 167.69 2 ca:0c:73:c3:bc:45 yes 0.00
Note that the br0 interface connects my host computer to the same bridge being used by the guest.
You can check that you are bridged rather than NAT routed to your own network by using traceroute 8.8.8.8. If the first node is your network's router rather than the guest's ip address your network should be working correctly.
See this documentation.
virt-manager
Be sure that you have installed virt-manager and hal. The hal package is a suggested dependency for virt-manager and is used to determine the network configuration of your system when creating or editing guests.
While having the br0 bridge defined as above I created a virtual machine with virt-manager as follows:
I was able to go directly to the rest of my home network and to the Internet from this guest. I was also able to ssh into it from the other (non-host, non-guest) Ubuntu computer on my home network.
Here's the very long kvm command run by virt-manager (for comparison with EApubs or anyone else having trouble with this):
/usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name precise -uuid f057a729-eda6-4b85-84dc-f100c9ae3789 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/precise.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/media/natty/home/gruber/ubuntu-kvm/tmpW8gSGB.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=18,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:0e:da:9b,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
Here's the network portion of the virtual machine description in /etc/libvirt/qemu/quantal.xml
<interface type='bridge'>
<mac address='52:54:00:b0:8e:aa'/>
<source bridge='br0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
According to this link, for performance and reliability, it may be best to set the network device model to virtio, you can do this by in the virt-viewer by pressing the i button, going to the NIC setting, and setting the "Device model" to virtio. You could also add this to the XML above by adding the line:
<model type='virtio'/>
In Summary
All this took on 12.04 was:
- Installing virt-manager, bridge-utils, qemu-kvm, and related packages
- Make sure each user wishing to use kvm are in the libvirtd group.
- Defining /etc/network/interfaces as above (which match the quoted article)
- Reboot, making sure Ethernet is plugged in and wireless (if any) is off.
- Either run kvm against an image directly with, e.g.
-device e1000,netdev=tunnel -netdev tap,id=tunnel,ifname=vnet0, or create a virtual machine with virt-manager, specifying network Bridge br0 under the Step 4->Advanced Options panel.
No further changes were needed to networking, capabilities, templates, or configurations.
To expose a service in your new guest to the Internet you should:
- Prepare and configure any firewall service you will need.
- Either assign a static address in your guest configuration or in your DHCP service.
- If you are using a NAT router open a port for the service you are implementing directing it to the guest's IP address.
Remember to test and re-enable the firewall service for your host computer. It may need any entry to forward traffic to the guest.
See https://help.ubuntu.com/community/KVM/Installation, https://help.ubuntu.com/community/KVM/Networking, and https://help.ubuntu.com/12.04/serverguide/libvirt.html.
If the behavior you are seeing is host can access the guest, and guest can access the host, but the guest can't access other machines on the network or visa versa... probably the host's firewall is blocking access.
See: https://bugs.launchpad.net/ubuntu/+source/ufw/+bug/573461
Specifically, this section: "The final step is to disable netfilter on the bridge:
# cat >> /etc/sysctl.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
EOF
These are the two scripts I use to create a bridge for qemu-kvm.
First, let the host become a IP router.
Script ip-router.sh:
#!/bin/bash
internetinterface="eth0"
username=`whoami`
if [ "x$username" != "xroot" ] ; then
echo
echo "You must be root in order to run this script..."
echo
exit
fi
if [ "x$1" != "x" ] ; then
internetinterface="$1"
fi
if [ "x$1" == "xdel" ] || [ "x$2" == "xdel" ] ; then
disable="1"
else
disable="0"
fi
if [ "$disable" == "0" ] ; then
echo "Enabling IP forward and setting up masquerade NAT on interface $internetinterface"
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o $internetinterface -j MASQUERADE
else
echo "Disable IP forward and setting down masquerade NAT on interface $internetinterface"
echo 0 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -D POSTROUTING -o $internetinterface -j MASQUERADE
fi
Then, create the tun-tap interface and bridge it with your default interface (usually the one with an Internet connection).
Script create-qemu-bridged-tuntap.sh:
#!/bin/bash
bridgename=br0
tapinterface=tap0
outinterface=eth1
if [ "x$1" != "x" ] ; then
outinterface="$1"
fi
ifaces=`awk -F: '{print $1}' /proc/net/dev | tail -n +3`
iffound="0"
for i in $ifaces
do
if [ "$outinterface" == "$i" ] ; then
iffound="1"
fi
done
if [ "$iffound" == "0" ] ; then
echo
echo "Can't find the output interface."
echo
exit 1
fi
outifaceip=`ifconfig | grep -A1 $outinterface | tail -1 | awk -F: '{print $2}' | awk '{print $1}'`
outifaceiptokens=`echo $outifaceip | awk -F \. '{print NF}'`
if [ "$outifaceiptokens" != "4" ] ; then
echo
echo "The selected output interface $outinterface doesn't seem to have a valid IP address."
echo
exit 1
fi
hostaddress="192.168.1.1"
guestaddress="192.168.1.95"
sudo tunctl -t $tapinterface
sudo brctl addbr $bridgename
sudo brctl addif $bridgename $tapinterface
sudo ip link set $bridgename up
sudo ip addr add $hostaddress/24 dev $bridgename
sudo route add -host $guestaddress dev $bridgename
sudo parprouted eth1 $bridgename
sudo ~/scripts/ip-router.sh $outinterface
I use these scripts daily, so they should work well also for you. You'll have to install some package in order to have all of this working. Using:
dlocate `which COMMAND`
you can see which package is required to have COMMAND. For example to see which package is required to have brctl, simply run:
dlocate `which brctl`
and you'll have:
bridge-utils: /sbin/brctl
Using the same approach for all the commands in these scripts, you should (at least) run this aptitude command line:
sudo aptitude install dlocate iproute parprouted iptables uml-utilities bridge-utils net-tools
Finally, you can launch the main script (as a normal user):
#> create-qemu-bridged-tuntap.sh eth0
Set 'tap0' persistent and owned by uid 0
Enabling IP forward and setting up masquerade NAT on interface eth0
Running ip addr you should see a br0 interface with IP address 192.168.1.1, as specified inside the create-qemu-bridged-tuntap.sh script:
#> ip addr
8: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
link/ether 36:76:ee:d6:63:b2 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 scope global br0
This is the host address as seen by the guest. Conversely, the guest will have as IP address 192.168.1.95 (again, this can be easily changed inside the main script).
Now, using virt-manager, you just have to setup your guest nic to use br0 as physical interface.
Inside the guest, you just have to give to eth0 an IP address of 192.168.1.95 and everything should run fine.
Slackware13:~> ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:00:F7:6A:78
inet addr:192.168.1.95 Bcast:192.168.1.255 Mask:255.255.255.0