Running Xen Under Qemu
12 Mar 2012
I’ve been trying to learn more about hypervisors lately since, well,
they’re awesome. Also, I’ve always wanted to learn
qemu, so I
decided to try using
qemu as a sort of hypervisor “prototyping”
area. In this post I’ll be documenting how I got
Xen running under
qemu with a Debian guest. It’s a dome within a dome within a dome
(within some more domes?). Suffice it to say there are a lot of
domes. At the highest level possible:
+------------------------------+ | Arch Linux | | +---------------------------+| | | qemu || | | +------------------------+|| | | | dom0 (Fedora16 + Xen) ||| | | | +------+ +------+ ||| | | | |domU | |domU | ||| | | | |Debian| ... | BSD | ||| | | | | | | | ||| | | | | | | | ||| | | | +------+ +------+ ||| | | +------------------------+|| | +---------------------------+| +------------------------------+
This is going to get really slow, quickly. And it’s going to be awesome.
Preparation and Setup
First, you need to know whether or not you’ll be running
KVM is a Linux kernel module that allows user space
programs to access hardware virtualization features of the CPU. You
can only use
KVM if you have an Intel CPU with VT-x extensions, or
an AMD CPU with SVM or AMD-V extensions. You can check
list or examine your
/proc/cpuinfo to check for virtualization
$ egrep '(vmx|svm)' /proc/cpuinfo
I’m running an Intel Core 2 Duo, T6600, which doesn’t have VT-x, so
I’ll be running
KVM support. This also means I can’t
emulate other architectures; I’ll only be able to emulate x86_64 (is
I’m running Arch Linux, so I install
qemu like so:
$ sudo pacman -S qemu
Create a Hard Disk for Virtual Machine
Since I want to have a persistent “hard disk”, I create a 20GB one now
$ qemu-img create -f qcow2 disk.img 20G
-f qcow2 tells
qemu-img to use the
qcow2 image format, which is
described in the
man page as follows:
qcow2 QEMU image format, the most versatile format. Use it to have smaller images (useful if your filesystem does not supports holes, for example on Windows), optional AES encryption, zlib based compression and support of multiple VM snapshots.
See the man page for more info about the various options. I’m especially interested in snapshots and the “backing_file” option, (also known as “overlay images”) which basically lets you create a baseline image and then create VMs whose disks are “deltas” of the baseline, resulting in lower disk usage and the possibilty of care-free experimentation.
Install the dom0
Xen requires that a dedicated guest, named “domain 0” (or dom0) be installed to take care of some of the hardware abstraction and guest OS management. This is usually a Linux or other Unix OS with special modifications for Xen support.
Initially I tried using the Xen Cloud Platform distribution but was having issues accessing the console of any guest machine I created. If I were using real hardware I would most likely use XCP for the dom0. It’s a stripped down Linux install with support for a bunch of cool tools like OpenXenManager, XenCenter, etc. as well as a bunch of ready-made templates for guest OSes.
Since XCP didn’t pan out, I ended up simply installing Fedora 16 to
use as the dom0. The overall installation took a very long time since
qemu is doing all the emulation in software on my machine. I
probably should have done a text-based install to try to keep things
as slim as possible, oh well.
After the install is complete, I shut down the machine, and make a “baselined” disk, which only stores the deltas from our original disk.
$ qemu-img create -b disk.img -f qcow2 overlay1.img
And bring the machine back up without the cdrom:
$ qemu-system-x86_64 -hda overlay1.img -m 1024
At this point I do the usual updates and then install the
package group to enable support for the hypervisor:
# yum update # yum install xen
I lay down an overlay at this point since it’s a good checkpoint that
I can come back to if I mess things up. To do this simply hit
ctl-alt-shift-2 to access the
qemu console, then type:
(qemu) savevm installed-xen
Install some Guests
If your CPU doesn’t support hardware virtualization, your guest OSes will need to be “paravirtualized” for Xen. This means that the kernel has been modified so that certain system calls are routed directly to the hypervisor. With hardware virtualization, on the other hand, those “unsafe” instructions instead cause a CPU exception and then are re-routed to the hypervisor. Almost paradoxically, hardware virtualization is slower than paravirtualization!. This is mainly due to the overhead involved with CPU exceptions. Here is a great whitepaper by the folks over at ok-labs if you want to understand more of the nitty gritty behind how hypervisors work.
Paravirtualization is a little more difficult to use since it involves modifying the guest operating system. However, more and more OSes are adding support built-in for paravirtualization. For example, recent versions of the Linux kernel have paravirtualization support (and it is enabled as a module by default in some distros).
I never got any guests to work :(