Virtualization on the command line with KVM

Virtual machines are just practical. You can use virtual computers to test new distributions, start Windows or revive older operating systems. One of the most popular open source virtualization software tools goes by the somewhat unwieldy name "Kernel-based Virtual Machine" (KVM [1] for short). Although primarily driven by Red Hat, it's the preferred virtualization solution in almost all major distributions.

Division of Labor

KVM consists of several components (Figure 1). The base is the kernel module that ensures that the virtual machine can access the actual hardware efficiently without getting in the way. This kind of a manager is called a hypervisor [2] or virtual machine manager (VMM). As of version 2.6.20, the KVM module is a standard component of the Linux kernel, which makes it a part of every current distribution.

Figure 1: The KVM module provides the necessary infrastructure for virtual machines.

In the "old days," the module services loaded virtualization software and most distributions used QEMU [3]. The program is considerably older then KVM and would build a complete PC, inclusive of processors. Nowadays, QEMU can use the KVM module and, thus, the physical PC's vital hardware components. The result is a considerably faster execution of the operating systems and programs that QEMU launches.

Some current distributions separate QEMU and its helper programs into multiple packages. OpenSUSE users need only the kvm package; but for Ubuntu 13.04 you need the qemu (or qemu-system ) package from Software Center. If in doubt, search for the kvm package for your distribution in the package manager.

Host

The installation itself is easy as pie. All you need is to integrate the KVM package through package manager. However, you can start a virtual machine easily if you simply meet a couple requirements on the actual computer.

First, the processor must be an Intel or AMD model that supports virtualization with special command extensions. Intel calls this technology VT; AMD calls it AMD-V. Most desktop and notebook processors in the past five years support this technology. An exception is a particularly underperforming processor, such as some from the Atom series. Intel provides its own list [4] of compatible architectures.

To test whether your PC supports virtualization, use the lsmod | grep kvm command, which should output two lines, one beginning with kvm and another with kvm_intel or kvm_amd , depending on your setup. If these two lines don't show up, you can try starting the module manually with the following two commands for Intel processors:

$ sudo modprobe kvm
$ sudo modprobe kvm_intel

You may also first need to activate the functions in the computer's BIOS or the firmware. If the CPU lacks the hardware virtualization functionality, you can still start the virtual machine, but the guest system it runs will crawl along at a snail's pace.

Next, you need as much main memory as possible, because now it needs to accommodate not only the host operating system but also the virtual machine. The absolute minimum is 2GB. However, the more RAM you have, the more virtual machines of memory-hungry operating systems you can start. Pay particular attention to the requirements of the host operating system. If you want to run 64-bit programs later in the virtual machine, the host Linux needs to be 64-bit. You can run 32-bit programs on a 64-bit host, but not the other way round.

Finally, test that QEMU is correctly installed by using the qemu -version command. If you get an error message in place of a version number, try qemu-kvm -version instead. Some distributions, openSUSE among them, call the associated QEMU qemu-kvm , in which case substitute qemu with qemu-kvm .

Quick Start

To start a virtual machine quickly with a Live system, use the command qemu -cdrom img.iso , where img.iso is the name of the ISO image that the Live CD/DVD uses to start the guest system. If you've already burned a CD or DVD, simply replace the filename for the device with the appropriate drive, such as:

$ qemu -cdrom /dev/cdrom

The desktop for the started system will now appear in a separate window. Most Linux distributions recognize that you're running a virtual machine that can be run as a normal application, but this can be a bit iffy in some cases. Ubuntu 13.04, for example, has the particular problem of swallowing the mouse pointer or not building the screen correctly, in which case you can add the -vga std option when running QEMU:

$ qemu -cdrom ubuntu.iso -vga std

This command applies another, simpler virtual graphics card to the simulated computer system.

In some cases, you need to click again in the virtual machine window. QEMU then "catches" the mouse pointer. You exit the window by pressing Ctrl+Alt. To toggle back and forth from full-screen mode, press Ctrl+Alt+F, which, unfortunately, doesn't always work and can result in a black virtual machine screen. If you have problems with the graphics, start the Linux guest system in Safe mode.

By default, QEMU provides a virtual graphics card of the Cirrus CLGD 5446 PCI variety. Although most operating systems can handle this GPU, 3D applications won't work. Setting up the network card normally also requires some attention. By default, QEMU provides a network interface card (NIC) with the Realtek chipset, for which all reasonably current operating systems provide a corresponding driver.

The built-in virtual router automatically provides an IP address from its DHCP server. The virtual system not only uses it to access the network or Internet, it accesses the host system with address 10.0.2.2 (Figure 2). If an SSH server is running on the guest, you can use Secure Copy (scp ) to transfer files.

Figure 2: Thanks to the built-in router, which includes a firewall, Ubuntu can access the Internet without any additional settings.

You should absolutely resist the temptation of closing the virtual machine window with the close icon on the title bar. For the guest system, it's as if you pulled the plug on the PC. Instead, power down the guest system as usual from the physical PC, which automatically closes all the windows.

Virtual Disks

If you want not only to test the Live CDs but also install operating systems, you need to have enough hard disk space. With QEMU and KVM, you can choose to use either the physical machine's hard drive for the virtual machine, or simulate one with a corresponding large enough file. The qemu-img utility can create such a hard drive image with:

$ qemu-img create -f qcow2 my_img.img 30G

This command creates the image in QCOW2 format in the my_img.img file, where the virtual hard drive uses a maximum of 30GB. Note that qemu-img also lets you convert among various formats (see "QEMU Conversion Artist").

The size of the disk image in QCOW2 format corresponds exactly to the scope of the files stored therein, with the image size growing according to its content. Only when full does it grow to the complete 30GB. This can happen relatively quickly if you download a video in the virtual machine or install software packages willy-nilly. The physical hard drive should always have enough free space for the virtual hard disk file.

You submit a newly created hard disk image by adding a final parameter when starting QEMU:

$ qemu -cdrom ubuntu.iso -m 1024M my_img.img

The -m 1024M option provides the virtual machine with an additional 1GB of RAM, although QEMU and KVM tap this off the physical main memory. So, be sure that you give the hosting system enough breathing room. Without the -m parameter, the virtual machine gets only 128MB of main memory.

Windows XP (Figure 3) can work reasonably well with that, but Windows 7 and 8 (Figure 4) should get at least 2GB. Ubuntu will be happy with just 512MB. By the way, you can always substitute -m 1G for -m 1024M .

Figure 3: An older Windows XP runs just as well on QEMU/KVM…
Figure 4: …as a newer Windows 7.

The Windows Vista, 7, and 8 installations can take rather a long time. Even if the physical PC is quite fast, let it be for at least an hour. The physical process then returns to its usual self.

In place of a QCOW2 image, you can use the ready-made hard disk images from VirtualBox (ending in .vdi ), VirtualPC (ending in .vhd ), or VMware (ending in .vmdk ). If you want to use a real hard drive instead, replace my_img.img in the previous example with the corresponding device file:

$ qemu -cdrom ubuntu.iso -m 1024M /dev/sdb

Be aware, however, that if the hard drive already has an installed operating system, it might start in the virtual machine but will find completely different hardware (Table 1). Windows especially will not only not install the matching drivers but also will want to be reactivated once again.

Table 1

QEMU: Virtual Hardware

Standard Components
BIOS SeaBIOS
Graphics card Cirrus Logic GD5446
Network card Realtek RTL8139 PCI
Host PCI bridge i440FX
PCI to ISA bridge PIIX3
Keyboard/Mouse PS/2
IDE interface 2
Floppy drive 1
Serial/parallel interface 1/1
Options (Added Manually)
Sound card Various models, among them Soundblaster 16, AC97, HDA
USB support PCI UHCI USB controller, virtual USB hub

Sound Support

The virtual machine creates sound only from the physical machine's sound card. QEMU can provide a few virtual models. You can add Intel HD audio cards found in many current PCs as follows:

$ qemu -cdrom ubuntu.iso -m 1024M -soundhw hda my_img.img

Because Windows XP unfortunately provides no drivers, you can apply an AC97 audio card for a guest system with that operating system, as follows:

$ qemu -cdrom ubuntu.iso -m 1024M -soundhw ac97 my_img.img

A classic one – recommended particularly for older systems – is Sound Blaster 16, with the command as follows:

$ qemu -cdrom ubuntu.iso -m 1024M -soundhw sb16 my_img.img

The qemu -soundhw ? command lists all other supported sound card models. The abbreviation at the beginning of each output line is the one to use as the value after -soundhw .

Media Changes

If you ever need to change the installation media, press Ctrl+Alt+2. You then end up in the monitor console at a special command line you can use to replace, convert, and manipulate the virtual machine hardware while still running. To replace a CD or DVD, first inquire about the current state of the devices using the following command:

(qemu) <info block>

This spits out a few cryptic lines, such as in Figure 5. The QEMU device name is one of the first entries.

Figure 5: With the "info block" command, the monitor console lists all block devices in the virtual machine. Among them is usually even a floppy drive.

In Figure 5, it's easy enough to figure out the CD drive as ide1-cd0 as the "integrated" ISO file along with the removable=1 designation. You can eject the CD or DVD with the following command:

(qemu) <eject -f ide1-cd0>

Here the -f option forces the ejection, in case, for example, the operating system is still holding onto the media.

Once the virtual disk is empty, you can substitute the replacement media, which, in the following example, is located in /home/tim in a file named cdrom2.iso :

(qemu) <change ide1-cd0 /home/tim/cdrom2.iso>

If a virtual machine hangs, you can start it up again in the monitor with system_reset . Also useful is the sendkey monitor command with which you can trigger a key combination on the virtual machine, such as:

(qemu) <sendkey alt-f1>

This command would open the start menu. Exit the monitor with Ctrl+Alt+1.

Boot Orders

The virtual machine always tries to boot from the first hard disk. If you want to boot from the CD drive, you need to specify the -boot once= option at startup, as in the following example:

$ qemu -cdrom ubuntu.iso -m 1024M -boot once=d my_img.img

The d refers to the CD/DVD drive, whereas c would select the first hard drive instead. Combining the characters with the -boot order option gives you a permanent boot sequence in the order indicated:

$ qemu -cdrom ubuntu.iso -m 1024M -boot order=dcn my_img.img

The virtual machine thus would always boot first from the optical media, then the first hard disk, and then look via Etherboot and PXE in the network for a bootable system. PXE, which stands for Intel Preboot Execution Environment, is the protocol for booting an operating system off the network. Etherboot is a GPL-licensed PXE implementation.

External Drives

If you want to use a USB device connected to the physical machine, you need to reach the host system in several steps. First, use the lsusb command on the host system, which provides a list of all attached USB devices (Figure 6). Find the device and jot down the Bus and Device at the beginning of the line.

Figure 6: To get at a USB device, keep track of the Bus and Device numbers.

In Figure 6, for example, the USB stick is on the line with Bus 001 and Device 003 for the Flash Drive. You then specify these values when you start the virtual machine, as follows:

$ qemu -cdrom ubuntu.iso -m 1024M -boot once=dcn my_img.img -usb -device usb-host,hostbus=1,hostaddr=3

The -usb option activates the USB support and you specify the device name after it. The usb-host is the USB device on the host and the bus and device numbers are after hostbus and hostaddr , respectively. You can also "plug" the USB device into a running virtual machine. Change to the monitor and use the usb_add command (Figure 7).

Figure 7: Add the device in the monitor…

The command has the general syntax usb_add host:<Bus>.<Device> , so the USB stick example would be as follows:

(qemu) <usb_add host:1.3>

For QEMU to reach the physical USB device (Figure 8), it has to access the device file. If it can't, a flood of error messages start appearing, in the manner of /dev/bus/usb/001/003: Permission denied . In such a case, it's easiest to start QEMU as root instead of a normal user.

Figure 8: …and access it from the virtual machine.

Conclusion

QEMU allows for countless other settings, many of which relate to the network. For example, you can set up port forwarding, via TUN/TAP devices that integrate virtual machines in the network or access Windows guests over an SBM server on directories of the host system. Multiple devices can connect to a private network, and you can remote manage individual ones per VNC and freeze a snapshot of its entire state.

The options necessary for the network setup are not exactly intuitive at times. If you want to experiment with it, take a look at the QEMU documentation [5], which is more or less a reference and therefore dry as a bone. Beginners might want to take a look at an older, albeit free, QEMU book by Robert Warnke and Thomas Ritzau [6]. More quick guides and how-tos are in the KVM wiki [7]. Unfortunately, many of the instructions, tips, and tricks on the Internet are outdated or even obsolete.

Once you internalize the many options, you can operate the KVM command line on a virtual machine faster than the required three mouse clicks in VirtualBox, etc.

Qemu-img Conversion Artist

The qemu-img tool creates not only a hard drive image but also can convert among various formats. The following command example creates an image in QCOW2 format from the VirtualBox ubuntu.vdi image:

$ qemu-img convert -O qcow2 ubuntu.vdi ubuntuqcow2.img

The qemu-img --help command shows which formats qemu-img can digest. Among them are the popular file formats such as VirtualBox (vdi ), Parallels, VMware (vmdk ), VirtualPC (vhd and vpc ), and Bochs or Images for Mac OS X (dmg ). The raw format is an uncompressed image of a hard drive that's created, for example, with dd . The qed format stands for QEMU Enhanced Disk Format, the designated successor to QCOW2 [8].

Depending on the size of the image and the performance of the computer, the conversion process can take some time. If the converted image is in QCOW2 format you can also encrypt it, as follows:

$ qemu-img convert -O qcow2 -o encryption=on ubuntu.vdi ubuntuqcow.img

Qemu-img now requests a password. The encryption itself is with AES and a 128-bit key. You only need to provide QEMU with the (encrypted) image as usual. The virtual machine then starts up, albeit with a black screen. From there, press Ctrl+Alt+2 to switch to the monitor, continue the processing with the c command, and type in the password for encrypting the hard disk.

As an alternative to encryption, QEMU can use the Zlib compression library for a QCOW2 image, as follows:

$ qemu-img convert -c -O qcow2 ubuntu.vdi ubuntuqcow.img

You pass even the compressed image as a normal image over to QEMU. Compression and encryption might save some hard disk space by creating smaller images that a stranger can't peek at, but both processes cost some computer power.

The QCOW2 format provides another handy feature: You can freeze the state of an image file. QEMU stores all subsequent changes in a second, smaller image, which can be useful if you're experimenting with new software a lot. The base system in the first image remains intact, while the second image captures all the changes. To get a fresh system, you only need to remove the second image. To use this function you first create a hard disk image as usual, as follows:

$ qemu-img create -f qcow2 base.img 30G

You start this in QEMU and install the desired operating system on it. Afterwards, stop QEMU and create a second, so-called overlay, image that captures your changes:

$ qemu-img create -f qcow2 -o backing_file=base.img overlay.img

Now when you start QEMU, you're using only overlay.img . The base.img must continue to be in the same directory, with the data contained within acting as a starting point. When you later want to discard the changes, simply delete overlay.img and start using base.img again. You can layer as many overlay images as you want on base.img If you want to transfer the state of overlay.img onto base.img , use the following simple command:

$ qemu-img commit overlay.img

Alternatively, use the commit command in the QEMU monitor. It may also be worthwhile converting a QCOW2 image into a new QCOW2 image: QEMU detects unused sectors and doesn't translate them into the target image – the resulting image file will become smaller than the original one. You can extend the method even further and embed the image files via -snapshot , as follows:

$ qemu -m 1024M -boot -snapshot base.img

QEMU now reads from the hard disk only. It saves all changes to a temporary file in the /tmp directory, which QEMU removes as soon as the virtual machine ends operation. The guest system, as with a Live CD, thus returns to its original state.

Should you doubt the integrity of the image file, the following command can help:

$ qemu-img check my_image.img

The command checks the specified image for errors. This works for QCOW2, QED and VDI formats only.