An Introduction to the Linux-based Assignments
Note: Please use the class discussion board (you can find
it on your blackboard) as a
communication and Q&A tool. Should you still need to send an
e-mail inquiry, address it to the TAs and cc the instructor.
|A few words about QEMU and Linux
Linux is a Free Operating System
kernel originally spear-headed by Linus Torvalds. It has been
adopted by many UNIX-like Operating Systems distributors. GNU/Linux is
a variant that is built around the Linux kernel and
software (compiler, libraries, shell, etc.) by the GNU Project. Debian is a Free GNU/Linux
distribution. We will be using Linux kernel 3.18.77 from the kernel repository.
We will be using a stable Debian release (Etch) as the
distribution with which we test our Linux kernel modifications.
QEMU (Q Emulator) is a processor
emulator that began as a project by Fabrice Bellard. It is also Free software,
can emulate many different CPUs, and it is stable and fast (QEMU relies
on dynamic binary translation to achieve reasonable performance, but
can also use a virtualizer to boost it even more). QEMU is a full
system emulator so it can give you access to the network and emulate
peripheral devices; we will be providing you with a file, which is the
virtual disk image where Debian is installed. We will be using
virtualize a whole Debian system with an i386 CPU.
The bundle of QEMU as the system emulator, Debian GNU/Linux as the
system, and the latest Linux kernel will be our testbed for Linux kernel
will be referring to it as QDGL from now on (QEMU - Debian GNU/Linux).
We will be referring to your desktop or whatever machine you run QDGL
on, as the host OS, and the virtualized Debian GNU/Linux as the guest OS.
The QDGL bundle is a bundle of Free
software and is also available as such
and without any guarantee.
You are not required to use QDGL; you can
build your own configuration or use any other tool you want, but you
will have to confirm your assignment works on QDGL. For the purpose of
this class, QDGL
has been pre-configured for your convinience, so it is suggested that,
though you can build your own configuration, you stick to the one
provided. Since this is the first time we will be using this platform,
we might experience bugs occasionally; we will need your cooperation
to fix them, so make sure you communicate the bugs and problems you
might experience. More importantly, make
sure you make backups of your
|QDGL: List of files you 'll need and
how to acquire them
In the QEMU/Linux assignments, you are required to use QDGL to
modify and test a Linux kernel. You will be using your personal cs/csug
accounts for development.
First you need to copy QDGL to your directory.
You will need ~562MB just to grab QDGL, and another ~400M as soon
as you untar and compile the kernel, so make sure you have at least 1GB of free
in your account (the instructor and system administrators have
made sure you have the appropriate allocation). We will be using a
stock kernel from the kernel repository, version 220.127.116.11
Below is the list of the contents of QDGL:
To acquire QDGL
- linux-3.18.77 :
The Linux kernel. You will have to untar it and compile it with any
modifications suggested per assignment.
- hda.img : Debian's virtual disk image. Your disk
is seen by the linux kernel as device /dev/hda - more about this
not touch this file.
- qemu : This directory holds the
binaries of your emulator, QEMU, plus other files QEMU needs to work
(virtual PC bios, video device, etc.). Do
not touch anything in this directory.
* Or anyone
with access only to
the undergrad network
- Undergrads* : Copy QDGL
to your home directory using the following instruction:
- Grads** : Copy QDGL to
your home directory using the following instruction:
-r ~cs456/QDGL ~
** Or anyone
with access only
to the grad network
|QDGL: Test driving your system
To take a look at the default Debian installation, move into your copy
of the QDGL directory (
and on your terminal run the following instruction:
./qemu/bin/qemu -m 64M -L ./qemu/share/qemu/ -hda
- ./qemu/bin/qemu : execute
- -m 64M : tell the
emulator that the emulated system will have a physical memory of 64 MB.
to increase this number if your host can accept a larger value
- -L ./qemu/share/qemu :
emulator where to find the virtual pc bios and other necessary files
for the system emulation
- -hda hda.img : tell the
emulator which file will act as the virtual hard disk image - this file
should appear as device /dev/hda in the guest GNU/Linux OS
- -nographic : tell the
emulator to use the ncurses
library to draw its interface on your terminal
Alternatively, if you choose to use the SDL interface, you can simply neglect the last
parameter (-nographic). If you have access to the X server of your Linux host,
you should also be able to see a virtual monitor in a separate window (this option uses
the SDL libraries, which you should already have installed) instead of the terminal.
This should boot Debian from the virtual disk, with the distribution
kernel. You will be prompted by the boot loader to choose the kernel to
boot (Press any key to continue)
- you can safely just wait 5sec, which will start the default kernel,
or you can pick it yourself in the grub menu. After the kernel
log messages, you will be prompted with the login ; the system is
called cs2456. Below are the two accounts which already exist in the
USER: root PASS:***
USER: cs2456 PASS:***
The passwords to those accounts will
be handed over in class. You should change your personal virtual
machine passwords as soon as possible. You should not enter as root
unless you absolutely need to.
Hitting the command
you can check the system information -- you should see Debian's
pre-installed kernel version
You should always make sure you turn off your client properly ; the
virtual disk is setup as a journaled ext3 filesystem, but properly
shutting down is essential to keep it "clean". Another good habit would
be to be taking backups of your virtual disk image at checkpoint
events. To turn off your system safely, if logged in as root just type
if logged in as cs2456, type
|QDGL : Compiling and testing a new
For the following we are assuming that you are working on a GNU/Linux
environment, such as cycle1.cs/csug.
Back to the QDGL directory, and as soon as you have test-driven your default
Debian box, you can untar your to-be-modified 18.104.22.168 kernel, by
linux-22.214.171.124.tar.bz2To compile your kernel run
newly compiled kernel image is at arch/x86/boot/bzImage
kernel directory (i.e., the directory where you untarred the
kernel sources: linux-126.96.36.199 by default and assumed from now on). To
test drive your new kernel run
./qemu/bin/qemu -m 64M -L ./qemu/share/qemu/ -hda
"root=/dev/hda1 console=ttyS0,115200n8 console=tty0" -kernel
Now if you login again and type
"root=/dev/hda1 console=ttyS0,115200n8 console=tty0":
passes the parameter in quotes to the
kernel. The kernel needs to know where the root directory will be to
mount it and boot properly. The kernel also needs to know it is displayed on a serial
console along with the console parameters if you choose to use a serial display.
linux-188.8.131.52/arch/i386/boot/bzImage: tells the
emulator where the external (to the image) kernel to boot lies (full or
uname -a you should see
that the new kernel version is
Congratulations, at this point you have managed to boot a kernel you
compiled yourself !
Every time you make modifications to your linux kernel, you should
rebuild it (
test it (
|QDG: Hidden options / tips / tricks
QDGL is free software and as such comes with no guarantees (please read
the Disclaimer). Though the simple procedure
suggested above should be enough
to get you through your assignments, we list below a few options that
might come in handy in case of bugs in the environment and in case you
want to try some higher-functionality option, which hopefully will make
your coding more productive.
- SDL :
To use the SDL interface, simply neglect the parameter -nographic and try
running QEMU as
./qemu/bin/qemu -m 64M -L
the default case
./qemu/bin/qemu -m 64M -L
"root=/dev/hda1" -kernel linux-184.108.40.206/arch/i386/boot/bzImage
if you want to boot your own kernel
- Networking : QEMU will
automatically try to bridge its virtual network device to the host's
network device. Should that not work, explicitely adding
-net nic,vlan=1 -net user,vlan=1 to
QEMU should get it right (we define how this bridging should work).
should safely bring your guest OS to the outside world and give you
access to the internet.
- Your Debian installation comes with an ssh server/client. If
your host has an ssh-server also running on port 22, then there
would be a port-clash. You could avoid this problem by passing the
-redir tcp:2222::22 to
QEMU. Then, connecting on port 2222 on the host OS (localhost) would get you to the ssh
server port of the guest OS (port 22, Debian in our case).
- Generalize the usage of the above instruction to accommodate
other port clashes that might appear, for example if you share your
host OS with someone else, pick a random 4-5 digit port as the bridge
port to your regular server port.
- Kernel config : The
procedure to build a kernel is actually first making the configuration
( for example
make menuconfig )
and then building it (
To save disk space, the provided kernel has been stripped of
unnecessary modules and is being compiled as a static binary (the Linux
kernel, though monolithic, is also for the most part modular, with
external loadable modules, but we are compiling it as a big binary).
You can change the configuration to fit your needs if you have to. One
interesting option that will increase the disk space you will need is
the kernel hacking options.
By turning them on you should get useful runtime debugging information.
Make sure you have enough disk space for this purpose (we estimated an extra
500M in our attempts).
- Debugging: QEMU can be
attached to GDB, the GNU debugger, to allow you to step through your
kernel. Simple printk (the
analogue of printf inside the kernel) should also work. However, it's
always nice to have extra functionality that can speed up your
compile-test cycle. If you want to try using the debugger, here is one
of the ways
you can do it:
- Compile the kernel enabling the "kernel hacking" options, more
specifically, enabling the debugging information. To do so, go to the
kernel directory, run
navigate to the "kernel hacking"
option and enable at least the "
Compile the kernel
with debug info"
option, plus all the options that you will find useful according to
your target. Recompile as discussed, and then start your kernel adding
the following two parameters, "
-s [port] -S"
(port will default to 1234 if not provided - it is optional), resulting
in something like the example below:
-m 64M -L
-append "root=/dev/hda1" -kernel linux-220.127.116.11/arch/i386/boot/bzImage
Those parameters tell QEMU to open a port (1234 by default) for gdb to
connect to, and then freeze CPU until it gets the signal to continue.
- Start gdb giving it as the executable to be debugged
which is the linux kernel binary (bzImage is the bz-compressed kernel
image). vmlinux is in the topmost linux kernel source directory,
as assumed here, so you can start as
- Connect gdb to qemu by the following instruction
target remote localhost:1234
(or port instead of 1234 if you specified it - and you should uniquely
if you share your host OS).
- Start debugging by setting your breakpoints etc.
- Code browsing : The Linux
kernel is huge, so you might want to always use a code
browser. You will find one at LXR.
Another option is using Eclipse
with the C/C++ plugins (CDT) for code browsing and editing.
- Debian : Find out how to
use apt utilities if you want
to add extra software to your disk image. For starters, open-ssh is
installed, so you can transfer files to and from your disk image.
- Host OS : QEMU runs on
Windows, GNU/Linux, and Mac OS X. We don't care if you use your desktop
and what kind of software you have, but you should be able to compile
your kernel there. There is no other limitiation: you can take QDGL
home and work with it. There are many reasons why you would prefer a
GNU/Linux host, however. For example, the virtual disk image is just a virtual
disk you can mount using a loopback device and add/delete files from;
all you need to know is which sector the first partition starts
at and find the exact offset to mount (offset=32256 in our case).
Of course, you can ask the TA
and the instructor for help. At the same time, we would like to
encourage you to
help each other, especially by using the discussion board at the blackboard. Before asking
any questions though, do yourself the favor and search for the answer;
most probably, one of the innumerable resources we have at hand
will have quite precise directions to get you moving.
- QEMU : You can visit
Read the manuals, read the options, explore them - you will make your
development cycle a lot faster.
- Debian : Again,
you can visit their website. Use
the man pages. Search the internet.
- Eclipse : Extra
Information about Linux kernel debugging using Eclipse and QEMU can be
(but please don't spend more time on setting up your environment rather
than coding for your assignments).
- QDGL : As you can
see from links like that, we are not the only ones who are using this setup to
teach/hack/develop OS. Dr. Tia Newhall 's QEMU
guide is meant for her OS class. Try searching the internet for
"QEMU and Linux kernel development" - you will see it is a very
successful combination .