Category Archives: What?!

GNS3 and Gentoo – fixing QEMU networking

I was hoping to have time to further familiarize myself with IPv6 this weekend –  but workplace emergencies, sleep, and a technical glitch slowed me down. Here’s the story of the technical glitch.

GNS3 - this thing rocks.

GNS3/dynamips can handle emulation of a number of routers, and I was happy to see that it can emulate host PCs as well. Unfortunately networking in GNS3  is done via UDP tunnels – something that requires a Qemu patch to implement.

For some reason QEMU won't really start if it's networked.

In Gentoo, simply building GNS3 from the Sunrise Overlay doesn’t provide sufficient support for this requirement. Emulated routers (and I’m assuming switches) work fine when networked together. Emulated host machines work fine when NOT connected to anything. But once you network things together you’ll discover that your Qemu host won’t start.

Here’s why – when you start a Qemu host, this is essentially what GNS3 runs:

/usr/bin/qemu -name QEMU1 -m 256 /mnt/virtualmachines/dynips/IPv6-tc/working/QEMU1/FLASH -hdb /mnt/virtualmachines/dynamips/IPv6-tc/working/QEMU1/SWAP -enable-kvm -net nic,vlan=0,macaddr=00:aa:00:8f:e4:00,model=rtl8139 -net udp,vlan=0,sport=20000,dport=10006,daddr=127.0.0.1 -net nic,vlan=1,macaddr=00:00:ab:fa:72:01,model=rtl8139 -serial telnet:127.0.0.1:3000,server,nowait -no-acpi -vnc :0

GNS3 attempts to use the -net udp option, something not offered in the default QEMU-KVM distribution. With no Gentoo USE flags to add the patch in either 0.13.0 or 0.14.50, I decided to simply upgrade to 0.14.50 and see if it was part of the default build:

# kvm –version
QEMU emulator version 0.14.50 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard
# kvm –help | grep “net udp”
#

Nope, it’s not. So the next step is to remove app-emulation/qemu-kvm from your system and build it by hand.

First, remove kvm and download the source from kernel.org:

# emerge -C app-emulation/qemu-kvm

# cd /usr/src/

# wget http://mirrors.kernel.org/gentoo/distfiles/qemu-kvm-0.13.0.tar.gz

Unpack and patch it, using the patch provided by the gns-3 team .

# tar zxvpf qemu-kvm-0.13.0.tar.gz

# cd qemu-kvm-0.13.0/

Testing the patch, I ran into a handful of issues:

# patch –dry-run -p1 < /mnt/nas/downloads/qemu-0.13.0-
mcast-udp.patch
patching file Makefile.objs
Hunk #1 FAILED at 25.
1 out of 1 hunk FAILED — saving rejects to file Makefile.objs.rej
patching file block/raw-win32.c
Hunk #1 FAILED at 93.
Hunk #2 FAILED at 347.
2 out of 2 hunks FAILED — saving rejects to file block/raw-win32.c.rej
patching file hw/e1000.c
Hunk #1 FAILED at 567.
1 out of 1 hunk FAILED — saving rejects to file hw/e1000.c.rej
patching file net/udp.c
patching file net/udp.h
patching file net.c
Hunk #1 FAILED at 30.
Hunk #2 FAILED at 1075.
2 out of 2 hunks FAILED — saving rejects to file net.c.rej
patching file net.h
Hunk #1 FAILED at 33.
1 out of 1 hunk FAILED — saving rejects to file net.h.rej
patching file qemu-options.hx
Hunk #1 FAILED at 996.
1 out of 1 hunk FAILED — saving rejects to file qemu-options.hx.rej

A quick modification and we’ve got a new patch file. This one applies nicely:

# patch –dry-run -p1 < /mnt/nas/downloads/gentoo-qemu-0.13.0.gns3.patch
patching file Makefile.objs
patching file QMP/qmp-commands.txt
patching file block/raw-win32.c
patching file config-all-devices.mak
patching file config-host.h
patching file config-host.h-timestamp
patching file config-host.ld
patching file config-host.mak
patching file hw/e1000.c
patching file libdis/config.mak
patching file libdis-user/config.mak
patching file libhw32/config.mak
patching file libhw64/config.mak
patching file net/udp.c
patching file net/udp.h
patching file net.c
patching file net.h
patching file qemu-doc.html
patching file qemu-img-cmds.texi
patching file qemu-img.1
patching file qemu-monitor.texi
patching file qemu-nbd.8
patching file qemu-options.hx
patching file qemu-options.texi
patching file qemu-tech.html
patching file qemu.1
patching file roms/seabios/config.mak
patching file roms/vgabios/config.mak
patching file x86_64-softmmu/config-devices.mak
patching file x86_64-softmmu/config-devices.mak.old
patching file x86_64-softmmu/config-target.mak
vonnegut qemu-kvm-0.13.0 #

So let’s apply it for real and then configure, build, and install our new Qemu:

# patch -p1 < /mnt/nas/downloads/gentoo-qemu-0.13.0.gns3.patch

# ./configure –prefix=/usr –target-list=i386-softmmu –enable-sdl

# make && make install

A quick check to ensure UDP tunneling is compiled in:

# /usr/bin/qemu –version
QEMU emulator version 0.13.0 (qemu-kvm-0.13.0), Copyright (c) 2003-2008 Fabrice Bellard
# /usr/bin/qemu –help | grep “net udp”
-net udp[,vlan=n]sport=sport,dport=dport,daddr=host
#

And voila – now not only does my host machine start when networked into my layout, it also has connectivity:

Yes, it finally works.

Next up – continue lab work with IPv6, and begin studying for my CCNP.

Packet Creation for IPv6

With most of my projects, I’ve noticed that before I can really begin to delve into them, I first have to learn something completely new.

I don’t do most of my work in Perl, but I can definitely make ends meet. My more recent work in Perl has dealt w/ RIP scanning and route-updating (a lot of DSL CPE tends to have RIP enabled). Packet creation isn’t that difficult, and using Perl I could bang out a script in a short period of time.

Usually for packet manipulation, I stay away from Perl – instead I’m a huge fan of Hping[23]. If you’re wanting to watch and play w/ the transport layer – an old laptop with hping and wireshark running is definitely the way to go. If you want to test MTU problems, or ACLs  –  Hping is fast and easy.

Hping is an amazing tool – one of my favorites. Hand crafted TCP and UDP traceroutes are easy as hell, and hand crafting TCP timestamp requests is easy too. Just read the manpages and you’ll find all sorts of interesting features of TCP/IP.

hping: A Great Tool
hping: Like a screwdriver set with most of the bits.

The downside to Hping is that it doesn’t seem to be under regular development (last release 2005?), and IPv6 isn’t supported yet (Aside from some basic third-party patches). Application layer creation is also left for other utilities to handle (But in all fairness – that’s not it’s job)

I was a late adopter, but about a year ago I made a new friend (and picked up some python along the way). My new best friend is Scapy. “Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more.” It runs in interactive and script-able mode, and thus far it’s proven to be quite powerful. It also supports IPv6.

Scapy understands a large number of Application level protocols (SNMP, TFTP, SMB, etc, etc), and makes it easy for one to craft such packets.  When I recently realized early versions of Cisco IP Router Export have corrupt headers – Scapy came to the rescue (albeit, only to rip the headers off, as fixing them wasn’t possible).  When I wanted to craft a quick and easy FreeRadius Packet of Death – Scapy came to the rescue.

Exploits Database
FreeRadius PoD on www.exploit-db.com

Another great thing about Scapy is that it’s easy to add “layers” (protocols). Recently I sat down with the RFC’s for OSPF and in about 2 days time had fully implemented OSPF in Scapy – learning a lot about the protocol along the way. (Dirk Loss had beat me and submitted it first, but that was besides the point).

Looking for a fun new world to poke, and tired of IPv4? Sign up for a 6-in-4 tunnel and go exploring. Just make sure you pack Scapy and a handy guide.

Customizing the WebConverger Kiosk – The fast way

Recently I was asked to assist a co-worker find a good locked down Kiosk solution for a local coffee shop. The project requirement was small – mostly just a simple browser.

There seems to have been a LOT of Linux based Kiosk projects out there, most of them now deprecated. After a bit of searching, I came across a good one – WebConverger.

Webconverger

Webconverger is a Live Debian build, created and maintained by Kai Hendry using the Debian Live-builder packages. The LiveCD includes the lightweight  Dynamic Window Manager (DWM) , the IceWeasel browser with Webconverger kiosk extension (to lock things down), and… well that’s about it.

Kai has excellent documentation on rolling your own Webconverger LiveCD using his GIT repository, although being pressed for time I opted to go the BraindeadProjects route: Just modify his already released ISO.

To begin modifying it in this manner, first mount the iso using a loopback device:

#mkdir /mnt/webconverger
#mount -o loop webc-7.2.iso /mnt/webconverger

Next mount the SquashFS image, also using a loopback device:

#mkdir /mnt/webconverger-filesystem
#mount -o loop -t squashfs /mnt/webconverger/live/filesystem.squashfs \
/mnt/webconverger-filesystem

As these two mounts are readonly, we need to create editable copies of each.

#mkdir /devel/isolinux
#rsync -av /mnt/webconverger/ /devel/isolinux/

#mkdir /devel/squashfs
#rsync -av /mnt/webconverger-filesystem/ /devel/squashfs/

Now we can go about modifying these two directories. Changing out the ISOLINUX boot splash image is an easy start. The image found under /devel/isolinux/isolinux/splash.png is actually an LSS16 image. To replace it, take or create an image 640×480 in size, and convert it to 14 indexed colors. (In GIMP, these options are under Image/Index)

Indexing Colors in Gimp

Once complete, save your image in PPM format.

Next, you’ll need the syslinux package installed on your machine. The syslinux package includes a handy utility to convert PPM to LSS16 (for use as a bootsplash image)

#ppmtolss16 < /tmp/myimage-boot.ppm > splash.png

Since you’re already working on the ISOLINUX side of things, I recommend looking at and revising your boot menu. Once I have an image that I’m happy with, I set the following options to prevent someone from rebooting the Kiosk and tampering with boot parameters:

menu background /isolinux/splash.png
default /isolinux/vesamenu.c32
noescape 1
nocomplete 1
prompt 0
timeout 15
allowoptions 0

While you can nest a number of ISOLINUX boot configs together, I generally keep it to one file that includes the above directives. Dont’ forget to include at least one label for a kernel to boot.

After updating the Boot Splash screen, have a look at /devel/squashfs/home/webc/pb.sh. This script is what causes IceWeasel to start, restart if closed, and sets the desktop background image (amongst other things). This script also downloads a background image from your homepage at boot – which can come in handy if you want to rotate daily ads.  I’ve personally modified my installation to always load the same background image, and fullscreen that image.

The webpage that appears each time IceWeasel starts is passed as kernel boot parameter (homepage). To update the homepage, simply edit the labels in the ISOLINUX  directory.

You will notice that when pressing the home button in the browser however, that you’re actually taken to an about: page that gives details about the current IceWeasel build. To configure this homepage, look at /usr/lib/iceweasel/browserconfig.properties

browser.startup.homepage=www.braindeadprojects.com

I personally like to lock things down a bit more than the standard release. For that reason I also add the following to /etc/iceweasel/pref/local.js

pref(“network.protocol-handler.external.snews”, false);
pref(“network.protocol-handler.external.news”, false);
pref(“network.protocol-handler.external.irc”, false);
pref(“network.protocol-handler.external.mail”, false);
pref(“network.protocol-handler.external.mailto”, false);

Another thing that may prove beneficial is to remove any and all remnants of xterm. As xorg does depend upon xterm, it will have to be forceably removed. This is best done in a chroot environment

#chroot /devel/squashfs/ /bin/bash
#dpkg –force-all -p xterm
#exit

Once you have your modifications complete, you will want to re-squash the squash filesystem. To do this, you’ll need squashfs-tools version 4 (Centos is currently distributing version 3, so do keep that in mind). Squashing using version 3 of the tools will result in a non-bootable kiosk.

#mksquashfs /devel/squashfs/ /tmp/webc.squashfs
#mv /tmp/webc.squashfs /devel/isolinux/live/filesystem.squashfs
#cd /devel/isolinux/
#mkisofs -o /tmp/my-webc.iso -b -r -J -l -cache-inodes -allow-multidot -no-emul-boot \
-boot-load-size 4 -boot-info-table -b  isolinux/isolinux.bin -c isolinux/boot.cat \
/devel/isolinux

Finally, isohybrid your ISO:

isohybrid /tmp/my-webc.iso

I highly recommend testing your ISO image in VirtualBox. Using VirtualBox (or any other virtualization option), saves you from constantly burning an image to  a CD or USB drive. Be mindful that you can skip the isohybrid step and test with VirtualBox, although you won’t be able to install it later using dd.

Once you have an image that you’re happy with, use dd to copy the ISO onto the hard drive of your Kiosk machine. Personally, I copy my ISO to a USB thumbdrive running the Gentoo Based  System Rescue CD, boot into it and then install onto the harddrive:

dd if =/livemnt/boot/kiosk/my-kiosk.iso of=/dev/sda

 

Of course, one could save time and simply use the WebConverger Customization Service… but why not use this as an opportunity to sharpen one’s skills.

Coming soon to – a walkthrough on how to build and customize a WebConverger ISO from Kai’s GIT repository (as opposed to re-rolling his ISO).

Back in the dark ages

Recently (don’t ask me why, seriously) I had to migrate a physical Redhat 9 server to a virtualized platform (KVM).

Yes, ideally one rebuilds the deprecated server anew, but due to time-constraints and a number of other issues that wasn’t a possibility. Unfortunately, sometimes you just have no choice but to kick the can down the street.

The migration from physical to virtual is simple – create the virtual guest, create an LVM logical volume for the disk,  format it and rsync the contents from the physical to the virtual drive. After that, install grub on the guest and voila – you’re done.

Keeping some consistency (although unnecessary), I went with an EXT3 filesystem on the “new” guest. Unfortunately, I came across the following snags:

  • The inode size used in Redhat 9 is 128 bytes, but modern systems (ie: the host I formatted the partitions from ) use 256 bytes.
  • There’s a number of attributes that weren’t present in Redhat 9.

Here’s the output of tune2fs on a Redhat 9 EXT3 partition:

# tune2fs -l /dev/hda1
tune2fs 1.32 (09-Nov-2002)
Filesystem volume name:   /boot
Last mounted on:          <not available>
Filesystem UUID:          9e6fa853-18bc-4c08-bb7c-51c74e0c11ae
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery  sparse_super
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              8032
Block count:              32096
Reserved block count:     1604
Free blocks:              17451
Free inodes:              7982
First block:              1
Block size:               1024
Fragment size:            1024
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         2008
Inode blocks per group:   251
Filesystem created:       Tue Oct 26 12:48:49 2010
Last mount time:          Thu Oct 28 15:58:10 2010
Last write time:          Thu Oct 28 15:58:10 2010
Mount count:              11
Maximum mount count:      28
Last checked:             Tue Oct 26 12:48:49 2010
Check interval:           15552000 (6 months)
Next check after:         Sun Apr 24 12:48:49 2011
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal UUID:             <none>
Journal inode:            8
Journal device:           0x0000
First orphan inode:       0

The following is output from a Centos 5.5 formatted EXT3 filesystem:

# tune2fs -l /dev/mapper/VolGroup00-LogVol00
tune2fs 1.39 (29-May-2006)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          7384cac8-b098-4c85-be6d-643443ae3d3d
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              6809088
Block count:              6807552
Reserved block count:     340377
Free blocks:              6204875
Free inodes:              6752521
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1022
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         32736
Inode blocks per group:   1023
Filesystem created:       Thu Oct 28 13:31:34 2010
Last mount time:          Thu Oct 28 17:43:57 2010
Last write time:          Thu Oct 28 17:43:57 2010
Mount count:              2
Maximum mount count:      -1
Last checked:             Thu Oct 28 13:31:34 2010
Check interval:           0 (<none>)
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      42eb54b3-8f66-4aef-8578-388c9863423c
Journal backup:           inode blocks

As you can see, some basic features and the inode size has changed in 10+ years. The solution:  format the “new” guest  to match the specs of the old:

#mke2fs -O has_journal,filetype,sparse_super,^ext_attr,^resize_inode,^dir_index \
-I 128 -j /dev/sda$disk

And that’s it- rysnc the files over to the guest VM, install grub, verify your fstab is good, and voila – you’ve transferred a physical server to a virtual one.