Recently my workplace ordered a batch of Ubiquiti products from DoubleRadius. Ubiquiti is loved by the WISP world, and (mostly) utilize Atheros boards – just as the Engenius products do. Both also have stock firmware that’s also based off OpenWRT. Being inspired, I purchased some equipment myself.

Part of the reason the Ubiquiti gear seems to be more popular is their excellent support and web forum. Engenius is notoriously difficult to get any answers out of. I’ve still yet to get a copy of their firmware source required by the GPL, and I gave up long ago trying to get anywhere. Luckily, the few questions I did have were answered by simply expirementing.

This weekend I began the migration to Ubiquiti. My first rollout involves a Rocket M2 and a 120 degree sector antenna. For the past few weeks I’ve been working on adding a more recent version of the excellent CoovaChilli to the device. I have located a version of the CoovaAP firmware that runs natively on a Rocket M2, however it lacked a number of features (non-Chilli related) that I was hoping for. After much searching, I’ve been able to locate a 5.3 SDK (which features AirControl and the built-in spectrum analyzer).

The AirView built in spectrum analyzer

As I mentioned, it took quite a bit of searching to obtain the SDK. According to Kevin Perry at Ubiquiti  “We’ve had to temporarily remove the SDKs from our site to comply with US export control regulations. Once approved, we will be able to continue providing SDKs. “  I’ve not heard anything new, and that correspondence is from September of 2011.

Of course, the Internet is a not only a compendium of 140-word-or-less Tweets about what someone just ate for dinner, it’s also a giant archive of the past. And with a bit of searching, I came across the SDK still hosted (albeit hidden) on Ubiquiti’s site.

Unfortunately the latest SDK is 5.5, and not 5.3. I’ve not had luck locating it (and it does contain the patch for the SkyNet worm), so I’m working off 5.3 for the time being. I’ve locked down access to the web interface to prevent infection from the worm.

The CoovaChilli Captive Portal

With the SDK in tow, I began the work of inserting CoovaChilli into the system. Ubiquiti ensures important services stay running by having them spawned (and respawned if needed) via inittab. The stock inittab on the system is basic, handling only login and a couple of runlevel items. Prior to /sbin/init starting, the system runs a bash script /init. This in turn handles filesystem mounts, makes a handful of device nodes, copies a number of files into place, partially configures sysinit, and sets the timezone. It also calls ubntconf which gleans information from the nvram variables, prior to executing the traditional /sbin/init.

ubntconf is a closed source application, although it’s pretty easy to glean most of it’s operations by simply running strings on it (It is an ELF not a script). Depending upon nvram variables that are set, different scripts are dropped into /etc/sysinit. Additionally, items are added to /etc/inittab like this:

null::respawn:/bin/pppd eth0
null::respawn:/bin/lighttpd -D -f /etc/lighttpd.conf
null::respawn:/bin/dropbear -F -d /etc/persistent/dropbear_dss_host_key -r /etc/
persistent/dropbear_rsa_host_key -p 22

Each of these entries will respawn the corresponding service should it crash. Using their system as a model, I created my own config creator “chilliconf” to add entries to inittab for the chilli daemon, in addition to the necessary iptables rules in /etc/sysinit, and a few modifications to their use of dnsmasq (I wanted to syslog DNS queries offsite). When I later added NProbe to my firmware, I also used my chilliconf configurator to add it to the inittab.

My chilliconf configurator, like ubntconf, is called from the /init script.

One thing of note – any daemon listed in the inittab file needs to actually run in the foreground (so calling the process a daemon isn’t entirely accurate). This is for the obvious reason that the initiating process fork()s then exits – so inittab will continually respawn the service until you run out of resources.

All original functionality is still there. It's a like a cyborg version of itself.

I used the 3.6SDK version of CoovaAP as a model for my web interface. Adding a tab was fairly easy – the web interface is maintained as the ubnt-web package in the SDK. Most of the web CGI is PHP/FI (with a couple of functions added to PHP to facilitate saving configs and reading values). The code is extremely readable, and easy to figure out.

When you save your settings in the Ubiquiti web interface, you’re presented with a blue message bar asking if you’d like to test or apply your settings. In either case, you’ll be disconnected for a moment while the system reconfigures itself. This appears to be done via a call to the original /init script. The blue bar is a bit of a problem when adding a new tab to the menu.

The background image for the blue bar (msg.png) is originally  745 pixels, and the screen width is set to 790 pixels. With an added tab, the screen width becomes 866 pixels. Using GIMP I resized this image to 821 pixels.

Resize the blue background to fit.

• Originally it was 745, with the original screen width of 790.
  • 790-745 = 45.
  • 866-45 = 821 — our newly resized image.

The web interface went through a few iterations before I had all the options I presently needed. CoovaChilli’s dnsdomainlocal option is problematic – causing any DNS query that returns an NXDOMAIN error for an A record to cause a redirect to the portal. Unfortunately the portal (from my experience) doesn’t properly track in this instance if the MAC associated with the request is authenticated – and redirects the user back to the UAMSERVER with the “res=notyet” variable. This means the user is prompted with a login screen unnecessarily.

In other words, visit “adfasjzc9vdsa0f9jdsaf.com”, and you’re presented with a login screen even after you’re logged in.  Adding the option to disable dnsdomainlocal fixes this problem.

The CoovaChili tab (with NProbe)

By default, I have the “dhcpradius” option enabled. With this feature, when a DHCP request is received by the CoovaChilli server, it forwards the MAC address to the radius server (as both the username and password) for authentication. If it doesn’t authenticate, the system redirects the user back to the UAMServer for a captive portal login.

Broadcasting throught the stratosphere

For the time being, the system is a bit overkill, but the options it presents for the future are many. CoovaChilli can rate-limit individual users based on login or MAC address. This alone opens the door for a pay-for-bandwidth model. If a user is found to be filesharing (legal or not) or otherwise consuming an inordinate amount of bandwidth, they can be easily ratelimited.

I still have a few minor items to add to the firmware (including the ability to pull RSSI information per client), but in the meantime it’s been fun working with the Rocket M2. The access point is now being tested in the neighborhood.

With Midtown Wifi I had the following goals:

1. Work more in the C language.
2. Build a stable and (mostly) embedded captive portal system with a minimal ToS acceptance screen.
3. Let the surrounding neighborhood use the Internet for free in exchange for helping me build and test the system.
4. Use the system as a way to introduce neighbors, let them post local interest items (missing pets, crime reports, events, etc).
5. Provide maps of recently reported crimes via the Harrisburg, PA online Police Blotter.

Over the years I’ve accomplished all of this, to one degree or another. Harrisburg, PA is in the midst of some serious financial problems, so their online police blotter has gone away  – preventing me from easily obtaining local crime information. People are what they are, and as Google+, MySpace, and any other Social Site knows — getting people to truly use your social portal is a trick that requires sheer genius. Getting them to log into it and push a “Free Wifi” button however, is easy.

The Midtown Wifi Login Screen (as of Feb 15, 2012)

How it works:

After connecting to one of the open access points, the end user is redirected (courtesy of a patched NoDogSplash) to a captive web portal. The web portal is based off Elgg, a fairly easy to use Social Network Engine written in PHP. I’ve made a few modifications to the base system, adding a more recent JQuery and JQueryUI (so that I can create interactive Dialogs), and writing a few plugins to handle Netflow display, wireless signal strength reports (per user), user speed tests, and to verify that they have a picture set before allowing them to use the Free Wifi.

By nature, people won’t set a profile picture when all they want is Free Wifi. I had to enforce a profile picture (“it doesn’t have to be you, it can be anything non-offensive”) to make the site NOT appear like a barren wasteland.

I eventually limited account creation strictly to the access points as registrations from outside those IPs were mostly just spam.

After a user creates an account and logs in, they are directed to the “Dashboard”, which is a listing of recent posts from any of the users. Most are quick “Hey you!”, but sometimes people post something more substantive. When my rear car window was broken, I used the system as a venting forum.

All the names and 1 dirty word have been removed.

I’ve consolidated most of my customizations relative to the wireless users into a single Elgg plugin I named “TSA Patdown”. Initially TSA Patdown only verified that a user had a profile image set, but now it does quite a bit more. Every 30 seconds I export Received Signal Strength Indication (RSSI) for each client from the  Engenius equipment. I collect this information, as well as information from a Javascript based speedtest widget I wrote to get an idea as to what kind of online experience each user is having.

I represent this information to myself on the following menu, with signal bars that I created using Blender:

Signal Strengths, Login times, MAC addresses and more.

I can further delve into information on a per-user basis by simply clicking on a name. I can also pull a full neighborhood report, graphing each clients RSSI values as well as their recent speedtest results.

The Javascript Speedtest Widget

Being implemented in Javascript, the speedtest results aren’t the same as you’d see when visiting a Flash based speedtest. The standard web method of performing such a test is to have the end-user download an image file or two (oftentimes two images simultaneously)- and at random intervals determine how much of the image has been downloaded by that timeframe.  With a single image download, it can perform multiple measurements at various intervals and determine available bandwidth much more accurately. Since there’s additional overhead in the underlying TCP/IP layers, it appears most tests also add padding to their calculation to make things more accurate.

Speakeasy Flash based speedtest

Flash has methods that will allow for such periodic sampling, Javascript however does not. This makes my Javascript implementation an overall average – so a report of 900Kbit/sec can easily represent 1.5Mbit/sec. (My results are much more akin to what Wireshark will report as throughput). I do plan to write a Flash based speedtest in the near future.

Wireshark measures throughput how I do: Bandwidth / (endtime - starttime)

In this example, the capture in Wireshark measures the throughput as 21.22Mbit/sec, nowhere near the 52.37Mbit/sec rating given by Speakeasy. The recent throughput information is all displayed in the signal screen:

Can you hear me now? How fast can I talk?

The Netflow section of my TSA Patdown plugin details the current traffic flow on the network. This screen updates dynamically as users surf the internet. (I’ll reiterate my past posts here: The netflow data is only packet endpoints… basically “this person called this person at this time”, but not the actual content of those conversations). I’ve also added a small port-based protocol disector that colorizes the flows and provides protocol information depending upon the packet you select. If you choose a NetBIOS packet, you’ll get something similar to this:

Flying by at the speed of light: PACKETS

The system monitors for NetBIOS names as well as DHCP hostnames that appear on Midtown Wifi. All of this information comes together to paint an accurate view of the network.

Clicking a Protocol Name (in this instance NetBIOS) will direct you to a Wikipedia article on the protocol and how it works. Unclassified protocols can be classified and colorized with a simple click. You can also specify the URL to load when the protocol name is clicked.

Pick a transport, a color, a name, a URL - and you've defined a protocol

The pie charts, RSSI graphs, and throughput graphs are all handled using the PHP JPGraph libraries. In the future, I intend to improve the graphs (there are tendencies for my labels to bleed off-screen or over each other).

The access points share their own ADSL line for bandwidth but maintain individual PPPoE sessions. The wiring in my home needs improvement (the house was built in the 1800′s, the Cat5 running through the house is obviously not that old but does have some serious issues) . Most of the exterior walls appear to be metal, which does hinder re-running the DSL line a bit.

I recently migrated my home network graphing from NetMRG to Cacti, and I’m using Cacti‘s (albeit poor) FTP export function to offload graphs pertaining to MidTown Wifi to the captive portal.

Graphs that have been imported from Cacti.

As you can see in the graphs, the system currently has 175 subscribers. I have deleted the bogus accounts that weren’t created through the APs. The high number of subscribers is largely the result of transient users (my home is on a major bus line, rental homes in the area turn over somewhat frequently, the local college is blocks away,  etc). A couple of users have duplicate accounts having apparently lost their credentials (as is evidenced by a few repeat MAC addresses).

To put the large number into proper perspective:  in the last 7 days there were 157 logins by 18 unique users. Unlike myself, most of the users don’t spend every waking moment on the Internet.

I’ve covered the access points and the firmware images in a number of previous posts, so I’ll let them speak for themselves. In the next few days (hopefully not weeks), I’ll be introducing my first Ubiquiti access point to the system with full details posted then. If you have any thoughts or input, by all means reach me in the comments section.

It smells of Goo-Gone and looks to have dandruff.

The 493AH features 9 ethernet ports, can accept POE on its WAN interface, has 64M NAND,  128M RAM, and can support 3 mini-PCI cards. Configuration can be performed via a serial interface and there is an external power connector available if POE isn’t used.

The device itself wouldn’t boot, only hang at the RouterBoot bootloader. Attempts to boot the NAND image failed, but the bootloader gives an easy option for downloading an image to it via TFTP.

A brand that's loved by WISPs

Looking around, it appears the 493AH is an Atheros AR7161 .. an architecture readily supported under OpenWRT. Sure, I could just re-install RouterOS… but let’s do that later.

To install OpenWRT to the 493AH, first format the NAND. This can easily be done via the bootloader (option e):

Waxing the memory

Next, use subversion to check out the Backfire version of OpenWRT:

mkdir ~/svn/

cd ~/svn/

svn co svn://svn.openwrt.org/openwrt/branches/backfire backfire

Building the image is fairly easy, all configuration is done via a “make menuconfig“. First, we’ll build a small initramfs. This will give us a tiny environment to boot into the device and later install our kernel with.

Ensure that you’ve selected the AR71xx target architecture…

Building an AR71XX System

Next, I opt for the default profile (to give me all the modules I should need)

I demand everything!

And finally, select the build of a ramdisk image:

Fun with arrow keys and a spacebar.

After you’ve made your selections, exit saving your changes, and run make. The build itself will take some time, but when you’re finished you’ll have the first key ingredient – a basic root filesystem embedded into the kernel. This image is essentially a “Live CD” that we’ll use to install our real kernel.

As with all of the images you create, you’ll find them under ~/svn/backfire/bin/ar71xx/ Our newly created image is openwrt-ar71xx-vmlinux-initramfs.elf

Next, we’ll want to build our actual system. To do this, re-run make menuconfig and select the packages that you wish to compile and include in your firmware image. After you’ve made all of your selections, change your Target Image to squashfs and exit saving your changes.

Planting a SquashFS Garden

A quick make later, and we now have a working rootfs and kernel – in addition to our initramfs to install the system with:

openwrt-ar71xx-vmlinux-initramfs.elf (Our temporary kernel)
openwrt-ar71xx-vmlinux.elf (The kernel)
openwrt-ar71xx-root.squashfs (Our Root Filesystem)

We now have almost everything we need.

Oh, the sticker says "Warranty Void if Removed"...

To install our kernel, we need a few additional tools. First off, we need to configure a DHCP server (I’m using ISC’s). Here’s an example from my dhcpd.conf file:

authoritative;
ddns-update-style interim;
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.20 192.168.1.40;
option subnet-mask 255.255.255.0;
option routers 192.168.1.1;
}

host routerboard
{
hardware ethernet 00:0c:xx:xx:xx:xx;
fixed-address 192.168.1.17;
}

 

Next, we’ll need a TFTP server. I prefer atftpd. No configuration is necessary, simply create a directory to serve files from and start the server:

mkdir /tftpboot/

chmod 777 /tftpboot/

atftpd ––daemon /tftpboot/

The only file that has to be loaded via TFTP is the initramfs kernel. Copy it to /tftpboot with the filename vmlinux and we’re ready to start.

cp ~/svn/backfire/bin/ar71xx/openwrt-ar71xx-vmlinux-initramfs.elf  /tftpboot/vmlinux

Power up the RouterBoard and quickly press the space bar. Select “boot over Ethernet” and it will download and boot the linux kernel.

Opting to boot via TFTP

Next ,we need to install the kernel and root filesystem. Here’s where I ran into my first problem – the kernel has no init variable specified so it panics. Thankfully it clearly states this: “Kernel panic – not syncing: No init found.  Try passing init= option to kernel.”

Gratitous Image of a Kernel Panic

Unfortunately the boot loader doesn’t appear to allow one to specify command line options for the kernel and I was unable to find a way to set this when configuring the kernel. (I vaguely recall seeing it when compiling for x86, but may be mistaken). Either way, the solution is simple:

Add your kernel parameters to a file (kernel-params in my instance) and use objcopy to insert it into the ELF file:

The options I used are:

root=/dev/mtdblock2 rootfstype=squashfs,yaffs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit

The toolchain supplied with OpenWRT contains a MIPs compatible version of objcopy that will allow you to add a kernel parameters section to the ELF file:

cd ~/svn/bin/ar71xx/

~/svn/backfire/build_dir/toolchain-mips_r2_gcc-4.3.3+cs_uClibc-0.9.30.1/binutils-2.19.1/binutils/objcopy ––add-section kernparm=kernel-params openwrt-ar71xx-vmlinux.elf

To install the kernel, configure an IP on your ethernet  (or bridge) interface, mount /dev/mtdblock1 and use scp to copy your kernel to the device (as “kernel”).

ifconfig br-lan 192.168.1.10

mkdir /mnt/boot

mount /dev/mtdblock1 /mnt/boot

cd /mnt/boot/

scp 192.168.1.1:~/svn/backfire/bin/ar71xx/openwrt-ar71xx-vmlinux.elf kernel

Next, install your squashfs root filesystem to /dev/mtdblock2. Unlike the kernel, This shouldn’t be mounted when installed.

cd /tmp/

scp 192.168.1.1:~/svn/backfire/bin/ar71xx/openwrt-ar71xx-root.squashfs .

cat openwrt-ar71xx-root.squashfs > /dev/mtdblock2

After the root filesystem is installed, reboot the device and welcome to OpenWRT on the RouterBoard 493AH

It's Alive!

I’m not quite sure what I’ll end up doing with the 493AH just yet. The neighborhood wireless system now consists of 2 Engenius EOC2610 units running firmware images based off OpenWRT… so there may be the potential to add it to the fray.  The 9 ethernet ports would make it ideal for a Quagga router (although I already have one). Installing the MikroTik RouterOS and working with MPLS is another options. Right now it sits on my desk at work as a “pretty cool paperweight with a lot of potential”.

If you have any suggestions – please let me know.

A penguin juggling multiple tasks and OSes

KVM has the capability to run both Windows and Linux operating systems (as well as numerous others), supports both CPU based virtualization and para-virtualization, and has virtual drivers that can run in the guest instance speeding up disk and network IO as well as balloon memory drivers to reduce the actual utilization of memory on the host machine.

I’ll stop here and say this – For home use, I’d probably suggest avoiding anything laid out in this article and simply use Virtualbox. I use it on my desktop extensively and have for many years, but for this article I’m focusing on building a cluster with KVM.

VirtualBox running Meego (another project of mine)

With the absence of a proper Storage Area Network (SAN), I’m utilizing DRBD (Distributed Replicated Block Device) to provide VM disk replication across both virtual nodes. This allows for live migration of a guest from front-end node to front-end node. Additionally, this architecture will still allow for replacement of storage OR supplementing of storage with a SAN in the future.

The VM Cluster Layout.

DRBD replicates individual LVM volumes (and not the raid array as a whole) across my 2 host nodes. Each virtual guest has it’s own logical volume assigned to it, which is accessed via DRBD block device interfaces  (/dev/drbd<number>).

In the example image above, jabber0 and web0 (virtual “guests“) are running on virtual0 (a virtual “host” or “node“), with web1 (another “guest“) running on virtual1 (a virtual “host” or “node“). The DRBD resource is set to Primary mode on the virtual host connected to a running guest, with the disk being copied to the Secondary (the non-running virtual host) Primary mode allows the virtual host (and it’s guests) to access the DRBD resource (and read/write from the connected logical volume).

As far as a virtual guest is concerned, there is no DRBD, only a /dev/vda or /dev/sda device.

Only during live-migration should the DRBD resources on both virtual hosts be placed into Primary (a situation called Dual Primary). As one virtual guest is paused prior to the other going active, data corruption will not occur.

The Virtual Server itself (Well, it looks like this)

Each node is presently a Dell PowerEdge 2950 with 32G of memory and over 1 Terrabyte of storage. With the DRBD replication this gives approximately 1 Terrabyte of storage (and not a combined 2 Terrabytes).

Each node has 4 gigabit ethernet interfaces.

There are presently three ethernet bridges on each node:

Connecting to a Guest:

Each guest is accessible via standard means (ssh) when configured correctly. Additionally, one can connect to each guest by VNCing to a unique port on the virtual host. (I do maintain a list of which DRBD ports and VNC ports are used for each of my virtual guests)

Configuring an LVM volume:

The “vmdisks” LVM volume group is approximately 1.3TB of disk storage, used to provide individual volumes to the guest instances. I use a logical volume of 30G for most guests.

To add a logical disk for guest usage is simple – and Must be done uniformly across all nodes:

lvcreate -L <size>M -n <name> vmdisks

Initial Configuration of the DRBD MD device:

The DRBD MD device is the actual block device that the Guest machine will interface with.

The following MUST BE DONE ACROSS ALL NODES, however only upon initial creation:

Update the /etc/drbd.conf file to add your new node (here’s an example):

resource <resource name>
{
net
{
allow-two-primaries;
}

syncer
{
rate 33M;
}

startup
{
become-primary-on virtual0.braindeadprojects.com;
}

on virtual0.braindeadprojects.com
{
device /dev/drbd<Next Available block>;
disk /dev/vmdisks/<LVM Volume Group>;
address 10.255.255.1:<Next available port>;
meta-disk internal;
}

on virtual1.braindeadprojects.com
{
device /dev/drbd<Next available block>;
disk /dev/vmdisks/<LVM Volume Group>;
address 10.255.255.2:<next available port>;
meta-disk internal;
}

}

After updating the config, create the block device and enable it:

#drbdadm create-md <resource name>
#drbdadm up <resource name>

At this point, all nodes have a record of this DRBD resource. /proc/drbd will have additional information.

The following must be done ONLY ON THE PRIMARY (MASTER) NODE:

#drbdadm — –overwrite-data-of-peer primary <resource name>

This will begin an initial synchronization across the nodes. Again, this is only run on the “Master node” (the virtual host node that is initially running the VM guest).  At this time, the DRBD resource is available on ALL nodes, however until the synchronization is finished, reads/writes will take slightly longer.

An important note on synchronization:

The syncer{} stanza in the resource config plays an important role in how fast a drive is synchronized. Default sync speed is roughly 340K/sec, which in turn causes a drive synchronization of a 30G drive to take appx 28 hours.

This can safely be set to 33M/sec in my environment, reducing sync-time to roughly 20 minutes, depending upon load.

Sync rate will play an important factor in instances where an entire node fails, and the resources of the failover node cannot keep up. In such an event, a 3rd node should be added to the system, with drives synced to it.

Creating the VM Guest:

I’m utilizing libvirt as a convenient way to provision and manage virtual machines.

libVirt, handling all your hypervisor and sardine needs since 2009

Creation of a guest is fairly easy, and can be done interactively or via a one-liner:

#virt-install –connect qemu:///system -n <Guest Name> -r <RAM in MB> –vcpus=1 \
–arch=<i686|x86_64|…> –vnc –vncport=<unused VNC port number> –noautoconsole –os-type linux –accelerate \
–network=bridge:<kickstart_br0|world_br0|world_br1> –hvm –disk path=/dev/drbd<resource number>,bus=virtio \
–<pxe|import|cdrom>

After which time the guest will automatically start, with it’s vnetX interface bridging with kickstart_br0.

I’ve installed DNSMasq on each host machine. It sits atop the kickstart_br0 interface, and assigns the VM an IP in the 192.168.6.0/24 network (via DHCP), and PXE boots/kickstarts it off a mirroring server. (The 192.168.6.0/24 network is MASQUERADEd in iptables so requests appear to come from virtual[01])

After kickstarting the guest, the reboot process tends to shut down the virtual guest so it may need to be restarted (Normal reboots are not prone to this shutdown). Once restarted, server configuration can be done via ssh from the host node, or VNC.

A KVM Trixbox guest: It's ALIVE!!!

Once the machine is built, customized and ready to be placed on the Internet, power down the VM guest and edit the XML config file (replacing kickstart_br0 with world_br0 or world_br1). If you find that the VM guest attempts to PXE boot once again, you may need to also change the boot device order (specifying hd instead of network)

You will also want to adjust the clock to source itself from the host machine.

# virsh
Welcome to virsh, the virtualization interactive terminal.
Type:  ‘help’ for help with commands
‘quit’ to quit
virsh # edit <guestname>
…
…
<os>
<type arch=’x86_64′ machine=’rhel5.4.0′>hvm</type>
<boot dev=’network’/>
<boot dev=’hd’/>
</os>
…
…
<interface type=’bridge’>
<mac address=’54:52:00:2d:21:10′/>
<source bridge=’kickstart_br0′/>
<target dev=’vnet1′/>
</interface>
…
…
<clock offset=’localtime’/>
…
…

To boot from CD:

<disk type=’file’ device=’cdrom’>
<driver name=’qemu’ type=’raw’/>
<source file=’/usr/src/systemrescuecd-x86-1.6.2.iso’/>
<target dev=’hdc’ bus=’ide’/>
<readonly/>
<address type=’drive’ controller=’0′ bus=’1′ unit=’0′/>
</disk>

After editing the XML, you can restart the VM.

virsh # start <guestname>

General usage:

I’ve made sure to install virt-top, an interface to the hypervisor similar to the “top” command. This gives a nice overview of the system:

#virt-top

virt-top: An overview of CPU and memory

The shell API for libvirt makes manipulating guest instances easy. Here are a few of the more frequently used virsh commands:

#list <–all> (Lists running and non-running guests)
#start <guestname> (Starts guest instance)
#autostart <guestname> (Marks guest to be autostarted at node boot)
#destroy <guestname> (Immediately powers off guest)
#suspend <guestname> (Powers down guest gracefully)
#reboot <guestname> (Reboots guest)
#edit <guestname> (Edits the guest XML config)
#migrate (See the migration section for more info)

Live migration:

Live migration between nodes can be done via ssh (with shared keys) or TLS. I’m currently utilizing the ssh method:

Prior to migration, the DRBD resource needs to be place in Primary on both nodes:

#drbdadm primary <resource name>

After doing so, the following is run on the SENDING node only:

#virsh migrate –live <GUEST> qemu+ssh://<RECEIVING NODE>/system

As part of the migration process, the sending node copies memory and kernel state via ssh to the receiving node. During the migration process, the guest instance on the sending node is active, with the guest node being marked as paused.

Once information is migrated, the sending node pauses it’s guest instance, with the receiving node un-pausing. Actual migration depends upon a number of factors, although is generally accomplished in under 35 seconds.

Following the migration, it’s essential to place the losing node into DRBD secondary mode. Should I accidentally try to start the guest on the losing node, this will prevent it from obtaining access to the disk (and corrupting data):

#drbdadm secondary <resource name>

Virtualizing Physical Machines:

Virtualizing a physical machine is extremely easy. Instead of PXE booting and kickstarting an install (–pxe),  I use the –cdrom /usr/src/systemrescuecd-x86-1.6.2.iso flag when creating the virtual guest. On each virtual host, I have a copy of the excellent Gentoo based SystemRescueCd.

When the system boots into the live CD, partition the drive (usually /dev/vda or /dev/sda) as you wish (taking advantage of LVM for non-boot partitions if possible).

Create a /mnt/migrate directory from the live cd, and mount your newly created partitions there.

mount /dev/sda2 /mnt/migrate
for dir in {boot,proc,dev}; do mkdir /mnt/migrate/$dir; done
mount /dev/sda1 /mnt/migrate/boot

(Do the same for /var and any other directories you have partitioned separately)

Utilizing rsync over ssh, synchronize all files from the physical host to the virtual one (taking care that you perform the action correctly, so as not to overwrite the original server). A handful of files and directories NEED TO BE OMITTED, namely:

/proc
/dev
/sys
/lost+found
(possibly others)

I generally use an rsync command similar to this one:

#time rsync –exclude ‘/dev/’ –exclude ‘/proc/’ –exclude ‘/sys/’ –exclude ‘/lost+found/’  -ave ssh <originalserverip>:/ /mnt/migrate/

The following devices need to be present in order to boot into the OS.  Either rsync them or manually make them with mknod.

/dev/console
/dev/initctl
/dev/null
/dev/zero

Another easy way to accomplish this is:

for file in {console,initctl,null,zero}; do cp -a /dev/$file /mnt/migrate/dev/$file ; done

Following the rsync, the virtual guest will need a bootloader and an updated initial ramdisk. Both of these are best done in a chroot environment:

mount -o bind /dev/ /mnt/migrate/dev/
mount -t proc none /mnt/migrate/proc/
mount -o bind /sys/ /mnt/migrate/sys/
chroot /mnt/migrate/ /bin/bash

Inside the chroot environment, you will need to update /etc/mtab and /etc/fstab to reflect your new partitioning (at the very least drives will likely change to /dev/vda). You will also need to update /boot/grub/device.map to reflect hd0 as a valid drive.

Once these changes have been made, grub installation should be as simple as:

grub-install –no-floppy /dev/vda

(Although sometimes grub needs to be further caressed. See the Gentoo Handbook section on installing the grub bootloader for good documentation)

If a simple grub-install does not work:

# grub
grub> root (hd0,0)           <– where hd0,0 is the first partition on the first drive
grub> setup (hd0)            <– install grub on the mbr of the first drive
grub> quit

With the Bootloader installed, we need to create a working initial ramdisk with necessary drivers. Look for the most recent installed kernel in grub.conf and create the ramdisk (replace the version numbers with yours):

mkinitrd -f /boot/initrd-2.6.18-194.32.1.el5.img 2.6.18-194.32.1.el5

To take advantage of virtio (network and disk) modules on newer kernels, you may opt to include them in the initrd:

mkinitrd -f –with virtio_pci –with virtio_blk \ –with virtio_net /boot/initrd-2.6.18-194.32.1.el5.img 2.6.18-194.32.1.el5

Please be aware that many files (IE: databases) should only be rsynced when shutdown. For this reason, it’s often best to shutdown MySQL, PostgreSQL, and various other services PRIOR TO RSYNCing to prevent database corruption.

How to physicalize a virtual machine:

In the event of a major issue, converting a virtual machine back to a physical machine has the same process as physical to virtual, but reversed.

Of Note:

While Fedora currently supports (automatically, out of the box) SELinux profiles/labels for KVM guest instances, Centos 5.6 does not. It will be incorporated in Centos 6, however… and I plan on migrating to that OS eventually.

Final Thoughts:

As with everything, there’s pros and cons to this methodology.

• While I’ve always preferred avoiding GUIs, the fact is they standardize on what steps happen in which order (limiting the potential for user-induced errors.)

• A high performance SAN (or perhaps an OpenFiler box) would make things much easier to configure and migrate, but at the same time introduce a possible single point of failure.

• Utilizing an automation engine (like puppet) could limit the number of steps needed to provision a virtual guest across all nodes.

• Outside of some possible restrictions (virtio drivers being specific to KVM, LVM2 support for Windows), migrating from the present day system to vmware, virtualbox, or <insert your favorite hypervisor here> — should be fairly easy, requiring simply creating a guest and pointing it at an LVM share.

All in all, the system has been in production for nearly a year now and is performing beautifully. And best of all, I’m saving on power and generating less heat.

I can still SEE YOU!

I’ve upgraded the nvidia-drivers a number of times, never actually fixing the problem. Other Gentoo users on the #gentoo channel of freenode have suggested migrating to gnash instead… and while I have contemplated this, I’ve noticed a number of things that don’t work well under gnash on my netbook.

Thankfully I’m not the only person to be experiencing this. Earlier today I came across a solution (that while not optimal) definitely fixes the problem:

No acceleration, but no bleeding either.

Disabling hardware acceleration thankfully stops the bleedthrough. (Just right click on a flash movie, select “Settings” and disable acceleration under “display”) You will need to restart your browsers for it to fully take effect.

GNS3 - this thing rocks.

GNS3/dynamips can handle emulation of a number of routers, and I was happy to see that it can emulate host PCs as well. Unfortunately networking in GNS3  is done via UDP tunnels – something that requires a Qemu patch to implement.

For some reason QEMU won't really start if it's networked.

In Gentoo, simply building GNS3 from the Sunrise Overlay doesn’t provide sufficient support for this requirement. Emulated routers (and I’m assuming switches) work fine when networked together. Emulated host machines work fine when NOT connected to anything. But once you network things together you’ll discover that your Qemu host won’t start.

Here’s why – when you start a Qemu host, this is essentially what GNS3 runs:

/usr/bin/qemu -name QEMU1 -m 256 /mnt/virtualmachines/dynips/IPv6-tc/working/QEMU1/FLASH -hdb /mnt/virtualmachines/dynamips/IPv6-tc/working/QEMU1/SWAP -enable-kvm -net nic,vlan=0,macaddr=00:aa:00:8f:e4:00,model=rtl8139 -net udp,vlan=0,sport=20000,dport=10006,daddr=127.0.0.1 -net nic,vlan=1,macaddr=00:00:ab:fa:72:01,model=rtl8139 -serial telnet:127.0.0.1:3000,server,nowait -no-acpi -vnc :0

GNS3 attempts to use the -net udp option, something not offered in the default QEMU-KVM distribution. With no Gentoo USE flags to add the patch in either 0.13.0 or 0.14.50, I decided to simply upgrade to 0.14.50 and see if it was part of the default build:

# kvm –version
QEMU emulator version 0.14.50 (qemu-kvm-devel), Copyright (c) 2003-2008 Fabrice Bellard
# kvm –help | grep “net udp”
#

Nope, it’s not. So the next step is to remove app-emulation/qemu-kvm from your system and build it by hand.

First, remove kvm and download the source from kernel.org:

# emerge -C app-emulation/qemu-kvm

# cd /usr/src/

# wget http://mirrors.kernel.org/gentoo/distfiles/qemu-kvm-0.13.0.tar.gz

Unpack and patch it, using the patch provided by the gns-3 team .

# tar zxvpf qemu-kvm-0.13.0.tar.gz

# cd qemu-kvm-0.13.0/

Testing the patch, I ran into a handful of issues:

# patch –dry-run -p1 < /mnt/nas/downloads/qemu-0.13.0-
mcast-udp.patch
patching file Makefile.objs
Hunk #1 FAILED at 25.
1 out of 1 hunk FAILED — saving rejects to file Makefile.objs.rej
patching file block/raw-win32.c
Hunk #1 FAILED at 93.
Hunk #2 FAILED at 347.
2 out of 2 hunks FAILED — saving rejects to file block/raw-win32.c.rej
patching file hw/e1000.c
Hunk #1 FAILED at 567.
1 out of 1 hunk FAILED — saving rejects to file hw/e1000.c.rej
patching file net/udp.c
patching file net/udp.h
patching file net.c
Hunk #1 FAILED at 30.
Hunk #2 FAILED at 1075.
2 out of 2 hunks FAILED — saving rejects to file net.c.rej
patching file net.h
Hunk #1 FAILED at 33.
1 out of 1 hunk FAILED — saving rejects to file net.h.rej
patching file qemu-options.hx
Hunk #1 FAILED at 996.
1 out of 1 hunk FAILED — saving rejects to file qemu-options.hx.rej

A quick modification and we’ve got a new patch file. This one applies nicely:

# patch –dry-run -p1 < /mnt/nas/downloads/gentoo-qemu-0.13.0.gns3.patch
patching file Makefile.objs
patching file QMP/qmp-commands.txt
patching file block/raw-win32.c
patching file config-all-devices.mak
patching file config-host.h
patching file config-host.h-timestamp
patching file config-host.ld
patching file config-host.mak
patching file hw/e1000.c
patching file libdis/config.mak
patching file libdis-user/config.mak
patching file libhw32/config.mak
patching file libhw64/config.mak
patching file net/udp.c
patching file net/udp.h
patching file net.c
patching file net.h
patching file qemu-doc.html
patching file qemu-img-cmds.texi
patching file qemu-img.1
patching file qemu-monitor.texi
patching file qemu-nbd.8
patching file qemu-options.hx
patching file qemu-options.texi
patching file qemu-tech.html
patching file qemu.1
patching file roms/seabios/config.mak
patching file roms/vgabios/config.mak
patching file x86_64-softmmu/config-devices.mak
patching file x86_64-softmmu/config-devices.mak.old
patching file x86_64-softmmu/config-target.mak
vonnegut qemu-kvm-0.13.0 #

So let’s apply it for real and then configure, build, and install our new Qemu:

# patch -p1 < /mnt/nas/downloads/gentoo-qemu-0.13.0.gns3.patch

…

# ./configure –prefix=/usr –target-list=i386-softmmu –enable-sdl

# make && make install

A quick check to ensure UDP tunneling is compiled in:

# /usr/bin/qemu –version
QEMU emulator version 0.13.0 (qemu-kvm-0.13.0), Copyright (c) 2003-2008 Fabrice Bellard
# /usr/bin/qemu –help | grep “net udp”
-net udp[,vlan=n]sport=sport,dport=dport,daddr=host
#

And voila – now not only does my host machine start when networked into my layout, it also has connectivity:

Yes, it finally works.

Next up – continue lab work with IPv6, and begin studying for my CCNP.

I don’t do most of my work in Perl, but I can definitely make ends meet. My more recent work in Perl has dealt w/ RIP scanning and route-updating (a lot of DSL CPE tends to have RIP enabled). Packet creation isn’t that difficult, and using Perl I could bang out a script in a short period of time.

Usually for packet manipulation, I stay away from Perl – instead I’m a huge fan of Hping[23]. If you’re wanting to watch and play w/ the transport layer – an old laptop with hping and wireshark running is definitely the way to go. If you want to test MTU problems, or ACLs  -  Hping is fast and easy.

Hping is an amazing tool – one of my favorites. Hand crafted TCP and UDP traceroutes are easy as hell, and hand crafting TCP timestamp requests is easy too. Just read the manpages and you’ll find all sorts of interesting features of TCP/IP.

hping: Like a screwdriver set with most of the bits.

The downside to Hping is that it doesn’t seem to be under regular development (last release 2005?), and IPv6 isn’t supported yet (Aside from some basic third-party patches). Application layer creation is also left for other utilities to handle (But in all fairness – that’s not it’s job)

I was a late adopter, but about a year ago I made a new friend (and picked up some python along the way). My new best friend is Scapy. “Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more.” It runs in interactive and script-able mode, and thus far it’s proven to be quite powerful. It also supports IPv6.

Scapy understands a large number of Application level protocols (SNMP, TFTP, SMB, etc, etc), and makes it easy for one to craft such packets.  When I recently realized early versions of Cisco IP Router Export have corrupt headers – Scapy came to the rescue (albeit, only to rip the headers off, as fixing them wasn’t possible).  When I wanted to craft a quick and easy FreeRadius Packet of Death – Scapy came to the rescue.

FreeRadius PoD on www.exploit-db.com

Another great thing about Scapy is that it’s easy to add “layers” (protocols). Recently I sat down with the RFC’s for OSPF and in about 2 days time had fully implemented OSPF in Scapy – learning a lot about the protocol along the way. (Dirk Loss had beat me and submitted it first, but that was besides the point).

Looking for a fun new world to poke, and tired of IPv4? Sign up for a 6-in-4 tunnel and go exploring. Just make sure you pack Scapy and a handy guide.

There seems to have been a LOT of Linux based Kiosk projects out there, most of them now deprecated. After a bit of searching, I came across a good one – WebConverger.

Webconverger

Webconverger is a Live Debian build, created and maintained by Kai Hendry using the Debian Live-builder packages. The LiveCD includes the lightweight  Dynamic Window Manager (DWM) , the IceWeasel browser with Webconverger kiosk extension (to lock things down), and… well that’s about it.

Kai has excellent documentation on rolling your own Webconverger LiveCD using his GIT repository, although being pressed for time I opted to go the BraindeadProjects route: Just modify his already released ISO.

To begin modifying it in this manner, first mount the iso using a loopback device:

#mkdir /mnt/webconverger
#mount -o loop webc-7.2.iso /mnt/webconverger

Next mount the SquashFS image, also using a loopback device:

#mkdir /mnt/webconverger-filesystem
#mount -o loop -t squashfs /mnt/webconverger/live/filesystem.squashfs \
/mnt/webconverger-filesystem

As these two mounts are readonly, we need to create editable copies of each.

#mkdir /devel/isolinux
#rsync -av /mnt/webconverger/ /devel/isolinux/

#mkdir /devel/squashfs
#rsync -av /mnt/webconverger-filesystem/ /devel/squashfs/

Now we can go about modifying these two directories. Changing out the ISOLINUX boot splash image is an easy start. The image found under /devel/isolinux/isolinux/splash.png is actually an LSS16 image. To replace it, take or create an image 640×480 in size, and convert it to 14 indexed colors. (In GIMP, these options are under Image/Index)

Indexing Colors in Gimp

Once complete, save your image in PPM format.

Next, you’ll need the syslinux package installed on your machine. The syslinux package includes a handy utility to convert PPM to LSS16 (for use as a bootsplash image)

#ppmtolss16 < /tmp/myimage-boot.ppm > splash.png

Since you’re already working on the ISOLINUX side of things, I recommend looking at and revising your boot menu. Once I have an image that I’m happy with, I set the following options to prevent someone from rebooting the Kiosk and tampering with boot parameters:

menu background /isolinux/splash.png
default /isolinux/vesamenu.c32
noescape 1
nocomplete 1
prompt 0
timeout 15
allowoptions 0

While you can nest a number of ISOLINUX boot configs together, I generally keep it to one file that includes the above directives. Dont’ forget to include at least one label for a kernel to boot.

After updating the Boot Splash screen, have a look at /devel/squashfs/home/webc/pb.sh. This script is what causes IceWeasel to start, restart if closed, and sets the desktop background image (amongst other things). This script also downloads a background image from your homepage at boot – which can come in handy if you want to rotate daily ads.  I’ve personally modified my installation to always load the same background image, and fullscreen that image.

The webpage that appears each time IceWeasel starts is passed as kernel boot parameter (homepage). To update the homepage, simply edit the labels in the ISOLINUX  directory.

You will notice that when pressing the home button in the browser however, that you’re actually taken to an about: page that gives details about the current IceWeasel build. To configure this homepage, look at /usr/lib/iceweasel/browserconfig.properties

browser.startup.homepage=www.braindeadprojects.com

I personally like to lock things down a bit more than the standard release. For that reason I also add the following to /etc/iceweasel/pref/local.js

pref(“network.protocol-handler.external.snews”, false);
pref(“network.protocol-handler.external.news”, false);
pref(“network.protocol-handler.external.irc”, false);
pref(“network.protocol-handler.external.mail”, false);
pref(“network.protocol-handler.external.mailto”, false);

Another thing that may prove beneficial is to remove any and all remnants of xterm. As xorg does depend upon xterm, it will have to be forceably removed. This is best done in a chroot environment

#chroot /devel/squashfs/ /bin/bash
#dpkg –force-all -p xterm
#exit

Once you have your modifications complete, you will want to re-squash the squash filesystem. To do this, you’ll need squashfs-tools version 4 (Centos is currently distributing version 3, so do keep that in mind). Squashing using version 3 of the tools will result in a non-bootable kiosk.

#mksquashfs /devel/squashfs/ /tmp/webc.squashfs
#mv /tmp/webc.squashfs /devel/isolinux/live/filesystem.squashfs
#cd /devel/isolinux/
#mkisofs -o /tmp/my-webc.iso -b -r -J -l -cache-inodes -allow-multidot -no-emul-boot \
-boot-load-size 4 -boot-info-table -b  isolinux/isolinux.bin -c isolinux/boot.cat \
/devel/isolinux

Finally, isohybrid your ISO:

isohybrid /tmp/my-webc.iso

I highly recommend testing your ISO image in VirtualBox. Using VirtualBox (or any other virtualization option), saves you from constantly burning an image to  a CD or USB drive. Be mindful that you can skip the isohybrid step and test with VirtualBox, although you won’t be able to install it later using dd.

Once you have an image that you’re happy with, use dd to copy the ISO onto the hard drive of your Kiosk machine. Personally, I copy my ISO to a USB thumbdrive running the Gentoo Based  System Rescue CD, boot into it and then install onto the harddrive:

dd if =/livemnt/boot/kiosk/my-kiosk.iso of=/dev/sda

Of course, one could save time and simply use the WebConverger Customization Service… but why not use this as an opportunity to sharpen one’s skills.

Coming soon to – a walkthrough on how to build and customize a WebConverger ISO from Kai’s GIT repository (as opposed to re-rolling his ISO).

Yes, ideally one rebuilds the deprecated server anew, but due to time-constraints and a number of other issues that wasn’t a possibility. Unfortunately, sometimes you just have no choice but to kick the can down the street.

The migration from physical to virtual is simple – create the virtual guest, create an LVM logical volume for the disk,  format it and rsync the contents from the physical to the virtual drive. After that, install grub on the guest and voila – you’re done.

Keeping some consistency (although unnecessary), I went with an EXT3 filesystem on the “new” guest. Unfortunately, I came across the following snags:

• The inode size used in Redhat 9 is 128 bytes, but modern systems (ie: the host I formatted the partitions from ) use 256 bytes.
• There’s a number of attributes that weren’t present in Redhat 9.

Here’s the output of tune2fs on a Redhat 9 EXT3 partition:

# tune2fs -l /dev/hda1
tune2fs 1.32 (09-Nov-2002)
Filesystem volume name:   /boot
Last mounted on:          <not available>
Filesystem UUID:          9e6fa853-18bc-4c08-bb7c-51c74e0c11ae
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery  sparse_super
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              8032
Block count:              32096
Reserved block count:     1604
Free blocks:              17451
Free inodes:              7982
First block:              1
Block size:               1024
Fragment size:            1024
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         2008
Inode blocks per group:   251
Filesystem created:       Tue Oct 26 12:48:49 2010
Last mount time:          Thu Oct 28 15:58:10 2010
Last write time:          Thu Oct 28 15:58:10 2010
Mount count:              11
Maximum mount count:      28
Last checked:             Tue Oct 26 12:48:49 2010
Check interval:           15552000 (6 months)
Next check after:         Sun Apr 24 12:48:49 2011
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal UUID:             <none>
Journal inode:            8
Journal device:           0×0000
First orphan inode:       0

The following is output from a Centos 5.5 formatted EXT3 filesystem:

# tune2fs -l /dev/mapper/VolGroup00-LogVol00
tune2fs 1.39 (29-May-2006)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          7384cac8-b098-4c85-be6d-643443ae3d3d
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              6809088
Block count:              6807552
Reserved block count:     340377
Free blocks:              6204875
Free inodes:              6752521
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1022
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         32736
Inode blocks per group:   1023
Filesystem created:       Thu Oct 28 13:31:34 2010
Last mount time:          Thu Oct 28 17:43:57 2010
Last write time:          Thu Oct 28 17:43:57 2010
Mount count:              2
Maximum mount count:      -1
Last checked:             Thu Oct 28 13:31:34 2010
Check interval:           0 (<none>)
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      42eb54b3-8f66-4aef-8578-388c9863423c
Journal backup:           inode blocks

As you can see, some basic features and the inode size has changed in 10+ years. The solution:  format the “new” guest  to match the specs of the old:

#mke2fs -O has_journal,filetype,sparse_super,^ext_attr,^resize_inode,^dir_index \
-I 128 -j /dev/sda$disk

And that’s it- rysnc the files over to the guest VM, install grub, verify your fstab is good, and voila – you’ve transferred a physical server to a virtual one.

Put it here temporarily?

It makes sense, you need to place to hold something for a moment – it’s right there and readily available. And since you’re not pressing the Enter key, it’s not going to be sent anywhere right?

Well, actually it is. After you stop typing, it immediately sends an HTTP POST request to it’s target (Google in this case). And while the search does takes place, it doesn’t update your browser (so you might not realize it even happens). Here’s a copy of the content in the packet:

GET /complete/search?output=firefox&client=firefox&hl=en-US&q=mysuperleetpassword HTTP/1.1
Host: suggestqueries.google.com
User-Agent: <omitted>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Does this matter? That depends on what you put there. You probably wouldn’t pick up the phone and call Google (or Yahoo!, or BING, etc) and tell the receptionist “Hey, my Facebook username is … and my password is …”, but you can very easily do this by simply pasting ANYTHING in that handy little search bar.

Here’s a quick video of me running a packet capture and typing something into the search area. Again, I only moved my cursor – never did I press Enter (View it fullscreen for better detail).

I wonder how much garbage accidentally falls into search engine pits like this. I’m also curious as to how many sites log mistyped passwords (think of it this way – you accidentally type your webmail password into Facebook or vice versa).

All the misguided traffic reminds me of  the pollution problem of 1.0.0.0/8.