All posts by Matthew Gillespie (admin)

Cisco Switch and Route Completed!

I recently took my CCNP Switch Exam (642-813), passing on my first attempt. While I mostly used GNS3/dynamips to simulate routers for my CCNP Route Exam (642-902), I actually put together physical hardware for the Switch Exam. My switch lab consists of a Cyclades ACS48 terminal server, an APC 7900 switched PDU, 3 Cisco 3550-EMI multilayer switches, a 24 port patch panel, a Cisco SPA942 IP phone, and 2 ZNYX PCI 4-port ethernet cards for my desktop (for a combined 8 NICs dedicated to virtualization).

8 strands of cat5e can fit in a 3/4″ conduit, but it’s a tight fit

Initially I had thought there were no emulators for Cisco switches, but upon news that GNS3 would be supporting switching in their upcoming 1.0 release, I started digging. Apparently it’s a well established fact that Cisco switching can indeed be emulated and there are leaked binaries compiled for Linux for a handful of layer 2 IOS images. Hewlett Packard also has a 64-bit VM emulator for their VSR1000 Virtual Services Router Series.


GNS3 CrowdHoster
I think it’s safe to say GNS3’s fundraising will be successful when it ends.


While IOU is nice to work with, there are a number of bugs in the images. First of all, while you can configure private VLAN’s, they don’t appear to work (and they’re not supported on my physical 3550’s). Testing dynamic arp inspection in the following images also doesn’t seem to work:

  • i86bi_linuxl2-upk9-ms-15.0.bin
  • i86bi_linuxl2-ipbasek9-ms-15.1B.bin
  • i86bi_linuxl2-ipbasek9-ms-15.1A.bin
  • i86bi_linuxl2-upk9-ms-12.2.bin

The utility iou2net (which is used to connect the unix socket to physical hardware) is sometimes wonky,  sending unidirectional traffic much of the time.  So for my exam, I stuck mostly with my physical hardware for switching, emulated routers via dynamips (in GNS3), and hosts spawned by VirtualBox Manager (to make moving them around to different switchports easier).

Lots ‘o NICs and Bridge Interfaces

I’m connecting my emulated routers to my physical switches and virtual machines using GNS3’s Generic NIO ethernet interface. Of course, my physical setup isn’t without it’s own issues – the tulip drivers (or my motherboard) appear to have occassional complications, flooding the kernel ring buffer with annoying “slowpath” messages and also preventing traffic. Kernel upgrades haven’t resolved the problem (there were some fixes between kernel 3.9.11 and 3.10.1, but they don’t appear to resolve my issues). Thankfully bouncing the problematic ports will fix the issue.

[2842987.323263] Call Trace:
[2842987.323268]  [<c0455341>] dump_stack+0x16/0x18
[2842987.323272]  [<c012796c>] warn_slowpath_common+0x48/0x5f
[2842987.323275]  [<c013284b>] ? del_timer_sync+0x2b/0x3d
[2842987.323277]  [<c0127992>] warn_slowpath_null+0xf/0x13
[2842987.323280]  [<c013284b>] del_timer_sync+0x2b/0x3d
[2842987.323284]  [<f8606ace>] t21142_lnk_change+0x331/0x507 [tulip]
[2842987.323294]  [<f86021e4>] tulip_interrupt+0x5d0/0x784 [tulip]
[2842987.323298]  [<c015c538>] ? clockevents_program_event+0xe5/0x103
[2842987.323302]  [<c0176516>] handle_irq_event_percpu+0x4d/0x158
[2842987.323305]  [<c0176647>] handle_irq_event+0x26/0x3f
[2842987.323307]  [<c01785fa>] handle_fasteoi_irq+0x63/0x8b
[2842987.323310]  [<c0102862>] handle_irq+0x67/0x71
[2842987.323313]  [<c01024ce>] do_IRQ+0x35/0x8e
[2842987.323317]  [<c045deec>] common_interrupt+0x2c/0x31
[2842987.323319] —[ end trace 80bd835791c79500 ]—


For studying, I worked off Chris Bryant’s really solid Bryant Advantage lectures, the official certification guide, and labwork, labwork, labwork. To play with some of the layer 2 and 3 protocols outside of IOS, I put together a small TinyCore LiveCD [gpg] that includes yersinia, hping2, scapy, and tcpdump. (I’ve packaged yersinia separately if anyone would like it here).

TinyCore Yersinia
Versions Prior to 0.7.3 can be problematic and hang in VM environments

I find it’s extremely helpful to see traffic in action and having 8 NICs helps to facilitate that. While you can easily sniff HSRP, VRRP, and GLBP traffic inside GNS3, seeing VTP and DTP frames can be accomplished by creating bridge interfaces on my desktop OS and using the patch-panel to place myself in the middle of switch-to-switch connections.

The exam itself was pretty typical. I find the simulations to be fun although I’m surprised at how bad the Cisco interpreter is. At one point I had placed a VACL on the wrong piece of equipment, went to remove it and place it on the correct switch but couldn’t remove it on the original switch. Trying every variation of “no” proved fruitless, and in the end I’m sure I lost points due to that mistake. What’s annoying is that I tried to correct it. Thankfully, I’m not alone and a LOT of people complain about the interpreter.

If you’re studying for the exam, here are my pieces of advice:

  • You’re paying to take the exam, so get the best grade possible. That means study, study, study.
  • Find a good practice exam engine (like Boson’s ExSim) and take a bunch of practice exams. You’ll find out what you don’t know, then lab, lab, lab.
  • Take some time off work to study if you can. I dedicated 12 hours a day 5 days (over a long weekend) before the exam to studying. Was it excessive, definitely, but there’s no harm in being over prepared (you’re paying to take the exam, work your hardest).
  • If you truly know the material, you know the wrong answers sometimes more clearly than you know the right ones. Always remove the wrong answers from your available options while taking the exam. “No, GLBP uses UDP port 3222 so that answer isn’t for HSRP”
  • Even when you’re absolutely certain of the right answer, read all of the answers before going to the next question. On multiple choice questions, I typically say in my head “No that’s not it, that’s it, that’s not it, that’s not it. Second answer is correct”.
  • Double check your work on the simulations before going to the next question. Make sure you do everything and completely, and test your configurations.
  • Lab work means breaking things. Don’t just configure a working protocol, purposely misconfigure the protocol. Make sure you see the nuances that no book will dedicate the column inches to detailing.
  • Look over the exam topics. I didn’t realize they existed until after I took my CCNA and the CCNP Route and Switch exams, and it really led to a lot of unnecessary anxiety (“What’s on this exam, do I know everything???? HELP!”)

So now, I’m in the process of studying for my final CCNP exam – the TSHOOT. Wish me luck!

GNS3 Crowdfunder

GNS3 - this thing rocks.
GNS3 – this thing rocks.

If you’ve not had the chance to contribute to the GNS3 crowdfunding event, I strongly encourage you to do so. The GNS3 team is working on version 1.o, and as you’ll see on their features list, they’ll be integrating a handful of exciting new features into the GUI — including switching support (ala Cisco IOU I’m hearing).

I woke up at 2am this past Wednesday to get in on the early-bird Premium package, and as of now there’s under 100 left. They met their fundraising goal within 24 hours but why not contribute to some really good FOSS.


Growing An Olive Router in a VirtualBox Environment

In the last few days, I’ve been working to build up my home networking lab. I’m using a fairly typical setup nowadays — real Layer 3 switches connected to a host machine that runs GNS3 as well as VirtualBox. Having recently installed Vyatta and Mikrotik RouterOS VM guests, I decided that I would play with something I’ve never touched before — Juniper’s JunOS.


The documentation for how to install JunOS into a Qemu environment is constantly replicated across the web, so I have no intention of doing so here. (Have a look here to get yourself started). In my first few attempts to create a VM guest, I followed the instructions thoroughly, but eventually I opted to move the full creation into VirtualBox. That’s when I learned a lot about the nuances between installing on either platform.

One of the first problems I encountered when trying to install into a VirtualBox container was an odd out-of-space message when installing my modified version of JunOS 8.5R1.14 :

WARNING: The /tmp filesystem that is created by the next stage of
WARNING: the installer does not have sufficient space. This package
WARNING: requires 159804k free (2k for configuration
WARNING: files and 159802k for the new software), but there is
WARNING: only -12140k available.

What makes this message odd is that I have plenty of disk space. It’s also odd that I only encountered this in my VirtualBox/vdi installations, not on Qemu/qcow2 installations. Before I dug any further, I rebuilt the VM using 10G of storage instead of the recommended 4G, and adjusted each of the individual partitions accordingly. This proved to be ineffective as I was once again greeted with this message.

The warning message is generated by the function “tmp_storage()” in the +REQUEST file inside jinstall-8.5R1.14-domestic.tgz. One of the expr calls is failing, but I never bothered to dig into why and instead removed it with a quick patch. The bootstrap installation now appears to be good and will prompt you to reboot.

At this point, ensure that you have serial port 1 redirected to a host pipe. If you don’t, shut down the VM and add one — you will need it to see what’s going on.

Serial Port
No need for a fancy schmancy DB9 connector

You can attach to the host pipe a few different ways, I prefer to have socat connect to it and create a /tmp/juniper0 PTY that I can hook into from minicom. Do keep in mind that socat doesn’t respawn automatically, so if you disconnect from the PTY link you may have to restart socat.

socat UNIX-CONNECT:/tmp/juniper_serial PTY,link=/tmp/juniper0,raw,echo=0,waitslave

Upon reboot,  the system would now install up to a point and then hang. After a few moments it was obvious something didn’t give.

A few reboots later I realized that this was going nowhere. Since my test build in Qemu alone did install, I opted to simply convert the qcow2 file to something I could import into VirtualBox.

qemu-img convert -O raw ~/GNS3/QEMU/olive-base.img JuniperBase.raw

VBoxManage convertdd JuniperBase.raw JuniperBase.vdi

VBoxManage modifyvdi JuniperBase.vdi compact

However this caused an interesting bug: Prior to installing JunOS, packet captures on the guest indicated that OSPF all-router multicast messages were entering the guest VM. After installation, that was no longer the case. I tried numerous times to resolve this, but with no luck.

Finally I opted to build a 4.11 FreeBSD instance and install Junos 9.6R1.13. My first attempt failed with the following message:

ELF binary type “0” not known

If you see this message, make sure that you’ve properly replaced checkpic with /usr/bin/true. In my case, I had accidentally just copied true to the directory and didn’t replace checkpic. After fixing my mistake, the bootstrap installation succeeded and prompted me for a reboot.

After reboot the full install commences. I was quite surprised when it completed without any problems. I was skeptical that this was going to fix the multicast problem, but lo and behold, post reboot multicast actually functions.

OSPF Image
Low and behold – it works!

So in the end, my working VirtualBox JunOS instance consisted of FreeBSD 4.11, JunOS 9.6R1.13 and VirtualBox 4.2.12 r84980.

Olive Pudding
Proof is in the Olive pudding.

Using ALSA plugins to stop scaring the cats.

The XMBCbuntu ZBOX server has really changed things in my house. It’s become hard to justify the $50+ cable bill seeing as how I only watch one television show when it originally airs (Breaking Bad).  I’m not invested in a Triple Play package – I get my Internet access from a 5Ghz Ubiquiti link to my office (in addition to an aDSL backup line), and for phone service I have an Asterisk server as a PBX with a SIP trunk to Teliax. Quite simply, cable doesn’t do much for me.

I’ve also cut down on shelves — my video and music collection is being converted to digital and packed away in the basement. Most books I purchase on my Nook. If I can’t buy a video or album new as  a DRM free digital download, I simply don’t.

The only issue so far with the XBMCbuntu set-top box has been audio – the Radeon HD 6250/6310 sound card seems to have no hardware feature to increase or decrease audio — only mute it. That means I turn the TV up really loud to watch an episode of The Wire, and when my wife switches back to the Nintendo Wii the sound is startling.

A whisper then a YELL!

Thankfully ALSA is more than just the ALSA mixer, and has an entire plugin architecture behind it.


Using an .asoundrc file, I’m able to connect my HDMI audio output to a softvol plugin and increase audio that way. No more cats running from the upstairs loft scared half to death.

pcm.hdmi_formatted {
type plug
slave.pcm “dmix:0,3” #card 0, device 3

pcm.hdmi_complete {
type softvol
slave.pcm hdmi_formatted hdmi_volume
control.card 0
max_dB 50.0

pcm.!default pcm.hdmi_complete


Quite simply, I’m binding the sound card with a softvol plugin, setting it’s maximum gain to 50dB. Set the volume using alsamixer, and store your settings using “alsactl store” so they persist across reboots.

Voilà -- a way to boost audio.
Voilà — a way to boost audio.


While this is the first time I’ve utilized ALSA this way, I’ve long found that while watching movies on my netbook during long flights, mplayer’s softvol feature is extremely useful. You might find this trick handy when trying to hear a film over a loud jet engine:

mplayer -softvol-max 100 RobotAndFrank.avi


Robot and Frank
A funny and bittersweet movie I watched on a recent flight.