Claim to Fame

This week I managed to get my face on national television! Campbell Live on Thursday the 18th November to be exact.

This article is probably the most recent news item on my work mates success: http://www.scoop.co.nz/stories/AK1011/S00356/bid-bot-could-have-christmas-presents-sorted.htm

In summary, he programmed a script in python to utilise the recently made available, trademe web API, in order to search for cheap and rare items on trademe with free/low shipping and bid on them. Check out Paul’s blog for more info if you are interested: http://bieh.net/2010/11/14/changes-to-the-bot/

Anyway, how did I manage to end up with 3 seconds of airtime? Well I happen to work next to this awesome graduate so I was the guy in the background hard at work. At one point I do turn around and look back at what is going on behind me (all < 2 seconds of if 😀 ).

Here is the video, enjoy:

Summer sun – PHP gripes

Summer is almost here. Sunny skies, warm beaches, walks to the bakery for lunch.

Nothing to complain about there.

However, there is something to complain about in php. I was trying to track down a bug in my php scripts at work today and I came across another php pitfall for the weary programmer.

I think my svn commit change log will best explain what tripped me up.

The red line shows the bug in my old code, where I forgot a dollar sign and misspelled the variable ‘$direction’.

This was not producing a syntax error but rather the ‘direcion’ was being taken as a string and multiplied by -1 in some way, shape or form.

A small amount of googling came up with someone who also experienced a similar problem:

http://www.php.net/manual/en/language.types.string.php#89757

Network Map Display Project

To be awarded my degree with honours I have to complete an honours project which is worth the majority of my years work. The way it works is, lecturers write blurbs about projects that they would like to supervise and then the students browse through and pick the one that they like.

I have settled upon a project titled “Improved Network Map Display”. It is going to involve research around the area of network topology data consolidation and visualisation.

I have had a (short) look around the web and as far as I can tell, there has not been anything like this done in its entirety. Visualisations exist (most of which are static) that allow certain individual levels of network topology to be visualised. The purpose of this project is to improve upon these levels and to allow for seamless transitions between them as you zoom in and out. This will require interviews with various different network engineers and users, and also a well thought out topology schema that will fit the purpose of complete consolidation.

Here is the ‘Karen’ weathermap that shows the data traffic flows over the network. See the live version: http://weathermap.karen.net.nz/

This is a good example of how an autonomous system view might look.

The ability to zoom in here, assumes that the autonomous systems are willing to share network topology information, which I don’t think is a realistic assumption to make.

ADSL Cap Delay

Up until a few months ago, we only had 20GB bandwidth per month at my flat. This meant that we could not make heavy use of the internet during the duration of the month.

We are on a plan that caps the internet speed down 64kbit/s when you exceed the monthly limit. However there seems to be an approximately 24 hour delay between going over the bandwidth limit, and getting capped.

Lets do a little math.

  • time = 24 hours = 24 x 60 x 60 seconds
  • internet speed = 800 kb/s

24 x 60 x 60 x 800 = 69120000 kb = 65.9GB

Thats right, I can potentially more than tripple my bandwidth allowance by saving up all the large downloads until the last day of the billing month.

I tested out this theory a couple months ago and managed to download 36GB in one day. The only reason this stopped was because I ran out of things to download and I was not at home at the time 🙂

ADSL usage screenshot

New Zealand Parliamentary Streaming Site Hacked

Source: http://www.stuff.co.nz/dominion-post/politics/4145506/Hacker-hits-House-site

The New Zealand parliament TV on demand website inthehouse.co.nz was hacked over the weekend. At 8am this morning this is what the website looked like:

When I saw this, it reminded me of the time where a clients website I managed running oscommerce got hacked. A similar although different author claimed the credit. I actually kept a copy of it.

Kind of scary when the database is full of credit card numbers! There is only so much you can do though when you are using a PHP based content management system. Here is a handful of things that you can do if you are using an open source community based application.

  1. Sign up to the mailing list to hear about issues fast
  2. Run updates and apply patches as soon as possible
  3. Ensure that you have set up permissions on your scripts and directories correctly
  4. Password protect the admin directory (and rename it if possible)
  5. Use file monitoring scripts to detect hacker activity (see http://www.oscommerce.com/community/contributions,4441)

1D Clustering Problem

I have been given a large amount of anomaly annotations that goes along with a set of internet trace data.

Here is a quick scatter plot I whipped up to get an overall idea amount and spread of data that I am dealing with.

Each of the ticks of the yaxis correspond to a snort id (see http://www.snortid.com). Time is along the xaxis and covers just under 2 minutes.

It is not appropriate here to display all of this data as event icons on top of the graph. This is only showing 2 minutes of data and the ability to zoom out to view days worth at a time is also required.

In order to effectively visualise this I am going to have to look into 1-dimensional clustering algorithms. This would allow for events which occur with small amounts of relative variation to be aggregated into one graph event summary.

Now, there are quite different cluster patterns for each of the snort ID’s so I am not sure how successful I am going to be.

Sparklines using Flot

A friend from university sent me a link to a article about ‘sparklines’. As the article title states, sparklines are intense, simple, word-sized graphics.

The article:
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001OR

This visualisation style fits in well with one of the projects I am working with that displays a 2D matrix of latencies between national servers. Currently it displays a single latency value (the mean RTT) inside each cell. It would be nice if a sparkline could be placed in each of the cells to give a summary of the last two hours of latency data.

The current system in place makes use of the Flot graphing library for producing graphs so I thought I would give sparklines a go using that.

It turned out to be rather straightforward. Here is the final result:

What I did:

  1. Turned off the grid – gets rid of all borders, grid lines, ticks and tick labels.
  2. Made a series with one data point equal to the last datapoint of the main series – a trick to colour the last point red.
  3. Set the max x axis option to 1 unit more than the max datapoint – stops the red point from getting cut off.
  4. Draw the graph with the options and series.
$(document).ready(function(){
var data = [ [0, 1], [1, 2], [2, 2], [3, 2], [4, 2], [5, 3], [6, 4], [7, 2], [8, 2], [9, 3], [10, 5], [11, 5], [12, 4] ];

  var options = {
    xaxis: {
      // extend graph to fit the last point
      max: data[data.length - 1][0] + 1
    },
    grid: {
      show: false
    }
  };

  // main series
  var series = [{
    data: data,
    color: '#000000',
    lines: {
      lineWidth: 0.8
    },
    shadowSize: 0
  }];

  // colour the last point red.
  series.push({
    data: [ data[data.length - 1] ],
    points: {
     show: true,
     radius: 1,
     fillColor: '#ff0000'
    },
    color: '#ff0000'
  });

  // draw the sparkline
  var plot = $.plot('#container', series, options);
});

Scalable div appending

Ridiculously Slow – being silly

for (var i = 0; i < 65000; i++) {
  document.body.appendTo(document.createElement("div"));
}

Fast – using a DocumentFragment

var fragment = document.createDocumentFragment();

for (var i = 0; i < 65000; i++) {
  fragment.appendChild(document.createElement("div"));
}

document.body.appendChild(fragment);

Probably obvious for most, but not for me on a Friday evening after working all day. 😛

BookBook Case

I have to get myself one of these!

My flat mate mentioned yesterday that I am a typical consumer. Buying things that I just don’t need.

This is not just any ordinary item though! This is a leather MacBook Pro case that looks like an old school, weathered book. It also has a key advantage over my current case. You never have to take the laptop out of its case. It can just be left in there and unzipped (with the silk bookmark imitation zippers) and opened when required. Just like a book.

Okay, so maybe I am being ‘consumerised’ 🙂

BookBook Case homepage

Macbook Pro Case

Clickable event support for Flot

I am in the process of developing a plugin for ‘Flot’ – the pure Javascript plotting library for jQuery.

The plugin needs to add functionality to Flot that allows clickable (or hoverable) events to be overlaid on the graph. An event is a certain occurrence that occurs at a specific time or range of time.

This is what I have so far:

The event pins and tooltip currently are divs overlaid on flot’s div after it has been drawn. JQuery UI has a nice bounce feature that lets you bounce the red pins as a one liner:

$('#someDiv').effect("bounce", { times:3 }, 300);

This bounces ‘someDiv’ 3 times in the period of 300ms. I came across an issue with firefox using this code. When the div finishes bouncing, the ‘mouseenter’ and ‘mouseover’ events are again triggered. This leads to the bounce animation being repeated if you place this code inside a mouse over event as I did. Interestingly enough, this does not happen in Chrome and Safari.

Here is the workaround I used for firefox.

div.hover(
   // mouseenter
   function () {
      // check if the mouse is not already over the event
      if ($(this).data("bouncing") == false ||
          $(this).data("bouncing") == undefined) {

          // check the div is not already bouncing
            $(this).effect("bounce", { times:3 }, 300);
            $(this).data("bouncing", true);
      }
    },
    // mouseleave
    function () {
      $(this).data("bouncing", false);
    }
);

Oh the joys of cross browser support.

This is working well so far. The problem is that I need some way of visualizing many more events on the same size graph. I am not sure how I will tackle this yet. I have set up a hierarchical structure to allow only events of specific importance to be shown at one point in time. I need to reduce the size of the red pins, or more likely, look at a different way of visualizing the events.