1D Clustering Problem

I have been given a large amount of anomaly annotations that goes along with a set of internet trace data.

Here is a quick scatter plot I whipped up to get an overall idea amount and spread of data that I am dealing with.

Each of the ticks of the yaxis correspond to a snort id (see http://www.snortid.com). Time is along the xaxis and covers just under 2 minutes.

It is not appropriate here to display all of this data as event icons on top of the graph. This is only showing 2 minutes of data and the ability to zoom out to view days worth at a time is also required.

In order to effectively visualise this I am going to have to look into 1-dimensional clustering algorithms. This would allow for events which occur with small amounts of relative variation to be aggregated into one graph event summary.

Now, there are quite different cluster patterns for each of the snort ID’s so I am not sure how successful I am going to be.

Published by

oughton

Senior software engineer with extensive experience in enterprise software development targeting wholesale financial organisations and cloud solution providers.

Leave a Reply

Your email address will not be published. Required fields are marked *