Hell if I know if anyone is going to bother reading this but I felt it was something I Needed to say.


TLDR: Mass surveillance in the form of logging everyone’s internet activity seems like a very expensive very bad idea that’s not likely to be as useful as our government wants us to believe.


Lately a lot of Governments seem to have become obsessed with mass surveillance, well honestly they probably always were but in light of the Snowden leaks it’s just now more people know about it.

The latest craziness from the UK government is the idea of getting ISPs to log every internet users internet usage their main argument is it will help them find missing children, catch paedophiles and stop terrorist attacks (Also I just goggled the word “terrorist” because I can’t spell bet that set alarm bells off at GCHQ).  The thing is lets presume for a moment that government get their way and every “connection” is logged, I’m not actually sure how helpful this would be.

There are also several problems with this proposal some technical some not, for a start there seems to be a misconception that the internet works like the telephone system it does not, in a traditional circuit switched voice system (So lets ignore VOIP for the moment) when you make a call a “path” though the network is created, essentially a timeslot on a transmission system is allocated to that call and will remain yours until you finish the call.  Therefore one phone call, one conversation one record, Easy right?

I’m going to attempt to explain the difference between a traditional phonecall and the internet here, if there are any network engineers reading this that have a better understanding of the protocols involved than I do please DO feel free to correct me on any mistakes I’ve made.

The internet works on packets, everything you send/receive is broken down into little packets of information (usually no bigger than 1500bytes or about 12000 bits which in turn is 12 Kilobits assuming I haven’t failed at math) each one of these little packets has to contain information about the source and destination protocol .etc as well as the actual data transferred.

There’s also 2 main protocols in use over the internet, TCP and UDP.  TCP is a stateful protocol this essentially means there is handshake and session established and then traffic to do with that session. However sessions will be created/torn down as needed,  UDP however is stateless which means there is no such session, it’s mostly just fire and forget which tends to be used in situations where you need to get the data out quickly and don’t care if some gets lost along the way, arrives in the wrong order.etc (Games and VOIP for example use this).

Right you may wonder why I went to the trouble of trying at least to explain that, well it comes down to this, remember I said earlier that a packet tended to be maximum of around 1500 Bytes or 12 Kilobits well as you’ve probably already worked out 12 Kilobits isn’t a lot and since we’re not able to send more than 1500bytes in a single packet over the internet (there are some exceptions but I won’t go into this) there’s only one way to send more than 12 Kilobits in a second is if we send more than one packet in a second.

So for example if I’m downloading something at 70Mbit/s (The speed of a good FTTC line) I’m, probably pushing at minimum around 6000 packets per second just on that one download alone and that’s ignoring all the other traffic I’m sending/receiving.

Then there’s gaming where I’ll be sending lots of smaller packets, 10K packets per second is quite possible to hit with a few gamers sharing an internet connection.

The point I’m trying to make is one connection can generate many 10’s of thousands of packets per second easily, carrier routers have special ASIC (Application specific integrated circuits) that are really good at looking up the destination of these packets and sending them on their way, that’s actually fairly easy.

However in order to do what the government want the ISP is now going to have to examine each one of these packets and log the source/destination at the very least, except the ISP isn’t dealing with just the thousands of packets per second coming from MY connection they’re dealing with thousands, if not millions of customers each sending potentially a large number of packets per second equating into billions of packets every second that they’d have to monitor and log.  That would have to be one hell of a database, costing many tens of millions of pounds consuming vast amounts of computing power.

Assuming however the ISP’s managed to overcome that problem and the government do get their wish, everyone’s connection records are stored.  Great you have records of almost every internet connection in the country, but all that tells you is what was done on that connection not necessarily who did it.  On top of that there’s then the problem of trying to separate what the user is doing from what the computers themselves are doing, there will be billions of Internet connection records created that are nothing more than machines communicating with machines such as your computer checking to see if there are any updates to install, facebook/whatsapp.etc on your phone checking in to see if there are any new messages.  That dodgy site you visited was that really you or is your machine infected with malware, did someone hack your Wi-Fi.etc.

Then there are things like DDOS attacks where computers (usually machines infected with malware) are used to send floods of junk data this would also create internet connection records so even more for the monitoring equipment to handle.

There is the potential to collect VAST amounts of data on everyone but the real question is if they do manage this are they actually going to gleam anything useful from it or will it just be a case of looking for the needle in the worlds biggest haystack.  Short of having massively powerful computers searching it constantly for patterns the only other time it’s going to possibly be useful is if you know who your target is, at which point does knowing they were on Facebook.etc actually really tell you much that’s actually of any use I ask?.

The final point I’d like to raise is should such a database exist containing everyone’s internet connection records, data on every interaction you’ve ever made on the internet, well it’s not only the government that will be trying to get their hands on that juicy info.  If those databases ever leaked then it would be a goldmine for criminals, suddenly they have far more insight into their victims and their habits.  Been looking at travel websites lately great they know that sometime soon you’re going to be away for an extended period.

  • Sean

    if there are any network engineers reading this that have a better understanding of the protocols involved than I do please DO feel free to correct me on any mistakes I’ve made.

    s/The internet works on packets/The internet works on bacon/