Big Data – Does Size Matter?
by Timandra Harkness
Bloomsbury, £16.99
Review by Iain Macwhirter
Like most journalists I am both fascinated and repelled by big data. On the one hand it seems amazing that Google can predict the spread of disease by examining millions of search questions. On the other, it's infuriating to get adverts for toilets all over my web pages because last month I was researching bathroom fittings.
There is a distinct whiff of snake oil about big data. Nate Silver, the big data psephologist, claimed it was statistically impossible for Scotland to vote Yes in the independence referendum. Well, it clearly wasn't because the result was a close shave for the Union. Big data is all about making big claims and hoping no one notices when they are wrong.
Despite its strapline, Timandra Harkness's book is not a patronising explanation of basic statistics for the numerically illiterate. Harkness raises some very big questions indeed, not just about the grandiose claims of the big data evangelists, but also about how in the age of universal surveillance we can defend the concept of privacy.
Big data is the huge sets of statistical information about us stacking up on the internet in social media, online search, government agencies, banks, insurance companies. It is now routinely collected, stored, shared, linked together and analysed, sometimes in a highly manipulative way to seduce us into buying stuff.
Millions of taxi rides in New York City, according to Harkness, have been analysed so big data analysts can collate this with information about the passengers' social backgrounds to draw conclusions about who pays the best tips. Allegedly. For this big data thing is always just a little suspect. Most tips, for example, are not recorded because they are made in cash.
I find it objectionable that computer nerds are allowed to track my taxi destinations, but perhaps that’s just me. Since our mobile phones are automatic tracking devices, anonymity is on a hiding to nothing. US police now have spy drones that can record what people say on city streets miles away.
Big data really is big surveillance, and officialdom loves it. Harkness reports that Glasgow is very big on the big data. Its Future Cities programme involves a new network of 400 digital cameras in public places, connected to traffic cameras. One control centre can now survey the entire city, controlling traffic flow, spotting disturbance and tracking suspected terrorists through cross referencing information coming from smart phones, criminal records, facial recognition, and number plates.
As Harkness says, this is an engine of social surveillance and control, straight out of Orwell. Who agreed to all this? Where was the public debate over this invasion of privacy? Big data really is the bastard child of Big Brother.
In America they call big data surveillance Predpol – or “predictive policing”. This is the conceit that law enforcement can predict where crime is going to take place. Algorithms search for areas where more crime is expected, based on criminal records, social and ethnic profiles, social media etc, and police are sent to saturate the area.
The problem here, as Harkness points out, is that Predpol is self-fulfilling policing. More police means more crime because the officers will pick up on offences that would otherwise go unrecorded in areas where there are fewer police to record them. It also focusses police effort in ethnic areas, causing resentment. Predictive policing reinvents racial profiling, one of the oldest prejudices of law enforcement, in digital guise.
Algorithms used by recruitment firms are also highly suspect, having ethnic and class assumptions built in to them. Machine intelligence tries to assess how good a job candidate is likely to be, not just on their grades, but on whether they associate with successful people on LinkedIn or Facebook. This supposedly “objective” assessment again reproduces a very old social bias.
According to Harkness, big data is even being used in America to determine sentencing. If the machine says a convicted criminal is more likely to reoffend, using these prejudicial algorithms, they get longer sentences. Harkness calls for “algorithmic accountability” so that the unaddressed assumptions in machine intelligence can be exposed.
Of course big data is useful for social research, but bureaucrats and police love it because they think it allows them to see into our private lives. God help us when the SNP's Named Persons get their hands on big data and start predicting where the next child abuse case is going to take place.
Why are you making commenting on The Herald only available to subscribers?
It should have been a safe space for informed debate, somewhere for readers to discuss issues around the biggest stories of the day, but all too often the below the line comments on most websites have become bogged down by off-topic discussions and abuse.
heraldscotland.com is tackling this problem by allowing only subscribers to comment.
We are doing this to improve the experience for our loyal readers and we believe it will reduce the ability of trolls and troublemakers, who occasionally find their way onto our site, to abuse our journalists and readers. We also hope it will help the comments section fulfil its promise as a part of Scotland's conversation with itself.
We are lucky at The Herald. We are read by an informed, educated readership who can add their knowledge and insights to our stories.
That is invaluable.
We are making the subscriber-only change to support our valued readers, who tell us they don't want the site cluttered up with irrelevant comments, untruths and abuse.
In the past, the journalist’s job was to collect and distribute information to the audience. Technology means that readers can shape a discussion. We look forward to hearing from you on heraldscotland.com
Comments & Moderation
Readers’ comments: You are personally liable for the content of any comments you upload to this website, so please act responsibly. We do not pre-moderate or monitor readers’ comments appearing on our websites, but we do post-moderate in response to complaints we receive or otherwise when a potential problem comes to our attention. You can make a complaint by using the ‘report this post’ link . We may then apply our discretion under the user terms to amend or delete comments.
Post moderation is undertaken full-time 9am-6pm on weekdays, and on a part-time basis outwith those hours.
Read the rules hereLast Updated:
Report this comment Cancel