It was a group of eccentric geniuses from Dundee, Aberdeen and Greenock who pioneered the art of hacking enemy communications for British intelligence. Neil Mackay uncovers their role at the heart of international espionage, and how a new book is bringing their remarkable story to light for the first time

THE 21st century is defined by the slippery, sinister, suspect nature of modern digital communications. We’ve had Russian hackers tinkering with Western democracies, the deliberate spread of destabilising online conspiracy theories by foreign intelligence agencies, state-sponsored cyber attacks from overseas governments, enemy spies propagating fake news on sites like Facebook and Twitter, and countries such as America running mass surveillance operations on the internet data of citizens.

We live in a paranoid era where what we read and write online is open to monitoring, exploitation and deception by spooks behind a computer screen. The shadow world of espionage is no longer one of dead letterbox drops and microfilm – it has become the realm of bits, bytes and social media.

Using communications as a weapon of war, though, is now well over a century old – this covert art is known in the spy game as Sigint, or signals intelligence. Today’s tactics and operations mostly centre on the internet, and are very different from those of the past. From the First World War to the Cold War, Sigint was once all about breaking wireless codes, tapping phones, monitoring letters, and sending out false information over the radio waves to confuse the enemy.

But who created this very secretive form of modern warfare? You might be surprised to learn that it was a bunch of eccentric and gifted Scots born and bred in the Victorian period, sons of the Empire, who created the concept of communications surveillance and founded what became GCHQ, Britain’s secret intelligence listening post, or Government Communications Headquarters, to give the organisation its formal title.

A new book, Behind the Enigma: The Authorised History Of GCHQ, reveals Scotland’s role in inventing Sigint and creating GCHQ. It is by John Ferris, professor of history at the University of Calgary in Canada, a Fellow at the Centre for Military and Strategic Studies, and an expert in war studies. His research assistant, Jock Bruce, has a simple answer when asked about Scotland’s role in Sigint: “We bloody invented the subject.”

Ferris agrees. “The Scots played a really disproportionate part,” he told The Herald on Sunday in an interview on Zoom from his home in Canada. He puts that down, in part, to the famed superiority of the Scottish education system of the late 1800s, which created the right mix of languages and technical skills needed for the dark arts of international espionage, Sigint and codebreaking.

Ferris was given unprecedented access to GCHQ’s secret archives while researching his book. He discovered that from winning the spy war against Imperial Germany to cracking the Enigma codes and hastening the end of Hitler, Scots were right at the heart of the action when it came to the secret world of Sigint.

The story of Scotland and signals intelligence begins around 1900 in the run-up to the First World War. At the time, Britain, says Ferris, had “control” of transatlantic telegraph cables, sending messages back and forth under the ocean between Europe and America. “As soon as the war breaks out,” he says, “it becomes obvious that Britain can intercept radio traffic from other countries; that Britain can try to attack that traffic through codebreaking.”

To put this new form of intelligence gathering on a professional footing, some “enthusiastic volunteers” are found. Britain establishes two codebreaking agencies: Room 40, run by the navy; and MI1b run by the army. Both are headed up by Scots, and both would lay the ground for what would eventually become GCHQ.

Room 40

In 1914, Winston Churchill was First Lord of the Admiralty and looking for someone to create a “cryptographic bureau” to spy on communications. He opted for Alfred Ewing, a scientist who was then Britain’s director of naval education. Ewing was the son of a Free Church minister from Dundee.

He had studied physics at Edinburgh University and worked on telegraph laying in Brazil as a young man. Later, he worked as a professor in Tokyo, before returning to academic life in Dundee. Back in his home city, he became something of a champion of social causes. By 1890, he was a professor at Cambridge and helped develop the steam turbine. His technical skills soon led to him being talent-spotted by the Admiralty and, after war broke out, he was appointed head of Room 40.

“Room 40 has a lot of success against German naval codes,” says Ferris. These codes had been captured early in the war by the Russians and Australians, and sent to Britain for espionage purposes. Room 40, under Ewing, quickly cracked Berlin’s “super encipherment”, and was soon attacking “German diplomatic traffic”, says Ferris, giving Britain a key insight into the Kaiser’s war strategy.

Room 40’s biggest intelligence coup was the Zimmermann telegram. In January 1917, Room 40 intercepted a message from Berlin’s foreign minister, Arthur Zimmermann, proposing a military alliance between Mexico and Germany. America was outraged and it helped pave the way for the US entering the war on Britain’s side in April.

Sigint was also monitoring attempts by Germany’s ally, the Austro-Hungarian Empire, to sue for a separate peace, as well as moves by the US to get the Germans to accept a compromise peace with Britain. It all gave the UK incredible diplomatic advantage.

Room 40 was key to the war at sea as well. Its work allowed Britain to “know more or less every day where the German navy was and what it was doing”. The Royal Navy feared German subs and mines in the North Sea, and without Room 40 the Admiralty would have been forced to send ships “out frequently on missions to try to find the enemy” – and that meant exposing sailors to danger. “Room 40 eliminates all of that,” says Ferris. The number of lives saved is incalculable.

The Battle of Jutland also showed the power of Sigint – though the lesson came in a brutal form. The German navy used an “elementary form of wireless deception” in the run-up to the battle in the hope of confusing the British over the location of its fleet. British intercept operators spotted the deception in the hours before the battle and warned the Admiralty, but the information wasn’t relayed to the British fleet. If there hadn’t been this “failure in terms of passing on intelligence”, as Ferris describes it, Jutland, which was disastrous, may have turned out rather differently. In other words, it taught military commanders to listen to their spies.

Military intelligence

IT’S a Scottish laird called Ian Malcolm Hay who ends up in charge of the army’s First World War signals intelligence unit, MI1b. Ferris describes Hay as “an amazing character”. When war breaks out, Hay enlists as an officer and is soon wounded, captured and repatriated in a prisoner exchange. Too injured to fight, his talents as a linguist see him recruited to head up military codebreaking.

“MI1b starts early in 1915 to attack diplomatic code books and by the end of the war it’s reading every code book in the world,” says Ferris. Hay’s spooks monitored US State Department communications – a key advantage, given Britain’s hope of bringing the US into the war. Both Room 40 and MI1b “are able to pick out talks between the Americans and the Germans”, Ferris explains.

At its peak, around half of MI1b’s codebreakers were Scots academics, mostly from Aberdeen University where Hay had studied theology. A bitter opponent of anti-Semitism, he would become a “British Israelite” in later life. Ferris refers to him as “an extrovert character with very different enthusiasms”. The Germans, Ferris notes, “lagged massively behind” Hay’s team when it came to Sigint. When the Germans finally start to break into low-level British communications in early 1915 their radio messages mention that they’ve cracked British systems. The irony is, though, that the British read those messages and so set up a “very successful system of cryptography which keeps the Germans contained”, says Ferris, and gives the UK “one-sided superiority”.

Blockaders

Another key asset was the War Trade Intelligence Department, with many of its staff comprised of young and gifted graduates from St Andrews. Ferris says it was something of a “tartan club”.

War Trade Intelligence, Ferris explains, handled “one billion German cables, wireless communications and letters during the war, and were able in real time to get useful intelligence out of that material which was essential for the blockade to work”.

The Royal Navy blockade wasn’t an easy task and there were some from neutral countries prepared to run it, shipping goods to countries like the Netherlands and Denmark, from where material could then be sent on to Germany.

“The blockade,” says Ferris, “really pisses the neutrals off, including the US.” Britain needs “clear evidence” that the wartime embargo is being breached if it’s to stop shipping, in order to keep neutral nations happy, and Sigint provides just that.

Instead of stopping ships randomly and infuriating neutral powers, British intelligence could pinpoint which ships were up to no good and pull them in for searching, usually around the Hebrides.

“What Sigint let’s you do,” says Ferris, “is say, ‘What are the goods on this ship, who owns them, is there anyone who owns these goods who we’ve evidence against, is there anyone on this ship who works for people shipping contraband?’ And that let’s you say, ‘Okay, we’re going to seize the following goods’.”

“What the War Trade Intelligence department did was provide hard proof in the form of intercepted cables, wireless messages and letters that ships had been sent by someone who’s involved with contraband.”

That meant countries like America “were willing to live with the blockade” as it was backed up with hard evidence. “The blockade could be a battle-axe normally,” says Ferris, “but Sigint allows you to reduce the collateral damage and make blockade more palatable for neutrals.”

.On the Western Front, Sigint didn’t really give anyone much of an advantage as most communications were by telephone, not radio. It took a while for telephone tapping to swing into action, but when it did it was the Germans who reaped the dividend. Before the Somme, Germany was able to listen in to British calls and discover the time and place of attacks. “No doubt that was part of why British losses were so heavy”, Ferris says.

Britain did, though, start intercepting communications between German spotter planes and artillery, and use that to guide British fighters to take out enemy aircraft.

Fighting Hitler

After the First World War, Room 40 and MI1b effectively merged, with Sigint now the domain of a successor agency called the Government Code and Cypher School (GC&CS). Hay from MI1b was in the running to take charge but he was a prickly character, seen by Whitehall mandarins as too determined to do things his way. The Royal Navy also wanted to make sure it ran Sigint, not the army, and Hay was an army man. So a Scot called Alastair Denniston stepped forward. He was a codebreaker from Greenock who helped set up the navy’s Room 40 with Ewing. The son of a doctor, he had studied in Paris and Bonn and was an Olympic hockey medalist.

Denniston was made operational head of GC&GS in 1919. His work would become central to the defeat of Hitler. The US cryptographer William Friedman once told Denniston’s daughter: “Your father was a great man in whose debt all English-speaking people will remain for a very long time, if not forever. That so few should know exactly what he did … is the sad part.”

Denniston began setting up a centre for breaking and reading foreign diplomatic communications. Ferris says: “He sees himself as being as much a codebreaker as a manager of codebreakers.” Perhaps Denniston’s greatest coup was the hiring of two men – Alan Turing, who broke the Nazi Enigma codes, gaining the allies access to “ultra” intelligence, the highest-level encrypted enemy communications; and Bletchley Park’s other great mathematician and codebreaker, Gordon Welchman. “That’s really one of the greatest decisions in the history of codebreaking,” says Ferris.

It’s hyperbole to describe anyone as “the man who won the war”, but Ferris says Turing and Welchman certainly hastened its end and thereby saved countless lives.

Denniston’s career doesn’t continue on such a high note though. As GC&CS becomes increasingly professionalised, he’s seen as a less than capable administrator. The Government wants codebreaking turned “from a craft to an industry” but what Denniston mostly wants to do “is sit at one of those craftsmen’s tables and do the work himself”.

Denniston was eventually sidelined for his deputy Edward Travis, who would become the first head of the organisation which we know today as GCHQ, the successor to Room 40, MI1b and GC&CS. With the end of Denniston, the ascendancy of Scots at the very top of Signit fades.

“Denniston is treated shabbily at the end of the war,” says Ferris. “He’s not given a specific honour for his work.” After he retires from codebreaking, Denniston had to become a teacher to pay the bills. “He deserves a lot more credit than he received,” Ferris adds.

Tragedy

One of the saddest of all stories relating to Scotland and the early incarnations of GCHQ is that of Malcolm Kennedy, who was born in Edinburgh in 1895. Kennedy was a spy, soldier and expert on the Japanese army. He joined GC&CS in 1935 and became the mainstay of its Japanese desk, working as a gifted translator rather than codebreaker.

In 1945, Kennedy was reading ultra-intelligence reports when he learned that the submarine his son was assigned to had been sunk in the Far East. But, as a spy, says Ferris, “Kennedy wasn’t able to even talk to his wife about it”. He had to keep the truth secret until the loss of the sub was officially announced. “It’s one of those personal tragedies that shows you how the secrecy involved in the business can be so taxing”.

Kennedy, an expert in targeting the Japanese, was outraged after the war when a US congressional investigation into Pearl Harbour began to release documents showing British and US spying efforts against Tokyo. Even the mere acknowledgement of the work of British codebreakers and Sigint, Kennedy knew, would alert enemy states to the UK’s capability and thereby limit the power of spies like him.

As the Cold War approached, there couldn’t have been a greater risk. “It did all cause the Russians to think, ‘Yes, we’re up against a serious threat’,” Ferris notes wryly.