In 1982 a rising young Whitehall official accompanied defence secretary John Nott to a hastily convened meeting with Margaret Thatcher, the prime minister. David Omand, Nott’s private secretary, carried a folder of top secret intercepts from the Government Communications Agency, GCHQ. The decrypted messages, sent by the Argentine navy, showed Leopoldo Galtieri’s junta was set to seize the Falkland Islands, British overseas territories deep in the south Atlantic. The capital Port Stanley fell a few days later.
Omand, whose later career carried him into most of the secret spaces of Britain’s deep state, including as director of GCHQ, tells the Falklands story at the opening of How Spies Think. His purpose, though, is not just to advertise the skills of the electronic eavesdroppers. The warning had come too late for the UK government to reinforce the Falklands’ defences. Earlier intelligence about Argentine signals had been misinterpreted, and Thatcher had failed to grasp Galtieri’s intentions when it still might have been possible to avert a war.
The Joint Intelligence Committee, a gathering of spooks and experts who sift and interpret secret information, had previously concluded that, in the absence of provocative action on the British side, Galtieri was unlikely to go to war to enforce Argentina’s long standing claim of sovereignty. The government was lulled into a false sense of security. As Omand puts it: “I have learned the hard way that intelligence is difficult to come by, and is always fragmentary and incomplete, and is sometimes wrong”. The seemingly magic “top secret” stamp testifies to the means of acquisition rather than the completeness of the information within.
Omand tells several such stories in this thoughtful handbook on the modern business of spookery — from the hoovering up of vast amounts of digital data by GCHQ and its American cousin the National Security Agency to the more conventional gathering of secrets by officers and agents of the Secret Intelligence Service, or MI6. He is clear about both the spies’ essential purpose — to forearm government as to the intentions of enemies and provide an important edge in the event of hostilities — and about its limitations.
The explosion of information in the digital age, with social networks offering adversaries channels of information that intersect and overlap with those of ordinary citizens, has made the agencies’ task at once more complex and politically sensitive. When does the search for the enemy demand surveillance of the innocent?
The fragments of information that arrive on the desks of ministers via electronic intercepts or dead-letter drops have to be pieced together — and then carefully interpreted. Success does not belong to the James Bond-types or to the boffins operating GCHQ’s vast computers. Rather it depends on proper analysis of the information they provide.
For all the occasional failures, Britain has a deserved reputation for being good at this stuff. Intelligence is an area where, in the phrase of a former foreign secretary, Douglas Hurd, the country punches above its weight in the world. GCHQ has earned the serious respect of the NSA and partners in the other three nations — Canada, Australia and New Zealand — which comprise the so-called “Five Eyes” intelligence consortium.
It wasn’t always like this. For a while, John Ferris tells us in Behind The Enigma, spying was regarded as bad form. As politician Thomas Macaulay said in the first half of the 19th century, what difference was there between “breaking the seal of his letter in the post office” and “employing a spy to poke his ear to the keyhole”? This was before the English discovered how much was to be learnt from kneeling at keyholes.
The present UK government is in the midst of a national stock take of the assets, economic and military, that it can deploy to prop up the nation’s global influence in the wake of departure from the EU. GCHQ, SIS and its domestic counterpart MI5, are assured of favourable treatment in a grandly named National Security Strategy and Strategic Defence Review. Britain is no longer a front rank power, but its spy agencies can still count themselves world class.
They are well-practised. GCHQ, previously the Government Code and Cypher school, recently celebrated its centenary, tracing its eavesdropping back to the days of the famous Zimmerman telegram in 1917. This was the intercepted message from the German foreign minister inviting Mexico to sign up with Berlin should America enter the war. The intercept itself — a telegram grabbed from an undersea cable running across the Atlantic — tells only half the story. The real coup lay in the use of the information. The judicious leaking of Zimmerman’s offer stirred anti-German outrage in Washington. Within a month the US had joined the war just as Berlin had feared.
Behind the enigma
John Ferris’s deep dive into the archives and algorithms of signals and communications intelligence traces the growth of GCHQ to the industrial-scale collection of data
how spies think
As author David Omand puts it: ‘I have learned the hard way that intelligence is difficult to come by, and is always fragmentary and incomplete, and is sometimes wrong’
Omand documents other triumphs — among them the treasure trove of secrets provided by Oleg Gordievsky, a KGB double agent, during the early 1980s. These included much more than invaluable information on Soviet spy networks and military dispositions. Gordievsky provided his SIS spymasters with a window on to the thinking of the Kremlin leadership. The west learnt that Moscow’s fears of a “first strike” by the US and Nato, and the concomitant risk it might seek to pre-empt it, were dangerously real. Thatcher was prompted to tone down her Iron Lady rhetoric and build a relationship with the Soviet leader Mikhail Gorbachev. Encouraged by the same intelligence, US president Ronald Reagan successfully pressed Gorbachev to agree to nuclear weapons reductions.
Omand’s lessons, however, are best expressed in the failures and half-failures — the most notorious about Saddam Hussein’s supposed weapons of mass destruction in the run-up to the second Iraq war. The first mistake was a disinclination to look at the world through Saddam’s end of the telescope. Doing so might have revealed Saddam’s bravado and told them that he believed his domestic authority in Iraq rested on a pretence that he still had WMD.
The second error saw the spooks carried along by groupthink — the consensus thinking that selects only partial evidence and locks out serious challenge. The intelligence from the Iraqi scientist codenamed Curveball was often questioned, but never sufficiently strongly to overturn the prevailing assumption that Saddam continued to defy the UN.
The Canadian historian John Ferris picks up Omand’s Falklands story in his official history of GCHQ — and goes on to show how the later intercepts it supplied were critical in the success of the British task force. The war was a closer run thing than often presented, and the information gleaned from Argentine signals and radar gave Britain’s commanders a vital edge in naval battles fought 8,000 miles from their base.
While Omand writes about the uses and misuses of intelligence, Ferris digs deep into the archives and algorithms of signals and communications intelligence. He traces the growth of GCHQ from a group of boffins in Room 40 of the Admiralty who cracked the Zimmerman telegram to the industrial-scale collection and mining of data that defines its present role as shield against the nation’s enemies.
It is a fascinating tale, if one sometimes told by Ferris with too much dense technical detail. It takes us with the codebreakers — mathematicians, linguists, teachers and philosophers and eccentrics — through the ages of radio, telegrams, telephone and satellites to the digital present. For all the flamboyance of its doughnut-shaped headquarters in genteel Cheltenham, GCHQ was long the most secretive as well as the fastest-growing of the three intelligence services. The great success of the Bletchley Park operation in cracking the German code machine Enigma during the second world war and the larger-than-life brilliance of mathematicians and cryptographers such as Alan Turing were stories well told. Beyond that, however, the shutters stayed down.
Any residual hopes the agency might remain in the shadows, however, were blown up by the American whistleblower Edward Snowden. Among the vast cache of top secret files released by the former contractor at the NSA were great numbers of private emails and other digital messages hoovered up by the operators at GCHQ.
The public learnt first hand that electronic snooping in the digital age involves a lot more than tapping phones of potential adversaries. In an era of jihadi terrorism, international crime networks, state-sponsored theft of intellectual property and undeclared cyber wars, GCHQ was collecting the private communications of the innocent in order to track the messages of enemies. In the minds of many, it had become an agency of “the surveillance state”.
The debate that followed exposed fault on both sides. The charge of an Orwellian conspiracy fell down with the evidence that most of the data was never examined. GCHQ’s task was to find needles in haystacks but it first had to collect the hay. On the other hand, the procedures to ensure that the agency searches were rigorously confined to those who would do the nation ill were patently too lax.
The furore prompted GCHQ to join MI6 and MI5 in commissioning an authorised history. Ferris was given access to many — though far from all — of the secret files and his narrative does justice to the role that the agency plays in safeguarding national security. Among all the technical detail, what’s missing, sadly, is much of an insight into the motives and emotions of the cryptologists and codebreakers — the personalities, the excitements and the disappointments as they join battle with Chinese cyber warriors, Russian subversion units and Islamist terror groups.
And, of course, the rest of us will never know the whole story. For as much as the agencies have stepped out of the shadows, they will always keep some of their secrets. But then that goes a long way to explain our enduring fascination with spooks.
Behind the Enigma: The Authorised History of GCHQ, Britain’s Secret Cyber Intelligence Agency, by John Ferris, Bloomsbury, RRP£30, 848 pages
How Spies Think: 10 Lessons in Intelligence, by David Omand, Penguin, RRP£20, 352 pages
Philip Stephens is the director of the FT’s editorial board
Join our online book group on Facebook at FT Books Café
Get alerts on Non-Fiction when a new story is published