It takes more than computers to crack ciphers

DW: If a cipher is a means or a code to cover up plain text, at what point did we decide as people, a species, that we needed to keep certain information secret? And why do we do that?

Craig P. Bauer: People have always wanted to keep certain things secret. In the early days of writing, writing itself was sufficient for secrecy. Very few people were literate. But as soon as literacy spread, there were secrets that people wanted to keep from other literate individuals, and codes and ciphers needs to be developed. So in virtually every culture that developed writing, secret writing followed close on its heels.

But cryptography, as you describe it, is not just about writing. There is also a mathematical approach, isn't there? And you start the book with what I found to be a very entertaining exercise for the reader - the job of deciphering an extract - and you show how the process starts with a statistical analysis of language, like which are the most frequently used letters or words. So how much is this about math?

I wrote this for a general audience, but I happen to be a mathematician, so every once in a while I throw in a little math. And a mathematical approach can be very important to breaking today's modern ciphers. But it's only a relatively recent innovation. For most of the history of cryptology, it wasn't mathematicians making the cyphers, it wasn't mathematicians attacking them, and the mathematics wasn't very sophisticated. Even today you have large numbers of criminals, using old methods - methods that go back hundreds of years and which can be broken by amateurs without any special mathematical training. So I'm hoping my book inspires people outside of my own field to take a look at some of these [unsolved ciphers]. We have many examples of important ciphers that were broken with no real formal training.

A great example is the Zodiac Killer, a man who claimed many victims in California in the late 1960s.

So give us an idea of that cipher, then.

Sure. He had sent several ciphers to newspapers along with taunting letters, ridiculing the police, and it was a pair of amateurs who broke the first one. It was Donald Harden and his wife Bettye Harden. He was a history and economic teacher at a high school and he had just read a couple of books on the subject, so no special training, and it was actually his wife, likewise with no experience, who had some of the key insights. She thought the killer was egoistical, he was demanding attention with his letters, and that he would probably begin his message with the letter "I." So guessing the first letter, using this psychological approach was a key step. She also thought the word kill would appear in the message, and it did. She was able to locate that with her husband's help. And usually with a cipher, once you make a couple of key steps it unravels more and more. It becomes easier and easier with each correct step you make.

And so apart from the statistical analysis, or that some things might be chance, you do really have to get inside the head of the person who has created the code. Some might create entirely fabricated symbols, or they may be covering up a language that you don't know to begin with. So tell us more about the psychological context you might need to crack a code.

In attacking most ciphers, the context of the message is extremely important, and part of that is the psychology of the person who created the message. Breaking a cipher is a mix of science and art, and more on the art end of things is a psychological element, trying to get inside the person's head and guess some words that they might use. If you can do that and get a little bit of an opening in a cipher, that's often something you can build on, and the whole thing may quickly unravel.

So one example is a cipher made by the great British composer Edward Elgar. Looking at the context, we see that in letters to relatives and friends, he would use a lot of words he just made up or humorously misspelled in his own way. So this is very important in attacking his unsolved cipher - perhaps some of these words, which we would not find in English dictionaries, appear there. So it's always the more we can learn about the case, the better chances we have of finding out what was really written.

And how do ciphers scale up? So if we go from a relatively basic form of cipher, like a mono alphabetic substitution cipher (MASC), which you explain in the book, and where for instance the letter "A" becomes the letter "C" in the cipher, the letter "B" becomes "D" and so on, how does that scale up and get more complex.

A code maker or cryptographer comes up with something that he hopes will be hard to break. But then a code breaker or cryptanalyst finds a weak spot, exploits it, and recovers the messages. So then the person making up the cipher has to go back and patch against it. Just like software is constantly patching security vulnerabilities or other flaws, so ciphers have to be constantly patched against attacks. In a simple MASC, the attacker can see that one letter appears more often than any other. In English, German, and many other languages that letter would be "E." So that's a weak spot. The code breaker can make that substitution and he's on his way towards a decipherment. So the person creating the cipher could patch that by having several different letters replace "E" to disguise its popularity. It's a constant back and forth that has escalated tremendously in the 20th century.

I should point out that computers were basically invented to crack ciphers, and now they are used to make ciphers. So we do need this interaction of powerful computers, but also the most powerful computers we have on the planet, the human brain. And combined, this is what allows us to get at and ferret out important secrets.

Craig P. Bauer is associate professor of mathematics at York College of Pennsylvania. He is editor in chief of the journal Cryptologia and has served as a scholar in residence the US National Security Agency's Center for Cryptologic History. His book, Unsolved! The History and Mystery of the World's Greatest Ciphers from Ancient Egypt to Online Secret Societies, is published by Princeton University Press (2017).