So the story goes, as told by Stephen Sigler, Nature May 1989:

"The US military studied fighter planes returning from missions to try to improve their survival rate and were considering adding heavy armour to those parts of the plane that tended to show the greatest concentration of hits from enemy fire, until statisticians pointed out the fallacy of that argument. The more vulnerable parts of the plane were those with the fewest hits; planes hit there tended not to return at all. The single most vulnerable part, the pilot’s head, was without serious scar in the sample of planes that returned."

It’s easy to draw correlations from what we see in front of us. But what we see usually represents just a small part of what has happened. Focusing on the evidence we can easily see at the expense of that we can’t leads to survivorship bias. To say it another way, when Bill Gates drops out of college and starts Microsoft it might seem like dropping out is a path to success for others too, but that ignores all the dropouts who didn’t create Microsofts and consequently you didn’t hear about.

Silent evidence — the evidence that we don’t or can’t easily choose to consider — is a term from Nicholas Nassim Talleb.

When trying to discover the veracity of the planes story I enjoyed Bill Casselman’s American Mathematical Society article on The Legend of Abraham Wald. The postscript points to some of the source behind the story including a mention of Stephen Sigler’s letter in Nature quoted above.

I covered survivorship bias before but like this story so much I thought it was worth doing again.

Survivorship bias

This is a fun thought-experiment highlighting the challenges of communicating over an unreliable channel — where your message may or may not get through, and where you won’t know if it has.

The situation is that two armies, either side of a hostile valley, are trying to coordinate an attack, and they want to attack at the same time to be sure to succeed. The trouble is, to coordinate they have to send brave messengers through the hostile valley to the other camp and the messengers may or may not be captured along the way.

To coordinate the attack you have to know that your message got through which you’d know if you received a confirmation. But how would the general sending the confirmation know that their confirmation got through unless they received a confirmation of the confirmation. And on…

The two generals problem

This is military strategist John Boyd’s framework for combat operations that also turns out to apply quite well to businesses and learning in general. It’s really not that far from the ol’ test-learn cycle or a good design process.

There’s lots of nuance to his framework, but some of the application of it that I like includes:

The faster you can do this loop — at your startup or in a fight — the more stuff you will get to try and the faster you will learn

If you’re in competition with another entity then completing your OODA loop fast, and acting to change the environment for them, disrupts their loop and forces them back to observe before they can act. It slows them down and creates confusion

I learned about it from Reid Hoffman’s excellent Masters of Scale podcast.

Sketchplanations tagged with war