Red Sox as an illustration of Bayesian Probability Theory

DaveWill the Red Sox win the World Series this year?  What are their chances?

What are their chances of going all the way, if they win their first game of the playoffs this Friday?

Aha!  I have a chance to apply Bayesian probability theory!

I recently reviewed Nate Silver’s excellent The Signal and The Noise: Why So Many Predictions Fail – But Some Don’t , which at its core, advocates we adopt Bayesian probability methods for forecasting complex events.  Like Red Sox World Series championships.

Nate Silver’s Big Idea

Silver’s big idea is for us to move away from “I have the explanation and I know what’s going to happen,” to a different way of understanding the world characterized by “I can articulate a range of outcomes and attach meaningful probabilities to the possible outcomes.”

Bayesian probability

Bayes’ theory, Silver explains, helps us come up with the most accurate probability of some event occurring.  Fortunately, it’s not too complicated.

The Red Sox, of course, defy all probabilities

As we approach the MLB playoffs I’m fully aware of the irony of applying rational Bayesian probability to something as totally irrational, magical, and unlikely as Red Sox playoff outcomes.

My childhood and young adulthood consisted of them repeatedly snatching defeat from the jaws of victory.  Both the Game Six World Series loss in 1986 to the Mets and the 2003 ALCS loss to the Yankees[1] defied all semblance of probability – we didn’t need a mathematical theorem to tell us that.

At the time, all we knew was that God personally intervened in baseball outcomes and that she enjoyed torturing us.  And we hoped that God had plans for our redemption, some day.

We know now that, like the biblical story of Job, Red Sox Nation suffered for a reason.  We now own the Greatest Sports Victory of All Time, coming back impossibly from devastating losses in the first 3 ALCS games in 2004 to vanquish the Yankees and sweep the Cardinals.[2]  No sports victory has ever been as sweet as that.  It was all so improbable.  No math could ever explain that magic.

Greatest Sports Moments Ever, reduced to probabilities
Greatest Sports Moments Ever, reduced to probabilities

And yet, I insist we try to learn Bayesian Probability today

Fine then.

To use it, we need to define three known (or assumed) variables, in order to come up with a fourth, unknown variable, which is the thing we want to know, the probability of an event.

The known or assumed variables will be:

  1. X = an initial estimate of the likelihood of an event.  This is called a ‘prior’ since it’s our best guess of some probability prior to further investigation.  Before the playoffs even begin, how likely are the Red Sox to become World Series Champions?
  2. Y = The probability that if some condition is met, the event will happen.  In other words, how probable is it that a team that won the World Series had originally won their first game of the playoffs?
  3. Z = The probability that if that same condition is met, the event will not happen.  For a team that did not win the World Series, how probable is it that they won their first game of the playoffs?

 

The unknown variable, what we’re trying to determine, is our closest approximation of the probability of the event happening.

4. I’ll call that unknown variable V.  What is the probability of the Red Sox winning the World Series, if they win their first game on Friday?

The math formula of Bayes’ theorem, using these four variables, is:

V = (X*Y)/(X*Y + Z(1-X))

I understand that formula makes no sense in the abstract, so that’s why we’ll illustrate it with the Red Sox.

We need an example using numbers, please

Since it’s that time of year, I’ll ask the key question on everyone’s mind right now:

If they win on Friday, October 4th – their first game of the playoffs, will the Boston Red Sox go all the way on to win the World Series?

We can now define variables and assign probabilites

The variable V (This is the unknown what we’re trying to solve for)

V is the probability that the Red Sox win the World Series this year, if they win their first game of the playoffs.

 

Variable X, our prior

I will make our prior –the initial estimate for the Red Sox winning the World Series – 15%.  If all 8 playoff teams had an equal chance of winning the World Series my prior would be 12.5%, the percent equivalent of 1 divided by 8.  But given that the Sox had the best record in baseball this year – and they have studs like Big Papi and Pedroia – I have to boost their prior to 15%.

 

Variable Y, the conditional probability that the hypothesis is true

One of the requirements for using Bayesian probability theory is that we insert a conditional probability. We can simply express this hypothesis as “If this happens, this other thing is made more likely.”

In our example I’ll make the non-crazy hypothesis that there is some positive causal relationship between teams winning their first game of the playoffs and teams that eventually win the entire World Series.

Let’s assume we know, from historical data,[3] that teams that won the World Series had previously won their first game of the playoffs 58% of the time.  That’s our variable Y.

 

Variable Z, the false hypothesis variable

The false hypothesis variable in this example would be made from the 7 of 8 teams that historically begin the playoffs but do not go on to win the World Series.  Of these non-champions, what is the probability they won their first game?  I’ll estimate this at 45%[4]

Putting it all together

Using Bayes Theorem, we can now revise our estimate of the Red Sox winning the World Series, after the first playoff game has been played.

If the Red Sox win on October 4th, we can plug in variables X, Y and Z to determine the new probability of a glorious Red Sox World Series victory, variable V.

Remember: V = X*Y / (X*Y + Z*(1-X))

Plugging in our known and assumed probabilities, we get the

following math:

V = (15% * 58%) / ((15% * 58%) + (45%*(100%-15%)))

Solving that in an Excel Spreadsheet we get

V = 18.5%

Summed up, if the Red Sox win their first game Friday[5], we would revise our probability of them winning the World Series up to 18.5% from 15%.

Intuitively, this makes some sense.  There should be only a modest increase in the probability of a World Series championship after one game.

There’s a small positive correlation between winning the first game in the playoffs and eventually winning the World Series.

But even if it’s a blowout one way or another, let’s not get carried away.  The chances of them going all the way is only up to 18.5%.

bloody sock
Martyrdom & bloody sacrifice go beyond rational thought

Anchoring effect of priors

We should note, and Silver emphasizes, that the anchoring effect of priors greatly influences our updated probabilities.  In plainer English, our starting point for how we think the Red Sox are likely to do limits our ending point.

If we start with a prior that the Red Sox only have a 5% chance of winning the World Series, then their chances of winning the championship only jump to 6.3% after taking the first game, using my same assumed inputs.

Again using the same assumptions, if the Red Sox were 75% favorites to win it all, then a first game victory pushes them up to 79.5% favorites using the Bayesian Theorem.

Next Steps

If we want to follow the rest of the Red Sox playoff outcomes probabilistically, we’d take our revised prior – let’s say 18.5% after Game One – and come up with updated probabilities for variables Y and Z for Game Two.  To use new Y and Z variables effectively we would need new historical data to determine the conditional probability of a World Series victory based on Game Two results.

Continued iteration

Nate Silver would advocate applying this constant iteration, revising our probabilities and priors as new information arrives, for a wide range of complex phenomenon that defy prediction.  Will Mike Napoli’s beard change weather patterns inside Fenway?  Is it not Nate Silver, but rather Big Papi who is the witch? Will super-agent Scott Boras release a karma-bomb press release on another client like he did with A-Rod during the 2007 World Series, effectively marking the beginning of the end for A-Rod?  The probabilities change as the events unfurl.

Or not

Or conversely, we could just ignore all math, attach ourselves to one big idea, and never let go.

Because unrevised big beliefs, like sports fandom, do have their attractions.

Please see related post Book Review of The Signal and the Noise by Nate Silver

 


[1] Fie on you New York! Shaking my fist.  Arggh!

[2] Incidentally, that 53 minute 30-for30 video of “the Greatest Sports Victory of All Time” I linked to on Youtube is totally awesome.  Gives me the chils.

[3] I’m not a baseball stats geek with easy access to this kind of data, so I’m just making up numbers for the sake of illustration.

[4] Again, a stats geek could come up with the correct historical data to suggest a more accurate probability for the false hypothesis, but just work with me here a little bit on my completely made up numbers.

[5] And of course if my numbers were based on real data, rather than just picked out of the clear blue sky.

Post read (12362) times.

Book Review: The Signal And The Noise by Nate Silver


I took a mandatory course in high school[1] called “Theory of Knowledge,” meant to help us consider ‘How do we know things?”

“How do we know things?” turns out to be one of those big philosophical questions – dating from the time of Plato & Aristotle – irritating all of us for the last few millenia.

What Nate Silver addresses more than anything in The Signal and The Noise: Why So Many Prediction Fail – But Some Don’t  is how we know things – in particular how we use and misuse information to understand and make predictions about complex phenomena such as baseball performance, political outcomes, the weather, earthquakes, terrorist attacks, chess, Texas Hold ‘em poker, climate change, the spread of infectious diseases, and financial markets.

I’ve written before that it’s Nate Silver’s world, and we just live in it.[2] The Signal and The Noise offers a 21st Century answer to the question of ‘how do we know things?’  Because most of us, and most media, do not yet think this way, Silver implicitly criticizes everything I hate about the Financial Infotainment Industrial Complex.

Big Ideas vs. Small Ideas

Silver argues effectively that we frequently go wrong in many areas when we adopt a single model or approach to a problem, when an evolving, flexible, multiple-input, probabilistic approach would serve us better.

The problem of political pundits

Silver repeatedly returns in The Signal and the Noise to criticize political pundits on a TV show called The McLaughlin Group, on which commentators from the left and the right appear to make bold political predictions.  Silver – among the most widely admired public forecasters of political outcomes – eviscerates this type of ‘prediction,’ citing data that shows these commentators make accurate predictions no more often than would a random coin toss.

But television rewards ‘bold stances’ and ‘big ideas’ of the type The McLaughlin Group traffics in, while largely ignoring more thoughtful approaches.

Silver labels and criticizes the “Big Idea” mindset that passes for political commentary on television in favor of a more modest, probabilistic, and empirical “Small Idea” mindset.  Small ideas, nuanced, uncertain, and modest, however, make for poor television ratings.

But Silver does have a Big Idea himself

For complex, hard to predict phenomena[3], Silver explains his preferred method, based on a probability theorem attributed to an 18th Century English minister Thomas Bayes.

No doubt Silver thinks many more of us should become familiar with this branch of probability and statistics math. [4]

Beyond the Bayesian theory, however, Silver encourages us to adopt a probabilistic world-view.   His big idea is for us to move away from “I have the explanation and I know what’s going to happen,” to a different way of understanding the world characterized by “I can articulate a range of outcomes and attach meaningful probabilities to the possible outcomes.”

Over time, as we refine our data gathering and multifaceted models, we can move our small ideas forward and become ‘less wrong’ about the world.

In the investment world the former style of traders – the one’s with big ideas and certainty – may have a good run of success, but generally get flushed out when markets turn.  The best traders I’ve ever worked with think and speak in the latter way, considering new possibilities as markets evolve.

Some parts of this remind me of Nassim Taleb

The habits of mind Silver’s book encourages are not dissimilar to Nassim Taleb’s empirical skepticism, although they differ greatly in style and in points of emphasis.  Taleb tends to be aggressively critical of everybody else’s models, whereas Silver more generously praises other theorists’ models and critiques his own.

Both Taleb and Silver share, however, a restless dissatisfaction with the inputs into our understanding right now.  Both would say we do not know enough. We have not considered enough factors to explain whatever phenomenon we purport to explain. Our models need improvement and perpetual skepticism.  The best we can do is to think probabilistically about future events.

Both encourage a learned humility about what we can know or patterns we think we observe in the world.

How does this relate to investing?

I’d estimate only about ten percent of Silver’s book explicitly addresses investing.  As I mentioned, The Signal And The Noise is really a “Theory of Knowledge” book rather than in investing book.

But because Silver thinks like the best financial traders, uses probabilistic math effectively, and writes more clearly than almost anyone, his ideas are worth applying to investing.

1. Attribution of success

Among people who invest their own or other people’s money, 99.5%[5] of us attribute successful outcomes to personal investing acumen, while attributing unsuccessful outcomes to circumstances beyond our control.

The noise surrounding our own success – misinterpreting a generally rising market as stock-picking skill for example – leads us to overestimate our ability to influence investment returns.  As a result, too many of us engage in security selection, or too many of us pay others to achieve superior investment results, despite the evidence that we’re overpaying.

2. Responsibility for failure

Conversely, our abdication of personal responsibility for losses – it must have been ‘the bad markets’ after all! – leads us to underestimate our own errors of judgment.

In both cases – success or failure – we’re prone to adopt an uncritical approach to the right level of responsibility for outcomes.

3. Efficient market hypothesis as an illustration of the Bayesian approach

Although Silver gives numerous examples of his Bayesian probabilistic approach to problems with numbers, one of his best examples is purely textual, on the efficient market hypothesis.  He lists seven increasingly accurate, yet also qualified and probabilistic statements, on what we know about efficient markets.

The series of increasingly accurate, yet ‘less bold,’ statements are not only a great illustration of his big idea but also the right lesson for us on investing, so I reproduce it in full here:

a)     No investor can beat the stock market.[6]

b)     No investor can beat the stock market over the long run.[7]

c)      No investor can beat the stock market over the long run relative to his level of risk.[8]

d)     No investor can beat the stock market over the long run relative to his level of risk and accounting for transaction costs.[9]

e)     No investor can beat the stock market over the long run relative to his level of risk and accounting for his transaction costs, unless he has inside information[10]

f)       Few investors can beat the stock market over the long run relative to their level of risk and accounting for their transaction costs, unless they have inside information[11]

g)     It is hard to tell how many investors beat the stock market over the long run, because the data is very noisy, but we know that most cannot relative to their level of risk, since trading produces no net excess return but entails transaction costs, so unless you have inside information, you are probably better off investing in an index fund.[12]

The first approximation – the unqualified statement that no investor can beat the stock market – seems to be extremely powerful.  By the time we get to the last one, which is full of expressions of uncertainty, we have nothing that would fit on a bumper sticker.  But it is also a more complete description of the objective world.

If you want a 21st Century theory of knowledge, teaching you ‘how to think’ about the major world problems of global warming, financial crashes, avian flu, and terrorism, as well as ephemera like poker, chess, sports betting and baseball, start with The Signal and The Noise: Why So Many Prediction Fail – But Some Don’t  by Nate Silver.

Please also see related post on Bayesian Probability and the Red Sox.

Please also see related post All Bankers Anonymous Book Reviews in one place!

The Signal and The Noise
The Signal and The Noise

 


[1] Readers who study at an International Baccalaureate (IB) high school will be familiar with the “Theory of Knowledge” course.  It’s a really great idea for a course, but I have yet to meet anyone who thought the experience of the course lived up to the idea that inspired it.

[3] Each chapter separately tackles baseball, political forecasting, weather, earthquakes, economic growth, infectious disease growth, sports betting, chess, poker, climate change, and terrorism – each in their own way posing a challenge of seeing into the future.

[4] The mathematics of Bayesian probability is relatively straightforward so I think I’ll try in a subsequent post to do it justice.

[5] I rounded down to be conservative, because that’s just good science.

[6] The original, powerful, efficient market thesis

[7] Because, clearly, some people sometimes do, for some period of time

[8] You can take some crazy stock-market risks and WAY outperform boring stodgy stocks much of the time.  We have to match up comparable investment risk levels.

[9] A theoretical ‘market-beating’ high volume trading strategy often looks less market-beating when you take into account the frictions of trading.

[10] Inside information sure is helpful, when trying to beat the market

[11] Maybe some can do it, like Warren Buffett, but it’s super rare.  Probably you can’t do it.

[12] So carefully hedged!  So qualified and full of doubts!  So true!

Post read (11238) times.