Book Review: On The Edge, by Nate Silver

Nate Silver published On The Edge: The Art of Risking Everything on August 13th and, well, I knew I had to jump it on the top of my reading list. 

On_The_Edge

Silver is best known for creating the election-forecasting blog FiveThirtyEight, which he reconstituted on Substack in 2024 as The Silver Bulletin

For newspaper-reading political obsessives like me, it’s Nate Silver’s world between now and November 5th, and we’re all just living in it. He aggregates and weights polling data, inputs other calibrated factors into his model, and suggests a probabilistic approach to electoral college results. 

In his new book, Silver gives coherence – and the name “The River”  – to a community of practitioners and thinkers who I admire, and who I try to channel in my own writing. 

Silver’s thesis is that there is a particularly successful and newly salient group of people from a variety of walks of life who share a common epistemology. 

Epistemology is the 50-cent word for a theory of knowledge or a way of thinking. 

The River

So this group, the River, primarily shares a way of thinking about risk. 

Among other things, they think probabilistically about risks in rational and objective ways, rather than emotional or traditional ways. They compare the probability of success versus the size of the rewards. They specifically seek to take risks when there is positive expected value – where the size of the reward is big enough to overcompensate for the probability of failure. They are competitive. They are strategically empathetic, by which he means they try to see how the other side of a contest is thinking. They update their views when new data comes in. They try to not be overly wedded to one world-view or one model of how things work. They can be contrarian in the face of societal consensus. 

The meat of the book is derived from interviews and observations of people who share this epistemology from the worlds of technology, private equity, trading, gambling, cryptocurrency and artificial intelligence. Silver is a member of The River, so he’s eager to explain the advantages of this method, as it serves him and others well when investing, gambling, sports-betting, election-model building or other risky endeavors. He’s also a careful journalist and writer, so he sifts through – especially in later chapters – how this type of thought can go wrong for individuals or the world.

Nate Silver

For an example of the latter, you get in this book very close-up conversations with Sam Bankman-Fried before, during and after his spectacular cryptocurrency rise with FTX, and his subsequent fall and fraud conviction

You also get the most in-depth explanation of the rise of artificial intelligence I have read to date, including an attempt at a technical explanation of how large language models (LLMs) like ChatGPT work for a non-technical reader like myself. You’ll get far more poker history and lore and strategy than you’ll ever need, as well as the methods and thought process of a sports better.

My Own Retrospective Guide

Narrative, connectivity, identity, justice, and status-quo pattern recognition, are examples of other useful intellectual techniques common in academia, government, and journalism. They also may be at odds with the hyper-rational, probabilistic, contrarian risk-orientation of The River.

What I hadn’t expected is that Silver’s new book would provide a kind of retrospective guide to my own mental aspirations when writing this column. I naturally gravitate to stories about practitioners from The River, probably because I think it’s a great corrective to the typical epistemology of traditional journalism. 

While there are quite a few members of The River and an extensive philosophical tradition – as explained in detail throughout On The Edge – the vast majority of us do not apply enough of these thought processes.

Silver dedicates two chapters to the rise of artificial intelligence, and especially the worries of leading rationalists like Eliezer Yudkowsky who see an existential threat from AI, something I became alarmed about last year

Silver is a major advocate for prediction markets like Manifold, Polymarket and PredictIt, which allow the collective bets of crowds on outcomes in a probabilistic manner, and with which I’ve become obsessed in the past few months.

The River’s way of thinking informs my view of why retail options trading is not likely to be profitable in the long run. 

My views on the organization GiveDirectly – which attempts to bring a rational and probabilistic mindset to philanthropy – stems from this same impulse. 

The Recent Texas Lotto Example

This one didn’t come from Silver’s book, but an excellent Hearst investigation of a lottery scheme in Texas is one of the best recent examples of River vs. non-River thinking from the Lone Star State.

The most commonplace piece of personal finance wisdom is to never buy lottery tickets. And this is true, you should not, precisely because the “expected value” of every lottery ticket you’ve ever bought is less than the price you pay. The more tickets you buy, the more you will lose over time, like any other game of chance at the casino. This is Expected Value 101.

On the other hand, if there were a theoretical lottery game in which the payout had a positive expected value, then you should play that lotto. In the real world this is extremely rare, and requires specific circumstances and some sophisticated techniques.

The investors and implementers behind a lottery scheme in 2023 are an example of people from The River who know how to calculate and exploit expected value opportunities, even with lottery tickets. You should look up the Hearst investigation yourself as its quite interesting, but the short version is this: 

For the April 22, 2023 Texas Lotto drawing, an investment group managed to spend an estimated $25.8 million to purchase every numerical combination possible in order to guarantee a win of the $57.8 million lump sum offered by the Texas Lotto, plus smaller prizes as well. Their expected value calculation depended upon the payout getting very large over many months without a jackpot, plus their confidence they had solved the technological and logistical problem of buying up every number combination over the course of two days. They basically brought an Oceans 11 approach to winning the Texas Lotto, and it was all legal. 

If you don’t know how to do that, you should not ever be buying lottery tickets. 

As a p.s. to the story, the Texas Lottery Commission will probably change the rules to prevent this kind of exploit in the future.

Improve Our Thinking

I’m not claiming to be particularly great at The River’s mode of thinking, but I am naturally attracted to it. I aspire to it.

My interest began as a childhood board game player, was enhanced by years working on Wall Street, and is kept percolating through hobbies like dabbling in poker, investing, and prediction markets. 

I’ve been exposed enough to it throughout the years to see it as something that can give me, and other people, a possible edge in understanding the world. Whether you’re a member of The River already, or just want to avoid the pitfalls of conventional thinking, I recommend Nate Silver as your guide.

A version of this post ran in the San Antonio Express-News and Houston Chronicle.

Post read (18) times.

Book Review: The Signal And The Noise by Nate Silver


I took a mandatory course in high school[1] called “Theory of Knowledge,” meant to help us consider ‘How do we know things?”

“How do we know things?” turns out to be one of those big philosophical questions – dating from the time of Plato & Aristotle – irritating all of us for the last few millenia.

What Nate Silver addresses more than anything in The Signal and The Noise: Why So Many Prediction Fail – But Some Don’t  is how we know things – in particular how we use and misuse information to understand and make predictions about complex phenomena such as baseball performance, political outcomes, the weather, earthquakes, terrorist attacks, chess, Texas Hold ‘em poker, climate change, the spread of infectious diseases, and financial markets.

I’ve written before that it’s Nate Silver’s world, and we just live in it.[2] The Signal and The Noise offers a 21st Century answer to the question of ‘how do we know things?’  Because most of us, and most media, do not yet think this way, Silver implicitly criticizes everything I hate about the Financial Infotainment Industrial Complex.

Big Ideas vs. Small Ideas

Silver argues effectively that we frequently go wrong in many areas when we adopt a single model or approach to a problem, when an evolving, flexible, multiple-input, probabilistic approach would serve us better.

The problem of political pundits

Silver repeatedly returns in The Signal and the Noise to criticize political pundits on a TV show called The McLaughlin Group, on which commentators from the left and the right appear to make bold political predictions.  Silver – among the most widely admired public forecasters of political outcomes – eviscerates this type of ‘prediction,’ citing data that shows these commentators make accurate predictions no more often than would a random coin toss.

But television rewards ‘bold stances’ and ‘big ideas’ of the type The McLaughlin Group traffics in, while largely ignoring more thoughtful approaches.

Silver labels and criticizes the “Big Idea” mindset that passes for political commentary on television in favor of a more modest, probabilistic, and empirical “Small Idea” mindset.  Small ideas, nuanced, uncertain, and modest, however, make for poor television ratings.

But Silver does have a Big Idea himself

For complex, hard to predict phenomena[3], Silver explains his preferred method, based on a probability theorem attributed to an 18th Century English minister Thomas Bayes.

No doubt Silver thinks many more of us should become familiar with this branch of probability and statistics math. [4]

Beyond the Bayesian theory, however, Silver encourages us to adopt a probabilistic world-view.   His big idea is for us to move away from “I have the explanation and I know what’s going to happen,” to a different way of understanding the world characterized by “I can articulate a range of outcomes and attach meaningful probabilities to the possible outcomes.”

Over time, as we refine our data gathering and multifaceted models, we can move our small ideas forward and become ‘less wrong’ about the world.

In the investment world the former style of traders – the one’s with big ideas and certainty – may have a good run of success, but generally get flushed out when markets turn.  The best traders I’ve ever worked with think and speak in the latter way, considering new possibilities as markets evolve.

Some parts of this remind me of Nassim Taleb

The habits of mind Silver’s book encourages are not dissimilar to Nassim Taleb’s empirical skepticism, although they differ greatly in style and in points of emphasis.  Taleb tends to be aggressively critical of everybody else’s models, whereas Silver more generously praises other theorists’ models and critiques his own.

Both Taleb and Silver share, however, a restless dissatisfaction with the inputs into our understanding right now.  Both would say we do not know enough. We have not considered enough factors to explain whatever phenomenon we purport to explain. Our models need improvement and perpetual skepticism.  The best we can do is to think probabilistically about future events.

Both encourage a learned humility about what we can know or patterns we think we observe in the world.

How does this relate to investing?

I’d estimate only about ten percent of Silver’s book explicitly addresses investing.  As I mentioned, The Signal And The Noise is really a “Theory of Knowledge” book rather than in investing book.

But because Silver thinks like the best financial traders, uses probabilistic math effectively, and writes more clearly than almost anyone, his ideas are worth applying to investing.

1. Attribution of success

Among people who invest their own or other people’s money, 99.5%[5] of us attribute successful outcomes to personal investing acumen, while attributing unsuccessful outcomes to circumstances beyond our control.

The noise surrounding our own success – misinterpreting a generally rising market as stock-picking skill for example – leads us to overestimate our ability to influence investment returns.  As a result, too many of us engage in security selection, or too many of us pay others to achieve superior investment results, despite the evidence that we’re overpaying.

2. Responsibility for failure

Conversely, our abdication of personal responsibility for losses – it must have been ‘the bad markets’ after all! – leads us to underestimate our own errors of judgment.

In both cases – success or failure – we’re prone to adopt an uncritical approach to the right level of responsibility for outcomes.

3. Efficient market hypothesis as an illustration of the Bayesian approach

Although Silver gives numerous examples of his Bayesian probabilistic approach to problems with numbers, one of his best examples is purely textual, on the efficient market hypothesis.  He lists seven increasingly accurate, yet also qualified and probabilistic statements, on what we know about efficient markets.

The series of increasingly accurate, yet ‘less bold,’ statements are not only a great illustration of his big idea but also the right lesson for us on investing, so I reproduce it in full here:

a)     No investor can beat the stock market.[6]

b)     No investor can beat the stock market over the long run.[7]

c)      No investor can beat the stock market over the long run relative to his level of risk.[8]

d)     No investor can beat the stock market over the long run relative to his level of risk and accounting for transaction costs.[9]

e)     No investor can beat the stock market over the long run relative to his level of risk and accounting for his transaction costs, unless he has inside information[10]

f)       Few investors can beat the stock market over the long run relative to their level of risk and accounting for their transaction costs, unless they have inside information[11]

g)     It is hard to tell how many investors beat the stock market over the long run, because the data is very noisy, but we know that most cannot relative to their level of risk, since trading produces no net excess return but entails transaction costs, so unless you have inside information, you are probably better off investing in an index fund.[12]

The first approximation – the unqualified statement that no investor can beat the stock market – seems to be extremely powerful.  By the time we get to the last one, which is full of expressions of uncertainty, we have nothing that would fit on a bumper sticker.  But it is also a more complete description of the objective world.

If you want a 21st Century theory of knowledge, teaching you ‘how to think’ about the major world problems of global warming, financial crashes, avian flu, and terrorism, as well as ephemera like poker, chess, sports betting and baseball, start with The Signal and The Noise: Why So Many Prediction Fail – But Some Don’t  by Nate Silver.

Please also see related post on Bayesian Probability and the Red Sox.

Please also see related post All Bankers Anonymous Book Reviews in one place!

The Signal and The Noise
The Signal and The Noise

 


[1] Readers who study at an International Baccalaureate (IB) high school will be familiar with the “Theory of Knowledge” course.  It’s a really great idea for a course, but I have yet to meet anyone who thought the experience of the course lived up to the idea that inspired it.

[3] Each chapter separately tackles baseball, political forecasting, weather, earthquakes, economic growth, infectious disease growth, sports betting, chess, poker, climate change, and terrorism – each in their own way posing a challenge of seeing into the future.

[4] The mathematics of Bayesian probability is relatively straightforward so I think I’ll try in a subsequent post to do it justice.

[5] I rounded down to be conservative, because that’s just good science.

[6] The original, powerful, efficient market thesis

[7] Because, clearly, some people sometimes do, for some period of time

[8] You can take some crazy stock-market risks and WAY outperform boring stodgy stocks much of the time.  We have to match up comparable investment risk levels.

[9] A theoretical ‘market-beating’ high volume trading strategy often looks less market-beating when you take into account the frictions of trading.

[10] Inside information sure is helpful, when trying to beat the market

[11] Maybe some can do it, like Warren Buffett, but it’s super rare.  Probably you can’t do it.

[12] So carefully hedged!  So qualified and full of doubts!  So true!

Post read (11264) times.