Book Review – The Signal And The Noise

Title: The Signal & the Noise

Author: Nate Silver

Publisher: Allen Lane (Penguin Press)

Date: September 27, 2012

Pages: 534

Cost: Rs. 600/-

ISBN: 978-1-846-14816-3

Reviewer: Abhimanyu J. L.

Nate Silver’s shot to fame was because of his successful prediction of the 2012 Presidential race in all fifty states of the United States of America. When the whole world was speculating as to who would become the next President of the United States , Nate Silver had Barrack Obama winning by a clear margin. Not only that, he had in the run up to the election, successfully predicted Republican party nomination races- results of which most people did not expect! This and many other details of his prediction brought the attention of the world onto this 35 year old genius of a statistician.

In his introduction to the book, Nate talks about the invention of the Gutenberg press and progress the world has made since then. He argues that lower cost of information led to the industrial revolution and further lowering of information costs (because of the World Wide Web) promises breakthroughs for mankind similar in impact to that of the industrial revolution. However, with the explosion of data, we find ourselves dealing with challenges we have not faced previously. Deciphering knowledge from this mountain of data allows for the title of this conversation: ‘The Signal and the Noise’.

The book  is divided into thirteen chapters and in each of those chapters Nate separately delves into the dynamics and  resulting challenges for forecasters and participants alike in fields such as Politics, Sports (Baseball, Basketball, Chess & Poker), Weather, Medicine, Economics (MBS fiasco, Economy & Stock Market), Climate, and Terrorism. Complexity, uncertainty and value (or lack thereof) of consensus views is core themes of this book (page 382).

The book is relevant to the CFA charter holders in that it talks about prediction, ways to think about data, cautioning us on common pitfalls while analyzing data, bias in data analysis and borrows other themes from Behavioral Finance. But, most importantly Nate relates his success to the success of Bayesian thought process. His arguments about conditional probability and assigning prior probabilities to forecasting events are simple and straight forward. He gives examples of analyzing information and predicting outcomes in one the chapters. He also argues about the short-comings of ‘Frequentism’ and how this has resulted in bouts of false positives in academic research. Frequentism features in level I & II curriculum of the CFA program and relates to finding confidence intervals, assumptions of distribution and ANNOVA table.

Throughout the book Nate also introduces us to characters such as Bob Voulgaris who makes his living by betting on outcomes of basket ball games. Bob earns a million dollars in a bad year by predicting the outcome of NBA games. Nate reveals how Bob uses Bayesian thought process to achieve these astonishing results.

Also spread throughout the book are ideas such as the Pareto Principle of prediction which assigns accuracy of prediction to the effort put into it, self-fulfilling models, and self-canceling models. Nate also walks the reader through incentives of forecasters in various fields and shows how these affect the forecasts and outcomes. The author also advocates betting markets for predictions so as to change incentive structures and increase their accuracy.

In conclusion, the author opines that there is more to be gained from Bayesian way of thinking. The complex systems we deal with, the information overload we encounter and the biases we are prone to, makes objectivity almost an unattainable goal. We comprehend through approximation and this has pitfalls, understanding ourselves and the way we distort signals will help us make better predictions.

–  A J L