MATHEMATICS AWARENESS MONTH

Editor’s Note: In celebration of Math Awareness Month on “The Future of Pre-diction,” we provide essays by Lisa R. Goldberg and Olle Häggström, andgreetings from some of our fellow mathematical societies around the world.

Lisa R. Goldberg

Prediction Is a Young SciencePredictions go back at least as far as the Oracle of Delphiand the astrologers of the Chinese Han Dynasty. Arguably,they are as old as the human race. However, our recentlyacquired capability to collect, store, and analyze datahas emphasized statistics and elevated prediction to ascience that pervades virtually everything we do. Despiterecent advances, prediction is a young science. We arejust beginning to explore its limits.

Scientific Prediction Can Be Humbling andConfusing

Time to eat some crow.—Nate Silver

On July 8, 2014, Germany won a stunning 7–1 victoryover Brazil in the World Cup semifinals. This was aproblem for superstar statistician Nate Silver, whosePoissondistribution-based Soccer Power Index (SPI)modelhad forecast a win for Brazil and had set the odds ofGermany beating Brazil by six goals or more at 1 in 4,000.

Silver has been using statistical models to predict out-comes of sporting events since the early 2000s, but he isbest known for predicting elections. In 2008, Silver and hiscolleagues at FiveThirtyEight correctly called every state inthe US presidential election with the exception of Indianaand the Second Congressional District in Nebraska, whichawards its own electoral vote. In 2012, FiveThirtyEightcorrectly called the US presidential election in all fiftystates (including the toss-up in Florida) and the Districtof Columbia. Silver’s book on statistics, The signal andthe noise, became an international bestseller. “Triumphof the Nerds” and other articles celebrating data scienceappeared, exceptionally, in the mainstream media.

Lisa R. Goldberg is founding partner of Berkeley Associates LLC,Director of Research at Aperio Group, and Co-director of the Con-sortium for Data Analytics in Risk at the University of Californiaat Berkeley. Her email address is lrg@berkeley.edu.

This article has benefitted from conversations with Bob Anderson,Jeff Bohn, Alan Cummings, Nick Gunther, Ola Mahmoud, BarryMazur, Caroline Ribet, Ken Ribet, Stephanie Ribet, Alex Shkolnik,Philip Stark, and Roger Stein.

For permission to reprint this article, please contact:reprint-permission@ams.org.DOI: http://dx.doi.org/10.1090/noti1377

In response to questions about SPI’s disastrous Brazil-Germany prediction, Silver explained:

Statistical models can fail at the extreme tails ofa distribution. There often isn’t enough data todistinguish a 1-in-400 from a 1-in-4,000 from a1-in-40,000 probability.

Statistical models rely on estimated distributions, out-comes, and likelihoods to make predictions. The tailsof a distribution are populated by rare or never-before-seen outcomes. It is virtually impossible to assess thecredibility of the estimated likelihood of a rare event.

Silver was not the only one to misforecast the outcomeof the 2014 World Cup. Many of the big investment bankshad used statistical methods to predict a win for Brazil.The Financial Times demoted data science from hero togoat with headlines like “Brazil’s World Cup eludes banks’best minds”. On the other hand, Microsoft was delightedto point out that its prediction engine, Cortana, hadcorrectly predicted the outcome in 15 of 16 World Cupmatches, including the Brazil-Germany semifinal.

Extreme Events May Seem Obvious in HindsightOn June 22, 2007, the investment bank Bear Stearns wasquietly working to rescue two of its hedge funds: the BearStearns High-Grade Structured Credit Fund and the BearStearnsHigh-GradeStructuredCredit EnhancedLeveragedFund. These funds were trading securities backed by sub-prime mortgages, which involve loans to homeowners bybanks who readily acknowledge the possibility that someof the homeowners might default. However, relatively fewconsidered the possibility that lots of homeowners mightdefault at the same time.

The rescue failed since the sub-primemarket hadbegunto collapse in response to a rapid rise inmortgagepaymentdelinquency. By mid-July, Bear Stearns disclosed that thetwo funds had lost nearly all their value. Smooth andupward-trending equity markets began to rock. This wasthe preamble to the global financial crisis that dominatedthe ensuing two years and which has been blamed for thedestruction of trillions of dollars, the loss of millions ofjobs, and the suicides of more than 10,000 individuals.

Despite plentiful data, the elaborate and expensive riskmanagement systems in place at many financial institu-tions in 2007 failed to alert investors to the impendingcrisis. There were exceptions. In The Big Short, MichaelLewis tells the tales of a few investors who noticed that

352 Notices of the AMS Volume 63, Number 4

MATHEMATICS AWARENESS MONTH

housing prices were inflated, acted on their observations,and profited. For the vast majority of us, however, thecrisis signals were obscured by the noise of everyday life.

I recall a San Francisco limousine driver telling meearly in 2007 about the houses he was flipping in Georgia.It seemed odd, but I did not ask myself what it implied.I could have known. That nagging I-could-have-knownfeeling that comes in the aftermath of a disaster is anexample of hindsight bias, and it depends on our selectivememory of what turned out to matter. In his bestsellingbook Thinking, Fast and Slow, Daniel Kahneman explainsthe behavioral roots of hindsight bias. These include theavailability heuristic, our tendency to rely on informationthat easily comes to mind; and the representativenessheuristic, our tendency to profile.

A Correct Prediction Can Also Be a Bad PredictionThe Monty Hall Problem goes like this:

There is a sports car behind one of three closeddoors, but the other two doors hide goats. Youget to select a door and keep whatever is behindit. After you make an initial choice, Monty Hallopens one of the two unselected doors, revealingthat it does not hide the sports car. He offers youthe opportunity to switch to the other unopeneddoor or stay with your original choice. Should youstick or switch?

If you stick with the door you originally selected, youmay be lucky and win the car. But the probability that thecar is behind the door you originally selected is 1/3, whilethe probability that the car is behind the unselected doorMonty Hall failed to open is 2/3. The prediction that thecar is behind the door you originally selected is bad, evenif it turns out to be correct.

A skilled predictor may never make a bad prediction,but shewill inevitablymakesomepredictions that turnoutto be incorrect. This conundrum is concisely summarizedin Nassim Nicholas Taleb’s Table of Confusion, whichpairs terms that tend to be mixed up: luck and skill,belief and knowledge, theory and reality, signal and noise,randomness and determinism.

Scientific Prediction Learns from MistakesIn 1948, major polls in the US famously predicted thatNew York governor Thomas Dewey would beat HarryTruman in the presidential election.

The predictions were based on quota sampling, whichtargets subsets of the population that are prespecifiedrather than representative. Thepredictionswere also stale.As George Gallup, co-chairman of the Gallup organizationexplained, “We stopped polling a few weeks too soon.”Modern election polls use random sampling rather thanquota sampling, and they run up to the last minute.

Today, scientific predictions tell us which movies,books, and restaurants we will enjoy, which diseases willafflict us in the future, whether a cancerous growth on oneof our feet will prove fatal, how drugs will interact, who

Incorrect banner headline on the front page of theChicago Daily Tribune on November 3, 1948.

will be our soulmates, and when wemight be wiped out byrising sea levels and ocean acidification. It can be difficultto distinguish good predictions from bad ones, since acorrect prediction need not be good. A prediction mightbe good if it is based on ample data and an algorithm thatlearns, if it fits in-sample and is validated out-of-sample,and if it is interpretable and appealing to common sense.It may be impossible to test predictions of the time andlocation of the next big earthquake or the efficacy andsafety of a cure for a rare disease or the time and causeof the next financial crisis.

But some scientific predictions have demonstratedtrack records. Advanced warnings of hurricanes andtornadoes have allowed communities at risk to evacuatebefore storms hit. Airline travel is remarkably safe, inpart, due to accurate weather forecasts. When GoogleNavigation tells you that, despite some traffic, you willreach your destination by 3:12 pm, there are reasons tobelieve it.

About the Author

Photoco

urtesy

ofKe

nRibe

t

Lisa R. Goldberg.

After starting her career in acade-mia in topology and complexdynamical systems, Lisa moved tomathematical finance in 1993 andhasn’t looked back. She and herhusband, AMS President Elect KenRibet, have two daughters. An avidswimmer, Lisa averages two kilome-tersperdayandhasswuma lifetimedistance of 25,000 kilometers.

April 2016 Notices of the AMS 353

MATHEMATICS AWARENESS MONTH

Olle Häggström

Our Desperate Need to Predict theFutureIn 1814, Pierre Simon de Laplace envisioned a demon whocouldpinpoint theexactpresentpositionsandmovementsof all particles and then calculate all of their futuretrajectories, thereby accurately predicting everything thatwill ever happen. He understood, however, that this taskwould be forever beyond mere human capabilities. Dueto what we now know of as sensitive dependence oninitial conditions, this brute force approach to predictingthe future does not work (other than in certain limitedsituations with limited time horizon). Instead, we’d liketo find regularities in the past and assume “by induction”

Whathappenswhen wesucceed increating amachine

thatsurpasseshuman ca-pabilities?

that they will continue into thefuture. Unfortunately, there seemsto be no way to defend induction—the extension of past regularitiesinto the future—without revertingto circularity by pointing out thatinduction has served us well in thepast and thus can be expected todo so in the future.

Nevertheless, we rely on induc-tion tomake predictions. We simplydo not know any other way. Goodapplied mathematicians and goodscientists know that the extent towhich these predictions are reli-able depends on how closely thestate of the system we are tryingto predict will remain within theenvelope of what has already beenobserved. The further we push at-

mospheric CO2 levels above those of the last severalmillion years, the less reliably we can predict the fu-ture climate. Moore’s law—the exponential curve that fitsseveral decades of computer hardware development sowell—eventually predicts the physically impossible.

Today we face unprecedented challenges of the follow-ing kind. We are beginning to develop new technologiesthat have the potential to take us very far outside theenvelope of the observed and the familiar—technologiesthat can bring us enormous benefits, but also catastro-phes on all scales, including the extinction of humanity.In my recent book Here Be Dragons, I discuss severalsuch technologies and argue for the need to act withforesight. Examples abound in biotechnology as well as in

OlleHäggström is professor ofmathematical statistics at ChalmersUniversity ofTechnology, amember of theRoyal SwedishAcademyof Sciences, and the author of four books. His email address isolleh@chalmers.se.

For permission to reprint this article, please contact:reprint-permission@ams.org.

nanotechnology. Here, for concreteness, let me focus onartificial intelligence (AI). What happens when we succeedin creating a machine that surpasses human capabilitiesin terms of general intelligence? Will it enter the kind ofrapid spiral of self-reinforcement known as an intelligenceexplosion or the Singularity? Will we be able to remain incontrol when we are no longer the smartest creatures onthe planet? Much has been written about this, for instanceby Ray Kurzweil, emphasizing the wonderful prospectsbrought by the Singularity, and by Nick Bostrom, empha-sizing the risks. Kurzweil likes to emphasize that we havealways lived with double-edged technologies and copedwell: “Fire kept us warm and cooked our foods, but alsoburned down houses,” Kurzweil told CNN in June 2015.The implicit suggestion here is that things will be finewith AI as well, but as physicist Max Tegmark recentlypointed out in a panel discussion with Kurzweil, theremay be a crucial disanalogy: we learned to use fire usingtrial-and-error, while if we do not get the first AI super-intelligence right (i.e., beneficial to humanity), we may notget a second chance.

These scenarios are so far outside what we are usedto thinking about as proper areas of extrapolation andprediction that any attempt in that direction may seemreckless. Yet, I claim, the amount of value at stakecompels us to try, as the alternative tends to involveimplicit and unfounded assumptions that nothing drasticwill happen and is tantamount to running blindfolded intoa minefield. Induction on its own is entirely inadequatefor making useful predictions here, but in combinationwith deduction, useful conclusions perhaps can be drawn,analogously to climate modelling, where empirically well-understood physical laws and regularities are combinedto predict the future (along with uncertainty estimates).Predicting the consequences of an AI breakthrough isfar more difficult than predicting the climate in 2100,but this should not stop us from doing the best wecan and accompanying the predictions with the rightamount of epistemic humility. Without a conscious effortto predict, we will rely on ill-founded and erroneousimplicit predictions of the status quo.

Onemore caveat.When I talk of the need for predictionshere, I really mean the need for conditional predictions,predictions conditional on various actions on our part. Aprediction like “a robot Apocalypse by 2100 has probabil-ity 20 percent” is of little use in itself; what we really needas a guide to policy and action is to understand, e.g., howpromoting or restricting various AI research directionsaffects that probability. This is analogous to how IPCCprojections are conditional on different greenhouse gasemission scenarios, thus guiding us in how choices ofemission policies affect future climate. As modelling ineconomics and other social sciences becomes more so-phisticated, it becomes increasingly tempting to includesocial aspects as endogenous to climate models. Just asthe models encompass physical feedback mechanisms(such as warming leading to reduced snow covering, lead-ing to more warming), it is also possible to include social

354 Notices of the AMS Volume 63, Number 4

MATHEMATICS AWARENESS MONTH

feedbackmechanisms (such aswarming leading to air con-ditioning, leading to more warming). While such studiescan be very interesting, there is also the risk that buildinghuman reactions into the system of differential equationscan lead to a deterministic view in which our free willseems to disappear. To be able to work effectively towardsimproving our prospects for a bright future, we need toretain a mindset according to which different actions anddifferent policies are both possible and consequential.

About the AuthorOlle Häggström, authorof four books, withthe first advance copyof Here Be Dragons: Sci-ence, Technology and theFuture of Humanity, Ox-ford University Press,2016.

Greetings from Some ofOur Fellow Math Societies

C. CilibertoL’Unione Matematica ItalianaEach year in April the Joint Policy Board for Mathematicsof the US sponsors the Mathematics Awareness Monthto stress the importance of mathematics in our societyand culture through a series of initiatives that highlightmathematical developments and applications. The 2016Mathematics Awareness Month will celebrate its 30thanniversary since this initiative started off in 1986.

Photoby

Giulia

Cilib

erto.

C. Ciliberto,President ofL’UnioneMatematicaItaliana.

This year the theme of the Mathe-matics Awareness Month will be “TheFuture of Prediction”. Human beingshave being trying to foretell the fu-ture since ever. In the far past, thiswas the job of charlatans, astrologists,prophets and oracles, who appealedto superstitions and pseudoscientifictools (the interpretation of the flightof birds, the form of tea leaves orcoffee grains at the bottom of a cup,the sound of a thunder, tarots, etc.) toconvince people that future could bepredicted. In more recent times thishas been taken over by scientists, whohave been trying to make predictionsbased on the currentmodel of interpretation of the nature.Natural sciences (like physics, chemistry, biology, etc.) andsocio–economical sciences (like economy, sociology, etc.)have been playing of course a crucial role in this respect.

But how can scientists anticipate the evolution of theclimate in the next twenty years if even a three-dayforecast is a serious challenge? How can we predictthe development of epidemics? How can we accuratelyforesee ourfinancial future? Tomake accurate predictionsabout the behavior of highly complex systems, we needa reference model to appeal to, and to collect, interpretand manipulate an enormous amount of data: of course,more a model is mathematically refined, more it isrobust and amenable to provide the right framework forreliable predictions. In the last century or so, thanks tothe development of more and more sophisticated tools,mathematics widely extended its domain of applicationsfrom physics (which was the classical one) to all othersubjects mentioned above, including human sciences.Never in the history of our civilization, is mathematics as

C. Ciliberto is president of the L’Unione Matematica Italiana. Hisemail address is cilibert@mat.uniroma2.it.

For permission to reprint this article, please contact:reprint-permission@ams.org.

April 2016 Notices of the AMS 355

MATHEMATICS AWARENESS MONTH

immanent as [it is] today in all our activities. In this sense,never as nowadays, mathematics is involved in the art ofpredicting the future. Therefore the subject of the 2016Mathematics Awareness Month is extremely timely.

The “Unione Matematica Italiana”, which is the refer-ence association of the Italian mathematicians, looks atthis fascinating theme with the greatest interests andwishes to this initiative the best success.

Jose Maria P. BalmacedaMathematical Society of the Philippines

Jose Maria P.Balmaceda,President ofMathematicalSociety of thePhilippines.

Warmest greetings to the AMSfrom Mathematical Society ofthe Philippines! We join theAMS,other reciprocating societies,and the worldwide mathematicalcommunity in celebrating Mathe-maticsAwarenessMonth in theUSand laud the choice of this year’sfocus on the ‘Future of Predic-tion’. Being in a region highlyvulnerable to natural disasters,environmental hazards and eco-nomic uncertainty, our membersare increasingly being relied uponto developmodels and techniquesfor forecasting and risk man-agement, and provide insightful

analysis of large amounts of data. We look forward tothe Notices’ excellent articles and commentaries on thesetopics.

Jose Maria P. Balmaceda is president of MathematicalSociety of the Philippines. His email address is jose.balmaceda@gmail.com.

For permission to reprint this article, please contact:reprint-permission@ams.org.

Francis E. SuMathematical Association of America

Cour

tesy

ofFran

cisE.

Su.

Francis E. Su,President of theMathematicalAssociation ofAmerica.

The power of mathematics is, to agreater extent than ever before, be-ing harnessed all around us in theinformation revolution, with mathe-matics and statistics as the powertools of a new economy. Phonesnow predict when you will gethome. Search engines predict thespread of disease. Numerical mod-elspredict theweather.Mathematicsmakes new inventions possible. Ihope you’ll use the theme of thisyear’s Math Awareness Month as agolden opportunity to amplify, foryour students and your friends, therole of mathematics at the center oflife in the twenty-first century.

Francis E. Su is president of Mathematical Association of Amer-ica and Benediktsson-Karwa Professor of Mathematics at HarveyMudd College. His email address is su@math.hmc.edu.

For permission to reprint this article, please contact:reprint-permission@ams.org.

356 Notices of the AMS Volume 63, Number 4