Thirty Years of Software Problems in the News
20
Thirty Years of Software Problems in the News Bryan Dosono, Syracuse University Andrew J. Ko, University of Washington Neeraja Duriseti, AT&T
-
Upload
bryan-dosono -
Category
Data & Analytics
-
view
498 -
download
4
description
How have the consequences of software problems changed over the past 30 years? To begin to answer this question, we analyzed 386,381 news articles reporting on software problems published between 1980 and 2012, spanning widely circulated newspapers to small trade magazines. Our results show that after an increase in reporting just prior to Y2K, news on software problems has declined in North America, but increased in the rest of the world. Most articles only report minor consequences such as frustration, confusion, anger, or at worst, having to delay some activity for a few hours, usually due to service outages in government, transportation, finance, and information services. However, about once per month, the news reports at least one death, injury, or threatened access to food or shelter due to software problems. Reports of these severe consequences are also increasing, due primarily to stories about transportation and government software.
Transcript of Thirty Years of Software Problems in the News
Thirty Years of Software Problems in the NewsBryan Dosono, Syracuse University Andrew J. Ko, University of WashingtonNeeraja Duriseti, AT&T
Related work• Software transforms society as it reshapes globalization,
social relationships, and other aspects of everyday work• Software isn’t perfect; it can fail spectacularly• Most water, gas, and electricity failures now involve
software failures (Rahman, Beznosov & Martí, 2009)• Security breaches are now more prevalent and more costly
than ever before (Garrison & Ncube, 2011)• Cars recalled more frequently than ever because of
software defects (Mark, Bagdouri, Palen, Martin, Al-Ani & Anderson, 2012)
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Research question•How have the consequences of
software problems changed over the past 30 years?
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Data collection• Gathered text to build a corpus for analysis• Three major news archives: ProQuest, LexisNexus, Factiva• Databases included 93 out of 100 most circulated newspapers• Only considered English news coverage
• Searched for words commonly referring to software problems• i.e., bug, defect, error, fault, flaw, malfunction, glitch, mistake• “Glitch” consistently referred to software problems (60%)• 386,381 articles published from 1980-2012 with word “glitch”
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Terms considered for retrieving articles
Term % About Software
# Articles With Term
# About Software Dominant Topics
glitch 60% 49,537 ~30,000 software problems
bug 11% 182,150 ~20,000 insects, illness
malfunction 8% 24,154 ~2,000 mechanical problems
defect 3% 92,300 ~3,000 taxes, birth defects
flaw 3% 106,240 ~3,000 construction, decisions
error 2% 629,832 ~12,000 sports, news, medicine
mistake 1% 586,720 ~6,000 decision making
fault 1% 265,383 ~3,000 politics
INTRODUCTION
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
METHOD
Clustering articles to stories• Developed new clustering algorithm• Converted each article’s words into a vector of tf-idf scores• Iterated through all articles in order of publication date• Compared each article to all subsequent articles published
within a 7-day window
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Story measurementsQuantitative variables
• Normalized article count• Frequency of news
source location by geographic region
Qualitative variables
• Categorized articles with specific codes among the three authors (Table 2)
• Three rounds of redundant coding until reaching satisfactory intercoder reliability
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Codes used to classify each story
Code Description
software system failure caused by a software defect
death one or more people lost their lives
harm one or more people were physically harmed
basic one or more people lost access to food or shelter
property one or more people permanently lost material goods, money, or data
delay one or more people had to postpone an activity due
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Top 10 stories by normalized article count
Software Problem Reported Year # ArticlesThe aftermath of the year 2000 millennium bug 2000 17,193E-voting defects in the 2004 US elections 2004 2,509Preparations for the year 2000 millennium bug 1998 1,745Medicare drug eligibility defect 2006 1,983Toyota Prius sudden acceleration bug 2010 3,782Delayed e-voting tallies in the 2008 US elections 2008 1,796Incorrect voting tallies from e-voting machines 2006 1,236Incorrect New Mexico voting tally 2000 1,003Atlanta Olympics results delivery to news delayed 1996 447E-voting defects in the 2002 US mid-term elections 2002 636
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Stories reported per month by region
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Reports of death
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Reports of physical harm
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Reports of lost access to food and shelter
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Reports of property loss
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Reports of delay
INTRODUCTION
METHOD
RESULTS
DISCUSSION
CONCLUSION
QUESTIONS
Discussion• Several possible causes could underlie the
observed news reporting trends• Software problems are becoming harder to observe• Most of the stories in our data are based on press
releases or investigations of public organizations• Software failures could be more common within
private organizations but are simply not reported
INTRODUCTION
METHOD
DISCUSSION
RESULTS
CONCLUSION
QUESTIONS
Discussion• New directions for software engineering practice• Software should be used in supplement to legacy
process systems rather than as a cure-all for problems
• Software design practices should consider the potential for failure in all parts of a system’s functionality
• Adapt frameworks on universal human needs into a software inspection checklist
INTRODUCTION
METHOD
DISCUSSION
RESULTS
CONCLUSION
QUESTIONS
Discussion• Current software lacks• A middle ground between working and not working• Tools to quickly isolate and identify causes for failure• Highly informative error messages to speed the
recovery process• Software should be designed with the recovery
process in mind
INTRODUCTION
METHOD
DISCUSSION
RESULTS
CONCLUSION
QUESTIONS
Conclusion• Consequences of software problems on society can
range from trivial to catastrophic• The majority of reported software problems are
brief inconveniences• Software engineering practices still need
improvement as we design software for the future
INTRODUCTION
METHOD
CONCLUSION
DISCUSSION
RESULTS
QUESTIONS
Any questions?• Acknowledgements• Thank you to Parmit Chilana for her comments on early
drafts of this work• This material is based in part upon work supported by the
National Science Foundation under Grant Numbers CCF-0952733 and CCF-1153625
• Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation
INTRODUCTION
METHOD
QUESTIONS
DISCUSSION
RESULTS
CONCLUSION
