The Evolution of Expertise in Decision Support Technologies: A

Challenge forOrganizations

Steve Diasio* & Núria AgellESADE Business School- Barcelona

GREC Research Group

*This research has been partially supported by the AURA research project (TIN2005-08873-C02), funded by the Spanish Ministry of Science and Information Technology and the Commission for Universities and Research of the Ministry of Innovation, Universities, and Enterprises of the Government of Catalonia.

Road Map

• Introduction and Motivation• Terms and Concepts• Leveraging Expertise in

Decision Support Technology– Expert Systems (ESs)– Group Decision Support

Systems (GDSSs)– Collective Intelligence

Tools (CI Tools)• Enhancing Decision-Making

and CI Tools• Conclusions

Empty trading pit @ CBOT

Introduction and Motivation

• Importance of decision-making • Researchers must evolve as technology

changes• Computer Supported Cooperative Work (CSCW)

provides a center point • An evolutionary view of three decision support

technologies that support the use of expertise.Expert SystemsGroup Decision Support

SystemsCollective Intelligence

Tools

Terms and Concepts

What is Expertise?• Multi-dimensional (Sternberg,

1997) with expert knowledge as the essential part (Tynjala, 1999)

• Short supply and difficult to represent

• Highly specialized or domain specific (Chi, Glaser, & Farr, 1988)

• Skills honed through practice (Jackson, 1999)

• Perform consistently more accurate in relation to others (Hartely, 1985)

Practical Knowledge Self-r

egula

tive

Know

ledgeFo

rmal

Kno

wled

ge

Expert Knowledge Dimensions

Expertise in Law

Formal Knowledge Practical Knowledge

Self-regulative Knowledge

•Reflective skill•Evaluation of action•Monitor argument and presentation to jury

•Factual knowledge•Learning of explicit information•In school or cases

•Intuition•Experience in legal setting•Tacit and difficult to express

Lawyer Expertise

Expertise by Means of Technology

• Expertise not limited to humans

• Technology built to capture knowledge or represent expertise (Barton, 1987; Liou & Nunamaker, 1990; Smith, 1994)

• Level of expertise can be augmented by increasing the amount of participants in the decision-making process

Expertise in Design

Level of Expertise in Systems Design

Number of People

Expert Systems

GDSSs

Collective Intelligence Tools

Objective: To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Playing a critical role for organizations and are a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts (Weiss & Kulikowski, 1984).

By mimicking and replicating the cognitive process of a human expert, novice users can be supported to perform as well as experts (Cascante et al, 2002).

ES are a technology that facilitates learning through the transfer of tacit and explicit knowledge (Yoon et al., 1995; Gregor & Benasat, 1999).

Leveraging Expertise Expert Systems

Attributes:

Objective: To capture the knowledge and contribution from the individual users to facilitate solutions to problems.

Occupies the center point for the aggregation of information and expertise from each participant.

Support the changing organizational structure, project basedteams, dispersed workforce, and greater emphasis on collaboration.

Aided groups to deal with to the changing dynamics characterized by greater knowledge, complexity, and turbulence (Huber, 1982; 1984).

Shown to reduce time, costs (Gallup, 1985), foster collaboration, communication, deliberation, and negotiations (Kull, 1982).

Leveraging Expertise Group Decision Support Systems

Attributes:

What is Collective Intelligence?

• The collective judgment of group can predict or forecast better than experts or groups of experts (Surowiecki, 2004)

• Diverging from traditional thought- high levels of expertise are the best source for decision-making

• Including many people in decision-making by harnessing lower levels of expertise for peak solutions (Page, 2007)

Objective: To facilitate the summative body of knowledge,information, and resources of its users.

Democratize decision-making by including many people in and outside the organization into the information gathering and decision-making process.

Prediction markets, incubates information scattered around the organization or network that allows non-experts to produce expert like results.

Challenges traditional roles of experts, may change answer givers to inquiry mediators in effort to harness the knowledge of the masses in decision-making.

Offer an additional tool in decision-making.

Leveraging Expertise Collective Intelligence Tools

Attributes:

Enhancing Decision-Making and CI Tools

• Past attempts have made steps (Aiken et al. 1991; Turban & Watkins, 1986).

• Opportunities for system integration to solve a wider spectrum of problems.

• AI techniques to CI Tools– Transforming from

passive to active agents– Intelligent components

to increase participation – Managing interaction

and collaboration between users

Ill- Structured

Problem Structure

Many

ES

Group Size SupportedFew

Well- Structured

GDSS

CI Tools

DSS

Decision Support Technologies *

*Figure adapted from Aiken et al. 1991

Shown Indicated

ExploredHighlighted

Conclusions

An evolutionary perspective of expertise supported by decision support technologies.

how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making

how organizational expertise in short supply can be augmented

issues of design for integration with existing methodologies

Thank You!

Steve Diasio & Núria Agellstephen.diasio; nuria.agell {@esade.edu}

ESADE Business School- BarcelonaGREC Research Group

References[1] M. Aiken, O. Liu Sheng and D. Vogel, “Integrating expert systems with group decision support systems”,

ACM Transactions on Information Systems (TOIS), Jan. 1991. (9)1, 75-95.[2] L. Bañón and K. Schmidt, “CSCW: Four characters in search of a context”, Proceedings of the First

European Conference on Computer-Supported Cooperative Work - EC-CSCW Gatwick, London, Sept., 1989, 358-372.

[3] A. Barton, “Experiences in Expert Systems”, The Journal of the Operational Research Society, The Practice of OR in Central Government Oct., 38(10), 1987, 965-974.

[4] J. Berg and T. Rietz, “Prediction Markets as Decision Support Systems”, Information Systems Frontiers, 5(1), 2003, 79-93.

[5] J. Berg, F. Nelson and T Rietz, “Accuracy and forecast standard error of prediction markets”, Working Paper, The University of Iowa, 2001.

[6] E. Bonabeau, “Decisions 2.0: The Power of Collective Intelligence”, MIT Sloan Management Review. Winter 50(2), 2009.

[7] B.G. Buchan, D. Barstow, R. Bechtel, J. Bennett, W. Clancey, C. Kulikowski, T. Mitchell and D.A. Waterman, “Constructing and Expert System”, Building Expert Systems, Hayes-Roth, F., Waterman, D.A. and Lenat, D.B. (eds), 1983. pp 127-167, Addison-Wesley, Reading, MA.

[8] L. Cascante, M. Plaisent, P. Bernard and L. Maguiraga, “The impact of expert decision support systems on the performance of new employee”, Information Resource Management Journal; Oct-Dec 15(4), 2002, 67-78.

[9] M. T. H. Chi, R. Glaser, and M. J. Farr, eds., The Nature of Expertise, 1988. Hillsdale, NJ: Erlbaum.[10] B. Cowgil, J. Wolfers, and E. Zitzewitz, “Using Prediction Markets to Track Information Flows: Evidence

from Google”, 2008, Working Paper.[11] K. Crowston and T. W. Malone, “Intelligent software agents”, Byte Magazine, Dec., 1988, 267-272.[12] G. DeSantics and R. Gallup, “A Foundation for the Study of Group Decision Support Systems”.

Management Science, 33(5), 1987, 589-609.[13] S. Diasio, “Collaborative Fusion: Expert, Group, and Collective Intelligence in Decision-making

Support”, unpublished paper, 2008.[14] R. Dye, “The Promise of Prediction Markets: A Roundtable”. McKinsey Quarterly (2) 2008.

References[15] E.A Feigenbaum, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering’,

International Joint Conference on Artificial Intelligence, 1977, 1014-1029.[16] B. Gallup, “The impact of task difficulty on the use of a group decision support system”. Ph.D. dissertation,

Dept. of Information and Decision Sciences, Univ. of Minnesota, 1985.[17] M. J. Gannon, Organizational Behavior: A Managerial and Organizational Perspective, Boston: Little, Brown,

1979.[18] T. Gill, “Early Expert Systems: Where are they now?”, MIS Quarterly. March, 1995.[19] R. Hanson, “Decision Markets”, IEEE Intelligent Systems, 14, 1999, 16-19.[20] R. Hartley, “Expert System Methodology: A Conceptual Analysis”, International Journal of Systems Research

and Information Science, 1, 1985, 11-22.[21] T. Ho, and K. Chen, (2007). “New Product Blockbusters: The Magic and Science of Prediction Markets”,

California Management Review. Fall, 50(1), Berkeley, California, 2007.[22] J. Howe, Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business, New York, Crown

Business, 2008.[23] G. P. Huber, “Group decision support systems as aids in the use of structured group management

techniques”, In DDS-82 Conference Proceedings, 1982, 96-108.[24] G. P. Huber, “Issues in the Design of Group Decision Support Systems”, MIS Quarterly, Sept. 8(3), 1984,

195- 204.[25] P. Jackson, Introduction to Expert Systems, (3rd) ed, Pearson Addison Wesley, 1999.[26] D. Kull, “Group decisions: Can a computer help?”, Computer Decisions, vol. 15(5), 1982, 64-70.[27] Y. Liou, and J. Nunamaker, “Using a Group Decision Support System Environment for Knowledge Acquisition:

A Field Study”, Proceedings of the ACM SIGBDP conference on Trends and directions in expert systems. Orlando, Florida, United States, 1990, 212 – 236.

[28] D. S. Murphy, and S. A. Yetmar, “Auditor evidence evaluation: Expert systems as credible sources”, Behavior and Information Technology, 15, 1996, 14-23.

[29] S. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Princeton University Press, 2007.

[30] E. H. Shortliffe, and B. Buchanan, “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”, 1984.

References[31] E. H. Shortliffe, “Computer-based medical consultation MYCIN”, American Elsevier, 1976.[32] H. Simon, Administrative Behavior: A Study of Decision- Making Processes in Administrative

Organizations, 4th ed. The Free Press, 1997.[33] B. Smith, Collective Intelligence in Computer-Based Collaboration, Hillsdale, NJ: Lawrence Erlbaum,

1994.[34] R. J. Sternberg, “Cognitive conceptions of expertise”, In P.J. Feltovich. K. M. Ford and R. R. Hoffman.

Expertise in context. Human and Machine, Menlo Park, CA: AAAI Press/ The MIT Press, 1997. 149-162.[35] J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective

Wisdom Shapes Business, Economies, Societies and Nations, Little, Brown, 2004.[36] P. Tynjala, “Towards expert knowledge? A comparison between a constructivist and a traditional

learning environment in the university”, International Journal of Education Research. 31, 1999, 357-442.

[37] J.P. Shim, M. Warkentin, J.F. Courteny, D.J. Power, R. Sharda, and C. Carlsson, “Past, Present, and Future of Decision Support Technology”, Decision Support Systems, 33, 2002, 111-126.

[38] S. Weiss and C. A. Kulikowski, A Practical Guide to Designing Expert Systems, Chapman and Hall Ltd, 1984.

[39] S. E. White, J. E. Dittrich and J. R. Lang, “The Effects of Group Decision Making Process and Problem Situation Complexity on Implementation Attempts”, Administration Science Quarterly, 25(2), 1980, 428-440.

[40] J. Wolfers and E. Zitzewitz, “Prediction Markets in Theory and Practice”, In S. Durlauf and L. Blume (eds) The Palgrave Dictionary of Economics, McMillan, 2nd ed, 2006

[41] M. Woodridge and N.R. Jennings, “Intelligent Agents: Theory and Practice”, The Knowledge Engineering Review, 10(2), 1995, 115-152.

Terms and Concepts

What is Expertise?• Multi-dimensional

(Sternberg, 1997)• Highly specialize or

domain specific (Chi, Glaser, & Farr, 1988)

• Skills honed through practice (Jackson, 1999)

• Perform consistently more accurate in relation to others (Hartely, 1985) Practical Knowledge Se

lf-reg

ulativ

e

Know

ledgeFo

rmal

Kno

wled

ge

Expertise Dimensions (Tynjala, 1999)

Objective To represent expertise to its users for decision-making when a human expert can not be found or is in short supply.

Playing a critical role for organizations and a source for competitive advantage (Gill, 1995).

Contributing to decision-making through their representation of knowledge and reasoning of human experts for its users (Weiss & Kulikowski, 1984).

By mimicking and replicating the cognitive process of a human expert novice users can be supported to perform as well as experts (Cascante et al, 2002).

Users of ESs learn faster for decision-making that would occur through development over time.

Leveraging Expertise

Expert Systems

GDSSs

CI Tools

Conclusions

• Showed an evolutionary perspective of expertise supported decision support technologies.

• Highlighted how organizational expertise in short supply can be augmented.

• Explored issues of design for integration with existing methodologies.

• Indicated how organizational use of expertise is changing which reflects the new roles of experts and non-experts in decision-making.

– Replicating the knowledge and reasoning of human expert (Cascante et al, 2002).

ExpertSystems

– To capture the knowledge and contributions from the individual users for group consensus and collaboration (DeSantis & Gallup, 1987).

GDSSs

– To facilitate the summative body of knowledge, information, and resources of its users and the premise that the collective judgment of many are better then individual experts or groups at predicting or problem-solving (Surowiecki, 2004; Page, 2007).

CI Tools

Leveraging Expertise

Box-Office Forecasting: An Analysis of Expert Systems and Collective

Intelligence Tools

FILM INDUSTRY APPROACHES TO LEVERAGING EXPERTISE

– New movies consisting of long production cycles, complicated logistics, and high production costs reinforce the need for correct and accurate forecasts for box office revenue.

– Box office success prior to movie production and release are ridden with high levels of uncertainty for customer demand (Sawhney & Eliashberg 1996, Eliashberg et al. 2000), which has lead to a high percentage of films failing; further emphasizing that film forecasting is largely nonscientific.

– With high financial stakes involved and significant failure rates it is understandable that movie studios and production houses are willing to invest in new methods to accurately predict box office success (Davenport & Harris, 2009).

Espagogix Ltd. a UK-based company

– Epagogix’s proprietary expert system, based on a neural network (algorithms) analysis, uses movie script attributes to correlate future success or failures on past movies produced.

– The expert system uses variables such as genre, star actors, technical effects, and release time to find statistical patterns to forecast financial success.

– In addition, to being able to forecast, Epagogix’s expert system is also able to make recommendations either to reduce costs based on production or increased opportunities for greater box off revenue success (Davenport & Harris, 2009).

Hollywood Stock Exchange (HSX)

– Hollywood Stock Exchange (HSX) (www.HSX.com) is an internet based stock exchange where participants trade shares of virtual stocks that bet on the future box office revenue of movies.

– Participants buy and sell shares of movie stocks (like a real stock market).

– Web, traders are able to make new predictions as they acquire new information, making trades reflect in real-time stock prices and predictions of the future market demand for the movie.

– The Web for combined with the collective knowledge of many has transformed the ambiguities and uncertainties of movie forecasting into a consumer interactive predictive market.