Proceedings of the 7th Annual ISC Graduate Research SymposiumISC-GRS 2013

April 24, 2013, Rolla, Missouri

Ravi ArvapallyDepartment of Computer ScienceMissouri University of Science and Technology, Rolla, MO 65409

IDENTIFYING OUTLIER OPINIONS IN AN ONLINE INTELLIGENT ARGUMENTATION SYSTEM FOR COLLABORATIVE DECISION SUPPORT

ABSTRACTThe intelligent argumentation system allows stakeholders to post their decision issues, alternatives and exchange their arguments over the alternatives posted in an argumentation tree. In an argumentation process stakeholders have their own opinions and very often these opinions might be contrasting and conflicting with others in the decision making group. Some stakeholders have opinions which might be outlier with respect to the mean opinion of the group. In this article, we present a unique approach to identify stakeholders with outlier opinions in the argumentation process. Our approach identifies outlier opinions both based on a stakeholders’ opinion as well as the collective opinion on a stakeholder’s opinion. The decision maker and other participating stakeholders have the opportunity to explore the outlier opinions existing in their group both from an individual’s perspective as well as from the group’s perspective. In a large argumentation tree, it is often difficult to identify stakeholders with outlier opinions manually. We also presented an experiment to evaluate our proposed method.

1. INTRODUCTIONIn a collaborative dialog process, stakeholders exchange their views and opinions with reasoning. In the dialog process as stakeholders exchange arguments, some change their opinion, some strengthen, and some weaken their opinions [18]. It is important to consider the opinion of each stakeholder in the argumentation process for collaborative decision support.In a collaborative decision making group, argumentation is a crucial step and each stakeholder has unique preferences, and opinions in the argumentation process. In a decision making group, some stakeholders approach the problem very uniquely and their opinions are farther away from any another stakeholder in the opinion dimensionality. We refer those opinions as the outlier opinions, since they are very different from the individual opinions of the group. In face-to-face discussions, one might be able to identify his peers with outlier opinions.By identifying the outlier opinions, the decision maker can closely investigate the arguments posted by that stakeholder, and consult that stakeholder to get in to more discussions. This can refine the opinions of the group and help them develop consensus within their group. It is the responsibility of the decision maker and the decision making group to understand

the underlying semantics of the outlier opinion. This problem is new to the domain of the argumentation systems. Few researchers have worked on identifying extreme opinions or ideas on Web-blogs and other social media such as YouTube [3].Earlier based on the similarity measurement, a framework was developed to identify the polarization groups in argumentation system [2]. But the framework in this section considers the dissimilarity of each stakeholder’s opinion with the mean opinion of the group and generates the ranked list of stakeholders based on their dissimilarity values.Our framework carries out two different methods in identifying outlier opinions. First, the outlier opinions are identified based on the aggregate opinion of a stakeholder which is computed from his arguments. In the second method, the collective determination score received by each stakeholder is computed and the outlier detection algorithm is used. The results produced by these two methods are later analyzed and compared. We used a simple distance based outlier detection algorithm to identify the outliers and inliers opinions in the intelligent argumentation system. In the argumentation process not only those viewpoints of a majority of participants need to be analyzed but also those unique viewpoints from minority of participants. Outlier opinions can accelerate new discussions and possibly refine opinions of participating stakeholders. This information can help the decision maker in taking more appropriate actions during the decision making process. The next section presents an update on the related literature work, followed by a brief background on the intelligent argumentation system. The opinion dissimilarity framework for identifying outlier opinions is presented which is followed by the experiment and conclusion of the article.

2. LITERATURE REVIEWThe following sub-section presents a brief introduction

about computer supported argumentation systems which is then followed by a brief literature on outlier detection algorithms.

2.1.Computer Supported Argumentation SystemsA computer supported argumentation system allows

stakeholders to post their alternatives over issues and exchange of arguments over the alternatives. An argumentation system is developed based on a model. Some systems are developed

1

based on formal models, and some on informal models. Our intelligent argumentation system is based on an informal model.

Araucaria [1], Belvedere [20], ConvinceMe [5], and Debatebase [6] are some of the argumentation systems that are developed to support argumentation among students in education. While some systems such as Carneades [30] are developed considering the application in law and justice. Several of these systems support basic support for constructing argument trees or well known as argument diagramming. The tradeoff between these systems is primarily based on two different criteria, ability to express well and usability of the system. Systems based on the formal argumentation models provide great ability to express. This is a great benefit to the participating stakeholders. At the same time these systems are less usable in reality. On the other hand, argumentation systems based on the informal models provides great usability but less expressing ability. CoPe_it! [10, 11] is one of the good web-based argumentation systems developed in recent times. This system provides different projections. The basic projection allows a group to discuss and construct trees. As one goes from one projection to the other, the formality of the underlying models gets stronger. This special feature allows for great expressiveness and more usable at the expense of switching projections.

Our major interest lies in the argumentation systems developed based on informal models and specially built for collaborative decision support. One could develop algorithms for processing arguments in informal model based argumentation systems and provide decision support. The Synergy system [28], Deliberatorium [12, 13], HERMES [9], CoPe_it! [10, 11] and the intelligent argumentation system [16, 17] are more easy to use in reality. The Synergy system is relatively a newer one. The MIT Deliberatorium allows a group to exchange arguments and construct argument maps. The researchers are currently working on implementing several deliberation metrics [14] that encourages a group to carry out more discussions and it provides sort of social dynamics information that exists in a group to the group stakeholders.

Several important social dimension aspects exist in face-to-face meetings and discussions. In group discussions and debates, with analytical ability one can identify people with outlier opinions. This identification helps others in the group to engage in more discussions to overcome the conflicts in opinions. Sillince and Saeedi [23, 24] discovered that several important aspects of face-to-face meetings and discussions such as social, emotional and symbolic requirements are missing in the current group decision support systems and computer supported argumentation systems. Our effort is to incorporate this special feature into the intelligent argumentation system. We are not aware of any computer supported argumentation system that identifies stakeholders with outlier opinions in group discussions.

2.2.Outlier Detection AlgorithmsAccording to Hawkins “Outliers are observations which

deviate significantly from other observations as to arouse suspicion that these are generated by a different mechanism [8]”. Outlier detection techniques are very crucial in the data

mining applications. The outlier detection algorithms have been well applied and used in the complex network systems. Gogoi et al., in their article classified the outlier techniques broadly in to distance-based, density-based, and machine learning based techniques [7]. In this article, we intend to use the outlier detection techniques in the argumentation network. There are several supervised outlier detection techniques and unsupervised outlier detection techniques.

For using supervised outlier detection techniques, one should have training data, in our case it would be more appropriate to use unsupervised outlier detection techniques. There are several other distance based statistical approaches to identify outlier detection techniques and there are also other soft computing outlier detection techniques using fuzzy systems. Several outlier detection algorithms such as K-nearest neighbor’s method [21], local-distance based methods [4, 25], angle based outlier detection [15] and density based outlier detection algorithms [19] are used in several applications so far but not in argumentation systems.

3. INTELLIGENT ARGUMENTATION SYSTEMThis section presents the elements of an argumentation tree and the argumentation reduction fuzzy inference system. The intelligent argumentation system supports several elements in its argumentation tree. A stakeholder can post decision making issue under a project, while a project node can have several child issue nodes. On issue node, one can attach several alternatives pertaining to the decision making issue. Figure 1 presents the position dialog graph that represents the position sub-tree in an argumentation tree. The position in Figure 1 is an alternative solution pertaining to the decision making issue. Arguments can be either directly or indirectly associated with the position. A stakeholder can post arguments supporting or attacking a position or an argument in an argumentation tree.

When a stakeholder posts an argument, he is responsible to post the degree of strength of his argument along with the argument text. This degree represents the strength and association with its parent argument. The strength of an argument ranges from -1 to +1. If a stakeholder’s argument is attacking its parent node, then the stakeholder can select a strength value from -0.1 to -1.0. If a stakeholder’s argument is supporting its parent node, then the stakeholder can select a strength value from +0.1 to +1.0. If an argument is neutral or indecisive, then a stakeholder can post zero strength. A stakeholder can support or attack other arguments or his own arguments. An individual can post evidences supporting his/her arguments. Figure 2 presents a snapshot of the intelligent argumentation system.

The argumentation reduction fuzzy inference system performs inference on the argumentation tree to compute the favorability (support) of an alternative in the decision making group. The inference system performs inference and ensures that at the end all the arguments in a tree are directly associated to its’ respective alternatives. This inference process is not visible to stakeholders.

2

Figure 1. Position dialog graph

Figure 2. A snapshot of the intelligent argumentation system

The inference process is performed using the strengths of the arguments and the inference rules. When an argument has to undergo inference process, the strength of the argument and the strength of it’s parent argument are provided as inputs to the argumentation reduction fuzzy inference system. The inference process is conducted based on the general fuzzy heuristic rules. The following examples are some of the general fuzzy heuristic rules.

• If argument B supports argument A and argument A supports position P, then argument B supports position P.

• If argument B attacks argument A and argument A supports position P, then argument B attacks position P.

Strong Attack (SA), Medium Attack (MA), Indecisive (I), Medium Support (MS), and Strong Support (SS) are the five fuzzy linguistic labels. These labels are based on the fuzzy membership functions and are not visible to the participants during the inference process. Based on the combination of these five variables, twenty fuzzy inference rules are developed. After the inference process, the aggregation on strengths of the arguments will be performed to compute the favorability of an alternative. This inference system will be used further in the framework section to compute a stakeholder’s opinion. For more information about the system please refer to some papers on intelligent argumentation system [16, 17].

4. FRAMEWORKFigure 3 presents the proposed framework for identifying and assessing outlier opinions in an argumentation process from both the individual and the collective viewpoint. An argumen-tation tree is provided as input for processing and analyzing ar-guments. The framework will generate the ranked lists of out-lier opinions. The first sub-section presents how the outlier opinions are identified from the individual viewpoint (method 1) and the second sub-section presents from the collective viewpoint (method 2).

4.1.Method 1 – Individual OpinionsStep 1 –In the first step of this framework, the system carries out the argumentation reduction inference process using the fuzzy argumentation reduction inference system on the argu-mentation tree. This process is carried out to compute a stake-holder’s aggregate opinions. In the argumentation tree the argu-ments are either directly associated or indirectly associated with their respective positions. After the argumentation reduction inference process, all arguments are directly associated to their respective positions [16, 17]. In Figure 4, stakeholder S2 has contributed three arguments under position 1. While one argument is directly associated with position 1, and the other two are associated with the arguments posted by stakeholder S1. The association between (Arg1, position 1) and (Arg4, Arg1) are considered for using the appropriate fuzzy inference rules. Based on the suitable fuzzy rule, the Arg4 is reduced level by level such that it is directly associated to position 1. The same procedure was conducted for Arg6. The system ensures that all arguments posted by a stakeholder are directly associated to an argument. The argument based fuzzy inference system then reassesses the strengths of the arguments based on the inference rules. The new strength that an argument is assigned is relative to the solution alternative. See Figure 5 for the argumentation tree after the fuzzy inference process. The favorability of stakeholder S2 for position 1 is the aggregate of the argument’s strength: Arg2, Arg4, and Arg6 (see Figure 5).

Step 2 – In the second step, the strengths of the arguments posted by a stakeholder are aggregated to compute the overall favorability of a stakeholder for that alternative. There by the system computes the stakeholder’s favorability for each position to compute a stakeholder’s favorability for that position. The favorability of a stakeholder for all the alternatives is represented as a vector, called as opinion vectors. Each element in the vector presents the favorability of stakeholder for a position.

Step 3 – The opinion vectors can be represented in the opinion dimensionality. The opinion vectors are normalized using the min-max normalization technique (Eq. 1). Min A and max A represent the minimum and the maximum values in the data set respectively. While we assign new_max A to +1 and new_min A to -1, since we want the new data to be normalized within the range of -1 and +1.

(1)

3

v '= v−min Amax A−min A

(new _max A−new _min A )+new _min A

Algorithm 1 - Distance based dissimilarity algorithm

Input: Opinion vectorsOutput: Ranked list of opinions based on the farthest from

the mean opinion of the group.

Step1 – Compute the mean vector (X) of the input opinion vectors (Y).

Step2 – Compute the Euclidean distance (Eq2.) from the mean opinion vector and all input opinion vectors.

Step3 – Sort and generate the ranked list based on the distance between opinion vector and mean vector.

Figure 3. Framework for identifying outlier opinions in the argumentation process

Figure 4. Sample argumentation tree before argument inference

Step 4 – The distance based outlier detection algorithm (Algorithm 1) will be applied on the vectors to generate a ranked list based on the opinion vector’s distance with the mean opinion of the group. We used a simple distance based outlier detection algorithm on our data (Algorithm 1).

Step 5 – Analyze results – The ranked list generated in step 4 is used for analyzing the results. The top stakeholder (opinion) in the list presents the opinion which is farthest from the mean opinion of the group. The last element in the generated list presents the opinion which is closest to the mean opinion of the group.

4.2. Method 2 – Collective OpinionsStep 1 – On each argument in the argumentation tree, the argumentation reduction inference process is carried out using the fuzzy argumentation reduction inference system. Each argument’s collective determination value will be derived, see Figure 6. In method 1, the argumentation reduction process was carried out on an alternative of a tree. But here, the argumentation reduction process is carried out on each and every argument to compute the favorability of each argument

from other arguments in the tree. In Figure 6, A1, A2, A3, A4, A5, and A6 represents arguments.

Figure 5. Argumentation tree after argument inference

Step 2 – In this step, the collective determination values of all arguments of each stakeholder under a position are aggregated. The aggregate collective determination of a stakeholder on all positions is derived and represented as a vector. This process will be carried out for all the arguments and stakeholders in the argumentation process. Each element in a vector represents the aggregate collective determination that a stakeholder’s arguments have received under a position.

Step 3 – The vectors are then normalized using min-max normalization technique (Eq. 1) to attain consistency in the data. This determines the total collective determination received by a stakeholder under a position. This process is carried out for all stakeholders across all positions posted under an issue in the argumentation tree.

Step 4 – The distance based outlier detection algorithm (Algorithm 1) that is presented in algorithm 1 is also used here to generate a ranked list of inliers and outlier opinions.

4

Figure 6. Computing collective determination of arguments

Step 5 – The first opinion in the ranked list presents the opinion (outlier) which is farthest from the mean opinion of the group. The last element in the generated list presents the opinion which is closest to the mean opinion of the group. The list generated here is from the group’s perspective.

5. EMPIRICAL EVALUATIONIn this section we present the empirical evaluation of the proposed method. The data in our experiments is from the experiments that were conducted in spring 2010 [22]. The decision issue was about the selection of software metrics program for a large scale organization for the given project described in the case study. Comprehensive metrics program, light metrics program, and no metrics program were the different positions posted.Initially the twenty four stakeholders in the decision making group were provided with the decision making issue and the relevant positions. The decision making issue and the positions were posted in the argumentation tree for stakeholders to participate. Stakeholders exchanged arguments over different positions for one week of time. After the argumentation tree was constructed, the developed framework was applied on the argumentation tree. Both the individual perspective and the collective perspective sub-frameworks were applied on the tree. This framework has then produced the results which are explained in detail in the following sub-section.

Results –Table 1 presents the ranked list of stakeholder opinions based on both the individual method and the collective perspectives. S18 is the opinion of stakeholder number eighteen. Stakeholder S18 ranks one in the outlier ranked list, while stakeholder S7 is ranked last in the list. Stakeholder S18’s opinion is farthest from the mean opinion of the group while stakeholder S7’s opinion is closest to the mean opinion of the group. From the collective perspective framework, S20’s opinion is ranked one (outlier) and stakeholder S2’s opinion is ranked last in the outlier list. A decision maker can also generate the top-K list of stakeholders with outlier opinion and inliers opinions using the results in Table 1. The information from Table 1 can be used to see the ranking and the overlap of rankings in the outlier opinion ranks.Stakeholder S12’s opinion is ranked as outlier number seven from both the individual method as well as the collective determination method. There are multiple cases one can analyze, for example, ‘From an individual’s perspective, his opinion is outlier, but from the collective perspective, his opinion is not an outlier’.

Table 1. Ranked list of outliers based on both individual and collective methods

Ranked list of outliers based on the stakeholders’ individual opinions.

Ranked list of outliers based on the collective determination values.

S18, S19, S21, S23, S9, S17, S12, S20, S16, S2, S8, S5, S4, S14, S15, S1, S10, S11, S22, S3, S24, S6, S13, S7

S20, S8, S23, S17, S22, S19, S12, S21, S13, S24, S5, S10, S9, S3, S4, S16, S11, S6, S14, S1, S18, S7, S15, S2

The results presented in Table 1 will fall in one of the cases presented above. This classification provides a better understanding of the opinions in a decision making group. This is one way of understanding the dynamics involved in a decision making group during an argumentation process. We do not claim that a stakeholder with outlier opinion is good or bad. Our model just identifies the outlier opinions and lets the group decide on how to use this information.

6. CONCLUSIONS It is important to consider every opinion in the argumentation process during decision making. Outlier opinions do exist in argumentation in a collaborative environment. Outlier opinions are the one which are farther away from the mean opinion of the group. When outlier opinions are contrasting with the rest of the group, this could lead to more argumentation and debate. It is also important to identify outlier opinions from both an individual’s perspective and a group’s perspective. We also learn from the experiment that there is a need to evaluate the method by using several other well established outlier detection techniques.

7. ACKNOWLEDGMENTS We would like to thank the Intelligent Systems Center at Missouri S&T for their support.

8. REFERENCES[1] Araucaria, URL: http://

araucaria.computing.dundee.ac.uk/doku.php#download, accessed on 2/15/2013.

[2] Arvapally, R.S., Liu, X. F., and Jiang, W., 2012, “Identifi-cation of faction groups and leaders in Web-based intelli-gent argumentation system for collaborative decision sup-port,” Proceedings of the International Conference on Collaboration technologies and Systems (CTS 12), pp.509 – 516.

[3] Bermingham, A., 2009, “Combining Social Network Analysis and Sentiment Analysis to Explore the Potential for Online Radicalisation,” Social Network Analysis and Mining (ASONAM 2009), pp. 231 – 236.

[4] Breunig, M. M., Kriegel, H. P., Ng, R. T. and Sander, J., 1999, “OPTICS-OF: Identifying local outliers,” Proceed-ings of the third European conference on principles of data mining and knowledge discovery, Prague, pp. 262-270.

[5] ConviceMe - Start a debate, URL: http://www.con-vinceme.net/, accessed on 2/15/2013.

5

[6] Debatebase, URL: http://idebate.org/debatabase, accessed on 2/15/2013.

[7] Gogoi, P., Bhattacharyya, D. K., Borah, B., and Kalita, J. K., 2011, “A Survey of Outlier Detection Methods in Net-work Anomaly Identification,” The Computer Journal, Vol. 54 (4), pp. 570-588.

[8] Hawkins, D., 1980, “Identification of Outliers,” Chapman and Hall, London.

[9] Karacapilidis, N., and Papadias, D., 2001, “Computer sup-ported argumentation and collaborative decision making: the HERMES system,” Information Systems, Vol. 26, pp. 259–277.

[10] Karacapilidis, N., 2006, ”CoPe_it!: Towards augmenting collaboration and learning in communities of practice,” Workshop in Modelling Meetings, Argumentation and Discourse, in Proceedings of the first International Con-ference on Computational Models of Argument.

[11] Karacapilidis, N., Tzagarakis, M., Karousos, N., Gkotsis, G., Kallistros, V., Christodoulou, S., and Mettouris, C., 2009,” Tackling cognitively-complex collaboration with CoPe_it!,” International Journal of Web-based learning and teaching technologies, Vol. 4(3), pp. 22-38.

[12] Klein, M., 2010, “Using Metrics to enable large-scale de-liberation,” Collective Intelligence in organizations: A workshop of the ACM Group 2010 Conference, Sanibel Island, Florida.

[13] Klein, M., 2012a,”How to harvest collective wisdom on complex problems: An introduction to the MIT Delibera-torium,” MIT Center for Collective Intelligence, URL: http://cci.mit.edu/klein/papers/deliberatorium-intro.pdf, Date accessed: 2/14/2013.

[14] Klein, M., 2012b, “Enabling Large-Scale Deliberation us-ing attention-mediation metrics,” Journal of computer supported cooperative work (CSCW), Vol. 21, pp. 449-473.

[15] Kriegel, H. P., Schubert, M., Zimek, A., 2008, “Angle-Based Outlier Detection in High-dimensional Data,” Pro-ceedings of the 14th ACM SIGKDD International Confer-ence on Knowledge Discovery and Data Mining (KDD '08), Las Vegas, NV.

[16] Liu, X. F., and Raorane, S., and Leu, M., 2007,”A Web-based Intelligent Collaborative System for Engineering Design,” in Li. W. D., Ong, S. K., Andrew, Y. C., and McMahon, C., Collaborative product design and manufac-turing methodologies and applications, Springer, pp. 37-58.

[17] Liu, X. F., Khudkhudia, E., Wen, L., Sajja, V., and Leu, M., 2009,”An Intelligent Computational Argumentation System for Supporting Collaborative Software Develop-ment Decision Making,” in Meziane, F., and Vadera, S. Artificial Intelligence Applications for Improved Software Engineering Development, IGI Global, pp. 167-180.

[18] Liu, X. F., Wanchoo, R., and Arvapally, R. S., 2011,”Em-pirical study of an intelligent argumentation system in MCDM,” Proceedings of the 2011 IEEE International Symposium on Collaborative Technologies and Systems (CTS 2011), Philadelphia, pp. 125-133.

[19] Ng, R. T. and Sander, J., 2000,”LOF: Identifying density-based local outliers,” Proceedings of the 2000 ACM SIG-

MOD international conference on management of data, Dallas, pp. 93-104.

[20] Paolucci, M., Suthers, D., and Weiner, A., 1995,”Belvedere: Stimulating Student’s Critical Discus-sion,” Proceedings of the Human Computer Interaction, Denver CO, pp. 123-124.

[21] Ramaswamy, S., Rastogi, R., and Shim, K., 2000,”Effi-cient algorithms for mining outliers from large data sets,” Proceedings of the SIGMOD international conference on Management of data, pp. 427-438.

[22] Satyavolu, M., 2010,”Contribution-based priority assess-ment in a web-based intelligent argumentation network for collaborative software development,” Thesis, Missouri University of Science and Technology, Rolla. URL:http://scholarsmine.mst.edu/thesis/pdf/Satyavolu_09007dcc8082f337.pdf , February 15th, 2013.

[23] Sillince, J. A. A., 1995,”A model of social, emotional and symbolic aspects of computer-mediated communication within organizations,” Journal of Computer Supported Cooperative Work (CSCW), Vol. 4 (1), pp. 1 – 31.

[24] Sillince, J. A. A., and Saeedi, M. H., 1999,”Computer-me-diated communication: problems and potentials of argu-mentation support systems,” Journal of Decision support systems (DSS), Elsevier, 26, pp. 287 – 306.

[25] Tang, J., Chen, Z., Fu, A. W., and Cheung, D., 2000,”A robust outlier detection scheme for large data sets,” Sixth international pacific – Asia conference on knowledge dis-covery and data mining, pp. 6-8.

[26] Toulmin, S., 1958,” The uses of argument,” Cambridge, Cambridge University Press.

[27] Turkett, W. H., Rose, J. R., and Huhns, M. N., 2003,”Massive Deliberation,” IEEE Internet Computing.

[28] Vesic, S., Ianchuk, M., and Rubtsov, A., 2012, “The Syn-ergy: A platform for argumentation-based group decision making,” Computational Models of Argument, Berheij, B. eds., IOS Press, pp. 501-502.

[29] Wiebe, J., Breck, E., Buckley, C., Cardie, C., Davis, P., Fraser, B., Litman, D., Pierce, D., Riloff, E., Wilson, T., Day, D., and Maybury, M., 2003,”Recognizing and Orga-nizing Opinions Expressed in the World Press,” Working Notes – New Directions in Question Answering, AAAI Spring Symposium Series, pp. 24 – 26.

[30] Gordon, T., and Walton, D., 2006, “The Carneades Argu-mentation Framework Using Presumptions and Excep-tions to Model Critical Questions,” Proceedings of the 2006 conference on Computational Models of Argument, pp. 195-207.

6