Learning From our PracticeConnecting Curiosity with Evaluation Tools to Learn

from and Assess ADR Programs and Practices

1

Thank you for joining this conversation.Thanks to the folks who organized this...

Cindy MazurPamela Pontillo Stephanie Fell

2

UB Program in Negotiations & Conflict Management

John Windmueller, Ph.D.

3

Scholarship

• PhD from ICAR in 2006• MA and BA in International

Relations• Teaching background at UB,

ICAR, CAR, & NCC• Research focus on civic

engagement and civil society; dialogue; conflict resolution evaluation and assessment methodologies; and emerging practices of narrative and frame-based conflict interventions

• Began practice work as a court and community mediator in 1993

• Practice work has included conflict resolution, facilitation, and consulting work in the contexts of large-scale community conflicts, organizational conflicts, and international development

Practice

4

The vast bulk of what we do as conflict resolution practitioners is not backed or guided by research.

We should and can change that.

5

6

7

http://www.xkcd.com8

I have some ideas about how conflict works and how to constructively intervene in conflict.

Those ideas might be wrong.

I should test them.

9

Community Sensemaking & Healing in the Wake of Political Violence

10

1991 Mount Pleasant Riot

11

9/11

12

13

14

Community Resilience Project

15

Dialogue

16

By dialogue I mean...

A deliberate process of conversation where emphasis is put on building understanding and empathy among the participants.

17

DebateArgument

Consensus BuildingAction Planning

Everyday Conversation

18

Theoretical parents of “modern” dialogue:

David BohmBohm, D. (1996). On dialogue. New York: Routledge. [originally a

series of papers done between 1985-1991]

Martin BuberBuber, M. (1970). I and thou (W. Kaufmann, Trans.). New York:

Simon and Schuster. [original in 1923]

19

• Quantum Physicist

• None of us see the whole picture.

• We need to improve the process and quality of our communication.

• Emphasis is on growing understanding of complex problems.

David BohmBohm, D. (1996). On dialogue. New York: Routledge.[originally a series of papers done between 1985-1991]

20

• Jewish, philosopher

• Distinguishes between I/it versus I/thou (Ich-Du and Ich-Es) ways of interacting and relating, and hence way of being, in the world

• Stresses the importance of empathy and relationship

Martin BuberBuber, M. (1970). I and thou (W. Kaufmann, Trans.). New York:

Simon and Schuster. [original in 1923]

21

What do we know about dialogue?

22

0

23

What we know about dialogue:

• Those who facilitate dialogue describe it in positive terms, anecdotally reporting that it raises empathy and understanding.

• There are many (again, anecdotal) reports of participants in dialogue reporting that they found the experience worthwhile.

• There are many (again, anecdotal) reports of dialogues that produced later collective actions by the dialogue participants.

24

Methodology

Grounded TheoryNarrative Analysis

25

Shifts Happen!

• Relational ShiftsMove from talking at to talking with over time

• Frame / Narrative / Discourse ShiftsThere is a substantive negotiation taking place over how to jointly make sense and meaning in the wake of political violence.

26

We need to get over ourselves!

Facilitators had the lowest ability in the room to influence the quality and direction of dialogue

27

Participants Matter!• Substantive diversity plays a significantly

positive role in dialogue shifts

• Participant skills have a greater influence than facilitators’ in shaping dialogue quality Open and revealing about their personal experiencesDiscussed what others saidMade personal connectionsReached out to silent membersRecognized both common ground & differencesForegrounded tacit frame negotiations

28

The Clever Models?

Failed

29

Research and evaluation can help prove the value and results of what we do.

Research and evaluation can challenge our assumptions and reshape our practice.

30

Evaluation & Assessment is something you do

every day

31

really

32

You probably even did it on your way here.

33

34

35

36

37

38

39

40

41

42

What would your ideal mediation dashboard look like?

43

44

Evaluation & Assessment 101

45

(that thing you already informally do every day)

46

Don’t Panic

It’s About Curiosity

It’s interesting, challenging, and fun.

47

48

49

50

51

Guiding Evaluation Principles

• Systemic inquiry

• Evaluator competence

• Integrity and honesty

• Respect for the security & dignity of evaluation stakeholders

• Consideration of general and public welfare

52

good evaluation...

• Can surprise you and disprove your assumptions

• Produces data you are curious about

• Produces data that can influence action

• Is designed with stakeholders

• Evolves

53

examples of data collection

54

55

Conflict Climate Assessment

56

57

Where can you get data?

58

Measurement can Matter“The score turned an intangible and impressionistic clinical concept—the condition of new babies—into numbers that people could collect and compare. Using it required more careful observation and documentation of the true condition of every baby. Moreover, even if only because doctors are competitive, it drove them to want to produce better scores—and therefore better outcomes—for the newborns they delivered.”

Virginia Apgar

Gawande, A. (2007). Better: A surgeon's notes on performance. New York: Metropolitan Books.

59

Why not anecdote?

60

what color is this slide?

61

Anecdote -> small N

62

The naked eye was inadequate for learning what you need to know to evaluate players. Think about it. One absolutely cannot tell, by watching, the difference between a .300 hitter and a .275 hitter. The difference is one hit every two weeks...If you see both 15 games a year, there is a 40 percent chance that the .275 hitter will have more hits than the .300 hitter...The difference between a good hitter and average hitter is simply not visible--it is a matter of record.

-- Bill James

Anecdote -> small N

63

There is no controversy in social science which shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction as this one. When you are pushing over 100 investigations, predicting everything from the outcome of football games to the diagnosis of liver disease, and when you can hardly come up with half a dozen studies showing even a weak tendency in favor of the clinician, it is time to draw a practical conclusion.

- Paul Meehl (cited in Ayres)

Expert judgement isn’t what it’s cracked up to be

64

Get data from many sources

65

Tools for Data Analysis

• Excel

• SPSS

• SAS

• JMP

• R

• StatCrunch

• Nud*ist

• NVivo

• AtlasTI

• HyperResearch

• TAMSAnalyzer

66

67

68

69

What does research have me curious about and questioning?

The role of mindset on learning/trainingOur (in)ability to predict satisfactionQuirky human decision making & heuristics The neurobiology of conflict resolution

70

Mindset1. Your intelligence is something very basic about you

that you can't change very much.

2. You can learn new things, but you can't really

change how intelligent you are.

3. No matter how much intelligence you have, you can

always change it quite a bit.

4. You can always substantially change how how

intelligent you are.

FIXED

GROWTH

Dweck, C. (2006). Mindset. New York: Ballantine.

Negotiation ability, like plaster, is pretty

stable over timeFIXED

GROWTHNegotiation ability is changeable and can

be developed

71

Mindset

Participants w/Growth mindset were more likely (88% vs 53%) to tackle challenging negotiation scenarios

Participants w/Growth mindset were typically twice as successful in negotiation scenarios

Participants w/Growth mindset earned higher course grades

Kray, L., & Haselhuhn, M. (2007). Implicit theories of negotiating ability and performance: Longitudinal and experimental evidence. Journal of Personality and Social Psychology, 93, 49-64.

Negotiation ability, like plaster, is pretty

stable over timeFIXED

GROWTHNegotiation ability is changeable and can

be developed

72

We are terrible at predicting the happiness that will result from different outcomes

73

Anchoring & Framing

Risk aversion

Status quo bias

Optimism and over-confidence

Those Quirky Decision Makers

Schwartz, B. (2004). The paradox of choice. New York: Harper Collins.

Thaler, R., & Sustein, C. (2008). Nudge. New Haven: Yale Universiy Press.

Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

74

How many African countries are members of the UN ?>< 10 -- average answer 20>< 60 -- average answer 45

Anchoring & Framing

75

Imagine that you have decided to see a concert where admission is $20 a ticket. As you enter the concert hall, you discover that you have lost a $20 bill. Would you still pay $20 for a ticket to the concert?

Imagine that you have decided to see a concert and already purchased a $20 ticket. As you enter the concert hall, you discover that you have lost your ticket. The seat was not marked and the ticket cannot be recovered. Would you pay $20 for the additional ticket?

76

Get $100

Flip a coin. If it’s heads, you get $200

Would you rather...

(Probability of success x Gain) - (Probability of failure x Loss)(.5x200)-(.5x0) = 100

Average amount it typically takes to induce the risk: $240

Risk Aversion

77

Risk Aversion: We are more sensitive to loss than gain

78

To keep your mug

To buy a mug

How much would you pay...

People bidding to keep mugs value them at almost twice the level of those bidding to get mugs

Status Quo Bias

79

<5 % of MBA students predict that they will be in the lower 50% of their class.

94% of professors report that they are better than the average professor.

Optimism & Overconfidence

80

The neurobiology of conflict resolution

?

81

So what next?

82

vs.

83

On the tail end of things...

Redelmeier, D., & Kahneman, D. (1996). Patients' memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain(116), 3-8.

84

Discomfort Stage 1

Discomfort Stage 2

Discomfort Stage 3

Discomfort Stage 4

8 5 8

8 5 8 3

A

B

85

All’s well that ends well...

86