DEPARTMENT OF SOCIOLOGY

The Past, Present and Future of ABM: How To Cope With A New Research

Method

Edmund Chattoe-Brown (ecb18@le.ac.uk)

Plan

• Disagree with me!• Trying to draw published ideas together:

May be a bit untidy. Sorry.• Past: How does a “discipline” organise

itself? Does the neoliberal university “matter” to this?

• Present: A million “toy” models reproducing the prejudices/division of disciplines.

• Future: Identifying workable “procedures” for doing ABM.

Puzzle 1

Does anybody

recognise this?

Puzzle 2

What about this?

Spot the crucial

difference!

The Past

• What would happen if we introduced everyone to ABM via Hägerstand rather than Schelling? (It is now on OPENABM at my suggestion.)

• Do you recognise: Clarkson, Gullahorn, Grémy, Dutton and Starbuck, Bernstein, Loehlin, Kalick and Hamilton? (And I am not looking hard yet!)

• And (just) now: Hegselmann, R. (2017) ‘Thomas C. Schelling and James M. Sakoda: The Intellectual, Technical, and Social History of a Model’, Journal of Artificial Societies and Social Simulation, 20(3), <http://jasss.soc.surrey.ac.uk/20/3/15.html>.

What Is “Supposed” To Happen?• “Progress”: People agree what research is important,

how to do “good” work, what “the problems” of the field are and so on. Over time, it becomes possible to teach a “canon” (like Marx, Weber and Durkheim) so a discipline develops a shared sense of identity.

• Too new? How new are we now?• Too old without success? (Critiques lost too!)• Too obscure?• Too pressured to publish?• Too independent? (Too spread out?)• What can we do about it now?

This Does Matter (At “Level 1”)• Hägerstrand: Independently calibrated

models really can be validated effectively. (Still happening, still being ignored: Abdou and Gilbert.)

• Grémy/Boudon: To justify using ABM look at the broad pattern of data. Is it explained more easily by a simple trend?

• Chattoe-Brown: The Zaller-Deffuant model looks nothing like real data. (I’ll come back to that.)

I Can’t Resist: Abdou and Gilbert (2009)

It Also Matters (At “Level 2”)• ABM can agree its own standards but what

happens if everyone else doesn’t find those standards credible?

• Do we want to be largely separated from the rest of social science (like System Dynamics) or increasingly integrated (like Social Network Analysis).

• This depends on what we think ABM “is”. IMO it is a research method. How we do ABM depends on what we think it is.

Connections• Anecdata: My friend in health HR.• Laurence Droy: My current PhD student.• Have people been telling us to “do data” since the

sixties without making an impression? (Does Dutton and Starbuck report a higher proportion of calibrated and validated models than Angus and Hassani-Mahmooei? Uh-oh?) Will the rest of the world get fed up with us sooner or later? Is later getting sooner?

• Working to survive the over-confidence and rejection phases. (Is over-confidence more likely now?) AI? Carley? Helbing? What is the next thing after the Next Big Thing?

What Do We Do?• This is the easy one: More reading, more citing,

more practical “use” of good examples (in teaching for example).

• “Recovery” replications: I’m currently doing one (with Simone Gabbriellini) on Norman Hummon’s “rational” ABM of social network formation (2000). Anyone heard of that?

• More attempts to “agree” teaching and contributions: ESSA sig on education?

• Other: Integrating “non English” ABM/simulation.• Just being aware of the issue?

Quote Maybe About “Empirical” ABM

• “Christianity has not been tried and found wanting; it has been found difficult and not tried.” (G. K. Chesterton)

Present• Opinion dynamics and “element selection”.• “Rationality” or deliberate decisions: Opinions and

attitudes. Is there a “fact of the matter” involved?• Media effects (and real events). Feedback loops?• Membership of groups/parties.• Psychology: But it doesn’t “agree” (i. e. “backlash”

effects.) Replication crisis?• (Dynamic) networks.• Multiple opinions and opinion structure/consistency.• “Debate”.• Probably plenty more.• Can’t just pick some you “like” or ...

The Challenge• Putting x in a model is usually “not implausible”

but leaving it out implies no effect at all (which is often very implausible).

• Different models for different domains doesn’t really help. Just “puts the problem back”.

• Models you can calibrate: Apparently no qualitative data. Is it true, for example, that shift from “pro” to “anti” traverses “don’t care?”

• What data exists to be explained? (Validation.)• Methodology: Even validation is better than

nothing.

Example: Chattoe-Brown (2014)

Let’s Science The Hell Out Of This• My model is “no good” because I

included elements arbitrarily and barely calibrated it.

• Ideally my article would not even need to have been written.

• But my model at least matches stylised patterns in data (turning points).

• Please, somebody beat me!

Complications• Lots of legitimate uses of ABM but most of

them are only “intermediates” to empirical application IMO.

• ABM is very good at formalising theories but a theory that is complete and coherent still doesn’t have to be “true” (or “apply”).

• Whatever you say an ABM is for (“interesting thought experiment”) you have to say what would count as a success that is more than personal opinion. (Part of wider “corner cutting” in academia?)

What Do We Do About This?• Admit it!• Develop methodology to compare models

(probably has to be empirical).• At least try to build models that will touch

data (even if you fail). No methodology without reality. (Survey data example.)

• Connection: If we did calibration and validation “well” in 1965 (even just once), what have we been doing since?

Future• ABM tends to take existing research

methods for granted: What do statisticians (and ethnographers) do and why do they do that?

• Research design.• Element selection.• “Procedural methodology”: This is

what I did and why I did it. Can we agree it works?

Example (Statistics)

• Are English people more reserved than Italians?

• Measure(s) of reserve.

• Pilot survey: What scale of difference do we find justifying sample size?

• Doing a good (for example unbiased) survey.

• Analysis: Almost the least of it.

• Make sure that the data you need for your analysis (here just comparison of means) will actually be produced by your survey.

What Is Research Design?• Lin, Z. and Carley, K. (1995) ‘DYCORP: A Computational

Framework for Examining Organizational Performance Under Dynamic Conditions’, Journal of Mathematical Sociology, 20(2-3), pp. 193-217.

• “In an attempt to systematically address what factors affect organizational performance, we built a dynamic computational framework for examining organizational performance in which organizations are composed of intelligent adaptive agents. Using this framework the user can contrast organizations with different designs, existing in different task environments, and subject to different stresses. We demonstrate the value of this model by examining how training and stress affect organizational performance.”

• Am I being unfair? Let’s look more.

Methodology: The Next Step• Many people know the “Gilbert and Troitzsch box” (or

“generative methodology”) but it isn’t so often followed.• We need to know exactly how this “works” in practice.• How much can we “fit” models? If we do this don’t we just

end up with a model that matches anything?• What does sensitivity analysis really tell us?• What happens if we leave something (media effects) out of

a model? This is OK for calibration and validation (maybe it works anyway) but for fitting it is “mis-specification”.

• Useful ideas from statistics: Over-fitting, mis-specification, out-of-sample testing, turning points. How to use these.

• Don’t be downhearted: Generative models may even predict better (aim to be causal).

Example: Switchable modelsA model in which we

change only one “process”: How “dangerous” is leaving

out processes?

The Goal

• Methodology will never take the creativity out of ABM.

• But we need to agree, for example, what counts as a “match” between real and simulated data.

• Procedures for converting “personal opinion” into standards that are hard to disagree with (but we also need to sort out exactly what we are disagreeing about much of which isn’t published.)

Example: Bravo et al. (2012)

Real on left: “We built an experimentLike model that exactly

replicated the original experiment with

calibrated parameters.”

What Can We Do About This?• ABM with “research designs”.• Being as self-critical as possible: Ask yourself why you

assumed something before someone else does.• Adopting good practice (parameter tables).• Reality checking: How many papers don’t show real

and simulated data? How many don’t reference “substantive” research? How many don’t make it clear how they want to be judged?

• Think about the progressive dimension: Two models can be “not implausible” separately but not together. Mark Knopfler: “Two men say they’re Jesus. One of them must be wrong.”

Vision• ABM competing with each other to improve validation

fit, “strengthen” calibration, test prediction and so on.• Collaboration between disciplines (based on shared

“process based” approach and methodology) to build empirically based “modules” for ABM to reduce “reinventing the wheel”.

• More agreement on what ABM apprentices “need to know” (and why they need to) not just about the “best” models but on “how to” as regards ABM building.

• ABM as a specialised but integral part of social science (and social science in alliance to improve understanding generally rather than competing to impose understandings.)

Current activities• Interesting Social Network Analysis in “proper” ABM

(for example changing populations): With SG.• ABM for torture: Brexit!• ABM for anti-microbial resistance (funded).• Integrating models of “place” and social networks (with

Laurence Droy).• “Switchable” models.• Opinion dynamics (with Flache, Deffuant, Edmonds et

al.)• Organisational ecology.• Target family size: Combining qualitative and

quantitative in ABM.

Now Read On 1• Abdou, M. and Gilbert, N. (2009) ‘Modelling the

Emergence and Dynamics of Social and Workplace Segregation’, Mind and Society, 8(2), pp. 173-191.

• Chattoe-Brown, E. (2014) ‘Using Agent Based Modelling to Integrate Data on Attitude Change’, Sociological Research Online, 19(1), <http://www.socresonline.org.uk/19/1/16.html>.

• Chattoe-Brown, E. (2017) 'Agent-Based Modeling', in Spillman, L. (ed.) Oxford Bibliographies in Sociology (New York, NY: Oxford University Press).

Now Read On 2• Chattoe-Brown, E. (in progress) ‘Agent Based

Modelling’. [Currently only from the author.]• Chattoe-Brown, E. (in progress) ‘Why Questions

Like “Do Networks Matter?” Matter to Methodology’. [Currently only from the author.]

• Hägerstrand, T. (1965) ‘A Monte Carlo Approach to Diffusion’, Archives Européennes de Sociologie, 6(1), pp. 43-67.