Comparative Investigation of Collaboratories: Cross-Cutting Themes

SCHOOL OF INFORMATION .UNIVERSITY OF MICHIGAN

Comparative Investigation of Collaboratories:

Cross-Cutting ThemesJune 20, 2003

University of Michigan

Ann Arbor


Reminder: Where We’ve Been UM group – 15 years of experience with

distributed collaboration SOC project

– ~40 Collaboratories at a Glance (C@G)– 10 in-depth studies

• Sept. 02: SPARC/UARC, CFAR, Bugscope, EMSL• June 03: NEESgrid, InterMed, GriPhyN, iVDGL, AfCS,

BIRN

“The Literature” Your input


What We Learned Here

Review Cross-cutting Themes– Modify

• Refine• Eliminate• Add

Framework for generalizations– What leads to success, failure?

Source of design prescriptions– How to do the next one?


Cross-cutting themes: From Prior Work

Collaboration readiness– Collaboration vs. competition in science– Bottom-up vs. top-down origins

Technology readiness– Experience with collaboration tools

Infrastructure readiness– Both technical and social

Common ground– Extent of shared knowledge; critical in interdisciplinary work

Coupling of work– The interdependencies among individuals


Collaboration Readiness Can a collaboratory be mandated by an

external agency (e.g., funding source)?– NEESgrid – collaboratory capability as a condition

of funding• High risk – details in presentation & discussion

History of collaboration– High energy physics vs. earthquake engineering

Science driven– AfCS– BIRN


Common Ground NEESgrid

– Differences in terminology between CS & EE communities

InterMed– Importance of establishing shared vocabulary– Boundary objects, pidgins

GryPhyN, iVDGL– Too much common ground Boundary objects as key

concept [G. Bowker] BIRN

– Attention to metadata, ontology


Cross-cutting Themes from SOC Analyses

What is success?– Detailed discussion in June 2001 workshop

What are the incentives for participation?– Survey study in progress

What kinds of collaboratories are there?– Taxonomy – presented later

How do collaboratories evolve?– Some ideas based on our taxonomy – presented

later


What is Success?

Use of the collaboratory tools Software technology Direct effects on the science Science careers Effects on learning, science education Inspiration for other collaboratories Learning about collaboratories in general Effects on funding, public perception


Measures of Success GriPhyN, iVDGL

– Persist beyond ITR funding– Spending less time on tools, more on science

BIRN– Cover story in Nature– Lots of publications

Multiple audiences– Beyond the scientists– Students, government, industry, general public

• Collaboratory NSF STC


Incentives

AfCS– Alliance with Nature

BIRN– Guidance re publications

LHC– Shift in time scale of experiments– Implications for careers


Evolution***** Ecology of collaborations

– Movement from limited to full collaboration Data – wisdom hierarchy [G. Furnas]

– Movement up and down over time and space– Relates to social vs. technical processes

Where did the field come from, where is it going?– Historical context as critical

Multi-tasking of individuals (G. Mark) Time scale issues

– AfCS – bioinformatics earlier?– BIRN –savings across successive BIRNs


The relationships

Wisdom Knowledge Information Data The world

Shared Instruments

Distributed ResearchCenters

Practice and Expertise

Community Data Systems


Cross-cutting Themes from SOC Analyses Do collaborations have an ideal size?

– Collaboratories allow for larger ones– How do they scale?

What are various organizational models for how to structure collaboratories?

How does the control and flow of resources affect collaboratory success?– The money flow; the relation to the sponsor(s)

How much flexibility should be designed in?– What kinds of early commitments?– How much flexibility will funders allow?


Ideal size ATLAS

– Collaboration of 2000• But very organized

– Beyond ATLAS?• Manhattan• Apollo

How many working groups can be supported?– Organizational science as source of clues– What does technology enable?– How to scale from literature on teams (G. Mark)


Flexibility

Retrenchment, redefining of goals– G. Bowker – may be key to success

Funding models– AfCS – enough flexibility? (A. Prakash)

Adapting to new developments– InterMed – 1995 shift to focus on

guidelines– AfCS – 2003 changing cells


Cross-cutting Themes from SOC Analyses How important are data issues in

collaboratories?– Data seems to be a central component of all

collaboratories For what kind of work do you need real-time

vs. asynchronous interactions? How important is security? What’s the mix of tailor-made vs. off-the-shelf

tools?


Data Issues Metadata Provenance Persistence, archiving Rationale for transformations

– NEESgrid, GriPhyN, iVDGL, AfCS, BIRN Details of size, usage – different software needs? What level of processing? Different disciplines may

vary [D. Sonnenwald] Data sharing across jurisdictional boundaries – BIRN

– IRB – data from humans– International


Cross-cutting Themes from SOC Analyses How crucial are platform issues?

– What is the emerging role of middleware? What is the role of emerging infrastructure

such as the Grid? How does one move from early prototypes to

production versions of collaboratories? Why isn’t there more reuse of collaboratory

tools? To what extent are the issues specific to

science domain or are general?


Moving to Production Versions Tensions between CS and domain users

– NEESgrid – “innovation vs. extrapolation”– GriPhyN & iVDGL

Moving beyond initial demo stages– Slow adoption

• InterMed– Sustaining the investment

• NEESgrid – NEES consortium infrastructure set up in advance

• GriPhyN, iVDGL – seeking a sustaining support process• BIRN

Incentives– “build hardware” [J. Leigh]

Diffusion of Innovation literature


Domain specificity

The unusual character of HEP– Long history – since Manhattan– Scale – LHC– Common knowledge, self-esteem, etc.


New Issues Human subjects issues

– IRBs across jurisdictional boundaries– Need for new approach?

Management– NEESgrid – management lags implementation– InterMed – need for tight management– GriPhyN & iVDGL – hiring project managers– AfCS – charismatic management– BIRN – governance manual; adding steering committee

Vision– Who’s vision– “Acephalous” projects (G. Bowker)– Leadership issues – charisma


New Issues What kind of technology?

– Specific applications vs. APIs– Generic collab vs collab in specialized tools (S. Poltrock)– Economics of the Grid (M. Cohen)

Standards as a unifying process– Politics of standards setting– BIRN in a box

“If you build it, they will come”– Highly flawed model

• NEESgrid• InterMed• GriPhyN, iVDGL

– Tied to incentives– Expectation management


New Issues Intellectual property

– Who negotiates?

– What are the arrangements? Evaluation

– Who does it?• Within the project – formative• Outside the project – summative

– What is it?• Cross sectional• Longitudinal

– Over what time period?• Lag effects, long term indirect effects

– Be sophisticated• “science” talk vs. “informal” talk (G. Bowker)


Biggest issues – my candidates What is success? Evolution – ecology Transition to production versions,

sustaining the vision Data issues How to manage collaboratories?


Some SOC Issues

Are we asking the right questions? Are we doing the right kinds of

analyses?– Measures– Control groups

Are our representations useful?– Resource diagrams

Mix of science and engineering

Comparative Investigation of Collaboratories: Cross-Cutting Themes

Documents

Transcript of Comparative Investigation of Collaboratories: Cross-Cutting Themes