Collaborative Evaluation in Multi-Stakeholder Projects

SBS research seminar

University of Portsmouth

18 May 2011

Colston Sanger

HEA BMAF Teaching Research and Development Award, 2009-10

Faculty Learning and Teaching Fellow 2008-9, 2007-8

sangerc@lsbu.ac.uk

Agenda

1. How it all began

2. Teaching agile project management in an agile way

3. Collaborative evaluation

4. Taking this back out into the „real world‟

5. Contributions, questions

1. How it all began …

How it all began …

Returning to HE

At University of Surrey, 2002-3

Within the BSc WBL and MSc CASS programmes

Working in an experiential, Gestalt-informed way

Self, peer and tutor assessment

At London South Bank University, 2006-

Within the MBA programme

The opportunity to develop a Managing Projects unit

A series of funded informal experiments

Faculty Learning and Teaching Fellowships, 2007-8, 2008-9

HEA BMAF Teaching Research and Development Award, 2009-10

2. Teaching agile project management in an agile way

Managing Projects

Managing Projects

Not Project Management

… as a team sport

Requires

involvement, engagement

Working on real or „near real‟

projects

Staying alive to change

Frequent decision points

Communication, simplici

ty, feedback, courage

Intended learning outcomes

Knowledge and

understanding

Describe …

Identify similarities,

differences, connections …

Intellectual skills

Evaluate…

Analyse …

Exercise appropriate

judgement …

Practical subject-specific

skills

Develop …

Demonstrate …

Transferable skills

Manage own learning …

Communicate effectively …

Work with others …

Recognise and support

followership, and be

proactive in leadership

Calibrating the process of

learning

Learning agreement

Check-in

Weekly project

progress reviews

End of unit project

retrospective

Staying alive to change

Frequent decision points

Communication, simplicity, feedback,

courage

And also forms of congruent or

„authentic‟ assessment

Near real projects

You are members of the Special Projects team at the London Borough of Cross River

Your Chief Executive had a GREAT IDEA …

A summer Street Ski Jump Festival!

Ski jump

San Francisco 29/9/05

A summer ski festival

The great glass elevator

Landmark urban

regeneration

A solution to the inner

London housing crisis

Unlimited student

residences for Cross

River University…

Castle House,

Elephant & Castle

The “Elevator”

A London village fête?

2010-11 Project Challenge

End of unit project

retrospective

3. Collaborative evaluation

What counts as good work?

„[In] the learning that

professionals do outside the

academy, learning outcomes are

rarely specified in explicit terms.

What is required of the learner is

embedded in a professional

practice…

Before learning can even

commence there is a need

for learners to identify for

themselves what they need

to learn, taking into account

a range of contextual

factors, and to judge what

counts as good work.’

(Boud and Falchikov, 2006)

What counts as good work?

1. Stakeholder Engagement

Not yet identified / Identified / Contacted / Agreed / Engaged

2. Critical Path

Didn’t know there was such a thing / Heard of it / Worked it out for this project / Know

where you are on/off it / Always on track

3. Investigation of resources

Doesn’t yet know what is required / Knows what is required / Has estimated when and

how much /Resources acquired for use

4. Added Value

Agreement fulfilled to spec / Skills development for participants / Gained Council and

employer support / National media recognition / Community takes over running of

event, social regeneration

5. Persuasiveness of presentation

Boring, mumbled, did not hold my attention at all / Confused, chaotic /

Understandable, but doesn’t add up / Coherent, but missing ‘So what? / Lively,

interesting, presents a compelling case

Scaling

4. Taking this back out into the „real world‟

Well then… What now?

Collaborative

evaluation in multi-

stakeholder projects

Not multi-criteria

decision analysis

Exploratory, grounded

theory approach

Not even sure what the

question is yet

Conversations with

experienced programme

and project managers

Some preliminary

data points

A good project is when you have a good plan. Good in the sense that it takes into account risks, resources, schedules, costs. And you also have all the prelims: stakeholder engagement, champion engagement, you train all your staff, everybody's working as a team, singing from the same song sheet, all those sorts of things…

Good achievement comes … when

everybody understands what you are

trying to do and they contribute to that to

make it happen. There's always a part of

luck, but I think you make your own luck.

And that's where the effort comes in,

because if the effort is put in properly then

things should go forward moderately well.

Major disconnect between the textbooks

and reality. The textbook will say: block your

time out … But the reality is, nothing works in

those smooth ways... Or [it assumes] you're

doing projects with reasonable people whose

time blocks can be compromised to yours.

It's a 'perfect world' situation: but the reality

is, people do their damndest to mesh with

your timetable, but it's not [always] going to

work.

Benefits realisation, post

project:

Generally, no-one in the

industries I work in tend to

pay me for post-project

monitoring. Therefore my

hands are basically free as

soon as the last invoice

has been paid.

Setting scope in an R&D project: When it's that kind of

project, you're basically hired to set the scope: to do the

background investigation, look at the needs, identify the

stakeholders, work out their influences. And then with their

input, the client's input and maybe a supplier input, you can

then define the scope. In that respect, that is a project in its

own right - and whether you get to do the project itself or

someone else takes it on, that's a different matter.

Balancing effort and output in multi-project

scenarios. Effort is basically costing money.

Outputs are added-value benefits. So it's really a

cost-benefit analysis in a strange sort of way. The

project manager is saying how does the client get

the best deal they can get ... It's about allocating

resources fairly. The reality is that whoever is

screaming loudest gets the most response.

Contributions, questions