8th October 2007

LancasterUniversity

int i, j, k;while…

Viva presentationViva presentation

SERE: Software Engineering for the Research Environment

Andre ObolerComputing Department

Lancaster University

272+212

Thesis focus• This thesis presents a new process based approach to

software engineering designed to meet the needs of academic Computer Science researchers.

• The core objective was to examine whether software engineering approaches could be adapted for the research environment so that they gain acceptance and enable improvement of the research process.

272+212

Limitations in scopeAcademic research environment:• While the approach and tools may have practical application in an

industrial R&D environment, this is outside the scope of this work

Computer Science Research:• While some process descriptors are generic in nature, the focus of this

work is for research that involves the development of experimental software

• We assume a graduate level of computer science background amongst the researchers, our approach seems to build on this and focus it on the needs of research

272+212

Hypothesis

• A RAISER-based approach (building on our past work) can improve the research process in computer science academia.

• The null hypothesis is that a new approach will

make no difference.

272+212

Outcomes

• The data from this thesis leads us to reject the null hypothesis on the basis that:– Empirical data shows an improved outcome– Qualitative feedback showed a positive experience– Process descriptors demonstrated benefits– The process perspective aids researchers in

reflection and enables further improvements

272+212

Contributions from this work• The Process Model for Supporting Research• The Personal Process Model• Formal Technical Reviews for research• Documentation standard for research• The Research Process Optimisation Framework• The high level framework for CARE First International Workshop on Improving Research

Productivity

272+212

PhD Process Review• This research was undertaken in an evolutionary manner. • Though necessary to development the SERE framework, the

final cycle (and post experiment analysis) leaves SERE much more mature than it was in the early cycles.

• Though unavoidable, three years of data on the process as it stands now would be an improvement over the data we have

• With plenty of scope for further development, even given a further three years the approach would continue to evolve

• Inclusion of PhD candidates in the participant sample would be another improvement, but was not possible in the scope of this research

• The cyclic nature of the work led to some difficulties in thesis structure, however the current mixed structure resolves these nicely (covering both the cycles and the big picture)

272+212

Experimental difficulties• By necessity participation was voluntary

+ve gives a true picture on adoptability-ve smaller than idea sample sizes

• Need to provide full support myself+ve first hand experience with all the data-ve limiting factor on sample size and what could be done

• Timing not always ideal but dictated by circumstances• Interview data not full processed

– No budget for transcription & very time consuming– Some interviews pre-selected then transcribed (other not transcribed)

• Getting survey responses from students was a problem• Limiting the thesis scope has been difficult as the work

touched upon many interesting and related topics.

272+212

Future developments• Expansion of provision to PhD students• Examination of the impact of a full Software

Development Lab• Examination of specialised group templates for

Personal Process Models• Examination of effectiveness and adaptation to

industrial Research & Development • Further development of CARE• Create of new and more specialised process

descriptors for different research areas

272+212

Viva discussion

There are further slides if you wish on:• Motivation and reply to detractors• The nature and needs of research • The research process optimisation framework• Improving knowledge transfer• The Personal Process Model• This research’s Personal Process Model

272+212

Motivation and reply to retractors

272+212

Objections

One may ask, why bother? • Current approaches have been working for

decades• This seems like a lot of extra work• Experienced researchers KNOW what they are

doing• The struggle and reinvention of ideas is part of

the learning process

272+212

Response• Increased pressure on science to deliver• Increased pressure on students to complete faster• Increased competition for resources• Computer science is not yet mature• Information Technology is now a part of life,

academia needs to be more scientific to hold value• We can and should do better.

272+212

Nature and needs of research

272+212

Research is…

• “original investigation undertaken in order to gain knowledge and understanding”[1]

• “creative work undertaken on a systematic basis in order to increase the stock of knowledge” [2]

272+212

Systematic yet Creative?

• Reducing wasted effort is a type of improvement that does not impact on creativity

• Not all effort is “wasted effort” so not all automation is “safe” – some might reduce scientific creativity

• Remember: research is creative work undertaken on a systematic basis, creativity and a systematic nature must coexist

272+212

The Research Process Optimisation Framework

272+212

The Research Process Optimisation Framework

a b c d e f

Knowledge (SKU)

Effort is the volume filling the pipe per Standard Knowledge Unit

Reflects the fact that some parts of the research process take more effort than others, and the amount of this that is “wasted” also varies.

272+212

…as a bar chart

0

1

2

3

4

5

6

7

8

a b c d e f

SKU

SEU

Our aim should be to improve the Effort : Knowledge ratio, either by reducing (wasted) effort or by increasing the amount of knowledge that can be gained from that effort.

272+212

Improving knowledge transfer

272+212

Improving knowledge transfer• How do we improve knowledge transfer?

– Capture process information– Share it– Develop it systematically– Allow feedback

• We must also ensure that:– Creativity is not inhibited– “Wasted effort” is not increased

(overly bureaucratic solutions provide one counter examples that both adds waste and inhibits creativity)

272+212

Approaches to knowledge transfer

Artifacts:• Research journal• Electronic research journal (so searchable)• Coding Guidelines• Personal Process Model (PPM)

Process:• Technical Reviews

272+212

Personal Process Model

272+212

Personal Process Model• Uses a a java source code file to set out the research process

as a set of classes

• Documents the key parts of the research process using comments

• Encapsulate and relate ideas using class relationships

• Generate external documentation, class diagrams and hypermedia using dOxygen

MYSTUFF

• Monitor changes in ideas / plans using diff

• Discuss changes with supervisor and colleagues, gain their wisdom

272+212

Key advantages - researcher• No context switching between coding environment and

research management environment• Builds on exiting software engineering knowledge• Uses an existing shared language for representation• Diagrams are easy to read and follow, but don’t need to be

created / updated by hand• Html output is easy to navigate and easy to share• Process Comments can link to internal code documentation

MYSTUFF

272+212

Key advantages - researchFor research as a whole, this approach means:

• More data is captured about the particular research• More data is shared about research approaches • Researchers share best practise and improve upon it

YOURSTUFF

MYSTUFF

272+212

This research’s Personal Process Model

272+212

My research model

272+212

My research modelNext slide will expand this part

272+212

Zooming in…

We are discussing this(please forgive the recursion)

272+212

References

[1] "RAE Circular 2/00" vol. 2006: Higher Education Research Organisation, 2000

[2] OECD, The Measurement of Scientific and Technological Activities - Frascati Manual 2002 : Proposed Standard Practice for Surveys on Research and Experimental Development Frascati, France: Organisation for Economic Co-operation and Development, 2002.