Nov 2013- 1 Quality Process for Research Software - Timothy C. Lethbridge Research Software Doesn't...

Nov 2013- 1Quality Process for Research Software - Timothy C. Lethbridge

Research Software Doesn't Have to be Buggy:

A Model-Driven, Test-Driven and Agile Process Applied in a University Research

Team

November 2013

Timothy C. Lethbridge


Motivations for This Talk

I was explaining to researchers at a conference how I have been able to maintain Umple– They were interested in the process I followed

It has been a very positive experience to experience ‘real’ software development– And not get bogged down it– While maintaining quality– And getting good feedback from students


Research Software: Often Short Life or ‘Not Production Quality’

Software developed by computer science researchers often only lasts as long as one student’s PhD

–Perhaps a little longer

It usually gets messy–The next generation of students often wants to rewrite it

The principal investigator wants to move on to other things

–The software becomes a drag–Lack of resources for maintenance–Grants are for research, not development


Why would we want research software to last for the long haul?

To really ‘get it right’

To act as infrastructure for countless experiments and idea exploration

To allow the professor and students to get really good at real software engineering


My Vision

Prodution quality software that persists for 20 or more years

Continual improvement of the process

Top-notch training, education, and research platform


My Approach

Adopt software engineering best practices in the software used in the research

Blend and mode-switch research approaches– Design based research– Action research– Grounded theory– Empirical experimentation


The Platform: UmpleModel-oriented compiler

– Adds UML and patterns to Java, C++ etc.– Textual UML– High quality code generator for class and state diagrams– Web-based + command line + Eclipse

Main site: http://www.umple.orgWeb app: http://try.umple.orgManual: http://manual.umple.org

Open source:– Google: http://code.umple.org– Mirrors on Github and SourceForge


Robustness maintained because of / despite contributions by >30 HQP

2 completed PhDs (+ 7 underway)4 completed Masters (+ 1 underway)22 undergraduate capstone projects (+ 6 underway)3 postdocs

UCOSP: Students from UBC, Simon Fraser, UAlberta, USask, URegina, Laurentian, Waterloo, Wilfred Laurier, Guelph, Bishops, Sherbrooke, UNB, Dalhousie

International contributions from Brazil.

Countless students taught in classrooms–Umple helps students learn UML (CSEET 2011)


Essential elements of the processRed ones are key

Open source

Multi-level test driven development

Model driven development

Carefully categorized issues list

Continuous integration

Lean live documentation for users and developers

Low management overhead

Marketing and publication


Open Source

Open to the world as early as possible

If there are spinoffs, let them be based on service, not software


Multi-level test driven development

Use test cases as specifications

Multiple levels in the layered architecture– For Umple: Parsing, metamodel instance, generation,

execution

In Umple, the compiler itself is a big testNote: We have not applied this yet to the user interface

http://qa.umple.org


Model driven developmentAlways start with the model and generate code from it

– It is possible– Requires indoctrination, and students sometimes have

to be asked to start again!

Simplifies code, reduces volume, enforces abstraction

That is what Umple is about, but we have also developed other systems in Umple– Key Umple benefits compared to other MDD tools

• Model is code; traditional code embedded in mode• No need to edit generated code• Bidirectional traceability


Carefully managed issues list

Very-easy, easy, moderate, hard, very-hard

Defect, enhancement, undergraduate project, graduate project

Priority: Critical, Vhigh, High, Med, Low, Vlow

Ownership of issue, but not code

http://bugs.umple.org


Continuous integration

Single master trunk for all work

Server builds then posts results– http://cc.umple.org

Test result browser– http://qa.umple.org– 100% test pass rate maintained 95% of the time– 100% test pass rate within an hour of a failure 99.9%

of the time– Only two ‘backouts’ in the last year (failed tests that

could not be quickly fixed)


Lean Live Documentation for Developers and End Users

Code commenting is key:– But comments transfer into documentation– Auto-generated diagrams are a result of MDD– http://metamodel.umple.org– http://grammar.umple.org

User manual is generated: http://manual.umple.org

If someone doesn’t understand something they update the wiki– http://wiki.umple.org– Exploration of requirements done in issues and wikis– Developers log experiences and progress for all to see


Low Management OverheadAs a professor, no time for much project management

– About 2 hours a week direct interaction with students actively working on the software

– I manage issues, set up policies, refine documents– Periodically I work on an issue

Code sprint (20 hours) once a semester

Code review of patches and commits– Focus on checking the tests– Delegation of inspection to others– Some refactoring


Marketing and Publication

Not just journals and conferences

We need users and feedback from users of our tools

Live mirrors: Google Code, Github,

Facebook, Google+, Twitter, Email lists, blogs


Use of our software in teaching

Several professors have taught software engineering concepts using our work– Introduction to software engineering– Advanced software design– Software engineering graduate course– Score project at ICSE– Others


Introducing students to Umple

1. Train in the use of the software

2. Development environment setup

3. Coaching on process

4. Code sprint

5. Have them create several patches for easy to moderate problems in several architecture areas

6. Make them a committer (happens for most students)


Conclusions

Make your academic software high quality by using–TDD–MDD–Continual intregration–Careful agile process

It works!

Benefits–You and your students get better at software engineering–You can do better research because your infrastructure is better


Questions?

This talk is available at:

http://bit.ly/1fMkhzZ

Or

http://www.eecs.uottawa.ca/~tcl/presentations/CSER-Nov2013-Talk-RshSWNotBuggy.ppt

Nov 2013- 1 Quality Process for Research Software - Timothy C. Lethbridge Research Software Doesn't...

Documents

Transcript of Nov 2013- 1 Quality Process for Research Software - Timothy C. Lethbridge Research Software Doesn't...