Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and...

MotivationDesign Test

DesignWizardEvaluation

Conclusion

Design Tests

João Brunet

Dalton Guerrero and Jorge Figueiredo

Federal University of Campina Grande (UFCG), Brazil.

09/28/2011

João Brunet 1/ 26



Conclusion

Agenda

1 Motivation

2 Design Test

3 DesignWizard

4 EvaluationAPI Usability

5 Conclusion

João Brunet 2/ 26



Conclusion

Motivation

Check code against design rules is an important activity toguarantee quality on source code

Design Review

Manual ProcessError-proneDoes not scale

João Brunet 3/ 26



Conclusion

The Problem

Gap between state-of-the-art and the state-of-the-practiceApproaches from the state-of-the-art require learning differentlanguages to express design rules

João Brunet 4/ 26



Conclusion

The Problem

Lack of structural conformance checking

Design decisions are violated

João Brunet 5/ 26



Conclusion

Design Test

Design Test

A test that checks whether an implementation complies with a givendesign rule expressed as an algorithm.

João Brunet 6/ 26



Conclusion

What’s the big deal?

It is written in the target programming language

Automated test

João Brunet 7/ 26



Conclusion

Example of Design Rule

Only controller package shall access dao package of a givencomponent.

João Brunet 8/ 26



Conclusion

Example of Design Test

João Brunet 9/ 26



Conclusion


Facts about the code (an API)

João Brunet 10/ 26



Conclusion


Facts about the code (an API)

Assertion routines

João Brunet 11/ 26



Conclusion

DesignWizard

João Brunet 12/ 26



Conclusion

Some methods of DesignWizard’s API

getAllPackages()

getAllSubClasses()

getStaticMethods()

getImplementedInterfaces()

...

João Brunet 13/ 26



Conclusion

Process

João Brunet 14/ 26



Conclusion

Process

João Brunet 15/ 26



Conclusion

API Usability

Methodology

“It is the comparison between what developers expect and what theAPI provides that is interesting when evaluating the usability of an API(Steven Clarke)”

Think Aloud Protocol For APIs to capture expectations, goals,difficulties and strategies

João Brunet 16/ 26



Conclusion

API Usability

Experiment

11 developers

5 real design tests (selected by the designer of the OurGrid)

OurGrid

Eclipse

API

Pseudocode vs. Code

João Brunet 17/ 26



Conclusion

API Usability

Experiment

João Brunet 18/ 26



Conclusion

API Usability

Experiment

Only one developer did not write one instance of design test.

João Brunet 19/ 26



Conclusion

API Usability

Experiment

...but, 10 tests were incorrectly written. The good news is that alldevelopers found the right method, but used it incorrectly.

João Brunet 20/ 26



Conclusion

API Usability

Results

Two weaknesses of API:

getAllClasses()

We decided to follow the Java Reflection API pattern

getDeclaredMethod (String name)

An appropriated name of the parameter would be signature

João Brunet 21/ 26



Conclusion

API Usability

Results

Think Aloud Protocol revealed that devolopers had no difficultiesto find the right method to achieve a given task

Results suggested changes to DesignWizard API anddocumentation.

João Brunet 22/ 26



Conclusion

API Usability

Threats to validity

Coding and designing expertise of developers

More complex design rules may produce different observations

The API was not fully evaluated

João Brunet 23/ 26



Conclusion

Final Remarks

Approach to check conformance between code and design rules

Evaluation regarding its usability

Design Tests and maintenance

Executable documentation of design rulesEnables developers changing functionality keeping in mindstructural constraints

João Brunet 24/ 26



Conclusion

Thank you

www.designwizard.org

João Brunet 25/ 26



Conclusion

Related Work

ArchJava

Reflexion Models

FindBugs

...

João Brunet 26/ 26

Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and...

Technology

Transcript of Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and...