20-21 August 2008, Queenstown, New Zealand

Kia ora Koutou

Greetings everyone!

20-21 August 2008, Queenstown, New Zealand

Testing for SuccessIndiver Nagpalindy@strakersoftware.com

About me

• CTO, Straker Software, New Zealand

• Involved in web development since 1996– development, content, project management

• Using ColdFusion for over a decade; Flex, 3 years

• Currently in love– Groovy (http://groovy.codehaus.org/)

– Testing processes– Getting Things Done (GTD) methodology

About Straker Software

• Adobe technologies - ColdFusion and Flex

• ShadoCMS (http://shadocms.com)– a large content management software written in ColdFusion

• ZoomFlex (http://zoomflex.com)– A rapid, Flex application generation software

• Been working at Straker for about five years

makeShadoCMS()makeZoomFlex()

ColdFusionFlex

Straker

Agenda

• Touch upon high-level issues in using testing

• Share some experiences of implementing a testing process

• Look at some tools that can help

• Basic idea: When you leave here, you have a fairly decent idea of what does it take to set up testing processes that work

Recognize the need for testing

• THE first step

• Ask questions like... what happens to the existing software if– New functionality has to be added– Fixing an issue in commonly used component– Departure of a developer

• How do you handle these?

• Ideally, how would you like to handle these?

We test anyway...

• All of us test our code

• In ColdFusion– Create a component (CFC), with methods– Create a script (CFM)– Instantiate the CFC in CFM– Fire methods of the instantiated CFC– Eyeball results

• Commonly used approach

But what is wrong with this?

• Unsystematic

• Unorganized

• Dependent on an individual

• Not repeatable in different situations

• Does not lend itself to automation

Why do we continue like this?

Organizational attitude

• Perhaps the most important one to change

• If an organization does not value testing, developers won’t do it

• Build a case for return on investment– Reduce time to market– Reduce ongoing support– Build quality software

• Identify in-house lobbyist

?

Consider this...

• 80% of the avoidable rework comes from 20% of the defects

• 90% of the downtime comes from 10% of the defects

• Disciplined personal practices can reduce defect introduction rates by up to 75%

• Peer reviews catch 60% of the defects

• http://www.cebase.org/www/frames.html?/www/researchActivities/defectReduction/top10/index.html

Individual factors

• All of the significant battles are waged within the selfSheldon Kopp (If You Meet the Buddha on the Road, Kill Him, 1976)

• Recognizing that the only Constant is Change– From being a developer to a technologist– From being “lazy” to proactive

• Quick exercise: Think of what the internet was when you first experienced it. Compare it to how you use it now.

Technological change

• Perhaps the easiest aspect

• Formal testing is becoming more mainstream in scripting languages

• Multiple frameworks: testing, mocking

• Integration with IDEs

• Integration with automation tools

Testing Frameworks #1

• ColdFusion Unit testing– MXUnit (http://mxunit.org)

– CFUnit (http://cfunit.sourceforge.net/)

– RocketUnit (http://rocketunit.riaforge.org/)

– CFCUnit (http://www.cfcunit.org/)

• HTML User Interface (UI) Testing– Selenium (http://selenium.openqa.org/)

– Canoo Webtest (http://webtest.canoo.com/)

Testing Frameworks #2

• Flex Unit Testing– FlexUnit (http://code.google.com/p/as3flexunitlib/)

– DPUnit (http://code.google.com/p/dpuint/)

– ASUnit (http://www.asunit.org/)

• Database Unit Testing– DBUnit (http://dbunit.sourceforge.net/)

Choosing a Framework

• Different needs for– Different organizations– Different technologies

• Defining criteria for choosing a framework is important

Choosing a ColdFusion Framework

• Integration with IDE (Eclipse)• Support for automation (Ant, Continuous Integration tools)• Reporting (Standard formats like JUnit)• Extendability (Creating own assertions)• Easy generation of dummy tests• Community support

• MXUnit

HTML UI Testing Framework

• Easy recording tests• Conversion of test scripts into Java/Groovy• Reporting

– Standard formats like JUnit• Support for automation• Community support

• Selenium

Writing tests

• Tests for new functionality– Test generators play a big role– Create tests, watch them fail, add test logic, watch them pass– Remember, the earlier you catch issues, the lesser the cost later

on

• Tests for existing codebase– Time consuming– Code does not lend itself easily to be tested– Setting aside regular time for writing tests - difficult to justify/

implement

Leverage IDE support

• Eclipse is a great IDE– CFEclipse for ColdFusion development– Plugins for other languages: Groovy

• MXUnit integrates beautifully– CFUnit comes pretty close

• Provide views for loading and running tests

• Setup for loading application and session scopes before running tests

Automation

• Manual running of tests does not scale

• Use Apache Ant for automating running of tests

• Use continuous integration server like Hudson to– Create build > Run tests > View reports

• Ensures that new changes do not break existing code

• Host on fast/multiple servers

Source Control Management

• How many use a source control management system (SCM)?

• Subversion is very good

• Works well with ColdFusion development teams– Haven’t made up my mind about next generation SCM tools like

Mercurial, Git, Baazar

• Great client tools -- TortoiseSVN, Subclipse, SmartSVN

• Host on fast servers -- either in-house or externally

Code Review Systems• Remember peer-reviews catch up to 60% of defects

• Establishing a Code Review System - Review Board

• Works in conjunction with an SCM system

• Allows peer review before code is checked into a source control system

• A little involved to set them up, but well worth the effort

SPEED

LIMIT

55

Groovy -- the language to learn

• Simple scripting language that compiles into Java

• Easy to learn; rapid application development

• Java libraries written in Groovy can be used directly in CF

• In-built support for Ant– Using Ant with scripting rather than XML

• Integrates very well with CI servers like Hudson

Testing Champion

• Someone in the organization needs to advocates testing

• Someone with perceived competence and experience

• Sets up testing processes and guidelines

• Reviews processes and refines

• Occasionally brings out the whip

Summing up

• “If you are not testing, you are crazy.” - Thomas Burleson

• Keep in mind the organizational, individual and technological

factors that impact testing processes

• Get organizational buy-in

• Choose frameworks carefully

• Testing and Continuous Integration go hand in hand

• Be the Testing Champion in your organization

• Embrace change

indy@strakersoftware.com

http://nagpals.com/blog/

Thanks for your time!