Test Case Passed! Who Cares!!! - Pradeep Soundararajan

8/4/2019 Test Case Passed! Who Cares!!! - Pradeep Soundararajan

1/40

Copyrights Pradeep Soundararajan http://testertested.blogspot.com [email protected]

Hello!

I am Pradeep Soundararajan from Bangalore, India


2/40


I test, manage, coach, consult, speak, write on software testing.


3/40


Here is a story of how I struggled to find bugs a few years

ago, which I thought might be of help to you.


4/40


No, this is not a story without a name, so here is the name

The snake in the monkeys shadow


5/40


oOPS! I think that wasnt a title that makes sense to this

context. OK, Let me rename it

Test case Passed! Who cares?

The title will appear above the above text in 45 seconds !


6/40


To start the story, here is an expected result of a test case

FR_1223 that I executed!

The application should handle the input andshould not hang or crash the application.


7/40


I executed that test and passed the test because the

application did handle the input and did not crash/hang the

application.

Are you curious to know what happened once I

passed the test?


8/40


The customer complained that although the input was

handled, the application took 15 seconds to perform the

operation.

but hey, the Spec didnt say that?


9/40


Well the test passed but why did I not catch the bug

reported by the customer?


10/40


I then added another point to the expectedresult:

Expected Result of test case FR_1223:

The application should not take more than 5 seconds tohandle it.

The application should handle the input and should not hangor crash the application.


11/40


I executed the test again in the next release and the test

passed!


12/40


Yet again, the customer came back logging another issue:

While the application now takes 5 seconds to handle, it fails

to process any input once I close and reopen the applicationafter an abrupt closure during those 5 seconds.


13/40


I then updated the Expected Result of test case FR_1223 to:

The application should handle any abrupt closure orthere should be a proper warning or error messageduring that operation.

The application should handle the input and should not hangor crash the application.

The application should not take more than 5 seconds tohandle it.


14/40


And in the next build I executed the test and it passed!


15/40


Our customer came back logging another issue: While the

application now handles abrupt closure by warning the user

of loss of data, the database elements are corrupted whenthe operation is iterated 5 times.


16/40


I then updated the Expected Result of test case FR_1223 to:

When such an abrupt closure is carried out multiple timesthe database should not be corrupted.

The application should handle the input and should not hang or crashthe application.

The application should not take more than 5 seconds to handle it.

The application should handle any abrupt closure or there should be aproper warning or error message during that operation.


17/40


Our customer was quite happy that he could not find

anymore issues with respect to that feature and most of his

expectations were met.

Wow! The story appeared to end.

I was simply wrong!


18/40


Our customer then passed the release to their customer. Our

customer came back saying their customer isnt happy

because the Submit button that performs an action ofprocessing the input is too small to be noticed for new users.


19/40


I then updated the expected result to:


All radio buttons associated with this operation should be ofsize X by X (as communicated by the customer in e-maildated October 22)

The application should handle the input and should not hang orcrash the application.


The application should handle any abrupt closure or there should be

a proper warning or error message during that operation.

When such an abrupt closure is carried out multiple times thedatabase should not be corrupted.


20/40


And in the next build I executed the test and it passed!


21/40


Yet again, our customers customer, reported an issue: It

appears that when this operation is being processed in IE,

the browser shows Not Responding and then recovers.


22/40


I then updated the expected result to:


The application should process the operation in a same way acrossmultiple browsers ( IE, Firefox, Safari, Web Monkey, Netscape )

The application should handle the input and should not hang or crash theapplication.


The application should handle any abrupt closure or there should be a properwarning or error message during that operation.

When such an abrupt closure is carried out multiple times the database should

not be corrupted.

All radio buttons associated with this operation should be of size X by X ( ascommunicated by the customer in e-mail dated October 22 )


23/40


The story went on for a while and today when I look back, I

realize I learned a few important lessons in testing.


24/40


I am wondering if I should share the lessons with everybody

instead of people whom I choose to share?


25/40


OK, so enter the password that I might have sent youto read the lessons, else you dont get to read them

and I am sorry about it.

Password: Check


26/40


Wait a minute! I changed my mind.

You can read them without entering the password

and here it goes


27/40


Lesson Number Minus Three

The specification could have been met long back but it is not the

specification that approves the product nor the ones who wrote

the specification but it is *humans*- who arent completely

bothered if specification is met but bother if it meets their

expectations without bothering if their expectation was

documented in the specification or not.


28/40


If you think your customer learns new things every day as you do,

then you must be aware that their expectations keep changing as

they learn, just like how it is happening for you.

Do you expect the same salary once you have learned that your

contribution to a project that you work has grown humongous?

If you expect the same you deserve the same

Lesson Number Minus F


29/40


If all expectations were to go in to a column of test case whose

heading is Expected Result then each element of the column

might be of several thousand words and hundred points as compared

to what we most commonly get to see even if we were to writethose test cases.

It might take about an hour just to read such a test, isnt that funny?

Lesson Number Minus Five


30/40


Lesson Number PLUS six

(Were you expecting a MINUS six? )

When I practiced using heuristics* and oracles** (not the database)while exploratory testing a product I found that I loaded my brain withthe list of oracles and heuristics and then shot every test with as many

expectations I could think of and I was able to find not just one bug withevery test I did but many.

Heuristic* A fallible method of solving a problem

Oracle** - A principle of mechanism by which we (humans and not the process)

identifies a problem


31/40


Lesson number minus seven-teen

I recommend other testers to not use the Heuristics and Oraclesapproach to their testing because I am a little selfish about being oneof the few testers who finds many important problems, quickly.

I also dont recommend you to go through Rapid Software Testingmaterial that helps you in learning a lot more about Oracles andHeuristics which can be found at www.satisfice.com/rst.pdf


32/40


Lesson number

eight If I had all lessons spoon fed tome, I would have become thelaziest tester in the world and I

think I did not become thatbecause I realized the one of thesolid ways to do a good testing, isby practicing doing good testing.


33/40


Lesson number Aight!

When I execute tests that have words like handle theinput looks reasonable correct results are

displayed

My opinion about those words might be different fromthose of my customers and even other testers.

Lesson contribution from Ben Simo www.questioningsoftware.com


34/40


Lesson number 8 ( not the word eight )

A different lesson dont worry about that!

I think we learned so many things with the story and Iguess we should have also learned that it is humanswho solve problems and not the process they make,which many think takes care of testing.

A process is a heuristic and can never be a bestpractice!


35/40


The first time I heard Heuristic and Oracles

I had the same feeling as you might be having now Greek andLatin

but I asked myself a question, If I can do a lot better testing by

learning that Greek and Latin, why not I learn it?

and here I am writing stories and articles about them after exploringthe basic power of those two words.


36/40


Thank you for your time!


37/40


This presentation is inspired by Michael Boltons Emotions and

Oracles Tutorial and the creative (in my opinion) style of presenting

was inspired by Ben Simos FAILURE presentation (that is not

public yet) and from the titles of the movie Monty Python and theHoly Grail.

I admit that I didnt do anything more creativethan those!


38/40


Some of you might have thought, after going through this

presentation (so far) that I am trying to say, I no longer struggle to

find bugs because I exploit the power of Heuristics and Oracles.

I am glad to say to those who thought that way,

I think you are wrong, I still do struggle but I might not struggle on

the same problems that I struggled last year or maybe yesterday


39/40


For more such stuff that might interest / bore you like this keep

visiting my blog http://testertested.blogspot.com

Lessons are not for free they are mixed with some

marketing, so bare with it


40/40


The END!

http://testertested.blogspot.com [email protected]

"Pradeep's first language is not English--his first language appears to betesting." -- Michael Bolton

Test Case Passed! Who Cares!!! - Pradeep Soundararajan

Documents

Transcript of Test Case Passed! Who Cares!!! - Pradeep Soundararajan