An Exploratory Study ofBacktracking Strategies

Used by Developers

YoungSeok Yoon (youngseok@cs.cmu.edu)

Institute for Software Research (ISR)

Brad A. Myers (bam@cs.cmu.edu)

Human-Computer Interaction Institute (HCII)

School of Computer Science

Carnegie Mellon University

CHASE 2012June 2nd, 2012.

CHASE 2012 2

What is Backtracking?• When developers write source code, they often need

to step back when something goes wrong

• We define backtracking as:

going back to an earlier state ofat least some part of the code,

either by removing inserted codeor by restoring removed code

CHASE 2012 3

Examples of Backtracking

Correcting minor mistakes

Tuning parameters

Trying out methodswhile learning an unfamiliar API

Experimenting with different classes(e.g. UI layout classes)

CHASE 2012 4

Existing Support for Backtracking

CHASE 2012 5

Existing Support for Backtracking

Conventional Undo Command• Only works for the most recent changes• Have to undo all changes in order• Undone changes are thrown away after making a new edit

CHASE 2012 6

Existing Support for Backtracking

Version Control• The desired version needs to be committed in advance• Unstable code should not be committed to the repository• Inappropriate when wanted and unwanted code is mixed

CHASE 2012 7

Existing Support for Backtracking

Manual Edit / Commenting Out• Have to manually find all the relevant code• Redo the work by hand

CHASE 2012 8

Research Question

How can we better understanddevelopers' backtracking behaviors?

As a first step towardssupporting more robust backtracking tools…

CHASE 2012 9

STUDY #1.LAB STUDY

CHASE 2012 10

Study Settings• 12 participants, all MS/PhD students at CMU• 2 hrs, THINK ALOUD• Code base: Paint program [Ko 2005][Fogarty 2005]

CHASE 2012 11

Tasks• Two sets of features each done in two different ways,

▪ intended to lead participants to backtrack

Feature 1 (F1) “Thickness Control”

F1-1

F1-2

Feature 2 (F2) “X, Y coordinates indica-

tor”

F2-1

F2-2

CHASE 2012 12

Commenting Out• 7 out of 12 participants preferred to comment out

rather than to delete code

• 3 main reasons for commenting out

Reuselater

e.g. activating one alternative at a time

Goodexample

leave the code as a good example

Badexample

remind themselves that the code was

not good

CHASE 2012 13

Problems while Backtracking• Difficulties in finding relevant code fragments that

have to be reverted when:▪ there are two or more similar looking code fragments▪ relevant code fragments are scattered across multiple

locations

• Forgetting to remove temporary code▪ e.g., Debug output statements such as:

• System.out.println• JOptionPane.showMessageDialog

CHASE 2012 14

Backtracking as Restoring Code• 2 participants accidentally deleted some code

• One participant failed to restore the code from memory after spending about 10 mins.

• The other succeeded using the undo commandbecause he knew:▪ what the code looked like▪ that it was deleted very recently

CHASE 2012 15

Backtracking as Restoring Code• When trying to restore deleted code, participants often

remembered some features about the desired code▪ the original location of the code▪ the surrounding code▪ the names of some code elements (variable, method, …)▪ what the code looked like

• Participants also reproduced the same code fragments from memory repeatedly▪ e.g. complex formatted string looking like

“(X, Y) = ” + x + “, ” + y + “)”

CHASE 2012 16

STUDY #2.ONLINE SURVEY

CHASE 2012 17

Demographics

• 48 professional programmers

• 13 yrs exp. on average

• recruited from online developer communities

Male92%

Female8%

CHASE 2012 18

Flexibility of their work

The details of the im-plementation code

Which elements of the API are used

Which APIs/libraries are used

The architecture of the code itself

The results that my code achieves

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Highly specified be-fore I start developing

Specified, but some opportunity to nego-tiate changes

Somewhat flexible, within broad con-straints

Highly flexible

Completely unspeci-fied; I can do what-ever I want

CHASE 2012 19

How often do they have to backtrack?

Trying to find an appropriate al-gorithm

Trying out various user interface designs

Fixing code just added, because it is not working

Figuring out how to use an API correctly

Tuning parameters

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

All the time

Frequently

Sometimes

Rarely

Never

CHASE 2012 20

Backtracking Strategies• For each backtracking situation, only a few strategies

were primarily used▪ e.g. “select & overtype” when tuning parameters▪ This hints at how to detect a certain backtracking situation

• Other strategies in the open-ended responses▪ using Boolean variables to temporarily turn on/off code▪ writing a small code snippet (outside the main project)▪ moving parameters out of the code▪ writing unit tests to see how the API works

CHASE 2012 21

Future Work• Conducting an extended version of this study

▪ A field study is ongoing, using our Eclipse logging tool called FLUORITE [Yoon 2011]

▪ If you are interested in participating, please let me know!

• Designing new backtracking tools, based on the needs identified in these studies

CHASE 2012 22

Conclusion• Backtracking happens a lot while coding

• The backtracking strategies used by developers are still manual and error-prone

• Better backtracking tools would help developers write code more correctly and effectively

CHASE 2012 23

Questions?

Thanks for funding from:

Contact Info: YoungSeok YoonE-mail: youngseok@cs.cmu.eduWeb: http://www.cs.cmu.edu/~yyoon1/

FLUORITE logging tool can be found at:http://www.cs.cmu.edu/~fluorite/