SurfExample- Recommendation of Exception Handling Code Examples

ON THE USE OF CONTEXT IN RECOMMENDING EXCEPTION HANDLING CODE EXAMPLESMohammad Masudur Rahman, Chanhcal K. RoyDepartment of Computer ScienceUniversity of Saskatchewan

IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2014), Victoria, Canada

2

Software Research Lab, U of S

EXCEPTION HANDLING

Normal program

code

Exception handler

code

3


EXCEPTION HANDLING STATISTICS

70%

30% IneffectiveEffective

32 applications Exception handling

Statistics (Cabral and Marques, ECOOP

2007)

70% of the handlers are generic or

ineffective

4


TRADITIONAL CODE SEARCH ENGINES

(2) Thousands of

results

(1) No logical ties with IDE

(3) Only keyword-matching search(4) No query support(5) No support for exception handling

5

IDE SUPPORT FOR EXCEPTION HANDLING

(6) Only adds a generic handler which is NOT much helpful for effective exception handling

6

SURFEXAMPLE: CONTEXT-AWARE CODE EXAMPLE RECOMMENDER FOR EXCEPTION

HANDLING

7


SurfExample

MOTIVATING EXAMPLE

Context code

Recommended code example

8


PROPOSED CONTEXT-AWARE CODE EXAMPLE RECOMMENDER FOR EXCEPTION HANDLING

Starta

b

c

d e End

GitHub Code Search

Source Files

Metrics CalculatorCorpus Score Calculator Rank ManagerCode search queryContext code Ranked code examples

Code examplesServer

Client

9


PROPOSED CONTEXT-AWARE CODE EXAMPLE RECOMMENDER FOR EXCEPTION HANDLING Distinguishing Features (3)

Graph-based structural relevance Static relationship and data dependency graph (Nguyen

et al., ESEC/FSE 2009) Graph structure matching

Handler quality Paradigm Novel idea to ensure quality of exception handlers Based on readability, count & quality estimate of the

handler actions Seamless integration of dataset

Exploits GitHub API for data collection Hundreds of popular and mature open source projects

from Eclipse, Apache and others

10


PROPOSED METRICS Structural Relevance (Rstr)

API Object Match (AOM) Field Access Match (FAM) Method Invocation Match (MIM) (Holmes and Murphy, ICSE 2005) Data Dependency Match (DDM)

Lexical Relevance (Rlex) Cosine Similarity (Takuya and Masuhara, SUITE 2011) Code Clone Measure (Roy and Cordy, ICPC 2008)

Exception Handler Quality (Qehc) Readability (RA) (Buse and Weimer, TSE 2010) Average Handler Actions (AHA) Handler to Code Ratio (HCR)

11


EXPERIMENT OVERVIEW

65 exceptions + context code

700+ GitHub code repositories

4400 code examples

Gold set solutions (50-60 work hours)

Peers

12

Software Research Lab, U of SRESULTS ON DIFFERENT RANKING ASPECTSRanking Aspects Metrics Top 5 Top 10 Top 15Structure (Rstr) MAPK

RecallPEH

38.07%50.00%69.23%

33.84%61.93%75.38%

32.64%69.32%81.54%

Content (Rlex) MAPKRecall PEH

35.00%45.45%66.15%

33.85%63.63%75.38%

33.08%70.45%81.54%

{Structure (Rstr),Content (Rlex)}

MAPKRecallPEH

43.08%51.70%69.23%

38.69%66.48%75.38%

37.33%74.43%81.54%

{Structure (Rstr), Content (Rlex), Quality (Qehc)}

MAPKRecallPEH

41.92%57.39%73.85%

39.92%68.75%81.54%

38.64%76.70%86.15%

[ MAPK = Mean Average Precision at K, PEH= % of exceptions handled]

13

Software Research Lab, U of SINFLUENCE OF DIFFERENT RANKING ASPECTS

14

Software Research Lab, U of SCOMPARISON WITH EXISTING APPROACHESRecommender Metrics Top 5 Top 10 Top 15Barbosa et al., SBES, 2012

MAPKRecallPEH

16.15%16.47%27.69%

14.69%25.57%38.46%

13.72%31.25%44.62%

Holmes & Murphy,ICSE, 2005

MAPKRecall PEH

4.62%11.36%24.62%

2.31%21.59%38.46%

2.31%27.84%47.69%

Takuya & Masuhara, SUITE, 2011

MAPKRecallPEH

21.54%15.34%33.85%

20.51%27.27%47.69%

19.74%30.68%47.69%

Bajracharya et al.,SUITE, 2009

MAPKRecallPEH

8.46%10.80%18.46%

7.95%15.91%27.69%

6.41%19.32%30.77%

Proposed approach MAPKRecallPEH

41.92%57.39%73.85%

39.92%68.75%81.54%

38.64%76.70%86.15%

[ MAPK = Mean Average Precision at K, PEH= % of exceptions handled]

15


MEAN PRECISION VS. RECALL CURVES

16


THREATS TO VALIDITY Subjective bias in the gold set development Subjected to the performance of GitHub Code

Search feature Limited size of dynamic corpus for

recommendation Limited number of exceptions for

experiments

17


CONCLUSION SurfExample combines

Graph-based structural features Lexical features Heuristic estimate of exception handler quality

SurfExample found promising based on experimental results

General-purpose code example recommender not much effective

18


FUTURE WORK Applicability or reusability of an example for

exception handling for current context More directed support– exception to be

thrown or to be handled

19


THANK YOU! QUESTIONS?

SPECIAL THANKS TO

20


PROVOCATIVE STATEMENTS Should the checked exceptions be removed

from Java? Was it a mistake? Who should be responsible for exception

handling, language designers or software developers?

21


REFERENCES[1] B. Cabral and P. Marques. Exception Handling: A Field Study in Java and .NET. In Proc. ECOOP,

pages 151–175, 2007[2] S. Bajracharya, J. Ossher, and C. Lopes. Sourcerer: An Internet-Scale Software Repository. In

Proc. SUITE, pages 1–4, 2009.[3] E. A. Barbosa, A. Garcia, and M. Mezini. Heuristic Strategies for Recommendation of Exception

Handling Code. In Proc. SBES, pages 171–180, 2012.[4] R. P. L. Buse and W. R. Weimer. Learning a Metric for Code Readability. TSE, 36(4):546–558,

2010.[5] R. Holmes and G. C. Murphy. Using Structural Context to Recommend Source Code Examples.

In Proc. ICSE, pages 117–125, 2005[6] T. T. Nguyen, H. A. Nguyen, N. H. Pham, J. M. Al-Kofahi, and T. N. Nguyen. Graph-based Mining of

Multiple Object Usage Patterns. In Proc. ESEC/FSE, pages 383–392, 2009.[7] M. P. Robillard and G. C. Murphy. Static Analysis to Support the Evolution of Exception

Structure in Object-Oriented Systems. TOSEM, 12(2):191–221, 2003.[8] C. K. Roy and J. R. Cordy. NICAD: Accurate Detection of Near-Miss Intentional Clones Using

Flexible Pretty-Printing and Code Normalization. In Proc. ICPC, pages 172–181, 2008.[9] H. Shah, C. Görg, and M. J. Harrold. Visualization of Exception Handling Constructs to Support

Program Understanding. In Proc. SoftVis, pages 19–28, 2008.[10] W. Takuya and H. Masuhara. A Spontaneous Code Recommendation Tool Based on

Associative Search. In Proc. SUITE, pages 17–20, 2011[11] Exception Handling Principles. URL http://howtodoinjava.com/2013/04/04/java-exception-handling-

best-practices.[12] Best Practices for Exception Handling. URL https://www.ibm.com/developerworks/library/j-ejbexcept

SurfExample- Recommendation of Exception Handling Code Examples

Education

Transcript of SurfExample- Recommendation of Exception Handling Code Examples