Automatic Detection ofPerformance Design and

Deployment Antipatterns inComponent Based Enterprise

Systems

by

Trevor Parsons

The thesis is submitted toUniversity College Dublin

for the degree of PhDin the

College of Engineering, Mathematical and Physical Sciences.

November 2007

School of Computer Science and Informatics

Dr. J. Carthy. (Head of Department)

Under the supervision of

Dr. J. Murphy

CONTENTS

Abstract vii

Acknowledgements ix

List of Publications xi

1 Introduction 11.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Thesis Contributions and Statement . . . . . . . . . . . . . . . . . . . . . 51.4 Key Assumptions and Scope . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Background 82.1 Performance of Software Systems . . . . . . . . . . . . . . . . . . . . . . 102.2 Component Based Software . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.1 Software Components . . . . . . . . . . . . . . . . . . . . . . . . 112.2.2 Component Frameworks . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 The Java Enterprise Edition . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3.1 Web Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3.2 Business Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.3 Enterprise Information System Tier . . . . . . . . . . . . . . . . . 14

2.4 The Enterprise JavaBean Technology . . . . . . . . . . . . . . . . . . . . 152.4.1 The EJB Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.4.2 The EJB Component Model . . . . . . . . . . . . . . . . . . . . . 162.4.3 EJB Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4.4 Deployment Settings . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.5 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.6 Software Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.7 Software Antipatterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.8 Performance Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.8.1 Workload Generation . . . . . . . . . . . . . . . . . . . . . . . . . 272.8.2 Profiling Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.9 Reverse Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.10 Design Pattern and Antipattern Detection . . . . . . . . . . . . . . . . . 362.11 Knowledge Discovery in Databases and Data Mining . . . . . . . . . . . 39

2.11.1 Frequent Pattern Mining and Clustering . . . . . . . . . . . . . . 40

i

3 Overview of Approach 423.1 Approach for the Automatic Detection of Performance Antipatterns . . 44

3.1.1 Research Methodology and Validation Criteria . . . . . . . . . . 453.1.2 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.1.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.1.4 Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.2 Antipatterns Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503.2.1 Antipattern Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . 51

4 Monitoring Required for Antipattern Detection 544.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564.2 Run-Time Path Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.2.1 Run-Time Paths Overview . . . . . . . . . . . . . . . . . . . . . . 574.2.2 Run-Time Path Tracing Motivation . . . . . . . . . . . . . . . . . 604.2.3 Run-Time Path Tracing Considerations . . . . . . . . . . . . . . 614.2.4 COMPAS Monitoring Framework . . . . . . . . . . . . . . . . . 624.2.5 COMPAS Extensions . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.3 Monitoring Server Resource Usage and Extracting Component Meta-Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.3.1 Using Java Management Extensions to Monitoring Server Re-

source Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.3.2 Automatically Extracting Component Meta-Data . . . . . . . . . 80

4.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.4.1 Applications of Run-Time Paths . . . . . . . . . . . . . . . . . . . 824.4.2 Alternative Representations for Component Interactions . . . . 834.4.3 Run-Time Interaction Tracing Approaches . . . . . . . . . . . . . 84

5 Reconstructing the Systems Design for Antipattern Detection 885.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905.2 Automatically Extracting Component Relationships and Object Usage

Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905.3 Reconstructing Run-time Container Services . . . . . . . . . . . . . . . . 965.4 Identifying Component Communication Patterns in Run-Time Paths

using Frequent Sequence Mining . . . . . . . . . . . . . . . . . . . . . . . 975.4.1 Frequent Itemset Mining and Frequent Sequence Mining . . . . 975.4.2 Support Counting for Run-Time Paths . . . . . . . . . . . . . . . 995.4.3 Further Criteria for Interestingness . . . . . . . . . . . . . . . . . 1025.4.4 Preprocessing for FSM Performance Improvement . . . . . . . . 1025.4.5 Closed Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . 1045.4.6 PostProcessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055.4.7 Component Communication Information for the Extracted De-

sign Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055.5 Data Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

5.5.1 Clustering Run-time Paths . . . . . . . . . . . . . . . . . . . . . . 1065.5.2 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.6 The Reconstructed Design Model . . . . . . . . . . . . . . . . . . . . . . 1075.7 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

5.7.1 Reverse Engineering . . . . . . . . . . . . . . . . . . . . . . . . . 1085.7.2 Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

6 Detecting Performance Design and Deployment Antipatterns 112

ii

6.1 Antipatterns Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1136.2 A Rule Engine Approach for Antipattern Detection . . . . . . . . . . . . 1146.3 Example Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

6.3.1 Antipattern Library . . . . . . . . . . . . . . . . . . . . . . . . . . 1176.3.2 Filtering Using Threshold Values . . . . . . . . . . . . . . . . . . 118

6.4 PAD Tool User Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1186.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

6.5.1 Antipattern Categorisation . . . . . . . . . . . . . . . . . . . . . 1196.5.2 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . 1206.5.3 Detection Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 1206.5.4 Antipattern Detection . . . . . . . . . . . . . . . . . . . . . . . . 121

7 Results and Evaluation 1247.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1267.2 COMPAS JEEM Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.2.1 Deducing System structure . . . . . . . . . . . . . . . . . . . . . 1277.2.2 Portability Assessment . . . . . . . . . . . . . . . . . . . . . . . . 1327.2.3 Performance Overhead . . . . . . . . . . . . . . . . . . . . . . . . 133

7.3 Analysis Module Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 1347.3.1 FSM Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . 1357.3.2 Applying FSM to Identify Design Flaws . . . . . . . . . . . . . . 1407.3.3 Data Reduction Results . . . . . . . . . . . . . . . . . . . . . . . . 142

7.4 PAD Tool Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1447.4.1 Antipatterns Detected in the Duke’s Bank Application . . . . . 1447.4.2 Antipatterns Detected in the IBM Workplace Application - Beta

Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1497.5 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

7.5.1 Overview of Contributions and Evaluation Criteria . . . . . . . 1517.5.2 Validation of Contributions . . . . . . . . . . . . . . . . . . . . . 152

8 Conclusions 1538.1 Thesis Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1548.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1558.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

References 171

A Antipattern Rule Library 172A.1 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

A.1.1 Category1: Antipatterns Across or Within Run-Time Paths . . . 172A.1.2 Category2: Inter-Component Relationship Antipatterns . . . . . 174A.1.3 Category3: Antipatterns Related to Component Communica-

tion Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176A.1.4 Category4: Data Tracking Antipatterns . . . . . . . . . . . . . . 177A.1.5 Category5: Pooling Antipatterns . . . . . . . . . . . . . . . . . . 177A.1.6 Category6: Intra-Component Antipatterns . . . . . . . . . . . . 178A.1.7 Adding Rules to The Rule Library . . . . . . . . . . . . . . . . . 178

A.2 Jess User Defined Functions provided by the PAD Tool . . . . . . . . . . 178A.3 Configuration Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

B JEEM and FSM Implementation Source Code 180

iii

LIST OF FIGURES

1.1 Typical Enterprise Architecture . . . . . . . . . . . . . . . . . . . . . . 3

2.1 Typical JEE Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Client Invoking an EJB . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3 Example EJB Deployment Descriptor . . . . . . . . . . . . . . . . . . . 202.4 Stateless Session Bean Lifecycle . . . . . . . . . . . . . . . . . . . . . . 212.5 Patterns, Antipatterns and their Relationship . . . . . . . . . . . . . . 272.6 JVMPI Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.7 The KDD Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.1 PAD Tool Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443.2 Run-time Design Meta Model . . . . . . . . . . . . . . . . . . . . . . . 463.3 Hierarchy of Antipatterns . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.1 Dynamic Call Trace (a) and Corresponding Dynamic Call Tree (b) . . 584.2 Example Run-Time Path . . . . . . . . . . . . . . . . . . . . . . . . . . 594.3 Run-Time Path Data Structure . . . . . . . . . . . . . . . . . . . . . . . 604.4 A Run-Time Path’s PathNode Data Structure . . . . . . . . . . . . . . 604.5 COMPAS Probe Insertion Process . . . . . . . . . . . . . . . . . . . . . 654.6 COMPAS JEEM Architecture . . . . . . . . . . . . . . . . . . . . . . . . 664.7 Intercepting Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.8 Remote Method Invocation . . . . . . . . . . . . . . . . . . . . . . . . . 744.9 The Sample Bean’s Home Interface . . . . . . . . . . . . . . . . . . . . 754.10 A Wrapper for the Sample Bean’s Home Interface . . . . . . . . . . . . 754.11 A Sample Bean Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.12 Run-Time Path with Tracked Object, as a Sequence Diagram . . . . . . 784.13 JEEManagedObject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.14 JDBCStats, JDBCConnectionStats, JDBCConnectionPoolStats . . . . . 81

5.1 Run-time Design Meta Model from Chapter 3 . . . . . . . . . . . . . . 915.2 Example Run-Time Path (a), Example Deployment Descriptors (b),

Extract of Component Data Structure (c) and Data Extracted to Pop-ulate Data Structure (d) . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

5.3 Example Run-Time Path (a), Extract of the TrackedObject Data Struc-ture (b) and Information Extracted to Populate the TrackedObjectData Structure (c) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

iv

5.4 Example Run-Time Path (a), Extract of the RunTimeContainerSer-vice Data Structure (b), and Information Extracted to Populate theRunTimeContainerService Data Structure (c) . . . . . . . . . . . . . . . 95

5.5 Class Diagram Showing Components Relationships . . . . . . . . . . 985.6 Example Transaction with Different Support Counting Approaches . 1005.7 Hidden Elements in Transaction and Corresponding Support Counts 101

6.1 Rule to Detect Simultaneous Interfaces Antipattern . . . . . . . . . . . 1156.2 Rule to Detect Needless Session Antipattern . . . . . . . . . . . . . . . 1166.3 Rule to Detect Bulky or Unusual Levels of Database Communication 117

7.1 AccountList Run-Time Path and UML sequence diagram . . . . . . . 1287.2 Diagram Showing Components in Duke’s Bank . . . . . . . . . . . . . 1297.3 Diagram Showing Components in PlantsByWebsphere . . . . . . . . . 1317.4 Portability Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 1327.5 Performance Overhead Test Results . . . . . . . . . . . . . . . . . . . . 1337.6 Test Results on K2 10 2 Database . . . . . . . . . . . . . . . . . . . . . 1367.7 Test Results on K2 100 2 Database . . . . . . . . . . . . . . . . . . . . . 1387.8 Test Results on Sun Database . . . . . . . . . . . . . . . . . . . . . . . . 1397.9 Test Results on IBM Database . . . . . . . . . . . . . . . . . . . . . . . 1407.10 Class Diagram of a Modified Version of Duke’s Bank with Commu-

nication Patterns Highlighted . . . . . . . . . . . . . . . . . . . . . . . 143

A.1 The Transactions-A-Plenty Rule . . . . . . . . . . . . . . . . . . . . . . 172A.2 The Conversational-Baggage Rule . . . . . . . . . . . . . . . . . . . . . 173A.3 The Sessions-A-Plenty Rule . . . . . . . . . . . . . . . . . . . . . . . . . 173A.4 The Needless-Session Rule . . . . . . . . . . . . . . . . . . . . . . . . . 174A.5 The Remote-Calls-Locally Rule . . . . . . . . . . . . . . . . . . . . . . . 175A.6 The Accessing-Entities-Directly Rule . . . . . . . . . . . . . . . . . . . 175A.7 The Bloated-Session Rule . . . . . . . . . . . . . . . . . . . . . . . . . . 175A.8 The Unusual-or-Bulky-Session-Entity-Communication Rule . . . . . . 176A.9 The Fine-Grained-Remote-Calls Rule . . . . . . . . . . . . . . . . . . . 176A.10 The Unused-Data-Object Rule . . . . . . . . . . . . . . . . . . . . . . . 177A.11 The Incorrect-Pool-Size Rule . . . . . . . . . . . . . . . . . . . . . . . . 177A.12 The Local-and-Remote-Intefaces-Simultaneously Rule . . . . . . . . . 178

v

List of Acronyms

ADL: Architecture Description LanguageAOP: Aspect Oriented ProgrammingAPI: Application Programming InterfaceAST: Abstract Syntax TreeBCI: Byte Code InstrumentationCCM: CORBA Component ModelCPI: COMPAS Probe InsertionDTO: Data Transfer ObjectEIS: Enterprise Information SystemsEJB: Enterprise Java BeansERP: Enterprise Resource PlanningFCA: Formal Concept AnalysisFIM: Frequent Itemset MiningFSM: Frequent Sequence MiningHTML: HyperText Markup LanguageHTTP: HyperText Transfer ProtocolJ2EE: Java 2 Enterprise EditionJ2SE: Java 2 Standard EditionJDBC: Java Database ConnectivityJEE: Java Enterprise EditionJMS: Java Message ServiceJMX: Java Management ExtensionsJNDI: Java Naming and Directory InterfaceJSP: Java Server PagesJSR: Java Service RequestJVM: Java Virtual MachineJVMPI: Java Virtual Machine Profiler InterfaceJVMTI: Java Virtual Machine Tools InterfaceKDD: Knowledge Discovery in DatabasesLQN: Layered Queuing NetworksMTS: Microsoft Transaction ServerOCL: Object Constraint LanguageOS: Operating SystemPAD: Performance Antipattern DetectionPMI: Performance Monitoring InfrastructurePOJO: Plain Old Java ObjectQN: Queuing NetworksRDBMS: Relational Database Management SystemsRMI: Remote Method InvocationRML Relational Manipulation LanguageSOA: Service Oriented ArchitectureSPA: Stochastic Process AlgebrasSPN: Stochastic Petri NetsSQL: Structured Query LanguageUML: Unified Modelling LanguageXML: Extensible Markup Language

vi

ABSTRACT

Enterprise applications are becoming increasingly complex. In recent times they havemoved away from monolithic architectures to more distributed systems made upof a collection of heterogonous servers. Such servers generally host numerous soft-ware components that interact to service client requests. Component based enterpriseframeworks (e.g. JEE or CCM) have been extensively adopted for building such ap-plications. Enterprise technologies provide a range of reusable services that can assistdevelopers building these systems. Consequently developers no longer need to spendtime developing the underlying infrastructure of such applications, and can insteadconcentrate their efforts on functional requirements.

Poor performance design choices, however, are common in enterprise applicationsand have been well documented in the form of software antipatterns. Design mistakesgenerally result from the fact that these multi-tier, distributed systems are extremelycomplex and often developers do not have a complete understanding of the entire ap-plication. As a result developers can be oblivious to the performance implications oftheir design decisions. Current performance testing tools fail to address this lack ofsystem understanding. Most merely profile the running system and present large vol-umes of data to the tool user. Consequently developers can find it extremely difficultto identify design issues in their applications. Fixing serious design level performanceproblems late in development is expensive and can not be achieved through ”code op-timizations”. In fact, often performance requirements can only be met by modifyingthe design of the application which can lead to major project delays and increasedcosts.

This thesis presents an approach for the automatic detection of performance designand deployment antipatterns in enterprise applications built using component basedframeworks. Our main aim is to take the onus away from developers having to siftthrough large volumes of data, in search of performance bottlenecks in their applica-tions. Instead we automate this process. Our approach works by automatically recon-structing the run-time design of the system using advanced monitoring and analysistechniques. Well known (predefined) performance design and deployment antipat-terns that exist in the reconstructed design are automatically detected. Results of ap-plying our technique to two enterprise applications are presented.

The main contributions of this thesis are (a) an approach for automatic detectionof performance design and deployment antipatterns in component based enterpriseframeworks, (b) a non-intrusive, portable, end-to-end run-time path tracing approachfor JEE and (c) the advanced analysis of run-time paths using frequent sequencemining to automatically identify interesting communication patterns between com-ponents.

vii

Dedicated to my parents, Tommy and Kay.

ACKNOWLEDGEMENTS

Firstly I would like to thank my supervisor, John Murphy for giving me the oppor-tunity to pursue this research, and also for all his help, encouragement and goodhumour along the way. I would also like to thank Liam Murphy who was alwaysavailable for dialog and who has effectively acted as a second supervisor over theyears. I would like to thank Andrew Lee for initially suggesting the notion of ”detect-ing antipatterns”, when I was back searching for research ideas. Also thanks to PeterHughes for his input and feedback during the early days of my work.

Next, I would like to thank my colleagues in Dublin City University, where this jour-ney first began. In particular, I would like to thank Ada Diaconescu, Mircea Trofinand Adrian Mos, from whom I learned so much during my initial two years as a re-searcher, for being fun colleagues, for always being available to bounce ideas off (evennow that you have all unfortunately left Dublin) and for teaching me the importantbasics of the Romanian language. Furthermore I would like to thank Adrian for as-sisting me in extending some of his research ideas and for inviting me to INRIA forinteresting discussions on my work. I would also like to thank Colm Devine, AdrianFernandes and Cathal Furey (three of the four horsemen) for their engaging lunchtime discussions (in the early days) on the human anatomy and other such topics.

Thanks also to my DCU/UCD colleagues, Dave ”the brickie” McGuinness, Jenny Mc-Manis, Gabriel Muntean, Christina Muntean, Christina Thorpe, Alex Ufimtsev, Oc-tavian Ciuhandu, Lucian Patcas, Olga Ormand, Jimmy Noonan, Hamid Nafaa, PetrHnetynka, Sean Murphy, John Fitzpatrick (for allowing me to introduce him to Las Ve-gas), John Bergin, Omar Ashagi (for teaching me basic Arabic) and Philip McGovernfor all being fun colleagues and to all those who I have had the pleasure of workingwith over the years. Also thanks again to Sean Murphy for taking my questions overthe years and especially for his help with some of the mathematical aspects of myresearch.

Furthermore, thanks to all those in IBM who helped me during my work and grantedme access to their environments. Thanks especially to Pat O’Sullivan and Simon Piz-zoli for their help, interest and invaluable feedback on my research.

A special thanks to Claire Breslin for her endless support and patience, and for re-minding me about the more important things in life.

Finally I would like to thank my parents, Tommy and Kay and brother, Tom, for theirconstant encouragement. I would especially like to thank my parents, to whom thiswork is dedicated. Without their unwavering love and support this work would nothave been possible.

ix

LIST OF PUBLICATIONS

Trevor Parsons, John Murphy. Detecting Performance Antipatterns in Component BasedEnterprise Systems. Accepted for publication in the Journal of Object Technology.

Trevor Parsons, John Murphy, Patrick O’Sullivan, Applying Frequent Sequence Miningto Identify Design Flaws in Enterprise Software Systems. In Proceedings of the 5th Inter-national Conference on Machine Learning and Data Mining, Leipzig, Germany, July18-20, 2007.

Trevor Parsons, John Murphy, Simon Pizzoli, Patrick O’Sullivan, Adrian Mos, ReverseEngineering Distributed Enterprise Applications to Identify Common Design Flaws. Pre-sented at the Software Engineering Tools For Tomorrow (SWEFT) 2006 Conference,T.J. Watson, New York, Oct 17 - 19, 2006.

Liang Chen, Patrick O’Sullivan, Laurence P. Bergman, Vitorrio Castelli, Eric Labadie,Peter Sohn, Trevor Parsons. Problem Determination in Large Enterprise Systems. Pre-sented at the Software Engineering Tools For Tomorrow (SWEFT) 2006 conference,T.J. Watson, New York, Oct 17 - 19, 2006. (Abstract only available)

Trevor Parsons, Adrian Mos, John Murphy. Non-Intrusive End to End Run-time PathTracing for J2EE Systems. IEE Proceedings Software, August 2006

Trevor Parsons, John Murphy. The 2nd International Middleware Doctoral Symposium:Detecting Performance Antipatterns in Component-Based Enterprise Systems. IEEE Dis-tributed Systems Online, vol. 7, no. 3, March, 2006

Trevor Parsons. A Framework for Detecting Performance Design and Deployment Antipat-terns in Component Based Enterprise Systems. In Proceedings 2nd International Middle-ware Doctoral Symposium, ACM Press, art. no. 7, Grenoble, France, 2005

Trevor Parsons. A Framework for Detecting, Assessing and Visualizing Performance An-tipatterns in Component Based Systems. First Place at ACM SIGPLAN Student ResearchCompetition Graduate Division, In OOPSLA’04: Companion to the 19th annual ACMSIGPLAN conference on Object-oriented programming systems, languages, and ap-plications, pages 316-317, Vancouver, BC, Canada, 2004.

x

Trevor Parsons, John Murphy. A Framework for Automatically Detecting and AssessingPerformance Antipatterns in Component Based Systems using Run-Time Analysis. The 9thInternational Workshop on Component Oriented Programming, part of the 18th Eu-ropean Conference on Object Oriented Programming. Oslo, Norway, June 2004.

Trevor Parsons, John Murphy. Data Mining for Performance Antipatterns in ComponentBased Systems Using Run-Time and Static Analysis. Transactions on Automatic Controland Control Science, Vol. 49 (63), No. 3, pp. 113-118 - ISSN 1224-600X, May 2004.

xi

CHAPTER

ONE

Introduction

Main Points

• Performance is a major issue during the development of enterprise applications.

• System complexity leads to a lack of understanding and consequently poor de-sign decisions are commonly made by developers.

• Poor system design is often responsible for a badly performing system.

• Current performance testing tools do not address performance design issues andare limited.

• There are a large number of well known design issues for enterprise systems.

• Antipatterns document well known design issues and their corresponding solu-tion.

• Thesis Contributions:

– An approach for the automatic detection of performance design and de-ployment antipatterns in systems built on component based enterpriseframeworks.

– A portable, low overhead, non-intrusive, end-to-end run time path tracerfor distributed JEE systems.

– A technique for the identification of interesting communication patterns ina collection of run-time paths.

1

1.1 Background and Motivation

In the past software developers had to be extremely careful when developing their ap-plications as resources were often scarce and the management of such scare resourceswas a complex issue. Modern advances in software technologies, however, have al-lowed for developers to concentrate less on issues such as performance and resourcemanagement, and instead developers have been able to spend more time developingthe functionality of their applications. An example of this can be seen in modern lan-guages (Java1, C#2) that provide garbage collection facilities, freeing developers fromthe task of having to manage memory, which had typically been a complex and timeconsuming exercise. Freeing developers from having to worry about what is happen-ing ”under the hood” allows them to concentrate more of their efforts on developingthe functionality of a system. This is even more obvious with enterprise level com-ponent frameworks (e.g. JEE3 or CCM4) whereby the framework can be expected tohandle complex underlying issues such as security, persistence, performance and con-currency to name but a few. Again the idea is to allow developers to concentrate onthe application functionality such that the time to market is reduced. A downsideof this advance in software technologies is that developers become less familiar withthe mechanics of the underlying system, and as a result, can make decisions duringdevelopment that have an adverse effect on the system.

Performance is a major issue for developers building large scale multi user enterpriseapplications. In fact recent surveys have shown that a high percentage of enterpriseprojects fail to meet their performance requirements on time or within budget 5 6. Thisleads to project delays and higher development costs, and results from the fact that de-velopers often do not have a complete understanding of the overall system behaviour.Figure 1.1 shows a typical enterprise application made up of a number of differentphysically distributed servers. Each server can in turn be made up of a large numberof software components that interact to service different client requests. Understand-ing the run-time behaviour of such systems can be a difficult task and consequentlyit is common that developers are unaware of the performance implications of theirdesign decisions.

Current development and testing tools fail to address this issue of understanding en-terprise system behaviour. For example most of today’s performance tools merelyprofile the running system and present performance metrics to the tool user. The

1The Java Technology, Sun Microsystems, http://java.sun.com/2The C# language, Microsoft, http://msdn2.microsoft.com/en-us/vcsharp/aa336809.aspx3Java Enterprise Edition, Sun Microsystems, http://java.sun.com/javaee/4The CORBA Component Model specification, The Object Management Group,

http://www.omg.org/technology/documents/formal/components.htm5Ptak, Noel and Associates, ”The State of J2EE Application Management: Analysis of 2005 Benchmark

Survey”, http://www.ptaknoelassociates.com/members/J2EEBenchmarkSurvey2005.pdf,6Jasmine Noel, ”J2EE Lessons Learned ”, SoftwareMag.com, The Software IT Journal, January, 2006.

http://www.softwaremag.com/L.cfm?doc=2006-01/2006-01j2ee

2

Figure 1.1: Typical Enterprise Architecture

volume of data produced when profiling even a single user system can be extremelylarge. When profiling multi-user enterprise applications, where a typical load may bein the order of thousands, the amount of data produced can be truly overwhelming.Often developers are required to sift through and correlate this information lookingfor bottlenecks in their systems. Furthermore, even when developers find issues intheir applications using these tools, it is common that they are unsure as to how to goabout rectifying the issue. There is a clear need for more advanced performance toolsthat not only profile the running system, but that also analyse the data produced toidentify potential issues in the application. While there has been research in the area ofdebugging tools (e.g. [95] [145] [55] [14] [47]) which allow for automatic low-level bugdetection, often it is the case that low-level optimizations or fixes will not be enough toenhance the system efficiency such that performance requirements are met. In manysituations an overhaul of the system design is required.

There are a large number of well known design mistakes that are consistently madeby developers building these systems. Such issues have been documented in the formof software design antipatterns [36]. Similar to software design patterns, which doc-ument best practices in software development, software antipatterns document com-mon mistakes made by developers when building software systems. However, as wellas documenting the mistake, antipatterns also document the corresponding solutionto the problem. Thus not only can they be used to identify issues in software systems,but they can also be used to rectify these issues by applying the solution provided. Amore complete and detailed definition of software patterns and antipatterns is givenin sections 2.6 and 2.7 respectively.

3

1.2 Thesis Overview

In light of the limitations of current performance tools and of the benefits of softwareantipatterns, we have developed an approach to automatically identify performancedesign and deployment antipatterns in systems built on enterprise component-basedframeworks. This approach takes the burden away from developers, of having tosift through large volumes of monitoring data in search of design flaws, and insteadautomates this process. Well known performance design flaws can be identified au-tomatically. Identified issues are presented with related contextual information and acorresponding solution to the problem such that the problem can be easily addressed.

The approach works by reconstructing the run-time design of the application undertest. The reconstructed design can be subsequently checked for well known pre-defined antipatterns. From a high level this is achieved (a) by monitoring the run-ning system to collect information required for antipattern detection, (b) by perform-ing analysis on the monitoring data to summarise it and to identify relationships andpatterns in the data that might suggest potential design flaws, (c) by representing theanalysed data in a design model of the system and (d) by loading the design into a ruleengine such that antipatterns (pre-defined as rules) can be detected. The approach hasbeen realised in the Performance Antipattern Detection (PAD) tool. The tool has beendesigned for the Java Enterprise Edition (JEE) technology.

The remainder of the thesis is structured as follows: Chapter 2 gives backgroundinformation on related technologies and related work. Chapter 3 gives a more de-tailed overview of our approach, discusses our research methodology and outlines anumber of criteria that we use to validate our work. In this chapter we also give anoverview of software design antipatterns, with particular focus on performance an-tipatterns. Chapter 4 outlines the different monitoring approaches that are requiredfor antipattern detection in a component based enterprise system, and how they canbe performed in a portable manner. Chapter 5 details a number of advanced anal-ysis techniques that are applied to identify interesting relationships and patterns inthe run-time data. In particular it presents an approach for identifying frequent or re-source intensive communication patterns between components using techniques fromthe field of data mining. In this chapter we also show how the data collected from en-terprise systems under load can be reduced and summarised. Chapter 6 shows howa rule engine approach can be used to identify antipatterns in the reconstructed run-time design of the system. In this chapter we also categorise JEE performance designand deployment antipatterns into groups based on the data required to detect them.Chapter 7 presents different sets of results from a range of tests that we have per-formed to validate our research. Finally chapter 8 gives our conclusions and ideas onfuture work in this area.

4

1.3 Thesis Contributions and Statement

The first major contribution of this thesis is an approach for the automatic detection ofdesign and deployment antipatterns in systems built using component based enter-prise frameworks [125] [129] [130] [131] [132] [133]. This approach builds on currentperformance tools by performing analysis on the data collected (i.e. run-time infor-mation and component meta-data). The analysis reconstructs the system design andidentifies performance design flaws within it. The approach has been implementedfor the JEE technology in the form of the PAD tool, however it could potentially beapplied to other component based enterprise frameworks (e.g. CCM). This solutionhas been successfully applied to both a sample and a real JEE application and has anumber of key advantages.

Firstly, it reduces and makes sense of the data collected by many of today’s perfor-mance profilers. This work makes use of statistical analysis and data mining tech-niques to summarise the data collected and to find patterns of interest that mightsuggest performance problems. Thus, it takes the onus away from developers whocurrently have to carry out this tedious task manually.

Secondly, while most of today’s performance tools tend to focus on identifying lowlevel hotspots and programming errors (e.g. memory leaks, deadlocks), this workfocuses on analysing the system from a performance design perspective. Since designhas such a significant effect on performance [43] it is essential that work is carried outin this area.

Thirdly, unlike with many of today’s performance tools, problems identified are anno-tated with descriptions of the issue detected, as well as a solution that can be appliedto alleviate the problem. This approach of identifying and presenting antipatterns todevelopers helps them understand the mistakes that have been made, and the under-lying reason as to why performance was affected. Developers can learn from usingour tool, and thus it may be less likely that the same mistakes are made in the future.This approach also allows developers to easily rectify the situation by applying thesolution provided. In fact, the antipatterns presented provide a high level languagethat developers and management alike can use to discuss such problems when theyoccur.

The second major contribution of this work is a portable, low overhead, non-intrusive, end-to-end run-time path tracer for JEE systems [128]. This is the first com-pletely portable approach for collecting end-to-end run-time paths across all serverside tiers of a distributed JEE application. It is non-intrusive insofar as it does not re-quire any modifications to the application or middleware source code. The monitor-ing approach instead makes use of standard JEE mechanisms to intercept calls madeto the instrumented components. A run-time path [44] contains the control flow (i.e.the ordered sequence of methods called required to service a user request), resources

5

and performance characteristics associated with servicing a request. Such informationis utilised to detect antipatterns by our PAD tool. By analysing run-time paths one caneasily see how system resources are being used, how the different components in thesystem interact and how user requests traverse through the different tiers that makeup the system. In fact these paths can also be used for Object Tracking, i.e. to monitorparticular objects’ life cycles across the different user requests. In this work we showhow run-time paths can be used to manually and automatically reverse engineer aJEE application. We also show how the reconstructed design can be used for eithermanual or automatic identification of performance design flaws. For example, thePAD tool makes use of run-time paths to identify the (run-time) component relation-ships, communication patterns and object usage patterns in a JEE system. Results aregiven for this monitoring approach which show that it produces a low overhead onthe instrumented system and that it can be applied in a portable manner.

The third and final major contribution of this work is a technique for the identifica-tion of interesting communication patterns in a collection of run-time paths [126].More precisely, we have applied a data mining technique, Frequent Sequence Mining(FSM) to identify sequences of interest (e.g. frequently repeating method sequencesand resource intensive loops) across a transactional database of run-time paths by us-ing alternative support counting techniques. In this work we also discuss scalabilityproblems (in terms of both the algorithm runtime and the amount of data produced)related to applying FSM to run-time paths and give solutions to these issues. Weshow how the sequences identified, can be used to highlight design flaws in enter-prise applications, that lead to poor system performance. The PAD tool makes use ofthis analysis technique to identify interesting component communication patterns ina JEE system that may indicate the presence of particular antipatterns.

Following the above contributions the fundamental thesis of this work can be statedas follows:

Performance design and deployment antipatterns can be automatically detected in componentbased enterprise systems by analysing run-time data and component meta-data.

1.4 Key Assumptions and Scope

The work in this thesis is focused on component based systems as defined in section2.2. As such, it is highly likely that the source code of the application to be analysedis not available in its entirety, as components may have been developed by third par-ties. Thus we assume source code is not available for analysis of the system. For suchsystems bytecode analysis may also be problematic due to security restrictions or li-censing constraints. Instead, we assume that a running implementation of the systemto be analysed is available such that dynamic data can be collected and utilised foranalysis.

6

We also assume that a realistic testing scenario is available which reflects how the sys-tem will be used in production. We do not address the issue of how such testing sce-narios can be obtained in this work, however, research in this area already exists. Forexample Weyuker and Voklos have outlined an approach for the development of per-formance test cases [176]. In this literature five typical steps required to develop per-formance test cases are outlined. Alternatively, Ho et al. [92] propose an evolutionaryapproach to performance test design based on their Performance Requirements Evolu-tion Model. The authors claim that more precise and realistic performance tests can becreated incrementally during the development process through customer communi-cation or performance model solving. In addition, agile development techniques suchas test-driven development [25] promote the design of test cases before developersbegin to code. Recently work has been presented which discusses how performancetests can be incorporated into the test driven development process [96] allowing forearly availability of performance testing scenarios.

Our approach is applicable to applications built on component based enterpriseframeworks. However our research has thus far only been applied to synchronouscomponents, and has not, for example, been applied to message driven beans whichare asynchronous components in the JEE technology. Thus, our scope is limited tosynchronous applications. Our plans for future work outline how this approach couldpotentially be applied to asynchronous components (see section 8.3).

7

CHAPTER

TWO

Background

In this chapter we introduce related research areas and technologies. We begin bydiscussing the research area of performance engineering. Next we give an overviewof component based software, giving a definition for a software component and dis-cussing component frameworks. We also give background information on the JavaEnterprise Edition technology which is the enterprise framework that our work hasbeen applied to. We focus specifically on the Enterprise Java Bean component tech-nology and give details in this area related to our research. We present an overview ofthe state of the art in performance tools discussing techniques for load generation andperformance profiling. We particularly focus on performance profiling tools for theJava technology. Furthermore we give an overview of software architecture, softwarepatterns, and software antipatterns. An overview of research in the area of reverseengineering is also presented. In this section we outline why previous approaches areless suitable for distributed component based applications. The current state of theart of research in the area of software pattern/antipattern detection is also discussed.Finally we introduce the area of knowledge discovery in databases, and data miningtechniques relevant in this work.

8

Main Points

• Current performance analysis techniques, e.g., modelling, are inaccurate andtime consuming when applied to component based enterprise systems. Thus,in industry performance analysis is usually deferred until performance testingbegins using the currently available performance testing tools.

• Component technologies such as EJB are increasingly being adopted to providefor flexible, manageable and reusable solutions for complex software systems.However poor system performance is common in these systems.

• System Architecture focuses on issues related to the overall system structure andis said to be non-local, whereas software design focuses on local issues .

• Enterprise design plays a significant role in a system’s overall performance. Bestpractices in design have been well documented in the form of design patterns.

• Well known design issues consistently occur in enterprise applications and havebeen well documented, along with their corresponding solution, in the form ofdesign antipatterns.

• Performance testing tools for complex-multi user enterprise applications are lim-ited and merely profile the running system, presenting vast amounts of data tothe tool user. There is a clear need from more advanced tools, that take the onusaway from the developer of having to sift through this data, and that automati-cally analyse the data produced.

• Detailed documentation is generally not available for enterprise applications.Thus, it can be difficult for developers to comprehend the overall applicationdesign.

• Current reverse engineering/design pattern detection/antipattern detectiontechniques are heavily based on static analysis and are unsuitable for compo-nent based systems

• Data Mining techniques can be applied to extract knowledge from vast volumesof data.

9

2.1 Performance of Software Systems

The performance of a software system has been described as an indicator of how wellthe system meets its requirements for timeliness [154]. Smith and Williams [154] de-scribe timeliness as being measured in either response time or throughput, where re-sponse time is defined as the time required to respond to a request and throughputis defined as the number of requests that can be processed in some specific time in-terval. Furthermore they define two important dimensions to software performancetimeliness, responsiveness and scalability. Responsiveness is defined as the ability ofa system to meet its objectives for response time or throughput. The ability to con-tinue to meet these objectives, as the demand on the system increases, is defined asthe systems scalability. The aim of performance engineering is to build systems thatare both responsive and scalable.

To date a vast amount of performance engineering research has focused on systemanalysis through performance models. Performance models are created based on sys-tem artifacts and various relevant estimates. Some of the most common performancemodel classes are Queuing Networks (QN) (or extensions, such as Extended QN andLayered QN), Stochastic Petri Nets (SPN), and Stochastic Process Algebras (SPA). Per-formance models can be evaluated using simulation techniques or analytical methods,in order to predict performance indices, such as throughput, response times, or re-source utilization. A comprehensive survey of modelling approaches for performanceprediction is presented in [15].

However modelling today’s enterprise applications with a high degree of accuracycan be a difficult and time consuming task. This results from the fact that these sys-tems are often very large and complex and made up of black box components, theinternals of which are generally unknown (e.g. application servers). Performancemetrics required to populate performance models can thus not be easily obtained.For enterprise applications accurate performance metrics can often only be obtainedthrough performance testing of a running system [58]. Recently Liu et al. [76] haveused a combination of performance modelling and benchmarking techniques to al-low for population of performance models of enterprise applications. Their initialresults give accurate performance prediction for the small sample systems. A draw-back of this approach is the lack of tool support which would allow for this techniqueto be easily reused. From our experiences with large software houses, it seems thatperformance modelling of enterprise applications is rarely performed. Most opt forperformance testing using available performance testing tools.

Work in the area of performance testing, however, has been very much lacking [176]and thus performance testing, especially in the case of large enterprise applications,can be a difficult task [76] [161]. This comes from the fact that today’s performancetesting tools are quite limited, insofar as they generally focus on simply collecting

10

data from a running system (i.e. profiling) and presenting this data to the user. Thesetools tend to focus on low level programming bugs and do not address many of theissues that lead to poor system performance (e.g. design issues).

2.2 Component Based Software

2.2.1 Software Components

There are numerous definitions of what software components are or should be 1. To bespecific, for the purpose of this thesis, we use Szyperski’s definition of a software com-ponent: ”A software component is a unit of composition with contractually specifiedinterfaces and explicit context dependencies only. A software component can be de-ployed independently and is subject to composition by third parties” [157]. Defining asoftware component as a unit of composition simply means that the purpose of a com-ponent is to be composed with other components. A component based application isassembled from a set of collaborating components. To be able to compose componentsinto applications, each component must provide one or more interfaces which providea contract between the component and its environment. The interface clearly defineswhich services the component provides and therefore defines its responsibility. Usu-ally, software depends on a specific context, such as available database connectionsor other system resources being available. For example, other components that mustbe available for a specific component to collaborate with. In order to support com-posability of components, component dependencies need to be explicitly specified. Acomponent can be independently deployed, i.e. it is self-contained and changes tothe implementation of a component do not require changes to other components. Ofcourse, this is only true as long as the component interface remains compatible. Fi-nally assemblers of component based applications, are not necessarily the developersof the different components. That is, components can be deployed by third partiesand are intended to be reused.

This definition of a software component leaves many details open, for example, howcomponents interact, what language(s) can be used for their development, for whatplatform. Component frameworks further define the notion of a component, by de-tailing these issues.

2.2.2 Component Frameworks

The key goal of component technology is independent deployment and assembly ofcomponents. Component frameworks are the most important step for achieving this

1Beyond Objects column of Software Development magazine, articles by Bertrand Meyer andClemens Szyperski, archived at http://www.ddj.com/

11

aim [157]. They support components conforming to certain standards (or componentmodels) and allow instances of these components to be plugged into the componentframework. The component framework establishes environmental conditions for thecomponent instances and regulates the interaction between component instances. Akey contribution of component frameworks is partial enforcement of architecturalprinciples. By forcing component instances to perform certain tasks via mechanismsunder control of a component framework the component framework can enforce itspolicies on the component instances. This approach helps prevent a number of classesof subtle errors that can otherwise occur.

There are numerous component frameworks that exist today. Examples include EJB,CCM, SOFA [140] and Fractal [37]. Each framework contains its own componentmodel, i.e. a set of features that components satisfy. Component models generally con-form to either flat component models or hierarchical models. Flat component models(e.g. EJB, CCM) define only primitive components whereby indivisible entities are di-rectly implemented in a programming language. Hierarchical models (SOFA, Fractal)also define composite components which are created via nesting of other components.The research in this thesis aims at solving issues related to flat component models. Inparticular we focus on EJB. EJB is part of a wider enterprise framework (Java Enter-prise Edition) for building enterprise level applications (see section 2.3). EJB has beenselected since it is a well established technology that is currently used in industry todevelop enterprise applications. There is also a body of work detailing best practicesand bad practices for this technology (see sections 2.6 and 2.7). On the other hand thehierarchical component models have mainly been used by the research communityand best practices in these areas are less well defined.

EJB is considered a contextual composition framework. Contextual compositionframeworks allow components to specify boundary conditions describing propertiesthat the runtime context must meet [166]. Composition performed by such frame-works is based on the creation of contexts and placement of component instancesin the appropriate contexts. For example, a component framework for transactionalcomputation can be formed by supporting transactional attribution of components(for example ”this component’s instances need to be used in a new transaction”) andtransactional enactment at the boundary of contexts. This approach can be used to cre-ate frameworks for any other properties such as security, load balancing, managementetc.

Any component instance in a context can potentially be accessed from outside its con-text. This context however, gets an opportunity to intercept all messages crossing thecontext boundaries. Intercepting instances (e.g. objects) inside a context remains in-visible to instances both external and internal to this context.

Current technology support for contextual composition includes Microsoft Transac-

12

tion Server (MTS)2, EJB containers, and CCM containers. We introduce the EJB tech-nology in the following sections. The run-time services we discuss in section 5.3 arecreated as a result of contextual composition.

2.3 The Java Enterprise Edition

The Java Enterprise Edition (JEE) is a component technology which defines a standard(or a set of standards) for developing multi-tier enterprise applications. JEE, formerlythe Java 2 Enterprise Edition (J2EE), is an enterprise component framework for theJava technology. The specification promotes a multi-tiered distributed architecture forenterprise applications. Figure 2.1 shows a typical JEE architecture consisting of 4main tiers: a client tier, a presentation or web tier, a business tier and an enterpriseinformation systems tier. JEE specifies different component types for implementingthe various enterprise application tiers. Naturally, clients reside in the client tier andcan be in the form of stand-alone Java applications or web browsers. In the followingsubsections we detail each of the server-side tiers and give details on the componentsthat they can consist of.

Figure 2.1: Typical JEE Architecture

2.3.1 Web Tier

The JEE web tier provides a run-time environment (or container) for web components.JEE web components are either servlets or pages created using the Java Servlet Pages

2Microsoft Corporation. Microsoft Transaction Server Transactional Component Services.http://www.microsoft.com/com/wpaper/revguide.asp.

13

technology (JSPs) 3. Servlets are Java programming language classes that dynamicallyprocess requests and construct responses. They allow for a combination of static anddynamic content within the web pages. JSP pages are text-based documents that ex-ecute as servlets but allow a more natural approach to creating the static content asthey integrate seamlessly in HTML pages. JSPs and Servlets execute in a web con-tainer and can be accessed by clients over HTTP (e.g. a web browser). The servletfilter technology is a standard JEE mechanism that can be applied to components inthe web tier to implement common pre and post-processing logic. It is discussed indetail in section 4.2.5.1.

2.3.2 Business Tier

Enterprise Java Beans (EJBs) 4 are the business tier components and are used to handlebusiness logic. Business logic is logic that solves or meets the needs of a particularbusiness domain such as banking, retail, or finance for example. EJBs run in an EJBcontainer and often interact with a database in the EIS tier in order to process requests.Clients of the EJBs can be either web components or stand alone applications. EJBis the core of the JEE platform and provides a number of complex services such asmessaging, security, transactionality and persistence. These services are provided bythe EJB container to any EJB component that requests them. More details on the EJBcomponent model are given in section 2.4

2.3.3 Enterprise Information System Tier

Enterprise information systems provide the information infrastructure critical to thebusiness processes of an enterprise. Examples of EISs include relational databases,enterprise resource planning (ERP) systems, mainframe transaction processing sys-tems, and legacy database systems. The JEE Connector architecture 5 defines a stan-dard architecture for connecting the JEE platform to heterogeneous EIS systems. Forexample a Java Database Connectivity (JDBC) Connector is a JEE Connector Archi-tecture compliant connector that facilitates integration of databases with JEE appli-cation servers. JDBC 6 is an API and specification to which application developersand database driver vendors must adhere. Relational Database Management Systems(RDBMS) vendors or third party vendors develop drivers which adhere to the JDBCspecification. Application developers make use of such drivers to communicate withthe vendors’ databases using the JDBC API. The main advantage of JDBC is that itallows for portability and avoids vendor lock-in. Since all drivers must adhere to the

3Java Servlet Technology, http://java.sun.com/products/servlet/4Enterprise Java Bean Technology, http://java.sun.com/products/ejb/docs.html5Java Connector Architecture, http://java.sun.com/j2ee/connector/6Java Database Connectivity Architecture, http://java.sun.com/products/jdbc/

14

same specification, application developers can replace the driver that they are usingwith another one without having to rewrite their application.

2.4 The Enterprise JavaBean Technology

The Enterprise Java Beans architecture is a component architecture for the develop-ment and deployment of component based distributed applications. It is designed tosimplify and reduce the costs of the development and management processes of large-scale, distributed applications. Applications built using this technology are capableof being scalable, transactional, and multi-user secure. EJB provides the distributedplatform support and common services such as transactions, security, persistence andlifecycle management. EJB also defines a flexible component model which allows forcomponents of different types that are suitable for specific tasks. Developers make useof the different component types to implement the application business logic. Subse-quently, EJBs are deployed and managed by EJB containers, as part of a JEE applica-tion server. EJB containers provide middleware services and manage the EJB lifecycleduring runtime. These processes can be configured via XML documents, referred toas EJB deployment descriptors. Physically EJB consists of two things [148]:

The specification7 which defines:

• The distinct ”EJB Roles” that are assumed by the component architecture.

• A component model

• A set of contracts: component-platform and component-client

A set of Java Interfaces:

• Components and application servers must conform to these interfaces. This al-lows all conforming components to inter-operate. Also the application servercan manage any components that conform to the interfaces.

2.4.1 The EJB Roles

The EJB specification defines the following roles which are assumed by the componentarchitecture:

• Enterprise Bean Provider: The enterprise bean provider is typically an applica-tion domain expert. The bean provider develops the reusable enterprise beansthat typically implement business tasks or business entities.

7The Enterprise Java Bean Specification version 2.0, http://java.sun.com/products/ejb/docs.html

15

• Application Assembler: The application Assembler combines enterprise beansinto larger deployable application units.

• Deployer: The Deployer takes the ejb-jar files produced by either the BeanProvider or the Application Assembler and deploys the enterprise beans con-tained in the ejb-jar files in a specific operational environment. The operationalenvironment includes the EJB Server and Container.

• EJB Service Provider and EJB Container Provider: The container supplies an EJBcontainer (the application server). This is the runtime environment in whichthe beans live. The container provides middleware services to the beans andmanages them. The server provider is the same as the container provider. Sunhas not yet differentiated between them.

• System Administrator: The system administrator is responsible for the upkeepand monitoring of the deployed system and may make use of runtime monitor-ing and management tools provided by the EJB server provider

2.4.2 The EJB Component Model

EJB is built on top of object technology (Java). An EJB component consists of a busi-ness interface, an implementation class, a home interface and configuration settings(defined in an XML deployment descriptor). All of these, except for the deploymentdescriptor, are Java artifacts (i.e. classes or interfaces).

The EJB implementation class contains the bean business logic written by the Enter-prise Bean Provider. The EJB implementation class is a Java object that conforms to awell defined interface and obeys certain rules. The interface it conforms to dependson the bean type. The rules are necessary in order for the bean to be able to run in acontainer.

Access to the implementation class can be obtained using a the EJB home interface.The home interface defines methods for creating, destroying and finding EJBs (i.e.lifecycle methods). The home interface can either be local or remote. Local interfacesallow access from clients within the same JVM whereas remote interfaces allow foraccess from remote clients (e.g. on another JVM running on the same machine or ona JVM running on a physically distributed machine). In fact an EJB component canhave both local and remote interfaces, however this is not recommended [161].

The bean implementation business methods are exposed through the business inter-face. Similar to the home interface the business interface can be exposed locally orremotely (or both).

An EJB component also requires configuration settings for deployment. These settingsare defined in a XML deployment descriptor. The information in the deployment

16

descriptor details the different container services that are required by the EJB. Forexample, a deployment descriptor can be used to declare how the container shouldperform lifecycle management, persistence, transaction control, and security services.

EJB 2.0 defines three different kinds of enterprise beans, namely session beans, entitybeans and message-driven beans.

Session Beans: A session bean is an action bean that performs work for its client,shielding the client from complexity by executing business tasks inside the server. Asession bean has only one client. When the client terminates the session appears toterminate and is no longer associated with the client. The life of a session bean spansthe length of the session (or conversation) between the session and the client. Sessionbeans are not persistent and typically they do not survive application server crashes,or machine crashes. They are in memory objects that live and die with their surround-ing environments. Session beans hold conversations with clients. A conversation is aninteraction between a client and the bean. The two subtypes of session beans are state-ful session beans and stateless session beans. Each is used to model different types ofconversations.

Stateful Session Beans: A stateful session bean is a bean that is designed to servicebusiness processes that span multiple method requests or transactions. Stateful ses-sion beans retain state on behalf of an individual client. If a stateful session bean’sstate is changed during a method invocation, that same state will be available to thatsame client upon the following invocation.

Stateless Session Beans: A stateless session bean is a bean that holds conversationsthat span a single method call. They are stateless because they do not hold multi-method conversations with their clients. Except during method invocation, all in-stances of a stateless bean are equivalent, allowing the EJB container to assign an in-stance to any client. Because stateless session beans can support multiple clients, theycan offer better scalability for applications that require a large number of clients. Typ-ically, an application requires fewer stateless session beans than stateful session beansto support the same number of clients.

Entity Beans: Entity beans are persistent data components. Entity beans are enterprisebeans that know how to persist themselves permanently to a durable storage (e.g. adatabase). They are physical, storable parts of an enterprise. Entity beans differ fromsession beans in a number of ways. They are persistent, and allow shared access. Theyhave a unique identifier, enabling a client to identify a particular entity bean. Entitybeans can also persist in relationships with other entity beans. Entity beans can be per-sisted in two ways, either using Bean-Managed Persistence, or Container-ManagedPersistence. Container-Managed persistent beans are the simplest for the bean de-veloper to create. All logic for synchronizing the bean’s state with the database ishandled automatically by the container. Thus, the beans do not contain any databaseaccess calls, and as a result the bean’s code is not tied to a specific persistent storage

17

mechanism (database). A Bean-managed persistent entity bean is an entity bean thatmust be persisted by hand. The component developer must write code to translatethe in-memory fields into an underlying data store.

Message-driven Beans: A message-driven bean is an enterprise bean that allows EJBapplications to process messages asynchronously. They rely on the Java Message Ser-vice (JMS) technology 8. Message-driven beans act as JMS message listeners. Themessages may be sent by any JEE component: an application client, another enter-prise bean, a Web component or by a JMS application or system that does not use JEEtechnology. A message-driven bean does not have component interfaces. The com-ponent interfaces are absent because the message-driven bean is not accessible via theJava RMI API; it responds only to asynchronous messages. One of the most impor-tant aspects of message-driven beans is that they can consume and process messagesconcurrently, because numerous instances of the MDB can execute concurrently inthe container. This capability provides a significant advantage over traditional JMSclients.

As discussed in section 1.4 we have not applied our research to asynchronous compo-nents. This is a direct result of the fact that our run-time path tracing approach (seesection 4.2) can not currently be used to monitoring message driven beans. Our plansfor future work suggest how this problem may be addressed (see section 8.3).

2.4.3 EJB Runtime

An EJB component contains a bean implementation class, a business interface, a homeinterface and an XML deployment descriptor all of which are supplied by the beanprovider. To integrate the component into the container environment the containerautomatically generates ”glue-code” that allows for the component to implicitly makeuse of the container services. In fact, enterprise beans are not fully-fledged remote ob-jects. When a client accesses an EJB, the client never invokes the methods directly onthe actual bean instance. Instead, the invocation is intercepted by the EJB containerand delegated to the bean instance. The interception is performed by the EJBObject.The EJBObject is generated by the container (either during deployment or at run-time)and provides the enterprise bean with networking capabilities and container servicessuch as transactions and security. The EJBObject replicates and exposes every businessmethod that the bean exposes. It is generated from the business interface supplied bythe bean provider. Similarly an EJBHome object is generated from the home interface.The EJBHome object exposes the same methods as this interface and acts as a factoryobject for EJBObjects. That is, the EJBHome Object is responsible for creating and de-stroying EJBObjects. In order to understand how the various component constituentswork together we give an example of the various steps that are performed by a client

8Java Message Service, from Sun Microsystems: http://java.sun.com/products/jms/

18

and by the container when a bean is invoked.

To create an instance of an EJB a client must first obtain an instance of a EJBHome ob-ject, generated by the container. The EJBHome object is bound to the component nameand available at run-time through the system’s naming directory, accessed through theJava Naming and Directory Interface (JNDI)9. Thus to invoke an EJB a client performsthe following steps (see figure 2.2):

(1) It first obtains a reference to the EJBHome object that the container has generated.The reference is looked up in the system-naming directory via JNDI. The client willcall the required construction method on the home object.

2) The EJBHome object instructs the container to create a new instance or retrieve anexisting instance of the component, and returns it to the client. The actual Java Objectreturned is an instance of the container-generated EJBObject class that corresponds tothe bean’s component interface.

3) The client invokes the business method on the returned EJBObject, transparently,through the component interface. The EJBObject performs the required container ser-vices (specified in the XML deployment descriptor) and calls the corresponding busi-ness method on the bean’s implementation object, instance of the bean provider’s beanclass.

Figure 2.2: Client Invoking an EJB

9Java Naming and Directory Interface (JNDI), http://java.sun.com/products/jndi/

19

Figure 2.3: Example EJB Deployment Descriptor

2.4.4 Deployment Settings

As shown in figure 2.2 the container generated EJBObject intercepts and delegatesall calls to the bean implementation. The EJBObject supplies the bean implementationwith any required services as specified in the deployment descriptor. Figure 2.3 showsan extract from a deployment descriptor which specifies transactional attributes for abeans methods. Such settings can have a major impact on the system performance 10

and should be carefully considered.

The container is also responsible for the management of the bean’s life cycle events.

10Performance Tuning EJB Applications - Part I by Mihir Kulkarni, February 2005,http://dev2dev.bea.com/pub/a/2005/02/perf tune session beans.html

20

Figure 2.4: Stateless Session Bean Lifecycle

21

The management of an EJB’s lifecycle is a complex process and differs from bean typeto bean type. Factors which influence the bean lifecycle management include the loadin the system and the container configuration settings 11. Figure 2.4 illustrates thelifecycle of a stateless session bean.

When the container starts the application it creates a pool of bean instances. The poolsize can be determined by setting the value in the configuration settings. If a beaninstance is required by a client, an instance is assigned from the bean pool. If no in-stances are available the container can create more instances until the pool has reachedits maximum size (which is also configurable). If the bean pool has already reachedits maximum size and there are still no instances available the client will be put in aqueue until an instance becomes available. The pool configuration settings can havea major impact on the system performance and should be tuned according to the ex-pected load on the system.

The lifecycle of a stateful session bean and of an entity bean are similar but slightlymore complicated than that of the stateless session (since they can both be passivated).More details on these lifecycles can be found in the literature [148]. It is sufficient tosay for the purposes of this thesis that the configurations settings in relation to EJBlifecycles can have a major impact on the system performance and need to be carefullyconsidered.

2.5 Software Architecture

A large number of definitions exist for the term software architecture 12. One of mostcited definitions is by Bass et al. [21] and states that:

”The software architecture of a program or computing system is the structure or struc-tures of the system, which comprise software elements, the externally visible proper-ties of those elements, and the relationships among them.”

Bass et al. [21] also outline a number of implications of this definition: Firstly, a soft-ware architecture is essentially an abstraction since it embodies information about therelationship between elements, and externally visible properties that are exposed toother elements, but it omits internal element information or information that does notpertain to the elements’ interactions. Secondly, the definition makes it evident thatsystems can and do consist of more than one structure. Thirdly, it is implied by thedefinition that every software system has an architecture since every system can beshown to compose of elements and relationships between them. Fourthly, the exter-nal behaviour of each element is part of the architecture and finally the definition isindifferent as to whether the architecture for a system is a good or bad one.

11PreciseJava, http://www.precisejava.com/12Software Engineering Institute, Carnegie Mellon, list of software architecture definitions,

http://www.sei.cmu.edu/architecture/definitions.html

22

A software architecture is important for a number of reasons. Firstly it becomes avehicle for communication among the system’s stakeholders [21]. System stakehold-ers are those concerned with the system (e.g. users, customers, software developers,management etc.). A software architecture is a common abstraction of the systemand can serve as a lingua franca, i.e. an intermediate language that all stakeholderscan use to discuss various aspects of the system. The different stakeholders of thesystem are often concerned with different system characteristics. An architecture pro-vides a common language in which these different concerns can be expressed. Sincestakeholders can be interested in different system characteristics it is important forthe architecture to provide different views [51] that let them consider the architecturefrom different perspectives. For example, a functional view might contain an abstrac-tion of the different system functions and their relations whereas a code view maygive an abstraction of the code in terms of objects or classes (or higher level subsys-tems or modules) and their relationships. Different stakeholders make use of differentviews to analyse the architecture according to their needs. Typical views include afunctional view, a concurrency view, a code view, a physical view etc [52]. Kruchten[104] introduced the 4+1 view model to describe a software architecture using fiveconcurrent views. Views are essentially a mechanism that allow for the separation ofconcerns within the architecture allowing for the analysis of the architecture from dif-ferent perspectives. Architecture description languages (ADLs) [50] can be utilised todescribe a software architecture. There have been many attempts to design such lan-guages. However while some have been employed in real word projects none havebeen widely adopted [21]. The literature [115] provides a comparison of ADLS.

Another important reason for system architecture is that it creates a realisation of earlydesign decisions and allows for system architects to analyse the suitability of these de-cisions in relation to the system requirements (e.g. performance, security, flexibility)[21]. These early design decisions manifested in the system architecture can not onlyimpact the quality attributes of the system but can also place constraints on the actualsystem implementation i.e. some technologies may be more suitable for particulararchitectures. The initial architecture can even have an impact on the organisationalstructure of the team (or teams) building the application [21]. One of the earliest de-sign decisions is often to choose a suitable architectural style. An architectural styledefines a vocabulary of components (e.g. clients, servers, databases) and connectortypes (e.g. procedure calls, database protocols), and a set of constraints on how theycan be combined [152]. Architectural styles are found repeatedly in practice to addresssimilar sets of demands.

Finally, software architectures are also reusable assets that can be applied repeatedlyto other systems exhibiting similar requirements [21].

23

2.6 Software Patterns

The current use of the term pattern in software engineering is derived from work byChristopher Alexander [168] in the field of contemporary architecture. Alexander’snotion of a pattern was adopted by a number of software engineering researchers [26][71] and became popular in this field mainly after work published by Gamma et. al[72]. Gabriel 13 gives the following definition of a pattern: ”Each pattern is a three-partrule, which expresses a relation between a certain context, a certain system of forceswhich occurs repeatedly in that context, and a certain software configuration whichallows these forces to resolve themselves.” This definition is consistent with Alexan-der’s original definition [168] which states that a ”pattern is a three part rule whichexpresses a relation between a certain context a problem and a solution.” Alexanderexpands his definition to say that a problem relates to a certain system of forces whichoccurs repeatedly in a context and that the problem solution can be considered as acertain configuration which allows these forces to resolve themselves. While patternshave been documented for a number of different domains (such as patterns for con-temporary architecture [168] or organisational patterns [54]) we are mainly interestedin software patterns. Software patterns are usually documented according to a patterntemplate. Common templates for describing patterns include the Alexandrian form[168] and the GoF form [72]. A given template contains a number of elements thatdescribe the pattern e.g. name, problem, context, forces, solution, examples, resultingcontext, rationale, related patterns and known uses 14.

Buschmann et al. [42] document a number of properties or benefits of patterns. Whilethey focus on patterns for software architecture many of the properties hold for soft-ware patterns in general e.g.:

• A pattern addresses a recurring problem that arises in specific situations, andpresents a solution to it [42].

• Patterns document existing, well proven experience. That is, they documentsolutions learned through experience and avoid the need for less experienceddevelopers to ”reinvent the wheel” time and time again [72].

• Patterns provide a common vocabulary and understanding for design principles[72]. Similar, to the way a software architecture can serve as a vehicle for com-munication (see section 2.5 above), pattern names can become part of a designlanguage and can act as a lingua franca facilitating discussion of design issuesand their solutions [42].

• Patterns support the construction of software with defined properties [42]. Pat-terns assist developers in meeting both functional and non-functional require-

13The Hillside Group, Pattern Definitions, http://www.hillside.net/patterns/definition.html14Patterns and Software: Essential Concepts and Terminology, by Brad Appleton,

http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html

24

ments since they can provide a skeleton of functional behaviour while at thesame time they can explicitly address non-functional requirements e.g. reuse-ability, maintainability etc.

Software patterns can be documented at various levels of abstraction. For example,Buschmann et al. [42] discuss patterns at three different levels of abstraction, i.e.,architectural patterns, design patterns and coding patterns or idioms. Architecturallevel patterns are concerned with system structure. They describe predefined sets ofsubsystems, specify their responsibilities and include rules and guidelines for organ-ising the relationships between them. Design patterns on the other hand tend to be atthe level of objects and classes (or micro-architectures) and are used for refining sub-systems or components of a software system. In the literature [72] they are definedas ”descriptions of communicating objects and classes that are customized to solve ageneral design problem in a particular context.” Eden and Kazman have also distin-guished between architecture and design stating that architecture is concerned withnon-local issues whereas design is concerned with local issues [62]. Coding patternsor idioms are lower level patterns specific to a programming language [53].

Since their introduction in the area of object oriented software development [72], pat-terns have been documented for a range of systems and technologies 15. For examplespattern catalogs exits in areas such as enterprise systems [67] [93], embedded-systems[142], telecommunication systems [171] to name but a few. Many technology specificpatterns (or idioms) also exist (e.g. for Java [80], Ajax [110] and Microsoft technolo-gies 16). In fact pattern catalogs even exist with particular quality attributes in mind(e.g. security [150], performance [154]). Alur et al. [7] provide a catalog of patterns forthe JEE technology which document best practices for the design and implementationof JEE applications. Other literature in this area also exists [113] 17. The design of aJEE application plays a major role in the overall system performance. For example, ithas previously been shown how the system design can influence a JEE system’s scal-ability [43]. In fact it is well known and recent reports 18 19 also indicate that poorsystem design is a major reason as to why JEE systems often fail to meet performancerequirements. Another reason why poor software design is particularly undesirableis that unlike with lower level software bugs, for example, poor software design canbe particularly difficult to rectify late in development and as such can lead to majorproject delays. Software design best practices documented in the form of patterns canbe used to help avoid design issues when developing JEE applications.

15Handbook of Software Architecture, http://www.booch.com/architecture/index.jsp16Enterprise Solution Patterns Using Microsoft .NET, http://msdn2.microsoft.com/en-

us/library/ms998469.aspx17The Server Side Pattern Repository, http://www.theserverside.com/patterns/index.tss,18Ptak, Noel and Associates, ”The State of J2EE Application Management: Analysis of 2005 Benchmark

Survey”, http://www.ptaknoelassociates.com/members/J2EEBenchmarkSurvey2005.pdf,19Jasmine Noel, ”J2EE Lessons Learned ”, SoftwareMag.com, The Software IT Journal, January, 2006.

http://www.softwaremag.com/L.cfm?doc=2006-01/2006-01j2ee

25

2.7 Software Antipatterns

Antipatterns, first suggested by Koenig [101], have been defined by Brown et al. [36]as: ”a literary form that describes a commonly occurring solution to a problem thatgenerates decidedly negative consequences.” The authors [36] also state that whendocumented properly an ”antipattern describes a general form, the primary causeswhich led to the general form; symptoms describing how to recognize the generalform; the consequences of the general form; and a refactored solution describing howto change the antipattern into a healthier situation.” Software design antipatterns thusprovide the opportunity for developers to learn from past experiences. They docu-ment software design mistakes that tend to consistently reoccur. However, as wellas documenting the mistake, antipatterns also document the corresponding solution.Thus they allow developers to identify design issues in their system, and to rectifythese issues with the corresponding solution provided in the antipattern description.Antipatterns are complementary to software patterns and often show situations wherepatterns are misused. In fact, as technologies evolve often patterns can become stale[60], i.e., what was once a best practice can in some instances become a bad practice.Examples from the JEE technology include the caching with a Service Locator pat-tern [7], which was recommended for J2EE 1.2 but is not recommended for J2EE 1.320. Another example is the Composite Entity pattern [7] which has become obsoletesince EJB version 2.x [61]. Figure 2.5 [36] shows the relationship between patterns andantipatterns.

Software design antipatterns, like software design patterns, have been documentedat a number of different levels. For example Brown et al. [36] introduced a num-ber of technology independent object oriented development antipatterns (as well ashigher level architectural and management antipatterns). Technology specific antipat-terns have also been documented (e.g. Java [160], J2EE [161] [61]). Antipatterns forsystems built on service oriented architectures (SOA) have also been recently doc-umented 21. As with software design patterns, some antipattern catalogs focus onparticular software quality attributes only. For example Smith and Williams have pre-sented a number of performance related antipatterns [154], while Kis has presentedantipatterns focusing on security [99]. For the purpose of this thesis we focus mainlyon performance related antipatterns for enterprise systems. In particular we focus onperformance antipatterns related to design and deployment for JEE applications.

Similar to software design antipatterns, Fowler and Beck introduced the notion ofcode smells [68]. Code smells are lower level symptoms of problems at the code level.

20B. Woolf. IBM WebSphere Developer Technical Journal: Eliminate caching in service locatorimplementations in J2EE 1.3.,http://www-128.ibm.com/developerworks/websphere/techjournal/0410 woolf/0410 woolf.html,October 2004

21SOA antipatterns,Jenny Ang, Luba Cherbakov and Mamdouh Ibrahim, November 2005,http://www-128.ibm.com/developerworks/webservices/library/ws-antipatterns/

26

Figure 2.5: Patterns, Antipatterns and their Relationship

They may not necessarily be a problem but often their presence in the code indicatethat problems exist. Catalogs of code smells are available in the literature 22 23.

2.8 Performance Tools

When design issues lead to poor performance, developers require testing tools to iden-tify why the system is performing poorly. Developers use performance testing toolsto try to understand system behaviour and to discover how their system makes use ofthe system resources. Application level performance testing tools fall into two maincategories, i.e. workload generation tools and performance profilers.

2.8.1 Workload Generation

In order to evaluate the performance characteristics of an application under devel-opment a realistic workload is required to mimic how the system would be utilisedby clients in a production environment. To achieve this a synthetic workload can beautomatically generated using a workload generator. Workload generators fall intotwo main categories, traced based approaches and analytical approaches [17]. Tracedbased approaches make use of server log files to characterise the workload of an ap-plication, whereas analytical approaches are based on mathematical models which are

22A Taxonomy of Code Smells, http://www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm23Smells within Classes, http://wiki.java.net/bin/view/People/SmellsToRefactorings

27

usually based on statistical methods [124]. There are advantages and disadvantagesassociated with both approaches. For example, traced based approaches are consid-ered relatively easy to implement and are based on activity from a known system.However disadvantages relate to the fact that this approach treats the workload as ablack box and as such insight into the workload characteristics can be difficult to ob-tain. Also it can be difficult to modify the workload to simulate future or alternativeconditions. Furthermore during development realistic logs may not be available tobase the traced based workload generation upon. Analytical approaches on the otherhand can be used to create synthetic workloads and do not suffer from the drawbacksoutlined above. However they can be more difficult to construct as an understandingof the characteristics of the expected workload is required. The most commonly usedworkload generators are the analytically based commercial profilers e.g. Apache’sJMeter 24 or Mercury Loadrunner25. The literature [136] gives a representative subsetof the workload generators currently available in the open literature.

2.8.2 Profiling Tools

Next we explain what we mean by the term profiling and discuss the different waysand levels of granularity in which profiling information can be collected (see section2.8.2.1). We also give an overview of the different categories of profilers that are avail-able for the Java technology (see section 2.8.2.2).

Profiling [169], is the ability to monitor and trace events that occur during run time.This includes the ability to track the cost of these events, as well as the ability to at-tribute the cost of the events to specific parts of the program. A profiler, for example,may obtain information about what part of the program consumes the most CPU time,or about the parts of the program which allocate the most amount of memory. Perfor-mance profilers can be used in conjunction with load generators to monitor a runningsystem and obtain the required information for performance analysis. Profilers areoften described as either exact profilers or sampling based profilers [135] [30]. Ex-act profiling also referred to as full profiling [64] or full instrumentation 26, capturesall events of a given type that are produced during program execution (e.g. methodinvocations). Sampling based profilers on the other hand select a part of the entireevent population with the aim of determining the characteristics of the whole pro-gram. Sampling usually involves selecting a subset of events for profiling based oncertain criteria (e.g. hot paths) or time intervals [64]. Exact profiling has the advantageof being more precise than sampling but carries a higher performance overhead.

24Jakarta Apache JMeter http://jakarta.apache.org/jmeter/index.html.25Mercury Loadrunner. http://mercury.com26http://profiler.netbeans.org/docs/help/5.5/custom instrumetation

28

2.8.2.1 Recording Information

Regardless of the profiling approach however, information must be recorded by theprofiling tool. Performance metrics can be recorded at different levels of granular-ity. At the lowest level hardware counters can be utilised to obtain performance met-rics from the underlying hardware on which the program executes [8] [10] [156] [88].Hardware counters can be utilised to record events, such as instructions executed, cy-cles executed, pipeline stalls, cache misses, etc. One of the main advantages of usinghardware counters is that an application can be profiled without the need to modify orinstrument it. Also the overhead associated with using hardware counters for profil-ing is generally quite low [10]. A disadvantage of hardware counters is that they relyon platform specific features and thus they are generally not portable across differenthardware. Another issue with this type of profiling is that the information may betoo low level for higher level program analysis. Information such as virtual memorymanagement requests, and signals caused by segmentation violations can be obtainedat the operating system (OS) level [88]. OS level information can be recorded by sys-tem level tools 27 or libraries 28. In situations where hardware counters or OS levelinformation is unavailable, or the information they produce is undesirable, informa-tion can be obtained at a higher level. For today’s enterprise Java applications suchinformation can be recorded at a number of different levels i.e. at the JVM level, themiddleware level or the application level.

JVM level information is generally recorded by either instrumenting the JVM or byusing an agent-based approach that requests notification of events from the virtualmachine. The former very often requires access to the JVM source code such that itcan be modified to record the information required [12] [33]. A disadvantage of thisapproach is that it requires an understanding of the complex JVM internals. Also thisapproach generally ties the user to a particular JVM and is thus not portable. One ofthe main advantages of instrumenting the JVM is that access to JVM level informationis not restricted as with the agent-based approaches.

Agent based approaches have been made popular through standard interfaces thatallow for a profiler agent to request performance related information from a runningJVM. The Java Virtual Machine Profiler Interface (JVMPI) [169] is an example of suchan interface (see figure 2.6). The JVMPI is a two-way function call interface betweenthe JVM and an in-process profiler agent. The profiler agent is responsible for commu-nication between the JVM and the profiler front end. The profiler agent can registerwith the JVM to be notified when particular events occur and upon notification cancall back into the JVM to obtain additional information. For example a notificationmay be received when a method is entered (or exited) and a call back may be made to

27Performance Monitoring Tools for Linux, David Gavin Jan, 1998,http://www.linuxjournal.com/article/2396

28Windows Management Instrumentation, http://www.microsoft.com/whdc/system/pnppwr/wmi/default.mspx

29

Figure 2.6: JVMPI Architecture

obtain the current stack trace at this point. The main advantage of standard interfacesis that they are implemented by different JVM vendors. While the JVMPI was an ex-perimental interface for Java 1.2, it was implemented by most JVM vendors and effec-tively became standard. Thus profilers built using JVMPI are portable across differentJVM implementations. A disadvantage of standard interfaces is that they are fixedinterfaces and, as such, can only enable predefined types of profiling [135] or eventnotifications. Another major issue with JVMPI in particular was that when using aJVMPI agent the JVM could not run at full speed and was required to run in a debug-ging mode. As such this profiling approach was not generally suitable for productionsystems. Another major draw back of the JVMPI approach was that notifications couldnot be tailored to profile selectively. If, for example, the profiler agent requested to benotified on method entry events, all method entry events would be reported to theagent. This lead to high overhead performance tools. The Java Virtual Machine ToolsInterface (JVMTI) 29 will replace JVMPI in Java 1.6 (the JVMPI is currently availablein Java 1.5). While at an architectural level the JVMTI looks similar to the JVMPI (i.e.it also consists of call back functions and a profiler agent) it is quite different and im-proves upon many of the limitations of JVMPI. Firstly, it allows for the JVM to runat full speed and does not require it to run in debug mode. It also promotes the useof bytecode instrumentation for many of the event based capabilities of the JVMPI.Using bytecode instrumentation one can be more selective when profiling the appli-cation and can instrument only the parts of the application that require analysis. Thisavoids the ”all or nothing” approach of the JVMPI and thus reduces the profiler over-head. In fact the JVMTI allows for dynamic bytecode instrumentation 30, which meansthat the application can be instrumented as it runs. An issue that remains, however,with both JVMPI and JVMTI is that they are native interfaces and while the profil-ing agents (which must be written in native code) are portable across different JVMs

29The Java Virtual Machine Tools Interface, http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html30Ian Formanek and Gregg Sporar, Dynamic Bytecode Instrumentation A new way to profile Java

applications, December 15, 2005, http://www.ddj.com/dept/java/184406433

30

they are not portable across different platforms. The java.lang.instrument interface 31

is another standard interface (as of Java 1.5) which allows for the interception of theJVM classloading process through a non-native agent. Since the agent is non-nativeit is portable across different platforms. Java.lang.instrument allows for the agent tomonitor the classloading process and to instrument the classes such that they can callback into the agent libraries.

Recording performance information for Java applications at the middleware level canalso be achieved through standard mechanisms. Java Management Extensions (JMX)technology 32 is a standard technology that allows for the management of Java re-sources through so called MBeans. MBeans, also known as managed beans, are Javaobjects that are used to represent and manage JMX resources. A JMX resource can beany application, device or Java object. In order to manage an MBean it must be reg-istered with a JMX agent. JMX agents directly control registered MBeans and makethem available to remote management applications. While the JMX technology canpotentially be used to manage a wide range of different resources it has been heavilyused, in particular, to manage the resources of JEE application servers. In fact, ac-cording to the JEE Management specification (Java Service Request 77 33) applicationservers are required to expose this data through the JMX technology. Profilers built us-ing JMX can collect data on the state of the different system resources (e.g. object poolsizes, thread queues, database connectivity information) and because JMX is standardthey are portable across the different application server implementations.

Non standard hooks or mechanisms have also been used to collect information at themiddleware level. Often middleware vendors provide these mechanisms to enhancethe capabilities of their products. For example, IBM provides non standard featuresfor the Websphere application server in the form of the Performance Monitoring In-frastructure (PMI). PMI is available for the Websphere application server and allowsfor the collection of performance information on the server resources. The informa-tion can be exposed to performance profiling tools through a number of different in-terfaces 34. The main issue with non-standard features, that allow for the collectionof performance information, is that they are not portable across different vendors’ im-plementations of the middleware, and thus can result in vendor lock in.

Where the information required is not available through standard or non-standardfeatures the middleware itself can be the subject of instrumentation. This can beachieved by manually modifying the source code if it is available [45] [105]. However

31J2SE 5.0 in a Nutshell, Calvin Austin, May 2004,http://java.sun.com/developer/technicalArticles/releases/j2se15,32Java Management Extensions Technology,http://java.sun.com/javase/technologies/core/mntr-

mgmt/javamanagement/33Java Service Request 77, J2EE management, http://www.jcp.org/en/jsr/detail?id=7734Srini Rangaswamy, Ruth Willenborg and Wenjian Qiao, IBM WebSphere De-

veloper Technical Journal: Writing a Performance Monitoring Tool Using Web-Sphere Application Server’s Performance Monitoring Infrastructure API, 13 Feb 2002,http://www.ibm.com/developerworks/websphere/techjournal/0202 rangaswamy/rangaswamy.html

31

as with JVM modification, instrumenting the middleware requires an understandingof the middleware behaviour which can be extremely complex. Also modification ofthe application server source code is not a portable approach.

Instrumenting at the application level to monitor performance can be achieved in anumber of different ways. The most basic way to instrument an application is to mod-ify the source code of the application to log the required information. IBM encouragesinstrumentation of the application at the time of development such that traces can berecorded through the Application Response Measurement API 35 36. This approachcan be cumbersome and can require a major effort during development. Another ma-jor drawback of this approach is that it can not be applied after development if sourcecode is not available.

More automated approaches have also been presented that remove the burden of hav-ing to manually modify the source. For example the Gprof profiling tool [79] allowsfor automatic instrumentation of an application at compile time for applications writ-ten in C or C++. It relies upon specific support from the gcc compiler to insert instru-mentation at the start of each method. Similarly binary rewriters and program trans-formation systems can be used to automate the instrumentation process [22] [134][155] [170]. It has been argued [135] that such approaches are cumbersome to usesince they operate at a low level. Binary rewriters for example require further codeto be written while program transformation systems require the creation of complexrewrite rules.

Application level instrumentation for Java systems can be achieved using bytecodeinstrumentation (BCI). BCI allows for instrumentation of the compiled bytecodes andthus access to the application source code is not required. JVMPI and JVMTI allowfor the instrumentation of bytecode before load time and at load time. Similarly thejava.lang.instrument interface allows for BCI at load time. The JVMTI even allowsfor BCI to occur at run-time (i.e. dynamic BCI). Although these interfaces provide amechanism for the modification of bytecodes, the actual instrumentation on the classfile is generally performed through a third party BCI library (e.g. [57]). Developerscould alternatively make use of Aspect Orient Programming (AOP) libraries 37 38 toinstrument their applications. AOP libraries are at a higher level than the BCI librariesand allow for modification of Java classes without a detailed knowledge of the byte-code class structure. It has recently been shown how the AspectJ AOP library can beeffectively applied to instrument Java applications for profiling [135]. AspectJ allowsfor both compile time and load time instrumentation and in fact makes use of BCIlibraries to instrument the application. Both BCI and AOP approaches are used by

35IBM InfoCenter, ARM 4.0 API concepts,http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp?topic=/eicaw/section1.htm

36The Open Group, Application Response Management Standard,http://www.opengroup.org/tech/management/arm/

37AspectJ, http://www.eclipse.org/aspectj/38Aspectwerkz, http://aspectwerkz.codehaus.org/

32

a number of today’s profiling tools to automatically instrument and profile runningapplications (e.g. [78] 39).

2.8.2.2 Java Profilers

Independent of the instrumentation techniques (e.g. event based agents, BCI, AOP),the most commonly used profilers for the Java technology can generally be split into 3main categories: basic profilers, graphically enhanced J2SE profilers and JEE profilers.Next we discuss each of the different categories. We begin by discussing the mostbasic profilers and finish with the most sophisticated.

The are a large number of freely available (and in many cases open source) basic pro-filers. The profilers in this category collect metrics and present them in a very basicformat. An example is the hprof profiler which comes as standard with the sun JVMs40. While the hprof profiler output looks quite basic, it makes use of the JVMPI/JVMTIinterfaces (depending on the tool version) and can collect a vast amount of informa-tion made available through these interfaces. Basic profilers, such as hprof, are mostsuited for analysing small scale single user applications since the information outputis presented in a basic format. In situations where a vast amount of information is pro-duced, as with multi-user applications, basic profilers are generally unsuitable since itcan be difficult to manage and understand the information produced.

A more sophisticated category of profilers are the (generally commercial) more graph-ically enhanced J2SE profilers. The profilers generally collect the same information asthe basic profilers but present the data in a more user friendly manner. Examplesinclude the Jprobe 41 and Optimizeit 42 profilers. These profilers tend to capture infor-mation at the Java class level (very often making use of JVMPI/JVMTI) and provideviews that help with performance analysis. For example these profilers generally pro-vide heap analysers that help with memory leak detection and thread analysers fordeadlock detection. Most also provide call graphs which show object dependencies.There are a large number of profilers that exist in this category 43. While these toolsare effective for single user applications, as with the most basic tools they are limitedwhen profiling larger multi-user enterprise applications. One of the main problems isthat they monitor the system at the class level. However, for enterprise systems thenumber of classes loaded by the JVM can be in the order of thousands. The classescan generally be broken down into the following categories: application level classes(written by the application developers), middleware classes (corresponding to con-tainer services) and lower level Java library classes. A major issue is that, while de-

39JRat the Java Runtime Analysis Toolkit, http://jrat.sourceforge.net/index.html40Kelly O’Hair, HPROF: A Heap/CPU Profiling Tool in J2SE 5.0, November 18, 2004,

http://java.sun.com/developer/technicalArticles/Programming/HPROF.html41Jprobe, http://www.quest.com/jprobe/42Optimizeit, http://info.borland.com/techpubs/optimizeit/43Java Performance Tuning Website, http://www.javaperformancetuning.com/resources

33

velopers are generally interested in obtaining information on their own code, it canbe very difficult for developers to distinguish their code from lower level middlewareand library calls. Another issue related to profiling enterprise applications is that of-ten these applications are distributed and made up of a number of different servers.The (J2SE) profilers in this category do not profile the application across JVMs andthus developers need to spend time correlating data from a number of profiler logs inorder to understand the entire system.

JEE profilers such as Performasure 44 or JProfiler 45 overcome this issue of muddlingmiddleware and application level calls by presenting data at a higher level of abstrac-tion. Many of these tools also profile across JVM’s and give a transaction centric viewof the applications. The JEE profilers are also generally known as Application Perfor-mance Management tools 46, since they are usually suitable for managing systems ina production environment.

The biggest problem with the current profiling tools, from the most basic to themost sophisticated is that they do not perform any meaningful analysis on the datacollected. The most sophisticated tools merely perform statistical analysis on per-formance metrics and give average resources consumed by the different compo-nents/classes. When profiling large enterprise applications the amount of informa-tion recorded can be truly overwhelming. It can be difficult and time consuming fordevelopers that have to analyse the data produced by these tools in search of partic-ular problems. Furthermore even when developers manage to identify issues oftenthey are unsure as to how to go about solving them. The work in this thesis comple-ments such profiling tools. By performing different analyses the data they produceperformance design and deployment antipatterns can be automatically identified.

Another problem with the current tools is that they tend to focus on identifying lowlevel performance bugs that exist in the system. For example the more sophisticatedtools provide views that help with memory leak detection and deadlock detection.The tools, however, do not focus on analysing the system from a design perspective.Thus design flaws can easily go unnoticed. While identifying and fixing low levelbugs in the system will very often improve the system performance, it is commonthat this is not enough to meet performance requirements. Often, an overhaul of thesystem design is required to improve performance significantly [43].

44Performasure, http://www.quest.com/performasure/45JProfiler, http://www.ej-technologies.com/products/jprofiler/overview.html46Jack Shirazi, Application Performance Management, July 2004,

http://www.javaperformancetuning.com/articles/apm1b.shtml

34

2.9 Reverse Engineering

The term reverse engineering covers a wide range of activities [163] including designrecovery, architecture recovery, feature analysis, impact analysis, clone detection etc.For the purpose of this research we are mainly interested in design and architecturerecovery.

To analyse the system design, developers can make use of design documentation.However unfortunately it is common that design documents are never created, orare created early in the development process and become out of date as the systemevolves. Furthermore, with agile development processes such as extreme program-ming, documentation is not generally maintained throughout the development pro-cess [24]. In such circumstances reverse engineering can be used to extract the designfrom the system such that a representation of the design is available.

Reverse engineering has been defined by Chikofsky and Cross [48] as ”the process ofanalyzing a subject system to:

• Identify the system’s components and their interrelationships and

• Create representations of the system in another form or at a higher level of ab-straction.”

Earlier approaches to reverse engineering mainly relied on static analysis (e.g. [122][143] [81] [158]). Static analysis techniques generally extract information from thesource code (such as component dependencies) without the need to execute the ap-plication. In fact a static call graph extractor has recently been developed for the EJBtechnology [165]. There are a number of issues related to applying source code/staticanalysis to enterprise applications. Firstly, the entire source code of enterprise appli-cations is often not readily available. This can occur as a result of outsourcing or fromusing third party components. This problem is however overcome in the approachpresented in the literature [165], by extracting a static model by analysing the byte-code of the application. Although if components have been developed by third partiessecurity or license issues may not allow for this approach to be applied. Secondly, andmore importantly, static models show all possible dependencies that could potentiallybe present during the system execution. With enterprise applications the number ofpossible dependencies can be very high and many of the potential dependencies maynever actually occur during execution. Thirdly, as claimed by the authors in the liter-ature [149], there is an inherent mismatch between static, code-based structures (suchas classes and packages) and the runtime structures that are the essence of most sys-tem descriptions. In fact, the actual runtime structures may not even be known untilthe program executes (e.g., Dynamic Linked Libraries not under direct control of theimplementers may be dynamically loaded). The authors go on to claim that determin-ing the actual runtime architectural configuration of a system using static analysis is,

35

in general, undecidable.

To overcome this problem dynamic analysis can be applied to extract run-time depen-dencies from the application. The advantages of dynamic analysis for reverse engi-neering purposes has been well documented [146]. For example, dynamic approachescan be more precise, insofar as only dependencies that are actually recorded duringexecution exist in a dynamic model. Dynamic analysis also limits the scope of investi-gation, similar to program slicing [23]. Dynamic analysis is also particularly useful inrelation to analysing the system from a performance perspective, since, performancemetrics can be obtained from the running system. This performance related informa-tion can not be obtained easily when analysing the system source code.

Dynamic analysis techniques [147] [177] have generally made use of program traceswhich can be analysed to extract dependencies. A major obstacle for performing thisanalysis on large distributed enterprise systems is that such traces can be difficult toobtain. Enterprise applications generally comprise of a large number of componentsrunning on physically distributed servers. Thus obtaining an execution trace for thesesystems is not a trivial task.

Recently Chen et al. [45] introduced an approach for the collection of run-time pathsin JEE systems in the form of the Pinpoint tool. A run-time path [44] contains theordered sequence of methods called required to service a user request, and thus con-tain information required for dynamic reverse engineering. A major drawback of thePinpoint approach is that it is intrusive and can only be applied for a particular appli-cation server implementation. In section 4.2.5.2 we discuss the pinpoint tool in moredetail and give related work in this area. We also show how this information canbe obtained in non-intrusive portable manner such that it can be applied to any JEEapplication running on different middleware implementations.

2.10 Design Pattern and Antipattern Detection

Research in the area of software pattern detection has spanned the past decade. Com-parison of pattern detection methods is often performed with respect to criteria suchas flexibility, efficiency and accuracy [139]. However efficiency and flexibility are of-ten unimportant if accuracy can not be guaranteed. Accuracy in relation to softwarepattern detection is usually defined in terms of precision and recall which are bothdefined in terms of true/false positives/negatives [139]. True positives tp are definedas pattern instances detected that exist in the system under test, while false positivesfp are defined as pattern instances detected that do not actually exist in the system.On the other hand, true negatives tn are defined as pattern instances not detected thatdo not exist in the system, while false negatives fn are defined as pattern instances notdetected that actually do exist in the system. Petterson et al. [139] define precision as

36

P = tptp+ fp

and recall as R = tptp+ fn

.

Over the past decade a wide number of software pattern detection approaches havebeen presented and generally use either static analysis or a combination of static anddynamic analysis. The static approaches (e.g. [5] [103] [98] [153] [151] [13]) aim toextract structural information such as class inheritance, interface hierarchies, associ-ations etc. Most approaches analyse the source code of an application [103] [11] [16][65] [98] [5] [13] [172], although some have used graphical representations as input[56] [27]. The application and design patterns are generally expressed using structuralrepresentations, and matching is performed using either meta-programming, graphre-write systems, logic programming, relational algebra, or parsing techniques [138].According to the literature [138], the matching techniques mainly impact the efficiencyand do not greatly effect accuracy. An alternative technique is presented by Buchli andNierstrasz [39] who use (static) formal concept analysis (FCA) to identify the presenceof groups of classes which instantiate a common, repeated pattern. Patterns identi-fied are analysed to find a match with known patterns. Tonella and Antoniol [164]have presented a similar approach based on FCA. The main advantage of approachesbased on FCA is that they do not require a knowledge base of design pattern rep-resentations. The combined (static and dynamic) approaches generally make use ofdynamic behavioural information to improve the accuracy of static analysis by reduc-ing the levels of false positives [89] [90] [91] [174] [175] [137]. It has been argued thatstatic analysis approaches are limited to the recognition of static parts of design pat-terns, and as a result, the dynamic behavior of patterns is disregarded, which leads tohigh levels of false positives during recognition [174]. Similarly software metrics havealso been utilised to reduce the numbers of false positives [11]. The literature [139]reports on the accuracy of the state of the art in design pattern detection techniques.Similarly, Vokac [172] gives a comparison of the accuracy of a number of detectionapproaches.

Efficiency is another important attribute of pattern detection approaches. With patterndetection the size of the exploration space for large software systems can be a majorissue. A combinatorial explosion can occur due to the great number of system classesand the multiple roles that classes can play in a specific design pattern. As a resultefficiency issues have often been reported in applying detection techniques in practice(e.g. [13] [70] [138]). A number of papers have been presented with particular focus onreducing the detection search space. For example, Niere et al. [123] increase efficiencyby replacing a number of constraints with a smaller number of weaker constraints.This approach allows more variants of a particular pattern to be identified during thedetection process and increases the number of false positives. They address this issueby attaching a ranking to each pattern candidate retrieved. Tsantalis et al. [167] takean alternative approach and partition the system into clusters of hierarchies. Theythen apply a similarity scoring algorithm to the smaller subsystems, rather than to theentire system. Petterson et. al [138] present a method to increase efficiency based on

37

planar graphs. Planar graphs are described as graphs drawable in two-dimensionswithout any crossing edges. Both the structure of a program and a design patterncan be represented as a graph. The detection of a pattern involves deciding if thepattern graph is isomorphic to a subgraph of the program graph. This is known as thesubgraph-isomorphism problem and is NP-Complete for general graphs. Petterson etal. use a filtering strategy to remove unneeded edges from program graphs with theaim of producing planar graphs. Detecting patterns in planar graphs can be solved inlinear time.

Less work has been undertaken in the area of design antipattern detection. How-ever many of the techniques for software pattern detection outlined above can alsobe applied to detect design flaws. In most of the approaches discussed above thepatterns and application are described in terms of structural (and in some instancesbehavioural) representations which are analysed to identify instances of the pattern.Design flaws can also be described in this way and thus the same approaches can beused to identify them. For example, work by Gueheneuc et al. [83] shows how defectscan be identified by creating abstract models of known design patterns. The models(representing entities and relationships) are used to identify similar sets of entities inthe source of the application, which represent distorted design patterns. Similarly,the work presented by Ciupke [49], extracts a model from the source code to iden-tify design issues. The model defines the different entities and relationships that mayoccur in the design of the program. The information contained in the model is con-verted to Prolog facts. Queries formulated as Prolog rules examine this informationand identify design issues. An approach by Moha et al. has suggested the use offormal concept analysis (which has also been applied for pattern detection) and soft-ware metrics, to identify and correct a well known deign antipattern (The Blob) [117].Metrics have also been applied by Marinsecu [114] for the purpose of design flaw de-tection. Munro [121] has applied metrics for the detection of bad smells. Alternatively,the ARBRIUM antipattern detection tool [63] has been used to detect antipatterns inuse case models by making use of the Object Constraint Language (OCL). Antipat-terns can be expressed using OCL and the integrity of the models are assessed.

The techniques outlined above, for both pattern and antipattern detection, mainly relyheavily on static source code analysis. Even where dynamic analysis is applied it isgenerally used only to add behavioural information to a static model of the systemsuch that detection can be more precise. There is a major issue applying such tech-niques to component based systems, since source code is often not available, or at leastmay not be available for analysis due to security or licensing constraints. Also, whenanalysing components at the component level we are not interested in their internalstructural make up (e.g. inheritance relationships) since components are consideredas black boxes and we may not be aware of their internal workings. We are more in-terested in how they communicate with each other and the underlying middleware.However since components can be subject to dynamic binding this information often

38

can be difficult to obtain statically. True inter-component dependencies can in manycases only be determined at run-time. Furthermore, since we are interested in per-formance antipatterns we need to be able to reason about how the system resourcesare being used. Such performance information can generally not be obtained by staticanalysis alone.

The goal of this research is to show how performance design flaws can be identifiedat the component level in enterprise applications using run-time data and componentmeta-data. The flaws we detect are presented to the application developer with sug-gested refactorings and contextual information. Our aim is to bring a greater level ofunderstanding to developers in relation to such problems than is currently availablefrom today’s enterprise performance tools.

2.11 Knowledge Discovery in Databases and Data Mining

When monitoring multi-user enterprise applications, the amount of data producedcan be truly overwhelming 47. Currently a major effort is often required by devel-opers who must correlate vast amounts of data in order to identify problems in theirsystems. Such tasks often require repetitive labourious work and should ideally beautomated so developers and system testers can put their minds to more useful tasks.

The field of Knowledge Discovery in Databases [66] combines techniques from pat-tern recognition, statistics, machine learning, databases, and visualization to auto-matically extract concepts, concept interrelations, and patterns of interest from largedatabases. The basic task is to extract knowledge, or information, from lower leveldata (databases). The basic tools used to extract patterns from data are called DataMining methods [87], while the process of surrounding the usage of these tools (in-cluding pre-processing, selection, and transformation of the data) and the interpreta-tion of patterns into ”knowledge” is the KDD process. This extracted knowledge issubsequently applied to support human decision-making, e.g., prediction and classi-fication tasks, summarize the contents of databases, or explain observed phenomena.The application of KDD systems alleviates the problem of manually analyzing thelarge amounts of collected data which decision-makers face presently. KDD systemshave been implemented and are currently in use in finance, fraud detection, marketdata analysis, astronomy, diagnosis, manufacturing, and biology. The KDD process(figure 2.7) is made up of the following sub processes: selection, pre-processing, trans-formation, data mining and interpretation.

The proposed antipattern detection approach (see chapter 3) in this thesis can be con-sidered as an instance of the KDD process. The monitoring stage collects data andforwards it on for analysis. The analysis stage implements pre-processing, transfor-

47Michael Baum, ”Migrating J2EE Applications from Development to Production Is it becoming morecomplex?”, Java Developers Journal, November, 2005, http://jdj.sys-con.com/read/152269.htm

39

Figure 2.7: The KDD Process

mation, and performs the data mining techniques. The detection stage interprets thediscovered patterns and presents the discovered knowledge. Data mining techniquesfrom the areas of frequent pattern mining and clustering are applied in our work.

2.11.1 Frequent Pattern Mining and Clustering

The aim of frequent pattern mining methods is generally to find all frequent itemsets,i.e. those that appear in at least a given percentage of all transactions. Agrawal et al.first introduced a solution to this problem in the form of the Apriori algorithm [4]. Thefrequent item set mining problem is usually formulated as follows:

Let I = i1, i2, ..., im be a set of items. Let D be a set of transactions T where eachtransaction is a set of items such that T ⊆ I. A unique identifier TID is given to eachtransaction. A transaction T is said to contain X, a set of items in I, if X ⊆ T. We canalso say that the itemset X appears in the transaction T. The itemset X is said to befrequent if it appears in at least a given percentage of all transactions.

The Apriori algorithm is based on the principle: an itemset can only be frequent ifall its subsets are also frequent. Since its introduction by Agrawal et al. [4] FrequentItemset Mining (FIM) has received much attention. Within the past decade, hundredsof research papers have been published presenting new algorithms or improvementson existing algorithms to solve these mining problems more efficiently [75]. Mostof these papers present algorithms which are proposed as being faster than previousexisting algorithms. Efficient techniques in the field of data mining are certainly im-portant, considering the recent advances in computing, communication and digitalstorage technologies which today make it possible to easily store massive volumes ofdata. However there is a real need to show how such algorithms can be applied todifferent domains to extract interesting and useful information, since it is irrelevanthow efficient an algorithm is, if it can not be put to a meaningful task.

In this research, we show how a generalisation of FIM, Frequent Sequence Mining(FSM) [3] can be applied to data collected from monitoring enterprise applications toidentify patterns of interest that can be used to identify design flaws in the system.The FIM and FSM problems are discussed in more detail in section 5.4.1.

40

A second data mining technique, clustering [87], is also applied in this research. Clus-tering is an unsupervised learning problem, whereby data is divided into groups ofsimilar objects. A cluster consists of a group of objects that are similar between them-selves and dissimilar to objects of other groups. In section 5.5 we show how clusteringcan be used to effectively summarise data produced during monitoring enterprise ap-plications. A comprehensive overview of current clustering techniques in the field ofdata mining can be found in the literature [28].

41

CHAPTER

THREE

Overview of Approach

In this chapter we give an overview of the work presented in this thesis, that is, anapproach for the automatic detection of performance design and deployment antipat-terns in component based systems. We briefly discuss the three main steps involved inthis process, i.e. monitoring, analysis and detection. We outline the research methodused, as well as a number of criteria that we have used to evaluate our approach. Wealso discuss how our approach meets these criteria. Following this, we further dis-cuss antipatterns detailing the advantages of automatically identifying them. In thischapter we also present a hierarchy of antipatterns, from high level abstract antipat-terns, to lower level technology specific ones. We focus in particular on performanceantipatterns and show that a high percentage of antipatterns in the JEE technology areperformance related. Finally we distinguish between design antipatterns, deploymentantipatterns and programming errors.

42

Main Points

• Automatic performance antipattern detection for component based systems re-quires three main steps: monitoring, analysis and detection.

• Our research approach makes use of empirical methods such as experiments andcase studies.

• A performance antipattern detection approach for component based enterprisesystems should meet a number of criteria. Such an approach should be portable,non-intrusive, end-to-end, applicable to distributed systems and produce a lowoverhead on the system under test. The analysis should be efficient. Detectionshould be user configurable and produce low numbers of false positives. Theabove criteria should scale when the approach is applied to real large scale en-terprise applications.

• Documented antipatterns conform to an antipattern hierarchy, from technologyindependent, antipatterns to technology and quality attribute specific antipat-terns.

• A high percentage of JEE antipatterns are performance related.

• JEE performance antipatterns can be categorised as being either performanceprogramming errors, performance design antipatterns or performance deploy-ment antipatterns.

43

Figure 3.1: PAD Tool Overview

3.1 Approach for the Automatic Detection of PerformanceAntipatterns

The approach presented in this thesis, for the automatic detection of performancedesign and deployment antipatterns, can be divided into 3 main modules (see fig-ure 3.1): a monitoring module, an analysis module and a detection module. Ourwork is concerned with identifying known antipatterns i.e. antipatterns that havebeen already documented (e.g. [161] [61]). Our work is not concerned with pat-tern/antipattern mining which is the task of identifying new, previously unknownpatterns/antipatterns in a systems 1.

The monitoring module is responsible for collecting information on the running sys-tem. Information is collected on the application components, and on the underlyingmiddleware such that a complete picture of the running system can be obtained andrecreated. Information collected is forwarded to the analysis module where analysisis performed off-line.

The analysis module is responsible for extracting a run-time design model of the sys-tem from the monitoring data. A system’s run-time design can be defined as an in-stantiation (or instantiations) of a system’s design decisions which have been made

1Patterns and Software: Essential Concepts and Terminology, by Brad Appleton,http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html

44

during the development process. A run-time design model captures structural andbehavioural information from an executing system. It contains information on thecomponents that are executed, as well as the dependencies and patterns of communi-cation between components that occur at run-time. Figure 3.2 gives a run-time designmeta model. The meta model describes the different relationships that are capturedin a run-time design model. For a typical component based system (e.g. a JEE sys-tem) the run-time design model will contain information such as: dynamic compo-nent dependencies, run-time paths (see section 4.2.2 for a definition), communicationpatterns and interactions between components and container services. A run-timedesign model can also be enhanced with related performance metrics (e.g. methodmemory consumption). The analysis module is responsible for populating the run-time design model by extracting the information from the monitoring module output.For example, during analysis component dependencies are identified and containerservices reconstructed. Also during analysis data mining algorithms are applied toidentify relationships and patterns of interest in the monitoring data (e.g. frequent se-quences). The analysis module is also responsible for summarising the data collectedduring monitoring. The summarised data can be captured in the model to give aconcise overview of the data collected during monitoring. For example, performancecharacteristics of components can be summarised using statistical analysis and addedto the run-time design model. The run-time design meta model is described in moredetail in section 5.6.

The output from the analysis module is a run-time design model of the system. Theextracted design model is analysed by the detection module. The detection module isresponsible for identifying any pre-defined antipatterns that exist in the model. An-tipatterns are pre-defined and stored in an antipattern library. The solution can beused to rectify any problems found. This approach has been realised for the JEEtechnology in the form of the Performance Antipattern Detection (PAD) Tool. Anyantipatterns detected by the tool are presented with contextual information and thecorresponding antipattern solution.

3.1.1 Research Methodology and Validation Criteria

Four main research methods have been outlined for software engineering [74], i.e., thescientific method, the engineering method, the empirical method and the analyticalmethod. The methods first presented by Basili [20] in the software engineering contexthave been described by Wohlin et al. [178] as follows:

• ”The scientific method: The world is observed and a model is built based on theobservation, for example, a simulation model.

• The engineering method: The current solutions are studied and changes are pro-posed, and then evaluated.

45

Figure 3.2: Run-time Design Meta Model

• The empirical method: A model is proposed and evaluated through empiricalstudies, for example, case studies or experiments.

• The analytical method: A formal theory is proposed and then compared withthe empirical observations.”

Traditionally software engineering research has made use of the analytical methods[74], the engineering method has been mainly used in industry and the scientificmethod has been used mostly in applied areas such as simulating telecommunica-tion networks [178]. More recently, however there has been a call for more empiricalresearch in the field of software engineering [162] [74] [144] [100].

A number of authors have suggested that rigorous experimentation is required toevaluate new technologies and their effects on organisations, processes and products[100]. Wohlin et al. [178] suggest three main empirical strategies that can be carriedout, i.e. experiments, case studies, and surveys. Experiments are often referred to as”research in the small” [100] since they are usually run in a laboratory environmentand are generally small in scale. The aim of an experiment is to manipulate one ormore variables while controlling the remaining variables at fixed levels. The effect ofmanipulation is measured. Experiments tend to be small since they require a highlevel of control over the test environment. Case studies on the other hand are referredto as ”research in the typical” [100] since they tend to look at what is happening in a

46

typical project. The level of control is generally lower in a case study than in an experi-ment. The objective is often to monitor a particular project or activity. Case studies aresaid to be observational studies whereas experiments are controlled studies. Finallysurveys are often referred to as ”research in the large” [100] since they try to capturewhat is happening over a range of projects. They are usually performed by obtaininginformation from a representative sample of the entire population through interviewsor questionnaires. The results are generalised to the population from which the sam-ple was taken.

In this thesis we make use of both case studies and experiments to evaluate our re-search. Experiments have been used to evaluate the overhead associated with ourmonitoring approach (see section 7.2.3) as well as to assess the efficiency of our anal-ysis module (see section 7.3.1). According to Wohlin et al. [178] the main strength ofan experiment is ”that it can investigate in which situations the claims are true andthey can provide a context in which certain standards, methods and tools are recom-mended for use”.

Case studies on the other hand have been used to show how our monitoring andanalysis outputs can be used to manually identify antipatterns (see sections 7.2.1 and7.3.2 respectively). Case studies have also been used to show how antipatterns canbe automatically identified in component based systems (see section 7.4). The mainadvantage of using case studies is that they incorporate qualities that an experimentcan not such as scale, complexity, unpredictability and dynamism [178].

Below we have outlined a number of evaluation criteria which we evaluate our workagainst. The evaluation criteria are the attributes which we believe are required, inorder for our approach to be considered useful. The evaluation criteria are as follows:

• Portable: The monitoring module should be portable across different vendors’implementations of the middleware. This is particularly important for JEE,whereby application servers are built by a large number of different vendors2. In fact it is not unusual for an enterprise application to be made up of web,application or database servers from a number of different software vendors. Tobe able to apply our approach to truly heterogenous systems it is required thatthe monitoring module is completely portable across the different implementa-tions.

• Non-intrusive: The monitoring module should also be non-intrusive, that is, itshould not require any modifications to the application or middleware sourcecode. This is important for component based systems, since often it may notbe possible to make such modifications. For example most commercial applica-tion servers are proprietary and access to the source code is unavailable. Also

2Application server product vendors, http://www.service-architecture.com/products/application servers.html

47

even with open source middleware 3 4 where the source code is available it maybe too time consuming for developers to gain an understanding of applicationserver behaviour, such that modifications can be made. Similarly, modificationsto the application components can be time consuming and cumbersome. Alsowith component based systems the application components may be from thirdparties (e.g. component vendors 5) and the source code may but unavailable orunmodifiable due to licensing constraints.

• Low overhead: It is important that the impact on the running system undertest is kept to a minimum. Performance tools that severely impact the systemperformance can not be run with realistic loads and thus can not obtain a realisticpicture of the system as it is intended to run in a production environment.

• End-to-end: Performance problems can occur in any of the different server sidetiers that make up an enterprise application. In fact often they occur as a resultof communication between the different component tiers. Thus it is importantto analyse the end-to-end design of enterprise applications and not just to con-centrate on tuning particular components (e.g. the database).

• Applicable to distributed systems: It is common that today’s enterprise appli-cations are physically distributed across a number of different machines. Theapproach we outline above should be applicable for such systems without theneed for a developer to correlate a wide number of different log files from thedifferent machines.

• Efficient: Although, the analysis and detection is performed off-line, the per-formance of these modules should not exceed an acceptable level, whichmight inconvenience users. For example analysis that takes a very long time(hours/days) would not be convenient for developers working to tight dead-lines.

• Low number of false positives: The number of false positives needs to be keptto a minimum. False negatives should also ideally be low, however false neg-atives can be difficult to measure since antipatterns may exist in a system thatwe are unaware of. This is especially the case when testing real systems wheredocumentation may be incomplete or unavailable.

• User configurable and extensible: Users need to be able to add their own mon-itoring information and antipattern descriptions since certain antipatterns maybe either application or domain specific. Users should also be able to configurethe antipattern detection process such that unimportant potential antipatterndetections are filtered out.

3JBoss Application Server, Red Hat,http://www.jboss.org4Jonas Application Server, Objectweb, http://wiki.jonas.objectweb.org/xwiki/bin/view/Main/WebHome5WebCab Components, http://www.webcabcomponents.com/

48

• Scalable: The above quality attributes should all scale when the tool is appliedto large systems and to systems under load.

Our approach, which has been implemented in the form of the PAD tool, meets therequirements outlined above. Next we discuss how this is achieved by giving a moredetailed overview of the monitoring, analysis and detection modules.

3.1.2 Monitoring

The monitoring module (chapter 4) is responsible for collecting run-time informationon the different components that make up an enterprise application such that a de-tailed representation of the system can be recreated and potential antipatterns iden-tified. The monitoring approaches allow for (a) identification of component relation-ships, (b) identification of communication patterns between components, (c) trackingof objects (and how they are used) across components, (d) the collection of componentperformance metrics, (e) the collection of component meta data (e.g. container servicesused, bean type) and (f) the collection of information on server resources. Using thisinformation we can extract the required relationships to reconstruct the system designfor performance antipattern detection. The monitoring module consists of three maincomponents, i.e. a run-time path tracer (section 4.2), a MEJB server resource moni-tor (section 4.3.1) and a meta-data extractor (section 4.3.2). The monitoring moduleis shown to be end-to-end (section 4.2.5.1), portable across different JEE implementa-tions (see section 7.2.2) and non-intrusive (see section 4.2.5.2). We also discuss howthe monitoring can be applied across physically distributed systems (section 4.2.5.3).The monitoring module output is forwarded to the PAD analysis module for auto-matic design reconstruction. Section 7.2.1, however, shows how this output can beused to manually reconstruct run-time design models of the system, and how usingthese design models, antipatterns can be easily identified by a developer. Finally, re-sults from performance tests show that the monitoring approach results in a very lowoverhead on the system being monitoring (see section 7.2.3).

3.1.3 Analysis

The analysis module (chapter 5) is responsible for extracting a run-time design modelof the system and does so by performs the following actions on the monitoring data:(a) it automatically extracts the component relationships and component object usagepatterns (section 5.2), (b) it automatically reconstructs the run-time container serviceboundaries (section 5.3), (c) it automatically identifies component communicationspatterns (section 5.4) and (d) it applies data reduction techniques (section 5.5). In par-ticular, in this chapter, we show how we apply the data mining technique, frequentsequence mining (FSM) to automatically identify interesting component communica-

49

tion patterns in the data collected. Specifically we show how FSM can be applied in anefficient manner to run-time paths (section 7.3.1). We also show how, by modifyingthe FSM algorithm, the algorithm output can be made more human readable (section5.4.2). In fact in section 7.3.2 we show how the output from the FSM algorithm canbe used to manually identify design flaws in JEE applications. The FSM algorithmtakes a user configurable input (known as the support value) which determines thethreshold for the patterns it identifies. The support value can be used to filter out lessinteresting patterns in the data. Finally the analysis module is also responsible forsummarising the monitoring data using clustering and statistical analysis techniques(see data reduction section 5.5). This data reduction allows for the design extractiontechniques to be scalable when large amounts of data are collected by the monitoringmodule (e.g. when the system is under load).

3.1.4 Detection

The extracted design model is used as input into the detection module (chapter 6). Ourdetection mechanism is based on a rule engine approach. The design model is con-verted to rule engine specific facts that can be loaded into the rule engine. Rules canbe input into the rule engine which describe the antipatterns that we want to detectin the model. The rule-engine makes use of forward-chaining rules which are in theform of ”if-then” statements. Rules are specified so that the rule’s conditions verify theoccurrence of a certain antipattern. Subsequently, when existing facts match a rule’scondition clauses, the rule action is fired indicating that the antipattern was detected(see section 6.2). Our predefined rules make use of configuration files which containuser specified thresholds (section 6.3.2). User defined thresholds can be utilised tofilter out antipatterns that developers feel are too insignificant to address. The PADtool also contains a rule library for JEE performance design and deployment antipat-terns (section 6.3.1) which users can extend by adding their own antipatterns rule de-scriptions. The PAD tool has been applied to a sample and real large scale enterpriseapplication. Our results (see section 7.4) show how the approach scales when appliedto a real enterprise system and that the number of false positives remains low.

3.2 Antipatterns Revisited

While the concept of software antipatterns is generally understood by developers, theindividual antipattern instances are not as widely known. As documented in the lit-erature [36], antipatterns consistently appear in software, especially in large complex(enterprise) applications [161]. The work in this thesis outlines an approach for theautomatic detection of these well documented issues that consistently occur in enter-prise applications.

50

The benefits of highlighting antipatterns are manifold. Firstly, software antipatternsdocument bad practices in software development as well as their corresponding so-lutions. Thus, detecting instances of antipatterns and highlighting them to the devel-oper, not only allows the developer to understand what is wrong with the application,but it also allows the developer to easily rectify the situation by refactoring the applica-tion with the solution provided. Secondly, antipatterns provide a high-level languagefor discussing design issues. Developers can use antipattern names to clearly and ef-ficiently discuss within the development team particular implementation issues thatmay arise. Also, exposing developers to performance related antipatterns helps themform a sense of performance intuition, since the underlying reason as to why a partic-ular antipattern had an adverse effect on performance is underlined in the antipatterndescription. Consequently developers are less likely to make the same mistake in thefuture.

In the following subsection we discuss a number of different antipattern categoriesand give a hierarchy of antipatterns that shows how the different categories that weare interested in are related. We also show that a high percentage of enterprise an-tipatterns tend to be concerned with performance issues. Finally in this sub sectionwe differentiate between design antipatterns, deployment antipatterns and program-ming errors.

3.2.1 Antipattern Hierarchy

Figure 3.3 gives an antipattern hierarchy diagram. At the top of the diagram we havehigh level technology independent software antipatterns. Brown et al. [36] introduceda number of such antipatterns concerned with a range of software quality attributes(such as reusability, maintainability, performance). More recently Smith and Williams[154] introduced general performance antipatterns which solely focus on performanceconcerns (i.e. level 2). The performance antipatterns presented in the literature [154]are high level and language-independent antipatterns. They describe situations wherea sub-optimal performance design choice has been made. Instances of the antipatternsdocumented by Smith and Williams, however, occur throughout different technolo-gies. Many of these problems are especially common in enterprise technologies whereperformance is often a major concern (level 3). JEE antipatterns have been presentedin [161] and [61]. The literature [161] concentrates on EJB antipatterns, while [61] listsantipatterns concerned with a number of aspects of the JEE technology (i.e. Servlets,JSP, EJB and Web Services). We have analysed the antipatterns from both sources.From a total of 43 antipatterns documented in [161] we have identified 34 (79%) ofthem to be performance related antipatterns (since according to the authors they havean impact on system performance). From a total of 52 antipatterns documented inthe literature [61] we identified 28 performance antipatterns (54%). The high propor-tion of antipatterns from [161] and [61], that are related to performance, is further

51

Figure 3.3: Hierarchy of Antipatterns

evidence that performance is an important software quality attribute for enterprisesystems, and that poor performance design is common in such systems.

We further divided all JEE performance antipatterns identified into 3 different cat-egories (level 4): (a) Performance programming errors, (b) performance design an-tipatterns and (c) performance deployment antipatterns. Performance programmingerrors (a) can be defined as common mistakes made by developers that result in de-graded system performance. They yield no design trade-off and always have a neg-ative impact on performance. Examples include memory leaks, deadlocks and im-proper cleanup of resources such as database connections. Generally developers areunaware of the presence of performance programming errors in the system. The Rot-ting Session Garbage antipattern [161] is an example of a performance programmingerror that is often made by developers using the EJB technology. This antipatternoccurs when a client fails to explicitly remove a stateful session bean when finishedusing it. The orphaned bean will continue to live in the application server using upsystem resources until it times out. Until then, the EJB container must manage the

52

bean, which can involve the relatively expensive job of passivating the bean to makeroom for other active beans. In many situations fixing programming errors alone willnot improve the overall system performance such that requirements are met. Often itis the case that the system design requires modification. Performance design (b) anddeployment (c) antipatterns can be defined as instances of sub-optimal design or sub-optimal deployment settings that exist within the application i.e. situations where aninefficient design choice has been taken. In such situations an alternative more effi-cient design choice exists. Developers are often aware of having made these choices,but can be unaware of their consequences. Performance design and deployment an-tipatterns can be used to identify and resolve these situations since they documentboth the sub-optimal design (or deployment setting) and its corresponding optimalsolution.

We are interested in both design and deployment antipatterns since, with componentbased frameworks such as JEE, many decisions that were inherent in the design andcoding of applications in the past, have been abstracted out into the deployment set-tings of the application. With EJB for example the granularity and type of transactionscan be specified in the XML deployment descriptors of the application. As a result,when making deployment time configuration settings, different design trade-offs thatcan significantly impact performance must be considered.

In section 6.1 we give the categories of performance design and deployment antipat-terns that our PAD tool can currently detect. In this section we categorise the antipat-terns further into groups related to the data required to detect them.

53

CHAPTER

FOUR

Monitoring Required forAntipattern Detection

This chapter outlines the various monitoring components that are required to capturethe information used for antipattern detection. The monitoring module is responsiblefor collecting information on the system under test such that a detailed representa-tion of the system can be recreated and potential antipatterns identified. To identifyperformance design and deployment antipatterns in component-based enterprise ap-plications we need to be able to identify (1) the components (and their types) thatmake up the system, (2) how the different components interact with each other and(3) how the components make use of the different middleware and system resources.The monitoring stage of the antipattern detection process obtains information on (a)component relationships, (b) component communication patterns, (c) component re-source usage, (d) component object usage, (e) component meta-data and (f) serverresource usage. Using this information we can extract the required relationships toreconstruct the system design for performance antipattern detection. The monitor-ing module consists of three main components, a run-time call path tracer, a serverresource monitor and a meta-data collector. In this chapter we detail the differenttechniques required to capture this information in a portable, low-overhead and non-intrusive manner. Our monitoring approaches are applied to a running applicationand do not require any analysis of the source code. The final section of this chapteroutlines related work.

54

Main Points

• A run-time path is a data structure for the representation of run-time calling be-haviour. It contains control flow, resource usage and performance characteristicsassociated with servicing a user request.

• Run-time paths can be applied for a number of different purposes, includingreverse engineering, antipattern detection, problem determination, workloadmodelling, hotspot identification and adaptive monitoring. Thus it is impor-tant that there are mechanisms and tools that allow for the recording of run-timepaths.

• There are four main requirements for run-time path tracing of component basedsystems: request identification, request tagging, distributed request trackingand a request interception or monitoring mechanism.

• Run-time path tracing requirements have been met in a non-intrusive, portablemanner for JEE in the COMPAS JEEM tool. COMPAS JEEM takes advantage of anumber of JEE standard mechanisms, and leverages and extends the capabilitiesof a number of open source projects to achieve this aim. The run-time path trac-ing approach is portable across the JEE technology and across other componenttechnologies.

• A solution for distributed run-time path tracing is presented for the JEE technol-ogy.

• The state of server resources can be monitored through the Java ManagementExtensions interface

• Functional and structural information on the JEE components can be automati-cally extracted from the component meta-data. This avoids the need for sourcecode analysis. Information on how the component interacts with the differentmiddleware services can also be obtained.

55

4.1 Chapter Overview

The monitoring stage of the antipattern detection process obtains information on (a)component relationships, (b) component communication patterns, (c) component re-source usage, (d) component object usage, (e) component meta-data and (f) serverresource usage. Using this information we can extract the required relationships toreconstruct the systems run-time design for performance antipattern detection (seefigure 3.2 on page 46). The monitoring module consists of three main components,a run-time path tracer, a server resource monitor and an meta-data extractor. Nextwe detail the different techniques required to capture this information in a portable,low-overhead and non-intrusive manner. Our monitoring approaches are applied toa running application and do not require any analysis of the source code.

The run-time path tracer collects run-time paths which contain the information (a) (b)(c) and (d) outlined above. More specifically a run-time path [44] contains the controlflow (i.e. the ordered sequence of methods called required to service a user request),resources and performance characteristics associated with servicing a request. Run-time paths are described in more detail in section 4.2.1. By analysing these paths onecan easily see how system resources are being used, how the different componentsin the system interact and how user requests traverse through the different tiers thatmake up the system. As part of this work a run-time path tracing tool has been de-veloped, called COMPAS Java End to End Monitoring (JEEM). COMPAS JEEM mon-itors the paths user requests take when traversing through the different tiers in thesystem. Resource usage and performance characteristics of each request can also beobtained using our approach. It builds upon the open source COMPAS monitoringframework 1 [120] [118] and leverages and extends the capabilities of a number ofother open source projects2 [45]. In this chapter we discuss how run-time path trac-ing has been achieved in a non-intrusive, low-overhead and portable manner. Section4.2.2 and 4.2.3 gives our motivation and the considerations that should be undertaken(independent of the underlying technology) for non-intrusive run-time path tracing.Section 4.2.4 presents the core instrumentation process and the extension capabilitiesavailable in the COMPAS monitoring framework. The end-to-end run-time path trac-ing approach leverages these extension mechanisms to build upon the COMPAS mon-itoring and instrumentation infrastructure. In section 4.2.5 we describe the extensionsmade to the COMPAS framework and the overall architecture of COMPAS JEEM. Inthis section we also detail the open source projects that we have extended and inte-grated with COMPAS to achieve our aims. In section 4.2.5.3 we detail how run-timepath tracing can be performed in a truly distributed manner. The idea of Compo-nent Object Tracking is introduced in section 4.2.5.8 and we discuss how this can beachieved through dynamic bytecode instrumentation and the COMPAS ByteCode In-

1COMPAS, http://compas.sourceforge.net/2P6Spy, http://www.p6spy.com/

56

strumentation (BCI) tool. The portability of the run-time path tracing approach acrossdifferent component technologies is discussed in section 4.2.5.9.

Section 4.3 details the remaining two components of the monitoring module i.e. theserver resource monitor and the component meta-data extractor. Firstly in subsection4.3.1 we detail how our server resource monitor obtains information on the serverresources at run-time through a Management EJB monitoring component. In this sec-tion we discuss how this is achieved in a non-intrusive and portable manner throughthe Java Management Extension interface. We also discuss the information obtainedthat is required for the run-time design model (i.e. (f) server resource information)and why this approach induces very little overhead on the running system. In subsec-tion (4.3.2) we outline how information on the different components can be extractedfrom the component meta-data (e). The component meta-data contains informationon how different components make use of the container services as well as structuraland functional information which is added to the run-time design model. This infor-mation can be obtained by analysing the component deployment descriptors. Thistask is performed off-line and is automated using an XML parser.

Finally in section 4.4 we compare our monitoring approach to related work. In partic-ular we detail: alternative applications of run-time paths, related data structures forthe representation of dynamic calling behaviour and related tracing tools and tech-niques that have been previously presented.

4.2 Run-Time Path Tracing

4.2.1 Run-Time Paths Overview

A run-time path is a data structure for the representation of run-time calling be-haviour. A run-time path has been previously defined by Chen et al. [44] as containing”the control flow, resources and performance characteristics associated with servicinga user request”. The authors also state that run-time paths are recorded at run-time bytracing requests through a live system, and that they span the system’s layers to di-rectly access component and resource dependencies. Next we describe run-time pathsin more detail and explain how they related to other representations of run-time call-ing behaviour. We also show a run-time path data structure and explain the variousfields that are contained with it.

A dynamic call trace [94] (see figure 4.1 (a)) is a recording of all calls and returns thatoccur in a program’s execution. Dynamic call traces are unbounded. Their size isproportional to the number of calls in an execution [8]. Figure 4.1 (b) shows a treerepresentation of a dynamic call trace, often referred to as a dynamic call tree [8] [94].Traversing the tree in preorder will give the ordered sequence of calls and returns asthey occurred during execution. Dynamic call traces/trees capture important infor-

57

Figure 4.1: Dynamic Call Trace (a) and Corresponding Dynamic Call Tree (b)

mation for system comprehension such as, the sequence of calls, the context in whicha call was made and repeating calls or loops. An issue with dynamic call traces how-ever is that they are unbounded and are space inefficient. More compact represen-tations, such as dynamic call graphs and calling context trees, have been previouslypresented to reduce the space overhead [8] [94] [79] however these representationsdo not capture important information that is contained in the dynamic call trace. Amore detailed comparison of these representations is given in section 4.4. Run-timepaths are similar to dynamic call traces. In fact they contain the same information asa dynamic call trace i.e. they contain the ordered sequence of all calls and returns thatoccur during execution. However they also contain related resource and and perfor-mance information for each call. In fact they can contain further information such asthe type of call made or the arguments passed in the call for example. Furthermorerun-time paths are generally associated with multi-user/multi-client systems. Theygroup together, into a single path, the sequence of calls that correspond to a givenuser/client request. As such, run-time paths isolate the calls that occur in responseto a given request. This is particularly important in multi-user/multi-client applica-tions whereby many simultaneous requests may be concurrently serviced. Chen et al.[44] have described this as providing ”a vertical slice of the system from a request’sperspective.” Finally, since they are associated with multi-user applications, run-timepaths often span a number of system layers. These layers can often be physically dis-

58

Figure 4.2: Example Run-Time Path

tributed across multiple hardware components. An example run-time path is givenin figure 4.2. The diagram shows a representation of a run-time path in a tree likestructure. An indent from one level to the next means that the call above the indentis a parent of the indented call below, and any other calls below with the same levelof indentation, until the parent call has returned. Figure 4.2 shows a sequence of callsthat were invoked to service a user request in a typical JEE application. It containsfurther information such as the type of call (e.g. a web tier call, a business tier call ora database call) and performance metrics.

Next we describe the run-time path data structure. A run-time path data structure(figure 4.3) essentially contains a path ID and a path node. The path ID is a uniqueidentifier that distinguishes the run-time path from other paths in the system. A pathnode data structure (figure 4.4) represents a call in the path. It contains an object thatholds information on the call details (e.g. component name, method name, argumentspassed, performance metrics) and an ordered list of path nodes (callees). By traversingthe path structure in a preorder fashion we obtain the ordered sequence of calls as theyexecuted in the application. Both figure 4.3 and 4.4 contain only the essential fieldsthat are required in these data structures. Typically a run-time path data structure cancontain a number of other useful fields and/or methods. For example it may contain acomponent quick list which gives a list of all components that make up the path. Alsoit may contain useful methods for access and creation. Similarly a path node datastructure may contain a number of other useful fields (e.g. call type) and methods.

59

class CallPath {

String pathID;PathNode Root;.........//other useful fields or methods}

Figure 4.3: Run-Time Path Data Structure

class PathNode {

Map Details;List Callees =new ArrayList();......//other useful fields or methods}

Figure 4.4: A Run-Time Path’s PathNode Data Structure

4.2.2 Run-Time Path Tracing Motivation

The analysis module presented in chapter 5 makes use of run-time paths to automat-ically extract information required for antipattern detection. Below are some alterna-tive uses that run-time paths can be put to.

A run-time path shows how a user request is serviced, giving the ordered sequence ofevents/calls and related performance and resource usage information. Thus run-timepath analysis can help greatly with system comprehension and can be applied for anumber of different purposes in this area. System comprehension can be a major issuefor developers of large enterprise applications. A lack of a system wide understand-ing can lead to badly designed systems which perform poorly and can be difficult todebug and maintain. In section 7.2.1 we show how run-time paths can be used toreverse engineer and quickly deduce the overall system structure of enterprise appli-cations. Analysing run-time paths to deduce system structure is advantageous for anumber of reasons. Firstly analysing application source code can be difficult since alarge number of files can be involved. Even when using code comprehension tools3 [77] determining the chain and order of method calls by stepping through sourcecode can be a complex and time consuming task. In addition, some paths might bevery difficult to determine from static analysis, in particular in systems which con-tain workflow engines that select execution targets based on run-time conditions. Incomparison COMPAS JEEM presents all calls from the main software componentsthat make up a user request in a single run-time path. It maintains the order of calls

3Juliet Java code comprehension tool, http://infotectonica.com/

60

and the call sequence can be easily observed by traversing the run-time path tree-likestructure (see figure 4.2). Another advantage of using run-time paths, for system com-prehension, is that they can be well represented in a diagrammatic format (see figure4.12). The diagram construction process can be easily automated reducing the effortrequired on the part of the developer further [97]. As shown in section 7.2.1, such dia-grams are useful for identifying performance design issues or antipatterns that mightexist in enterprise applications.

Run-time paths have also been put to other uses such as fault detection and diagnosis,workload modelling, and adaptive monitoring. In section 4.4 we will discuss thisrelated work in more detail.

4.2.3 Run-Time Path Tracing Considerations

Today’s enterprise applications are generally required to handle high loads of concur-rent users. Run-time path tracing in such systems involves tracing each of these userrequests as they pass through the different software tiers that make up the entire ap-plication. To achieve this user requests must be identifiable across the entire requestand the order of the calls that make up the requests needs to be maintained.

A system needs to meet the following requirements to be able to perform run-timepath tracing:

R1 the system must be able to identify new user requests entering the system

R2 the system is required to tag the request with request specific information (RSI) sothat calls can be mapped to the originating requests

R3 the system requires the ability to piggy back the RSI with the request, such thatthe request can be tracked across the entire system

R4 a monitoring framework that provides for request interception is required

Next we detail how run-time path tracing can be achieved through the above require-ments. The user request identification mechanism (R1) is required to determine whena new request enters the system such that it can be (R2) tagged with request specificinformation (RSI). The RSI contains a unique ID so that the request can be identifiedand distinguished from other concurrent requests in the system. The RSI also con-tains a sequence number that is used to order the calls that make up the request. Thesequence number is incremented (by the tracer’s run-time path ordering logic) everytime a component method is called and is reset upon a new user request.

User requests also need to be tracked (R3), so that, as the request passes through thedifferent software components that make up the system, it can be identified and theorder of its calls maintained. In order to be able to track the request the RSI must be

61

attached to it, in such a way as to be available at every call point along the request.This can be particularly challenging when a system is distributed across a network.

A monitoring framework (R4) is required to intercept and log calls made to the soft-ware components that make up the request. The RSI, along with component detailsare logged upon a method invocation. The component details describe the compo-nent that is called, giving information such as what component method was invoked,what arguments were passed etc. The logged information can be subsequently usedto construct the run-time paths offline. The reconstruction process makes use of theRSI data to (a) determine what calls make up each run-time path and (b) to order thecalls.

Thus far, the above requirements have generally been met in a manner that involvesinstrumenting either the software application itself or the underlying middleware [82][45]. Manual instrumentation of the application source is undesirable since instru-menting the code can be time consuming and cumbersome and the effort must berepeated for each new system under development. Since the JEE specification doesnot explicitly address the requirements for run-time path tracing, middleware layerapproaches for JEE have involved using non-standard mechanisms provided by spe-cific vendors that are not portable across the technology and that often require thesource code of the server to be available (such that it can be recompiled). For examplethe JBoss group recognised the need for R4 by providing its interceptor-based com-ponent architecture [105] which allows for the interception of calls to business tierJEE components. IBM has also recently addressed the need for R3 through its WorkArea Service [102]. The biggest issue with using non-standard mechanisms, however,is that developers get locked in to using a particular middleware implementation andlose the flexibility of being able to quickly change their underlying middleware imple-mentation should the need arise. Consequently, such approaches can not be appliedto systems that are made up of application servers from numerous different vendors.

In the following sections we present an approach that, through the COMPAS monitor-ing framework, takes advantage of a number of JEE standard mechanisms to meet therequirements outlined above for JEE systems in a non-intrusive and portable manner.While our implementation focuses on the JEE technology, only the framework instru-mentation process and request tracking are JEE specific, the remainder of the workpresented below can be applied to other component technologies (e.g. CCM). This isdiscussed in detail in section 4.2.5.9.

4.2.4 COMPAS Monitoring Framework

COMPAS [118] is intended as a foundation for building enterprise-level performancemanagement solutions for component-based applications. It was designed to pro-vide a common monitoring platform across different application server implementa-

62

tions. This is achieved by leveraging the underlying properties of component-basedplatforms in order to enable non-intrusive instrumentation of enterprise applications.Two main techniques are used to minimise the overhead of the monitoring infrastruc-ture: asynchronous communication and adaptive activation. The former is employedin the entire infrastructure by the use of an event-based architecture with robust mes-sage handling entities that prevent the occurrence of locks in the target application.The latter technique uses execution models captured from the target application todrive the activation and deactivation of the monitoring probes. By appropriately min-imising the number of active probes (i.e. through the alert management and adaptivemonitoring process), the total overhead is reduced while preserving complete targetcoverage (see [119] [118]). As the COMPAS infrastructure is designed to be used asa foundation for performance management tools, its design is highly extensible andbased on decoupled communication mechanisms.

The most important functional entity of the monitoring infrastructure is the monitor-ing probe. The probe is conceptually a proxy element with a 1 to 1 relationship withits target component. In JEE, target components are JSPs, Servlets, EJBs and EIS com-ponents deployed in a target application. The probe is implemented as a proxy layersurrounding the target component with the purpose of intercepting all method invo-cations and lifecycle events. The process of augmenting a target component with theproxy layer is referred to as probe insertion. This section focuses on the core COMPAScapabilities, initially limited to EJB components only, whereas the following sectionspresent the extensions that allow COMPAS to instrument and monitor the remainingJEE target components, grouped under the name COMPAS JEEM.

4.2.4.1 Portable EJB Instrumentation

COMPAS uses component meta-data to derive the internal structure of the target en-tities. The component meta-data is placed in deployment descriptors that containstructural as well as behavioral information about the encompassing EJBs. By lever-aging this data, it is possible to obtain the internal class-structure of each component,which is needed for instrumentation. The instrumentation is performed by a ”proxylayer” attached to the target application through a process called COMPAS Probe In-sertion (CPI). As all the information needed for probe insertion is obtained from themeta-data, there is no need for source code or proprietary application server hooks.Therefore, the effect on the target environment is minimal and user intervention in theprobe insertion process is not required. COMPAS is in this respect non-intrusive, as itdoes not require changes to the application code or to the runtime environment as themajority of other approaches do.

The CPI process examines the Target Application’s structure and uses componentmeta-data to generate the proxy layer. For each component in the target application,a monitoring probe is specifically generated based on the component’s functional and

63

structural properties. The CPI process leverages deployment properties of componentframeworks to discover and analyse target applications. Therefore, CPI is conceptu-ally portable across component frameworks such as EJB, .NET or CCM. Indeed, givena component system written for any such platforms, COMPAS would be able, withminimal modifications, to insert monitoring probes that match each of the applicationcomponents.For components written for the EJB platform, the following meta-data is extracted andused to generate a monitoring probe for a target component (TC):

• Component Name (bean name, for EJB)

• Component Interface (Java interface implementing the services exposed toclients, for EJB)

• Locality (local or remote, for EJB)

• Component Type (stateless session, stateful session, entity or message-driven,for EJB)

• Component Interface Methods (Java methods in the business interface, for EJB)

• Component Creation Methods (ejbCreate() methods, for EJB)

COMPAS provides a Probe Code Template (PCT), modifiable by the user, which rep-resents a template for the generated monitoring probes. It consists of extensible logicfor initiating event-handling operations and placeholders for component-specific in-formation. Using the PCT and the extracted meta-data, the CPI process generates oneprobe for each Target Application Component. The placeholders in the template arereplaced with the values extracted from the meta-data. The proxy layer is an instanti-ation of the PCT, using the TC meta-data values.The proxy layer (probe) is a thin layer of indirection directly attached to the TC. Tofulfill its instrumentation functionality, the Probe employs the Instrumentation Layerthat has the capability of processing the data captured by the Probe and performingsuch operations as event notifications. The Instrumentation Layer uses the COMPASProbe Libraries for implementing most of its logic. A Modified Component (MC) re-sults after the CPI process has been applied to a TC, and this will enclose the originalTC. In addition, it will contain the Probe and Instrumentation Layer artifacts. In orderto ensure a seamless transition from the TC to the MC, the CPI transfers the TC meta-data to the MC. The MC meta-data will only be updated so as to ensure the properfunctionality of the proxy layer (the bean class property must be updated to indicatethe insertion of the Probe class). The CPI process is illustrated in figure 4.5.

At run-time, probes collect and analyse performance and lifecycle events from the tar-get components and communicate with the central monitoring authority, the COM-PAS Monitoring Dispatcher. The Monitoring Dispatcher is the client-side entity re-

64

MC Probe TC PCT Probe Instrumentation Layer COMPAS Probe Libraries

TC + = (CPI)

COMPAS Monitoring Dispatcher

network Figure 4.5: COMPAS Probe Insertion Process

sponsible for mediating client access to the COMPAS probes by providing an abstrac-tion layer over the lower-level communication and control operations. It containshandlers for efficient processing and transformation of probe notifications into COM-PAS events. In addition, the Monitoring Dispatcher provides a control interface thatallows transparent relaying of commands to the monitoring probes. The central roleof the Monitoring Dispatcher is client-side data processing as illustrated in figure 4.6.

4.2.4.2 COMPAS Extension Points

COMPAS Monitoring contains an instrumentation core and a set of extensions forcoordinating and handling instrumentation events. The extensions are built upon thepluggable architecture of the instrumentation core by leveraging the COMPAS Frame-work Extension Points (FEPs) based on loosely coupled asynchronous communica-tion. Possible extensions that can be added to COMPAS include support for low-levelinstrumentation sources such as virtual machine profiling data, as well as high-levelfunctional extensions such as elaborate data processing capabilities for complex anal-ysis of the monitoring data. Complex decision policies for improving the alert man-agement and adaptive monitoring process can also be implemented as extensions.

Since the COMPAS infrastructure spans all the JEE server-side tiers and the client tier,it provides two types of framework extension points:

65

Figure 4.6: COMPAS JEEM Architecture

• Server-Side FEP: facilitates the extension of the functionality available to or pro-vided by a COMPAS Monitoring Probe. In addition, it facilitates the creationof new probe types. The majority of extensions introduced by JEEM use server-side FEPs. In particular, JEEM makes use of Servlet filter probes and JDBC driverprobes to feed information into the COMPAS infrastructure. Apart from theseprobes in the web and EIS tiers, JEEM also enhances existing EJB probes in or-der to generate appropriate run-time path extraction information. Other exam-ples of server-side FEPs include better time-stamp extraction techniques or ad-vanced anomaly detection algorithms used in the COMPAS problem diagnosisprocesses [118].

• Client-Side FEP: facilitate the extension of the functionality available to or pro-vided by the COMPAS Monitoring Dispatcher. The run-time path constructionand analysis module introduced by JEEM uses such a client-side extension andbenefits from the information made available by the distributed infrastructure tothe client-side. Other examples of extensions that can be added using client-sideFEPs include specialised GUI consoles or integration into wider-scope perfor-mance tools.

As indicated, the additional functionality that COMPAS JEEM provides over the basicCOMPAS framework uses several FEPs, across all JEE tiers. The following sectionspresent in detail these JEEM extensions.

66

4.2.5 COMPAS Extensions

As detailed in section 4.2.4 the current implementation of the COMPAS frameworkhas the ability to monitor the business tier of JEE applications. We have made a num-ber of extensions to the COMPAS framework using the COMPAS FEPs to allow forthe tracing of run-time paths across all tiers of a JEE system for multi-user workloads.The extensions made fall into three different categories, namely, monitoring, run-timepath tracing and analysis. The overall architecture of COMPAS JEEM is shown infigure 4.6.

4.2.5.1 Monitoring

The COMPAS JEEM monitoring approach is completely portable and non-intrusiveand has the ability to trace system wide JEE run-time paths. Basically it works by firstintercepting the calls made to the different components that make up the system andsecond, recording their sequence. The interception of calls is performed by extend-ing the COMPAS framework, using server-side FEPs, so that monitoring on all threeserver side tiers is performed.

COMPAS uses the idea of a proxy layer to monitor EJB components [119]. The proxylayer is a thin layer of indirection directly attached to the EJBs that captures any re-quests to and responses from the components. When extending COMPAS we appliedthe idea of intercepting requests to and responses from components to the web andEIS tiers. This is performed in the web tier using intercepting filters and in the EIS tierusing wrapper components. The EJB probes, intercepting filters and wrapper compo-nents can also be referred to as Interception Points (IPs).

Intercepting Filters A chain of pluggable filters can be created to implement com-mon pre and post-processing tasks during a web page request by implementing theIntercepting Filter pattern [7]. This patterns allows for the creation of pluggable fil-ters to process common services in a standard manner without requiring changes tocore request processing code. The filters intercept incoming requests and outgoing re-sponses, which allows for pre and post-processing respectively. The servlet 2.3 speci-fication 4 includes a standard mechanism for building filter chains based on the Inter-cepting Filters pattern. Consequently web applications running on servlet specifica-tion compliant web servers can be augmented with filters for pre and post processingwithout the need to modify source code.

Next we briefly detail the Intercepting Filter pattern which we use to monitor the webtier of JEE applications. A UML sequence diagram of the Intercepting Filter pattern isshown in figure 4.7.

The pattern contains the following participants: the Client, the FilterManager, the Fil-

4Java Servlet Specification, http://java.sun.com/products/servlet/download.html,

67

Figure 4.7: Intercepting Filter

terChain, one or more Filters and finally the Target resource. The FilterManager sitsbetween the client and the Target resource. When the client makes a request for theTarget resource the FilterManager intercepts this request and creates a FilterChain. AFilterChain is an ordered collection of independent Filters. The Filters perform thepre and post processing logic and are mapped to a particular Target resource. TheFilterManager next makes a request to the FilterChain to process the filters and subse-quently the Target resource.

The Intercepting Filter pattern can be implemented using a number of different strate-gies. Servlet specification compliant web servers implement the Intercepting FilterStandard Filter Strategy [7] whereby filters can be added declaratively through XMLdeployment descriptors. The web server container contains the FilterManager and isresponsible for creating the different FilterChains that can be specified through theXML deployment descriptors.

To monitor the web tier we have implemented a monitoring filter. The monitoring filterintercepts incoming requests and outgoing responses and thus can log the beginningand end of user requests. The monitoring filter can be added to the web applicationunder test automatically (see section 4.2.5.1 below).

Before the servlet 2.4 specification filters could only be applied to the initial client

68

request. Filters could not be applied to inter servlet communications. As a resultthe web tier was monitored as one software component on web servers prior to 2.4.However the 2.4 servlet specification allows for filters to be applied to calls betweenservlets (i.e. forwarded requests) 5 and thus our monitoring filters can collect the entireweb tier run-time path trace for servlet specification 2.4 compliant web servers. Sincethis has been mandated by the servlet specification all future JEE environments willsupport this feature.

EIS Tier Wrapper Components The monitoring of the EIS tier is performed usingwrapper components. This approach takes advantage of the proxy design pattern[72]. EIS tier monitoring works by wrapping the JDBC database driver with a mon-itoring layer. JDBC drivers implement the JBDC interfaces and classes of the JDBCAPI. For each original JDBC driver class, a wrapper class that exposes the same inter-face as the driver class is created. Rather than registering the JDBC driver class withthe application server as normal, the packaged wrapper classes are registered withthe application server. Calls made to the database layer from the application serverare thus intercepted by the wrapper classes. The original JDBC driver is instead regis-tered with the wrapper classes (declaratively using a properties file). Any calls madeto the wrapper classes can thus be delegated to the original driver classes. The wrap-per classes contain monitoring logic that captures all JDBC requests and responses.The wrapper classes have been implemented by extending the P6Spy open sourceframework 6.

As outlined above COMPAS JEEM’s monitoring framework consists of a number ofIPs. The IPs capture performance metrics and resource usage information along withthe control flow data required for run-time path tracing.

Adding The Monitoring Framework to a JEE Application: Adding the aforemen-tioned filters and wrapper components does not require the server code or the appli-cation source code to be available nor does it require special JVM hooks. Instead weleverage standard JEE mechanisms and make use of the component meta-data that isstored in XML files as mandated by the JEE specification. The monitoring componentscan be automatically added to the packaged application. The basic instrumentationscript (i.e. the CPI process) provided by the original COMPAS monitoring frameworkhas been extended to (1) unpack the entire JEE application, (2) parse the XML deploy-ment descriptors of the application (to obtain structural and functional information onthe different components that make up the system) and (3) repackage the applicationwith the COMPAS JEEM monitoring components inserted.

5Jason Hunter, ”Servlet 2.4: What’s in store”, Java World, March 2003,http://www.javaworld.com/javaworld/jw-03-2003/jw-0328-servlet.html,

6P6Spy, http://www.p6spy.com/, accessed June 2006

69

4.2.5.2 Run-time Path Tracing

The ability to trace system wide run-time paths for multi-user workloads has beenadded to the capabilities of COMPAS JEEM. A generic run-time path tracing approachhas been outlined in section 4.2.3. This approach has previously been implemented byChen et al. in the Pinpoint open source project [45] [44]. However, a major drawbackof the pinpoint implementation is that it relies on instrumentation of the middlewareand it is therefore tied to its target application server, JBoss. In the following sectionswe detail the Pinpoint implementation of the above approach and how we applied itin a non-intrusive capacity.

Pinpoint - Intrusive run-time path Tracing: Pinpoint is a framework for problem de-termination in internet service environments. It includes a runtime path tracer forthe JBoss application server. The requirements R1 R3 and R4 outlined in section 4.2.3have been met in the Pinpoint implementation through instrumentation of the mid-dleware. The following areas in the middleware were instrumented: the web server,the application server, a Remote Method Invocation (RMI) library and the JBoss JDBCResource Adaptor [105]. Next we briefly detail the above instrumentation points, andtheir responsibilities. R2 is the only requirement that has been met in a non-intrusivecapacity by the pinpoint framework and is described in section 4.2.5.2.

User requests enter the system in the web tier. Pinpoint instruments both the httpserver and the servlet container. The instrumentation in the http server is responsi-ble for determining when a new request enters the system (R1). It injects a unique ID(R2) value into the local thread of the new incoming request. The servlet container in-strumentation is responsible for logging calls to and responses from each JSP/servlet(R4). The servlet container instrumentation is also responsible for the runtime pathordering logic in the web tier.

The application server instrumentation is performed in the Pinpoint project usingJBoss interceptors [105]. The instrumentation logs calls to and from EJB components(R4) and performs runtime path ordering logic when a method is called. The JDBC Re-source Adaptor instrumentation has similar responsibilities in relation to JDBC calls.

Pinpoint also allows for tracing of run-time paths that span a number of threads(which for example might be caused by a remote method invocation) by instrument-ing a JBoss RMI library such that the unique request ID is marshalled across the remotecall from the local thread to the new (remote) thread (R3).

Non-Intrusive Run-time Path Tracing We require a non-intrusive approach for run-time path tracing. In the following paragraphs we discuss how all the requirements(outlined in section 4.2.3), monitoring (R4), user request identification (R1), user re-quest tagging (R2) and user request tracking (R3) can be met in a non-intrusive capac-ity.

70

An application can be easily monitored (R4) by instrumenting middleware. Where themiddleware source code is available it can be instrumented and recompiled such thatthe components running on top of the middleware are monitored and the requiredinformation logged (for instance performance metrics). The issue of monitoring thecomponents in a non-intrusive manner is overcome using the COMPAS frameworkalong with the monitoring extensions that we introduced above in section 4.2.5.1.

The run-time path tracing approach discussed in section 4.2.3 requires user requesttagging (R2), i.e. tagging each user request with a unique id. User requests are taggedby inserting a unique value into the local thread object when a new user request entersthe system. The pinpoint tracing library performs this in a non-intrusive manner: AThreadLocal object (introduced into Java in version 1.2) is associated with each threadin the system. The ThreadLocal objects can be used to hold a particular value (orobject) for the lifetime of the thread. The pinpoint tracing library can be used to injecta RequestInfo object into the ThreadLocal object. The RequestInfo object contains the RSI(i.e. the unique Id of the user request and the sequence number of the current methodcall).

Determining when a new user request arrives into the system can be easily achievedwhen instrumenting the middleware. If user requests are restricted to entering thesystem in the web tier, then the HTTP server (a component of the web server) can beinstrumented such that each new request for a http connection can be tagged. A newrequest for a http connection represents a new user request. However, determiningnew user requests non-intrusively, does not allow manipulation of the HTTP server.To overcome this we implemented a point of entry detection mechanism. Point of entrydetection is required to find if a call to a particular component is the point of entryinto the system i.e. is this call as a result of a new client request. We have extended thePinpoint tracing library to allow for point of entry detection (R1) in a non-intrusivemanner.

Our point of entry detection mechanism works by monitoring the depth of the callsmade on the current thread. We have extended the pinpoint RequestInfo object to in-clude a Depth field to monitor the level of the current method call. The depth value isincremented when entering a method and decremented upon method exit. A depth ofzero indicates that the current method is the first method called in the run-time pathand is thus the point of entry into the system. When a call is made to a method witha depth of zero a unique ID can be assigned to that thread and remains associatedwith that thread until the call depth again reaches zero. Our point of entry detectionmechanism is advantageous since (1) it is non-intrusive and (2) it can be applied toany of the tiers in a JEE system and unlike with the pinpoint approach outlined abovenew user requests must not be restricted to the web tier. The point of entry detectionlogic can be inserted into any of the IPs in figure 4.6 to identify new requests enteringthe system at any of the different tiers.

71

The remaining issue to be addressed in relation to non-intrusive run-time path tracingis user request tracking (R3). While intrusive approaches can afford the luxury ofinstrumenting RMI libraries to marshall request IDs across threads, this can not beachieved non-intrusively. Our approach instead makes the assumption that the webserver and application server are co-located. The database can be deployed eitherlocally or on a remote machine. This assumption is justifiable, since it is often the casethat enterprise systems are deployed on co-located servers during development. Itis common that such systems are only deployed in a distributed environment late inthe development process when production testing takes place. In fact recently it isbecoming more and more common for the web tier and business tier to be co-locatedeven in (often clustered) operational environments. The web and business tier canbe co-located to take advantage of faster local calls between the tiers. Thus, theseenvironments typically contain multiple identical machines, each running an instanceof the web server and the application server, optimised for local inter-communication.

Servers that are co-located run on the same Java Virtual Machine (JVM). Thus, whenservers are co-located, for a particular user request that consists of local calls, the samethread is used across the web server, EJB server and database driver. Thus using ourapproach the user request can be tracked across the different tiers, since the user re-quest can be identified by its unique ID stored in the ThreadLocal object. An issuearises, however, if the run-time path includes remote calls to components that are de-ployed locally. For instance a servlet may call an EJB that is deployed locally, throughits remote interface. In this case it is possible that a new thread is spawned when theremote call occurs. In fact according to the RMI specification 7: ”A method dispatchedby the RMI run-time to a remote object implementation may or may not execute ina separate thread.” In practice, however it seems that a new thread is not spawnedwhen a remote call is made to a component that is in fact local to the callee. Thiscan be clearly seen from the results of our portability tests (see section 7.2.2). We be-lieve that application server vendors take this decision not to spawn new threads sincespawning a new thread when invoking local components through their remote inter-faces would most likely prove wasteful. The fact that new threads are not spawnedallows for the tracing of user requests across all JEE tiers using the approach we out-lined above.

A problem arises, however, if calls are made across distributed JVMs. In this situationa new thread is spawned on the remote JVM to handle any remote calls. The newthread does not contain the RSI and thus calls made to the remote component cannotbe traced.

7Java Remote Method Invocation Specification, http://java.sun.com/j2se/1.4.2/docs/guide/rmi/

72

4.2.5.3 Tracking RSI in Distributed Environments

In this section, we analyze how calls from a client to a server can be tracked in adistributed setting. The approach we take is based on piggy-backing the RSI ontothe request from a client to a remote server. In the following subsections we detailhow this has been achieved in a portable manner for JEE systems. This approach haspreviously been implemented in the WebMon [82] profiling tool, a tool for profilingJEE web applications. In the following sections we discuss how this can be appliedfor COMPAS JEEM.

4.2.5.4 Piggy-backing data

There is no standard mechanism in JEE for piggy-backing data along a remote request.Piggy backing mechanisms however are available in other technologies. For exampleCORBA’s Portable Interceptors 8 allow for piggy-backing of data along a request. TheCORBA Portable Interceptors can be used to track user requests in distributed envi-ronments. For example, it has previously been shown that they can be used to attachadditional security information onto requests [179].

We discuss how this problem can be solved for JEE systems by adding an additionalparameter to the remote request and allowing the client to pass the extra request datathrough that additional parameter. This can be achieved without access to the appli-cation’s source code as follows: A components’ meta-data specifies the componentinterfaces. The interfaces can be reconstructed and rewritten from their bytecode rep-resentation (or by analysing the XML deployment descriptor) to add a method thatallows for the passing of the additional information using an additional request pa-rameter. Next the client component is also instrumented, to add this additional pa-rameter, and the server component instrumented, to pass it to the instrumentationinfrastructure. In the next section, we give a detailed description how this techniquecan be used to pass RSI across distribute component boundaries.

4.2.5.5 Client Side Interception

When a client component calls a distributed component in JEE, the following occurs:The client calls a stub which is a client side proxy object. The stub masks the lowlevel networking issues from the client and forwards the request on to a server sideproxy object. The server side proxy element, known as a skeleton, masks the lowlevel networking issues from the distributed component. It also delegates the remoterequest to the distributed component. The distributed component then handles therequest and returns control to the skeleton, which in turn returns control to the stub.

8Common Ojbect Request Broker Architecture: Core Specification, Version 3.0.3,http://www.omg.org/technology/documents/

73

The stub, in turn, hands back to the client (see figure 4.8).

Figure 4.8: Remote Method Invocation

For client side interception an IP is responsible for adding an additional parameterto the remote request. Adding a client side IP can be accomplished by replacing theclient stub used by the application with a custom wrapper. This custom wrapper pro-vides the functionality to intercept the client’s request invocation before invoking theserver side EJB. The custom wrapper performs the necessary task of extracting theRSI from the ThreadLocal object (or adding it if it is the point of entry into the sys-tem) and sending this information as an extra parameter in the remote call. A samplehome interface and a custom wrapper that accomplishes this is shown in Listings 4.99

and 4.10. A similar wrapper has to be provided for the bean’s remote interface.

In order for the custom wrapper to be able to intercept the client side calls it mustreplace the stubs generated by the Java Virtual Machine. With JEE, the client obtainsa reference to a remote component using the Java Naming Directory Interface (JNDI)naming service and obtains the stub by casting the reference to the component’s inter-face from a generic type to an RMI-IIOP interface type using the PortableRemoteOb-

9Listings 4.9, 4.10 and 4.11 are based on an example first written by Dr. Thomas Gschwind

74

ject.narrow method (see Listing 4.11).

public interface SampleHome extends EJBHome {Sample create()

throws RemoteException, CreateException;}

Figure 4.9: The Sample Bean’s Home Interface

public class WrappedSampleHomeimplements SampleHome{

private SampleHome home;

public WrappedSampleHome(SampleHome home) {this.home=home;

}

public Sample create()throws RemoteException, CreateException {

// intercept request before invocationWrappedSample obj=new WrappedSample(home.create());

// intercept request after invocationreturn obj;}

public void remove(Object p0)throws RemoteException, RemoveException {home.remove(p0);

}

... // other methods provided by home stub}

Figure 4.10: A Wrapper for the Sample Bean’s Home Interface

The JVM generated stub can be replaced with the wrapper by using a customPortableRemoteObjectDelegate object. This is supported through a standard mech-anism in the JEE specification and can be configured as part of the Java runtime en-vironment. Thus this approach is portable across different application server imple-mentations.

4.2.5.6 Server Side Interception

On the server side the IP is in the form of a proxy layer (or wrapper) as described insection 4.2.5.1. Using this approach all calls to and from the bean can be intercepted.The extra parameter (RSI) sent by the client is handled in the wrapper which overloads

75

public class SampleClient {public static void main(String[] args)

throws Exception {

Properties props=System.getProperties();Context ctx=new InitialContext(props);Object obj=ctx.lookup("Sample");SampleHome sh =(SampleHome)PortableRemoteObject.narrow(obj,SampleHome.class);

Sample s = sh.create();System.out.println

(s.callBusinessMethod("SomeString"));s.remove();

}}

Figure 4.11: A Sample Bean Client

each method such that it contains an extra parameter. This extra parameter is used topiggyback the data across the network. In the context of run-time path tracing oncethe data has been sent across the network it can be attached to the current thread asoutlined in section 4.2.5.2.

4.2.5.7 Analysis

COMPAS JEEM contains run-time path tracing logic which non-intrusively logs in-formation when methods are invoked. The monitoring framework uses the COMPASMonitoring Dispatcher to asynchronously log the information to a remote machine.This allows for the information to be stored and analysed off-line. Analysing the dataremotely (as opposed to analysing it locally) reduces the performance impact on themonitored system.

There are two main phases of information extraction from the collected data: run-timepath construction and advanced run-time path analysis. It is important to note thatthe analysis part of the framework is completely independent of the approach usedfor run-time path extraction. The only requirement is that the necessary information(see Run-Time Path Construction below) is provided.

Run-Time Path Construction On every invocation of a component method, informa-tion is logged. In the pinpoint framework this is known as an observation. The loggedinformation contains the following data:

• user request (unique) ID

• sequence number

• call depth

76

• component details

• performance data

Run-time paths are constructed by grouping all observations by user request id. Eachsuch group represents an unordered list of the calls that make up a single run-timepath. By ordering the list according to the sequence number of each observation, theobservations can be arranged in the order that they occurred during program execu-tion. A run-time paths data structure is subsequently created by traversing each or-dered list. This completes the construction of the run-time paths. Further analysis canbe applied to this information for a number of purposes. For example, by analysingthe run-time paths developers can easily construct UML diagrams that can help to de-duce the overall system structure (see sections 7.2.1). This output also forms the basisof the automatic antipattern detection approach and allows for the identification ofcomponent relationships, component communication patterns and the reconstructionof container services (see chapter 5). Next we discuss how component object trackingcan be performed using this data.

4.2.5.8 Tracking Component’s Object Usage

Objects can also be tracked along run-time paths to allow for analysis of their usagein the system. Such analysis can lead to identification of inefficient object usage (seesection 7.4.1). Figure 4.12 shows an example run-time path which tracks the lifecycleof instances of the AccountDetails data transfer object.

The COMPAS JEEM tool is currently being extended (the new version is called COM-PAS ByteCode Instrumentation [BCI]) to track selected objects across run-time paths.The new version of the tool makes use of dynamic byte code instrumentation, throughthe java.lang.management interface, to dynamically instrument selected objects. Toolusers can select particular classes to be tracked. When an object of the selected classis created it is identified along with its corresponding position in the run-time path.Whenever another method (other than the constructor [or creator method for EJBs])is accessed this is also logged. Thus we can identify where objects are created alongthe run-time paths and at what points they are accessed. We can effectively see howobjects are created, used and passed along the run-time path. Figure 4.12 show whereinstances of the AccountDetails object are created and accessed along a JEE call path.The object was created by an entity bean in the application tier and passed to theweb tier where a single method was accessed. This information is required to identifya range of common antipatterns (for example to identify variations of the excessiveobject allocation antipattern [154] which manifests itself in a number of different an-tipatterns in enterprise applications). We have developed a proof of concept of this in-strumentation by manually instrumenting objects that we required for tracking. The

77

Figure 4.12: Run-Time Path with Tracked Object, as a Sequence Diagram

COMPAS BCI approach which makes use of dynamic bytecode instrumentation isonly available for newer JVMs (Java 1.5+).

4.2.5.9 COMPAS JEEM Portability

One of the main advantages of our approach is that it is portable across the JEE spec-trum of platforms and can be applied independent of the underlying middleware im-plementation. However much of this approach is portable across different componenttechnologies (such as CCM). In fact only the instrumentation process (CPI) and therequest tracking (using the ThreadLocal approach) are specific to JEE. Therefore muchof our implementation can be reused across technologies as long as an instrumenta-tion process and request tracking approach can be provided. The CORBA PortableInterceptors, for example, allow for the insertion of custom monitoring code (i.e. in-strumentation of the application) and the piggy backing of data along a request (whichallows for request tracking). Thus our approach could also be applied in the CORBAenvironment by making use of the Portable Interceptors provided for through theCORBA standard and by reusing much of our implementation to perform run-timepath tracing (i.e. the tagging logic, the point of entry detection mechanism, the run-time path ordering logic, the monitoring dispatcher, and the run-time path analysismodules).

78

4.3 Monitoring Server Resource Usage and ExtractingComponent Meta-Data

In this section we outline how the final two monitoring components have been im-plemented, i.e. the server resource monitor and the component meta-data extractor.We give details on the information they extract and how this relates to the run-timedesign model. We also discuss why the approaches are portable and why they do nothave a major impact on the system performance.

4.3.1 Using Java Management Extensions to Monitoring Server Re-source Usage

In enterprise frameworks such as JEE the different components that make up the ap-plication interact with the underlying middleware. The state of the server resourcesthat service these components can significantly impact the overall system performance(e.g. thread pools, database connection pools, object pools). According to the JEEManagement specification (JSR 77 10) application servers are required to expose thisdata through the Java Management Extensions (JMX) technology 11. The foundationof the JMX technology is the Managaed Bean (MBean). JMX allows for a centralisedmanagement of MBeans, which act as wrappers for applications, components, or re-sources in a distributed network. This functionality is provided by a MBean serverwhich serves as a registry for all MBeans. MBeans provide attributes, operations, andnotifications. A client (e.g. a management console) can read and change the attributes,invoke operations, and receive notifications when certain events occur.

The JEE management model exposes MBeans for all well-known concepts from JEE.Examples include the JVM, EJBs, Sevlets and JDBC Drivers. The JEEManagedObject,shown in figure 4.13 12, is the base model of all managed objects in the JEE Manage-ment Model. JSR77 compliant application servers provide access to all managed objectinstances through a Management EJB (MEJB) component. The MEJB is a stateless ses-sion bean, which can be accessed either locally or remotely. The JSR77 also defines aPerformance Data Framework which specifies a data model as well the performancedata requirements of the JEE Management Model. Performance statistics on the var-ious parts of the JEE application server can thus be easily obtained at runtime. Forexample, figure 4.14 shows the JDBCStats type that is expected to be provided by aJDBC resource.

We collected various performance statistics specified by the Performance Data Frame-work for the different JEE component pools (i.e. the thread pool, the database con-

10Java Service Request 77, J2EE management, http://www.jcp.org/en/jsr/detail?id=7711Java Management Extensions Technology,http://java.sun.com/javase/technologies/core/mntr-

mgmt/javamanagement/12Figures 4.13 and 4.14 are from the JEE management specification

79

Figure 4.13: JEEManagedObject

nection pool, and the various EJB pools) and use this information to populate therun-time design meta model shown in figure 3.2 on page 46. Pool statistics include in-formation on the pool size, free pool size, pool queue size, passivation levels etc. Thisinformation can be directly added to the design model and can be utilised to identifydeployment related antipatterns (see section 7.4.1). The monitoring module makes useof a custom JMX monitoring tool to log such information. Since the data recorded bythe tool is already made available by the application server, the performance impactof a JMX monitoring tool is minimal.

4.3.2 Automatically Extracting Component Meta-Data

Component based enterprise frameworks are particularly suited for antipattern de-tection since (a) they specify a component model that defines how the different com-ponents types in the model should be used (e.g. using entity beans for persistence)and (b) they generally contain meta-data on the components that make up the system.EJB meta-data contains structural and functional information on each of the EJB’s de-ployed in the system. For example, information on the EJB component type (i.e. isthe bean a stateless session bean, a stateful session bean, an entity bean or a message

80

Figure 4.14: JDBCStats, JDBCConnectionStats, JDBCConnectionPoolStats

driven bean). The meta-data also contains information on the container services re-quired by the bean’s business methods (e.g. whether the bean requires security checks,whether the bean should be invoked in a transactional context, whether the bean canbe accessed remotely). This meta-data is contained in the XML deployment descrip-tors that are used to configure the application during deployment (see figure 2.3 onpage 20). Thus the meta-data can be obtained without having to access the sourcecode of the application. The data can be used to reason about the behaviour of cer-tain components. For example, if from our run-time profiling we see that a particularcomponent is frequently communicating with a database, from the meta-data we cancheck the component type. If this component turns out to be a stateful session bean wecould flag this as a potential antipattern, since stateful session beans are not designedto frequently access persistent data (as outlined in the component model specified by

81

the EJB specification). The fact that component based enterprise frameworks specifyhow certain component types should behave allows us to automatically identify thistype of unusual behaviour. Without this information automatic antipattern detectionis more difficult. For example, if instead we were monitoring an application made upof Plain Old Java Objects (POJO’s) with no information describing how we expect theobjects to behave, it would be difficult to flag unusual behaviour. In such situationsdomain or application specific information could instead be supplied by the appli-cation developers. The PAD tool extracts the component meta-data from the XMLdeployment files of the JEE applications automatically using an XML parsing library13. This is in fact achieved during the COMPAS instrumentation process (see section4.2.5.1).

The information obtained through the PAD tool’s meta-data extractor is used to pop-ulate the run-time design model (figure 3.2, page 46). Functional and structural in-formation is obtained on the various components (e.g. the component type, the com-ponent’s methods, the interface type). This information can be directly added to therun-time design model. Information on the container services that are required by thedifferent components is also collected. In section 5.3 we discuss how these servicescan be reconstructed using the component meta-data and run-time path informationand subsequently added to the run-time design model.

4.4 Related Work

In this section we discuss and compare related work to the work presented in thischapter. Firstly we outline how run-time paths have been previously used for a num-ber of different purposes. We follow this by discussing alternative representations fordynamic calling behaviour that have been previously presented and compare these torun-time paths. Finally we discuss tracing tools and techniques that are related to therun-time path tracing approach presented.

4.4.1 Applications of Run-Time Paths

In section 4.2.2 we have discussed a number of uses to which run-time paths havebeen put to, for example, reverse engineering, information extraction for antipatterndetection, or system comprehension. Run-time paths however have also been utilisedin a number of other ways.

Chen et al. [45] have applied run-time paths to deal with fault detection and diagnosisin large distributed systems. To detect faults they characterize distributions for nor-mal paths and look for statistically significant deviations that might suggest failures.To isolate faults to the components responsible (diagnosis) they search for correlations

13Java XML Libraries, http://java.sun.com/xml/

82

(using data mining and machine learning techniques) in the run-time paths betweencomponent use and failed requests. They have also applied run-time path analysis toassess the impact of faults and to help with system evolution. Traditional approachesto this problem have relied on static dependency models. However such dependencymodels often do not capture the dynamic nature of today’s systems. Chen et al. usedynamic tracing (i.e. run-time paths) to capture the dynamic nature of today’s con-stantly evolving internet systems.

The literature [19] presents the use of run-time paths to address performance issues.The Magpie project 14 collects run-time paths for distributed systems. It also measuresresource consumption. Magpie then builds stochastic workload models suitable forperformance prediction, tuning and diagnosis.

Another application of run-time paths (outside the area of system comprehension) canbe seen in recent work on autonomic monitoring [119]. Here Mos and Murphy intro-duce an adaptive monitoring framework whereby instrumentation is performed bylow-overhead monitoring probes which are automatically activated and deactivatedbased on run-time conditions. The approach makes use of dynamic models to ac-tivate and deactivate monitoring probes, such that the monitoring overhead is keptto a minimum under normal conditions. If a there is a sudden degradation in sys-tem performance monitoring probes are automatically activated to identify the bottle-neck component(s). The dynamic models are utilised to determine which monitoringprobes should be activated to identify the bottleneck. The dynamic models consist ofthe monitored components and the dynamic, ordered relationships between them (i.e.the dynamic models are equivalent to run-time paths).

4.4.2 Alternative Representations for Component Interactions

In this subsection we discuss alternative ways of representing component interactionsrecorded at run-time. The authors Jerding et al. [94] discuss a spectrum of representa-tions that can be used for this purpose, from the most accurate and space inefficient, tothe least accurate and most efficient. Dynamic call traces are said to be most accurateand space inefficient while call graphs are least accurate and space efficient [8] [94].A dynamic call trace is a recording of all calls and returns that occur in a program’sexecution. They capture important information such as the sequence of calls, the con-text in which a call was made, and repeating calls or loops. However extracting thecall trace may not be feasible for long running programs since they are unboundedand their size is proportional to the number of calls in an execution. Call graphs onthe other hand are space efficient but are less accurate. A call graph is a compact rep-resentation of calling behavior that summarizes all possible component relationships.While they are bounded by the size of the program, there is much interesting informa-

14Microsoft Research. Magpie. http://research. microsoft.com/projects/magpie/.

83

tion about calling behavior that is dropped to gain compactness. The sequencing ofcalls, the context in which certain calls are made, and repeated calls are all examplesof calling behavior that is lost. Call graphs are said to be context insensitive whilecall traces are context sensitive [8]. Context sensitive profiling associates a metric withsequence of executed calls. This can be used to separate measurements from differentinvocations of components [8]. Profiling tools that are context insensitive can onlyapproximate a program’s context dependent behaviour. In fact the inaccuracy of pro-filers such as gprof [79] and qpt [106] can be attributed to the fact that they used callgraphs to represent context dependent profile information in a context-independentmanner [141]. Another issue with call graphs is that they can contain infeasible pathsthat never actually occurred during execution. A calling context tree is an intermedi-ate representation that is more accurate than a call graph and less inefficient than thedynamic call trace in terms of space requirements. In fact, it is a more compact repre-sentation of the dynamic call trace and can represent all contexts in a given trace. Thecalling context tree discards any redundant paths in a trace while preserving uniquecontexts. Metrics from identical contexts are aggregated in order to improve spaceefficiency. Calling context trees are context sensitive and thus do not suffer from thesame inaccuracies that can occur with call graphs. The breath of a calling context treesare also bounded by the number of methods or procedures in an application. In theabsence of recursion the depth of calling context trees are also bounded [8].

As outlined in section 4.2.1 run-time paths are essentially dynamic call traces, thatcontains performance metrics, and that are recorded for each user request in a multi-user distributed application. They contain the same important information containedin dynamic call traces, that describes the calling behaviour of an application. How-ever they are also unbounded and thus can be space inefficient. To solve this issue aset of run-time paths can be easily represented by a calling context tree in the sameway that a dynamic call trace is [8]. An issue with calling context trees however isthe fact that the do not maintain the exact sequence of events as they occurred dur-ing execution. For this reason we have not used them in this work, since we requirethat the exact sequence of events is maintained to identify particular antipatterns (e.g.communication patterns discussed in section 6.1).

4.4.3 Run-Time Interaction Tracing Approaches

The closest run-time path tracing approach to our work is the pinpoint problem deter-mination tool. In fact we have made use of the pinpoint tracing approach in COMPASJEEM and have extended it such that it can be applied in a non-intrusive and portablemanner. Pinpoint has been discussed in detail and compared to our approach in sec-tion 4.2.5.2. Essentially its main drawback is that it instruments the middleware andthus it is not portable across different middleware implementations.

Webmon [82] is a similar tool that also instruments the middleware to trace run-time

84

paths. However as well as monitoring the server side components, webmon also in-struments client side components. This is performed used a cookie-based approachand allows for webmon to trace paths beginning in the browser, rather than on theserver side. Since instrumentation is performed on the client side this approach givesa more accurate measurement of a client’s actual wait time when a request is madeto the system. Server side only approaches record the amount of time that the sys-tem takes to process the request but does not record the time spent communicatingbetween the client and the system. The main drawback of the webmon approach isthat it is not portable. Another issue outlined by the authors is the fact that the instru-mentation process was very time consuming.

The magpie project [19] also monitors paths through distributed systems. The initialapproach [19] uses a unique identifier to identify requests and propagates this iden-tifier from one component to the next. A later version [18] avoids the need for globalidentifiers and instead correlates component events based on an application specificevent schema. Both versions make use of kernel, middleware and application level in-strumentation. The magpie project has been implemented for Microsoft technologies.

Another approach closely related to our work has been recently presented in the lit-erature [78]. The InfraRed tool 15 makes use of AOP to instrument the applicationand has the ability to capture run-time paths, (although it presents this informationin the form of a calling context tree). Similar to our approach, InfraRed makes useof a ThreadLocal object to record the sequence of calls. Their approach is potentiallyportable across different middleware implementations and is implemented using As-pectJ 16. It has also been shown to have a low overhead. However, the authors suggestan alternative approach to remote call tracking and do not marshal data across the net-work. Instead they simply relate the calls via the method name. Using this approachit is not clear how the order of calls can be maintained across remote calls when thesystem is under load.

Briand et al. have presented an approach for reverse engineering sequence diagramsfrom multi-threaded [34] and distributed java applications [35]. They record tracesusing an AOP monitoring approach. To distinguish between local threads they makeuse of a threadID identifier, however this information is not stored in a threadlocalobject. Instead a list of threadIDs are maintained in a text file. To allow for distributedtracing they marshall data across the network using aspects. Similar to our approachthey intercept remote calls such that information can be sent across the network suchthat one can identify the client method called that called a particular remote method.The information they send across the network however differs from the informationwe use. They make use of the identifier of the client node (clientNodeID), the threadidentifier (clientThreadID), the class name (clientClassName) and the object identi-

15InfraRed, http://infrared.sourceforge.net16The AspectJ Project, http://www.eclipse.org/aspectj/

85

fier (clientObjectID). However the approach essentially works in the same way sincetheir information is used to be able to identify the client thread responsible for remotecommunication. Their approach has not been applied to JEE applications.

JFluid [59] (now part of the netbeans profiler 17) is another tool which produces callingcontext trees. JFluid also makes use of a per thread data structure, which the authorscall a ThreadInfo object, to collect data for construction of the calling context tree.JFluid can perform dynamic bytecode instrumentation using a modified JVM. Toolusers can selectively profile their applications by selecting a number of root methods.Instrumentation is injected into all methods that are reachable from the root methodspecified by the user. This results in a reduced set of methods (i.e. a subgraph) that areprofiled. However an issue can potentially arise that could result in a distorted callingcontext tree. This issue comes from the fact that any method within a subgraph ofprofiled methods can be called from a non-profiled method outside of the subgraph.To make sure the calling context tree is not distorted, the instrumentation has to detectwhether a method is called in the profiled subgraph context or not. To achieve this theJFluid tool maintains the depth of calls within the subgraph (in the ThreadInfo object)to identify if calls are made to instrumented methods from non-instrumented meth-ods. Their approach is based on the same principles as our point of entry detectionmechanism, which also maintains the call depth in a per thread data structure (i.e. theThreadLocal object).

Hall [85] has implemented an approach to collect call paths in the form of the CPProfJprofiler. CPProfJ makes use of sampling to periodically interrupt a running appli-cation and records the runtime stacks of all threads currently in the system. Fromthe information recorded call path profiles are constructed. Call paths are usuallyrepresented by a list of methods enclosed in parentheses, e.g. (method1, method2,method3), and are defined as representing ”all stack traces that could be present wheneach of the calls within the call path is present (in call path order)” [84]. Call path pro-files report statistics about how much run-time can be attributed to the different setsof stack traces. Unlike with run-time paths however call paths are not end-to-end anddo not maintain the exact order of all events from the beginning to the end of a re-quest. They are thus not suitable for our purposes. In fact call paths have mainly beenapplied to address lower level issues than we focus on, such as inter-thread depen-dencies.

Finally data mining techniques have also been applied to extract dynamic dependen-cies from JEE systems. Agarwal et al. [1] use a data mining approach to extract re-source dependencies from monitoring data. The output is similar to that of COMPASJEEM. The approach is also non-intrusive since the data required to extract the depen-dencies is obtained from existing system monitoring data. Their approach relies onthe assumption that most system vendors provide a degree of built in instrumenta-

17The NetBeans Profiler Project, http://profiler.netbeans.org/

86

tion for monitoring. A major drawback of this approach however is that it is statisticaland unlike COMPAS JEEM it is not exact, and at higher loads the number of falsedependencies increase significantly.

87

CHAPTER

FIVE

Reconstructing the SystemsDesign for Antipattern Detection

In this chapter we discuss how we automatically extract the different relationships(that make up a reconstructed design model of the system) from the monitored data.In particular we detail how we automatically identify inter-component relationships,inter-component communication patterns and object usage patterns. In addition, weshow how run-time container services can be reconstructed and added to the designmodel and how our analysis approach summarises and reduces the amount of dataproduced during monitoring. A number of advanced analysis techniques are appliedto meet these aims. In particular we apply two techniques from the field of data min-ing, i.e. Frequent Sequence Mining (FSM) and Clustering. FSM is applied to run-time paths to identify interesting component communication patterns, for example,resource intensive loops. Issues related to applying this technique to run-time pathsare discussed and solutions are provided. Clustering and statistical analysis tech-niques are applied for the purpose of data reduction. Following this we describe therun-time design model of the system that captures the information extracted duringanalysis. Finally, we discuss and compare related work.

88

Main Points

• Dynamic component dependencies and component object usage informationcan be automatically extracted from runtime paths.

• Run-time container services can be automatically reconstructed from run-timepath information and component meta-data.

• FSM can be used to automatically identify component communication patternsin run-time paths. However, applying FSM to run-time paths creates scalabilityissues in terms of both the FSM output and the algorithm run-time.

• FSM with non-overlapping support counting can be applied to allow for a morehuman readable output of the FSM algorithm.

• FSM can be applied to identify both frequent sequences and resource intensivesequences.

• Preprocessing techniques can be applied to run-time paths to reduce the run-time path lengths an ultimately the FSM algorithm run time.

• All run-time paths recorded during monitoring can be clustered into groups togive a concise method level and component level view of the system.

• The extracted run-time design model contains the following information:

– component dependencies

– component communication patterns

– component object usage patterns

– component structural and functional information

– reconstructed container services

– run-time server resource information

89

5.1 Chapter Overview

In this chapter we discuss how we automatically extract the different relationships(that make up a reconstructed design model of the system) from the monitored data.In particular we detail how we automatically identify inter-component relationships,inter-component communication patterns and object usage patterns. In addition, weshow how run-time container services can be reconstructed and added to the designmodel and how our analysis approach summarises and reduces the amount of dataproduced during monitoring.

More specifically section 5.2 outlines how the run-time component relationshipsand component object usage patterns can be extracted from the monitoring data bytraversing the run-time paths collected. In section 5.3 we detail how the run-time con-tainer service boundaries can be automatically reconstructed by analysing the run-time paths and the component meta-data. The output of this analysis shows how thedifferent container services are being used at runtime by the application components.The automatic extraction of component communication patterns requires more com-plex analysis than the analysis techniques in sections 5.2 and 5.3. For this purpose wemake use of a data mining technique, frequent sequence mining (FSM). In section 5.4we discuss how we applied FSM to run-time paths, to identify sequential patterns ofinterest (e.g. frequent sequences, resource intensive sequences) within the data. Weoutline how the FSM implementation was modified to identify sequences of interestand to allow for a more human readable output from the algorithm. Algorithm run-time performance issues that arise with our approach are also outlined and a numberof preprocessing techniques are described that can be used to alleviate these problems.Furthermore in this section we also discuss how postprocessing techniques can be ap-plied to give more value to the FSM output. Section 5.5 shows how the amount of dataproduced during monitoring can be significantly reduced by applying clustering andstatistical analysis techniques. This summarised data can be used to further enhancethe design model. The penultimate section (5.6) in this chapter presents a descriptionof the reconstructed design model of the system, and the information that it captures.Finally section 5.7 compares the work presented in this chapter to related research inthe areas of reverse engineering and data mining.

5.2 Automatically Extracting Component Relationshipsand Object Usage Patterns

Run-time paths capture the run-time behaviour of an application and can be easilyrepresented in a graphical format. Figure 4.12 on page 78 shows a run-time path con-verted to a UML sequence diagram which captures the relationships between the dif-ferent components for a given user action. Run-time paths are represented at a code

90

Figure 5.1: Run-time Design Meta Model from Chapter 3

level by a tree like data structure. A root node represents the first component in thepath, which has an ordered list of callees. Each callee is itself a node which can alsohave an ordered list of callees. The run-time path structure can be traversed to identifythe different component relationships that exist within it. The run-time path structureis traversed by visiting each node as it appears in the path. For each node along thepath we identify the direct caller of this node and the node’s callees. By analysingall run-time paths collected we can build up a collection of all the component rela-tionships that existed during execution (of a particular test run). This informationcan be represented in a UML class diagram which shows the overall system architec-ture of the executed components and 7.3). During analysis instances of a Componentdata structure are created which contain this information. The Component data struc-ture also contains information extracted from the components’ deployment descrip-tors (see section 4.3.2). The instances of the Component data structure are added to therun-time design model. The run-time design meta model diagram which describesthe run-time design model from chapter 3 is shown again in this chapter in figure 5.1for convenience.

91

Next we give an example run-time path and example deployment descriptors for thecomponents in the path. We show the information that would be extracted duringanalysis of this run-time path. Figure 5.2 (a) gives a sample run-time path and figure5.2 (b) gives an extract of the deployment descriptors of the components that makeup the run-time path (note there is no deployment descriptor for the database compo-nents). The run-time path contains three different components. For each componentan instance of the component data structure (figure 5.2 (c)) is created. The componentdata structure contains information such as the component name, type, list of calleecomponents, list of caller components and a list of the executed component meth-ods. Figure 5.2 (c) gives an extract from a Java implementation of the data structure,omitting details related to constructors and accessor methods (note all data structurespresented in this chapter are presented in this way). Two tasks are performed to pop-ulate this data structure: (a) the component type and component name are extractedfrom the deployment descriptor and (b) the run-time path data structure is traversedto identify a component’s callers, callees and executed methods. The PAD tool usesrecursion to traverse the run-time path data structure. Figure 5.2 (d) shows the infor-mation which would be extracted for each component from the sample run-time path(a) and deployment descriptors (b). By repeating this process for all run-time pathsrecorded a picture of all the component relationships that existed during the test runcan be built up.

Object usage patterns can also be identified by traversing the run-time paths (pro-duced using COMPAS BCI, see section 4.2.5.8). For each object type that we track,we can mark any points along the path where an instance of this object is (a) createdand (b) accessed. This information can be stored in a TrackedObject data structure. Aninstance of this data structure contains information on the object type, a list of thecall paths where it has been created and accessed, a corresponding list of the objectmethods accessed in each path and (depending on the granularity of the informationrequired) the points/positions along the path at which the objects were accessed andcreated. A diagrammatic representation of this information is shown in figure 4.12 onpage 78.

Next we give an example run-time path where an object is tracked and we showthe information that is extracted and added to the run-time design model. Figure5.3 (a) gives a run-time path where the AccountDetails object is being tracked. Foreach tracked object type an instance of the TrackedObject data structure (figure 5.3 (b))is created. The TrackedObject instance is populated by traversing the run-time pathdata structure recursively. Each node along the path is visited and counters are in-cremented where instances of the tracked object type are accessed or created. In factindividual counters are maintained for each different accessor method. Figure 5.3 (c)gives the data that would be extracted for this example. When the run-time path istraversed the tracked object name and path ID are identified and the total number ofaccesses and creations are counted. Also for each accessor method the method name

92

Figure 5.2: Example Run-Time Path (a), Example Deployment Descriptors (b), Extractof Component Data Structure (c) and Data Extracted to Populate Data Structure (d)

93

Figure 5.3: Example Run-Time Path (a), Extract of the TrackedObject Data Structure(b) and Information Extracted to Populate the TrackedObject Data Structure (c)

94

Figure 5.4: Example Run-Time Path (a), Extract of the RunTimeContainerService DataStructure (b), and Information Extracted to Populate the RunTimeContainerServiceData Structure (c)

95

is recorded along with the number of corresponding accesses made to this method.The above information is added to the TrackedObject instance, which maintains a listof objects, where each object in the list contains tracking statistics for a given run-timepath (RTPStatsList).

5.3 Reconstructing Run-time Container Services

Contextual component frameworks provide services to components at run-time (e.g.checking and ensuring that component boundary constraints are being met). In EJBsuch boundary constraints include security restrictions and transactional isolationchecks. For example, an EJB component method may have the following transactionalattribute: (transaction) Required (i.e. the method will participate in the client’s trans-action, or, if a client transaction is not supplied, a new transaction will be started forthat method). Such attributes are defined during deployment, (specified in the XMLdeployment descriptor) and do not change during run-time. By analysing the differ-ent component attributes, along with run-time paths, it is possible to reconstruct thedifferent container checks as they occur along the paths. For example, by analysingthe transactional attributes of each component method along a particular run-timepath, one can easily reconstruct the transactional boundaries (i.e. where transactionsbegin and end) along the path. This information can be used by developers to easilyidentify how the container services are being used by their application. Since a highproportion of the application run-time can be spent executing such container services[9] it is important that the services are utilised in an efficient manner. Inefficient use ofsuch services can lead to well known antipatterns (e.g. the large transaction antipat-tern [61]). A RunTimeContainerService data structure is created during analysis whichcontains information in relation to the reconstructed services. The information can in-clude the service type, the path ID in which the service occurred and the service startand end points along the path, as well as the methods that make use of the service.

Next we give an example run-time path and the related component service require-ments. We show how the run-time service is reconstructed and the information that isextracted and added to an instance of the RunTimeContainerService data structure. Fig-ure 5.4 (a) gives an example run-time path and corresponding transactional require-ments for each EJB component method in the path. The transactional requirementsare extracted from the XML deployment descriptor. For each new run-time containerservice reconstructed an instance of the RunTimeContainerService data structure (figure5.4 (b)) is created. To populate the data structure the run-time path data structure istraversed and the required information extracted. Figure 5.4 (c) gives the informationthat would be extracted for the example run-time path. When a service is identi-fied its start and end points are recorded, along with the methods within the serviceboundaries. This information along with the path ID is added to the RunTimeContain-

96

erService instance, which maintains a list of objects, where each object in the list con-tains run-time container service statistics for a given run-time path (RTPStatsList). If areconstructed container service is identified which is equivalent to one which has al-ready been identified (and a RunTimeContainerService data structure has already beencreated) a new instance of the RunTimeContainerService data structure is not created.Instead the reconstructed container service statistics are added to the instance that hasalready been created. Two reconstructed container services are equivalent if the con-tain the same sequence of methods in the MethodList, are of the same Length andboth are of the same ServiceType.

5.4 Identifying Component Communication Patterns inRun-Time Paths using Frequent Sequence Mining

Identifying component relationships allows for the construction of class diagramswhich show a high level architecture of the application, as in figure 5.5. What is notclear from the class diagram in figure 5.5, however, is the communication patterns orfrequency of calls between the different components in the system. This type of infor-mation is often required when trying to identify particular performance issues in theapplication. It is important to be able to identify parts of the application where thereare high or unusual levels of communication between components as knowledge ofsuch patterns can lead to opportunities for optimizations (see section 7.3). Commu-nication patterns between components (e.g. long running loops) can be identifiedby manually analysing run-time paths. An issue with manually analysing run-timepaths from enterprise applications, however, is that (a) the paths can be very longand (b) there may be a large number of different paths that exist for a given system.Rather than having to analyse the paths manually, it is desirable to be able to performautomatic analysis, such that the amount of data to be examined by developers canbe reduced. From a performance design perspective we are interested in finding fre-quently occurring method calls across the run-time paths that might suggest instancesof potential performance design flaws in the application. That is, frequent sequencesof method calls that are resource intensive. Identified design flaws can be refactoredor optimised such that the overall system performance can be improved.

5.4.1 Frequent Itemset Mining and Frequent Sequence Mining

Frequent Itemset Mining (FIM) [2] is particularly suited to finding patterns in trans-actional data. Transactional data in the data mining context refers to a database oftransactional records. For example a database of different customer shopping trans-actions on a given day (known as market basket data) or a database of individualsbanking transactions. In relation to this research a transactional database refers to a

97

Figure 5.5: Class Diagram Showing Components Relationships

database of run-time paths. However it is important not to confuse a transaction inthe data mining context with the meaning of a transaction in the computer softwarecontext [148]. Figure 5.7 shows a transaction in the data mining context with 9 items.For the purpose of this research each item represents a component method call in arun-time path. However the item could as easily be used to represent other informa-tion such as shopping items in a customer’s basket. A transactional database consistsof numerous such transactions.

An issue with FIM in relation to applying it to run-time paths is the fact that it does nottake the order of items in the paths into account. Since a run-time path maintains theorder of the events that constitute it, it is important that our analysis technique alsorespects this order. Mining frequent item sequences [3] in transactional data considersthe order of the items in the transactions and thus is more suited to finding patterns inrun-time paths. FSM is a general case of FIM. FIM is concerned with discovering allfrequent itemsets within a transactional database. Most FIM algorithms work on thefollowing principle: an itemset X of variables can only be frequent if all its subsets arealso frequent (i.e. the downward closure property). Using this principle the generalapproach is to find all frequent sets of size 1. Assuming these are known, candidatesets of size 2 can be generated (i.e. sets {A, B} such that {A} is frequent and {B}

98

is frequent). The frequency of the candidate sets can then be calculated by scanningthe database. Infrequent candidate sets are removed. This gives the frequent setsof size 2. Next the candidate sets of size 3 can be constructed and their frequencycalculated. This process can be repeated until no candidate sets are generated. This isknown as the Apriori algorithm [2]. Calculating the frequency of a candidate itemsetis referred to as determining the support count of the candidate. The support count ofthe candidates are calculated by reading in each run-time path from the database anddetermining if each candidate is contained within the run-time path. Where this is truethe support count is incremented. For item sets the containment relation correspondsto the set inclusion (⊆) relation. In the case of item sequences, a sequence 〈a1, a2....an〉is contained in another sequence 〈b j1, b j2....b jm〉 if there exists integers 1 < j1 < j2 <

... < jn <= m such that a1 = b j1, a2 = b j2, ...an = b jn [3].

5.4.2 Support Counting for Run-Time Paths

The support counting performed in FSM is useful for identifying frequent sequencesthat occur across different transactions. However an issue with this approach is thatit does not take into account sequences which might repeat within a given transac-tion i.e. if a sequence appears twice in a particular transaction the support count willhave only been incremented by 1. In some situations it can be useful to also take intoaccount repeating sequences within the same transaction. This is especially the casewhen applying FSM to run-time paths since often we would like to be able to iden-tify repeating sequences of method calls within a given run-time path (e.g. loops).Weighted support counting can be applied to increment the support of an item, suchthat for every instance of the sequence within a given run-time path the support isincremented. A consequence of this is that the support count performance sufferssignificantly (see section 7.3.1). With non-weighted support the support count scanseach run-time path and stops as soon as it finds the item sequence. With weightedsupport however each run-time path is searched completely for all instances of theitem sequence.

Another issue that arises from using weighted support is that the output generatedcan be difficult to comprehend. This results from the fact that, with weighted supportcounting, the support is incremented for every instance of the sequence that occurs ina given transaction. This problem can be clearly seen from a simple example. Figure5.6 shows a single transaction containing 9 items. For the purposes of the example wegive the non-weighted support count and weighted support counts for this transac-tion in respect to the item sequence 〈1, 2, 3〉. For non-weighted support the supportcount value is 1, whereas for weighted support the value is 10. The weighted supportcount can be represented mathematically as being at least: (l+(n−1))!

(n−1)!(l)! where l is the se-quence length and n is the number of non-overlapping sequences in the transaction.Figure 5.6 shows the different sequences that are counted as part of the weighted sup-

99

Figure 5.6: Example Transaction with Different Support Counting Approaches

port. The figure visualises the sequences as they appear in the transaction by repeat-ing the transaction with the sequences highlighted in different colours. The bracketednumber that follows each transaction indicates the number of sequences that we visu-alise per line. Clearly it would be difficult to visualise all sequences without repeatingthe transaction. Furthermore mentally constructing the different sequences that arecounted using weighted support can be a cumbersome and confusing task. This iseven more evident when either the size of the sequence grows or when the number ofnon-overlapping sequences within the transaction increases. For example in transac-tion 〈1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5〉 the item sequence〈1, 2, 3, 4, 5〉 has a weightedsupport count of 21. Relating the weighted support count to the different sequences asthey appear in a transaction can be particularly difficult when applying FSM to run-time paths from real enterprise applications where individual transactions may havethousands of items.

To address this issue we introduce the notion of non-overlapping weighted support.Non-overlapping weighted support works by incrementing the support count fornon-overlapping sequences only. This approach improves upon the weighted sup-port counting in two main ways: (a) the performance of the algorithm improves fordatabases with long transactions (see section 7.3.1) and (b) the output of the algorithmis more easily understood by somebody trying to interpret the results (without infor-mation loss). For instance, in the example given in figure 5.6 the weighted supportfor the sequence 〈1, 2, 3〉 is 10 where as the the non-overlapping weighted support is

100

3. While relating the 10 instances of the sequence 〈1, 2, 3〉 to their actual occurrencesin the transaction can be difficult, for non-overlapping weighted support this taskcan be quite simple. Figure 5.6 shows the non-overlapping sequences in the transac-tion. Since the sequences do not overlap humans can easily visualise them. The non-overlapping weighted support counting works in a similar way to weighted supportcounting. For weighted support counting the transactions are read in one at a time.Each candidate sequence is compared with the transaction. For every instance of thecandidate sequence that is found in the transaction the support is incremented. Fornon-overlapping weighted support we prevent the support count being incrementedfor overlapping candidate sequences by keeping track of the start and end position ofthe last candidate sequence that was identified within the transaction. The supportcount is only incremented where the start position of the current candidate sequenceis greater than the end position of the previous candidate sequence, that the supportcount was incremented for.

We say there is no information loss since from the non-overlapping weighted supportcount output one can easily work out the value of the weighted support count by cal-culating the following equation: (l+(n−1))!

(n−1)!(l)! where l is the sequence length and n is thenon over-lapping weighted support. In situations where a sequence within the trans-action has within it a duplicate element (or elements) of the candidate sequence therewill be some information lost, insofar as the real weighted support will be higher than(l+(n−1))!(n−1)!(l)! depending on at what position along the transaction the duplicate element(s)

occur as in figure 5.7. We call these elements hidden elements.

Figure 5.7: Hidden Elements in Transaction and Corresponding Support Counts

101

5.4.3 Further Criteria for Interestingness

In general for FSM the interestingness [87] of a sequence is directly proportional to thefrequency of the sequence. However when analysing run-time paths from a perfor-mance perspective we are most interested in finding frequent sequences of methodsthat consume a high amount of system resources. In this situation the criteria for in-terestingness is not the frequency of the sequence but is in fact the amount of systemresources consumed by the methods that make up the sequence. In order to accountfor the resource consumption per method we can add this information to the algo-rithm input file such that the transactional database contains not only the item IDs butalso their corresponding resource consumption information (since resource consump-tion per methods is collected during monitoring and contained within the run-timepaths). This information can be utilised during the support counting stage to influ-ence the outcome of the FSM algorithm. During support counting, rather than sim-ply incrementing the support count, we instead add the resource consumption valuesof the methods in the candidate sequence to the support counter. This weights thesupport count value by the resources consumed such that sequences that consume ahighest amount of resources are found. Section 7.3.1 gives results of applying this ap-proach to run-time paths and shows how it can be used to quickly identify frequentlyoccurring resource intensive sequences. An issue with this approach however is thatthe downward closure property is invalidated since there may be instances where anitem sequence may be frequent even though some of its sub sequences are not [159].The literature [159] suggests a solution to this problem using the weighted downwardclosure property for weighted association rule mining. Future work will investigatethe applicability of this approach to FSM. Our results show that even though we in-validate the downward closure property the approach can be effective in finding se-quences of resource intensive methods that occur across the transactions (see section7.3.2).

5.4.4 Preprocessing for FSM Performance Improvement

In order to apply FSM to run-time paths it is required to convert the run-time pathsfrom their text format to a flat format or representation (which is used by the FSMimplementation [32], described in section 7.3). This conversion involves assigninga unique ID to each distinct method in the run-time paths and listing these IDs ina flat format in the same order as they would appear in the run-time path. How-ever for enterprise systems the run-time path length can be very long (in the order ofthousands of items). Run-time paths of this size can have a major impact on the algo-rithm performance particularly in relation to FSM implementations that make use ofweighted support or non-overlapping weighted support counting approaches. In or-der to reduce the running time of the algorithm we applied a number of preprocessing

102

techniques to reduce the run-time path lengths.

5.4.4.1 Using Parental Information to Split Run-Time Paths

The flat run-time path file contains the method IDs that make up the call path. From aflat run-time path one can determine the calls that make up the path and the sequenceof the calls. Information is lost in the translation from a run-time path to the flat for-mat, however, since we can not determine the caller (or parent) of each method in theflat run-time path file. This information is clearly available in the original run-timepath. Parental information is interesting from a performance analysis point of view,since frequent methods that have the same parent often provide opportunities for op-timization (e.g. loops, since every method in a particular loop has the same parent).In order to be able to identify such patterns in the data easily we can add the parentalinformation to the flat run-time paths such that it is available in an intermediate rep-resentation. This is performed in a preprocessing step. During this process each itemin a flat run-time path is augmented with the position of his parent (or caller) method.This information can be utilized, before applying the FSM algorithm. Since we are in-terested in method sequences with the same parent, at the preprocessing stage we cansplit the run-time paths in the intermediate representation into a number of smallersubpaths by their parent information. i.e. for each parent we create a new subpaththat contains all the children of the parent. The FSM algorithm can then be applied tothe subpaths to identify frequent or interesting sequences with the same parent. Thishas two main advantages: (a) it allows us to quickly focus on sequences of methodsthat have the same parent and which we are particularly interested in and (b) it re-duces length of each run-time path significantly which improves the performance ofthe search (see section 7.3.1). A downside to this approach is that frequent sequenceswith different parent methods are not identified.

If it is required to keep a list of the run-time path IDs that a frequent sequence ap-peared in, it is possible to reconstruct the split run-time paths after the algorithm hascompleted and to list the frequent sequences along with the run-time paths in whichthey appear. Maintaining such a list is a common feature of mining algorithms andis also very useful in run-time path analysis where a developer may want to quicklyanalyse the run-time paths in which the frequent or resource intensive sequences oc-curred.

5.4.4.2 Consecutive Item Replacement

Consecutive Item Replacement (CIR) can be used to further reduce the run-time pathlengths. CIR is an iterative approach whereby we perform a number of runs of theFSM algorithm. The first run of the FSM algorithm is aimed at finding a small numberof the most frequent sequences in the database. The amount of time spent in the first

103

run is generally quite short and can be achieved by either setting a high support count(close to the min. support count for the most frequent sequences of size 2 or larger)or by limiting the size of the sequences that are generated to a particular length. Thenext step is to identify if any of the frequent sequences found, in the first run, occuras consecutive sequences in the database. If this is true the consecutive sequences canbe replaced in the database by a new item ID (replacement item) which we use to rep-resent this sequence in the next iteration of the FSM algorithm. Replacing frequentconsecutive sequences reduces the length of the run-time path and as a result the nextiteration of the FSM algorithm will be faster than the last. This iterative approach canbe repeated until either an acceptable performance run-time (for the FSM algorithm)is attained or until no further improvements can be made. The performance improve-ment depends on a number of factors such as the run-time path length, length of theconsecutive sequences replaced, number of consecutive sequences replaced, databasesize etc. This approach is especially effective in databases of run-time paths wherelong running loops may appear. On completion of the final iteration of the FSMalgorithm any replacement items that exist in the algorithm output can be replacedwith the original consecutive sequences to give the real frequent sequences that occurwithin the data.

5.4.5 Closed Sequences

In some instances a large number of frequent sequences can be generated as a resultof the FSM algorithm (see section 7.3.1). Ordering the generated sequence by theirsupport value will however highlight the most significant sequences (i.e. the mostfrequent or resource intensive). Often however a sequence will have the same supportas many of its subsequences, and thus they will all be presented together, if orderedby support. This gives much redundant information which can limit the value of theresults produced. By identifying closed sequences only this problem can be avoided.

Closed Mining has been mainly applied to the FIM problem. An Itemset I is closed ifno proper superset of I exists that has the same support as I. The set of closed itemsetsis a compact representation of the frequent itemsets. All frequent itemsets togetherwith their supports can be generated if only the closed itemsets and their supportsare known. Closed Itemset Mining can significantly reduce the number of sequencesoutput and avoids the problem we outlined above. It has been shown that closeditemsets can be generated in a post processing step [181]. However, more efficientapproaches have been presented which modify the FIM algorithm (e.g. [182]). Closedmining can also be applied to sequences [180]. Our plans for future work involveinvestigating these approaches such that they can be applied to run-time paths.

104

5.4.6 PostProcessing

A number of postprocessing techniques can be applied to the data produced fromFSM. Postprocessing can be applied to give more value to the sequences such thatinteresting sequences can be more easily identified. For example the confidenceof a sequence (rule) can be calculated as follows sequence: confidence (s1 =>

s2) = support(s1) ∪ support(s1)/support(s2) [40]. The confidence, C, of the se-quence rule above holds true if: C% of run-time paths that contain s1 also contains2. The confidence of a rule is sometimes referred to as its accuracy. This infor-mation is particularly useful when analysing design of enterprise systems since itcan be useful to identify methods that appear together with a certain level of confi-dence. For example if we have remote method A and remote method B, and the rule:support(MethodA) ∪ support(methodB)/Support(a) = 100%, we can be sure (with aconfidence of 100%) that method A and method B will always occur together. Seeingas A and B are both remote methods and seeing as they occur together, there may bean opportunity to combine these methods into a single more coarse grained remotemethod call and thus cut down on network traffic.

Another postprocessing technique that can be applied, in relation to run-time paths,is adding parental information, which is contained in the paths, to the identified se-quences. For example parent components and methods of the sequence can be iden-tified, i.e. components and methods that called the sequence in the run-time paths.This information can be stored along with the sequence for future use.

5.4.7 Component Communication Information for the Extracted DesignModel

In chapter 7 we show how the information extracted by the FSM algorithm can also beused for automatic identification of performance antipatterns. For automatic antipat-tern detection the analysis module creates a FrequentSequence data structure whichcontains information relating to the frequent sequences identified in the run-timepaths (see figure 5.1). An instance of the data structure contains the path IDs of thesequence (i.e. in what paths the sequence occurs), the items (i.e. the component meth-ods) that make up the sequence, the parents of the sequence, the support of the se-quence (i.e. how often the sequence occurs, or how much resources were consumed).The support of the sequence can be broken down further to reflect how often the se-quence occurs in the different run-time paths. Confidence rules can also be createdwhere appropriate (e.g. for sequences containing remote methods).

105

5.5 Data Reduction

The amount of data produced during monitoring large scale enterprise applications isoften too large for easy human analysis. Thus it is required that the data be reducedand summarised such that it can be presented to developers and system testers in amore meaningful format. Summarising the data also allows for the data to be moreeasily integrated with a design model.

5.5.1 Clustering Run-time Paths

Run-time paths collected from enterprise applications can often be too long, or theremay be too many, for easy human analysis. Considering the number of software com-ponents and sub components in an typical enterprise application (this can be in theorder of hundreds), the number of paths that can be taken through the system is gen-erally quite large. Similarly the length of a run-time path corresponding to a partic-ular user action can also be very long (since a large number of component methodcalls may be needed to service a user request). The issue of path lengths is somewhataddressed through frequent sequence mining, since repeating sequences in a givenpath can be identified and represented in a more concise manner (see loops in figure4.12 on page 78). To address the issue of having a large number of paths we can applyanother data mining technique called clustering [87]. Clustering is the classificationof similar objects into different groups, or more precisely, the partitioning of a data setinto subsets (clusters), so that the data in each subset share some common traits.

Although there can be a large number of unique paths through the system many ofthese paths can be similar and may be constructed of many of the same sub-paths.Using basic clustering analysis we can reduce the number of paths significantly intocommon path groups. In section 7.3.3 we show how this can be achieved for run-timepaths, collected from monitoring a JEE system, when we cluster paths together that (a)call the same component methods and (b) make use of the same components. Duringanalysis we create a Cluster data structure (see figure5.1). An instance of a Cluster datastructure contains all the run-time path IDs that belong to a particular cluster. Thisdata structure also contains information relating to the clustering criteria (e.g. for (a)above the data structure contains the list of component methods that are called by therun-time paths in a particular cluster). Instances of the cluster data structure can beadded to the design model. This allows a developer to see (a summary of) the differentpaths that are taken through the system (e.g. at a method or component level), withoutthe need to analyse all run-time paths recorded.

106

5.5.2 Statistical Analysis

A wide range of performance metrics can be collected when monitoring large enter-prise applications. For example our monitoring module can collect performance met-rics such as component memory usage, component CPU usage, and a wide range ofserver resources (object instance pools, thread pools, instance caches). During a testrun the amount of information that is collected can be large, especially when a systemis under heavy load. Statistical analysis can be used to summarise resource usage andserver resource information. For example for each method we can get the averagemethods response time/CPU usage/memory usage, maximum and minimum val-ues, as well as the standard deviation. The same analysis can be applied to queues forserver resources, e.g. object pool sizes, database connection pools. The data structurescreated during analysis can be enhanced with statistical values for the componentmethods/server resources that they contain. The statistical values are calculated byanalysing the performance metrics collected with the run-time paths and by analysingthe server resource usage information that is collected.

Although the JEE Management Specification provides for a Performance Data Frame-work (see section 4.3.1) which mandates how server resources should be collectedand presented, in practice statistical information is often not available. Thus, we arerequired to perform this analysis for server resources.

5.6 The Reconstructed Design Model

The output from the analysis module is a detailed design model that captures the re-lationships and patterns identified during analysis, as well as the reconstructed con-tainer services, path clusters and statistical information. We call this a run-time designmodel since it captures a snapshot of system design decisions that are realised as thesystem executes during our test run. Figure 5.1 gives a diagram of the different datastructures that are contained in the design model. Figure 5.1 contains 8 entities withthe following relationships: A Component entity can call zero or more other Compo-nents, is made up of zero or more component Methods and can be associated with oneor more object Pools. A RunTimePath entity is made up of one or more componentMethod calls. The RunTimePath can contain zero or more FrequentSequences (made upof a sequence of one or more Methods) , the FrequentSequences can in turn belong to 1 ormore RunTimePaths. Likewise a RunTimePath can contain zero or more TrackedObjectsand TrackedObjects can be tracked in one or more RunTimePaths. A RunTimePath canbelong to a single Cluster and a Cluster is made up of one or more RunTimePaths. Fi-nally, a RunTimeContainerService consists of one or more Method calls within particularservice boundary constraints (determined by meta-data) and can be present in one ormore RunTimePaths.

107

The extracted design model gives two main views into the system. A transactionalview and a hierarchical view. The transactional (RunTimePath) view of the systemshows the main paths (Clusters) through the system, the most frequent communicationpatterns along these paths and how particular (Tracked) data objects are being createdand used along the transactions. From the transactional design view one can alsodetermine the container level services that are used along the different transactions.A more abstract hierarchical view of the system can also be obtained from the designmodel, by analysing the Component details and the component callers and callees. Thisshows the different component types and relationships that make up the system. Themodel also shows how the components are being made available by the underlyingcontainer through the different container pools.

5.7 Related Work

In this section we discuss and compare related work. We focus on two related areas inparticular, reverse engineering and data mining.

5.7.1 Reverse Engineering

Our work applies analysis to reverse engineer the system’s run-time behaviour andstructure from data collected by our monitoring module. Our approach mainly makesuse of dynamic data to extract the required information for design recovery. In thissubsection we outline and compare a number of related approaches.

Our research is closely related to work presented by Schmerl et al. [149]. In this litera-ture the authors present an approach for architecture recovery using runtime analysis.Their approach requires an engineer to create a mapping specification between lowlevel and architecture events. A run-time engine takes as input the mapping specifica-tion and monitoring events from a running system and produces architecture events.Finally, the architecture events are used to automatically create an architecture de-scription. The work focuses on identifying architectural structure and does not focusheavily on architectural behaviour. The authors have applied their approach to dis-cover the architecture of an EJB application. A number of drawbacks exist howeverfor this approach. Firstly, an effort is required by an engineer to develop the mappingspecification. For example it took an expert in their mapping specification languagethree days to construct a mapping for a sample EJB application. The engineer is re-quired to understand the format of the monitoring data, and the workings of the ap-plication to be analysed. Furthermore, in the case of EJB applications, the engineerrequires an understanding of the application server on which the application is run-ning. Secondly, the approach is not portable as a new specification is required perapplication, albeit the authors claim only minor modifications of the specification are

108

required for applications running on the same middleware. For example, in the caseof the EJB application, they claim that the same specification could be easily modifiedwith little effort such that it could be applied to another EJB application running onthe same application server. However this would not be the case if the system wasrunning on an alternative application server. Thus the approach is less portable acrossdifferent middleware implementations. However, a key advantage of this approach isthat it can be applied to any system that can be monitored to provide low level datasuch as object creation, method invocation and instance variable assignment.

Briand et. al present work on reverse engineering sequence diagrams from distributed[35] and multi-threaded [34] java applications. Their approach, similar to our analysismodule, is based on populating instances of a meta model with information collectedby monitoring a running system (their monitoring approach has been discussed insection 4.4). More precisely, the data collected at run-time is used to create an instanceof a trace meta model which in turn is transformed into an instance of a scenario metamodel. The scenario meta model is an adaption and simplification of the UML se-quence diagram meta model. The output of their approach is scenario diagrams. Theyhave performed a case study to show how reverse-engineered scenario diagrams canbe used to check the quality of an applications implementation and its conformance tothe intended design. Our monitoring tool, COMPAS JEEM, produces a similar output(in the form of run-time paths), that when represented diagrammatically results in asequence diagram. We use this information (along with collected performance metricsand component metadata) to populate an instance of our run-time design meta modelwhich gives a structural and behavioural view of the overall system design from a per-formance design perspective. Their approach also addresses the issue of identifyingrepeating sequences in a trace as a result of loops by instrumenting the control-flowstructures. On the other hand our approach makes use of FSM to identify repeatingpatterns. An advantage of their approach is that they can distinguish the executionof loops from incidental executions of identical sequences in different contexts. Ourapproach can not make this distinction, however, it is less intrusive since it does notrequire such low level instrumentation.

Static and dynamic data has been applied by Richner and Ducasse [146] to recoverhigh-level views of the system architecture through user defined queries. Both thedynamic and static information is modelled in terms of Prolog facts. They provide alogic layer made up of rules above the database of facts to allow sophisticated queriesabout the structure and behaviour of the application. The main advantage of theirwork is that it allows an engineer to drive the investigation of the application throughan iterative process, much like Murphy’s static relexion approach [122]. In compari-son our approach makes use of mainly dynamic information. Although source codeanalysis is not applied, we also make use of component meta-data to obtain structuraland functional information on the different components. We also extract informationpertaining to the components’ required container services.

109

The reconstruction of container services using component meta-data and, what theauthors call, binding graphs has previously been applied by Trofin and Murphy [166]to identify redundant boundary checks within contextual container services. Bindinggraphs (similar to call graphs) contain component dependencies and can be collectedstatically or dynamically. We reconstruct the container services to identify antipat-terns rather than for redundancy checking, and in contrast to their approach our re-construction process requires the order of calls to be maintained. Thus we make useof run-time paths rather than binding graphs for our reconstruction process.

5.7.2 Data Mining

In this thesis we have applied both FSM and clustering to data produced from moni-toring component based enterprise applications. FSM is applied to identify interestingcomponent communication patterns. We have modified the support counting tech-nique of the FSM algorithm to identify repeating sequences within a given run-timepath such that patterns as a result of long running loops, for example, can be identi-fied. Previous work has addressed this issue of counting the support of repeating se-quences within data using a sliding window approach [111]. However this approachhas been applied to event streams and is not suitable for data that fits naturally intodistinct transactions (e.g. run-time paths).

We have also weighted the support counting to take into account performance metrics(and not merely frequency) such that resource intensive patterns can be identified.The idea of weighting support count to take account of alternative criteria other thanfrequency has been previously been applied to association rules in the literature [159].Other examples of weighting the support with criteria other than frequency include[107] and [109] where the support is weighted by time values.

In this work we introduce the notion of non-overlapping weighted support. Non-overlapping weighted support allows for a more human readable output of the FSMalgorithm when repeating sequences within a single run-time path are accounted for.Similar to our non-overlapping weighted support approach, mining frequent closedsequences can produce more compact results [173]. In fact the sequences producedusing our non-overlapping weighted support approach can be further compacted byfiltering the results produced for closed sequences only. Maximal sets are a subset ofall closed sets and maximal mining techniques can be applied to reduce the outputvolume further [41].

To reduce the run-time of FSM when applied to long run-time paths we have ap-plied a number of preprocessing techniques. In particular we have made use ofparental information to split the run-time paths into shorter subpaths. We have alsoapplied a technique called Consecutive Iterative Replacement (CIR) which replacesfrequent consecutive item sequences with replacement items to reduce the run-time

110

path length. Reducing the run-time of sequential pattern mining algorithms by takingadvantage of consecutive sequences has been previously applied and documented inthe literature [108]. The authors propose an extension of the SPADE algorithm for FSMcalled GO-SPADE. One advantage of our Consecutive Item Replacement approach isthat it works by modifying the database of run-time paths and no modification of theFSM algorithm is required. To the best of our knowledge the use of parental informa-tion has not previously been applied to improve the run-time of FSM algorithms byreducing the length of the algorithm input data.

In this work we have also made use of clustering to reduce and summarise the datacollected during monitoring. That is, we have applied clustering and statistical anal-ysis to run-time paths to group the large volume of different paths produced. Chenet.al [45] have previously applied clustering and statistical anslysis to run-time pathscollected from monitoring enterprise systems for the purpose of problem determina-tion (i.e. identifying faulty software components). They make use of data clusteringand statistical analysis techniques to correlate failures of requests to the componentsmost likely to have cause them.

111

CHAPTER

SIX

Detecting Performance Designand Deployment Antipatterns

In this chapter we discuss how antipatterns can be detected in an extracted run-timedesign model. The detection process is based on a rule engine approach wherebyantipatterns are pre-defined, in an antipattern library, as rule-engine specific rules.Examples of how rules can be written for antipattern detection are provided. Theapproach is easily extensible and user configurable. In fact users can configure the de-tection module to filter uninteresting antipatterns through antipattern threshold con-figurations. User defined rules can also be added to the antipattern library such thatdomain or application specific antipatterns can be identified. The PAD tool usagemodes (i.e. single user and multi user) are also discussed.

In this chapter the JEE performance design and deployment antipatterns have alsobeen categorised into groups related to the data required to detect them.

The final section of this chapter discusses related work.

Main Points

• JEE performance design and deployment antipatterns are categorised intogroups based on the data required to detect them.

• Antipatterns are defined as rules and added to an antipattern library. The an-tipattern library can be extended with user defined antipattern descriptions.

• Example rules are provided.

• User defined threshold values can be used to filter out insignificant antipatterns.

• The PAD Tool can be used in both single and multi user mode. For the detectionof certain antipatterns multi user mode is required.

112

6.1 Antipatterns Categories

In this section we categorise the antipatterns, that we detect, based on the similaritiesin the type of data used to detect them. This categorisation was a result of analysingJEE performance design and deployment antipatterns from the literature 1 [161] and[61]. In the following section we give examples of rules that can be used to detectantipatterns from a number of the different categories.

1. Antipatterns Across or Within Run-Time Paths: In order to detect the antipat-terns in this category, information is required on how often particular compo-nents or services occur within the same, or across the different run-time paths.For example, a large number of session beans occurring across the different run-time paths may signify the existence of the Sessions-A-Plenty antipattern [61](i.e. the overzealous use of session beans, even when they are not required). An-other example of an antipattern in this category is the Large/Small transaction[61] antipattern whereby an inefficient transaction size is set resulting in eitherlarge, long living transactions or many inefficient short living transactions.

2. Inter-Component Relationship Antipatterns: The inter-component relation-ship antipatterns can be identified by analysing the relationships that existbetween the different components in the system. Where inappropriate rela-tionships exist antipatterns can be detected. A typical example might be theCustomers-In-The-Kitchen Antipattern [161] where web tier components di-rectly access persistent objects (e.g. Entity Beans). Other antipatterns that can bedetected by analysing inter-component relationships include the Needless Ses-sion Antipattern (discussed in section 6.3), the Transparent Facade Antipattern[61] and the Bloated Session Antipattern [61].

3. Antipatterns Related to Component Communication Patterns: This categoryof antipatterns can be identified by analysing the communication patterns be-tween particular component types. For example a high level of fine grainedchattiness between remote components (i.e. the Fine Grained Remote Calls An-tipattern, which is also known as the Face Off Antipattern [161]). Another ex-ample might be an unusually bulky or high amount of communication betweenthe business tier and the database which could signify the existence of an Appli-cation Filter/Join Antipattern [161]. Other examples include the Eager IteratorAntipattern [161].

4. Data Tracking Antipatterns: The data tracking category of antipatterns can bedetected by analysing how data objects are created and used across particularrun-time paths. A typical example of this antipattern is an unused data object,

1Precise Java Website, http://www.precisejava.com

113

whereby a data object is created and passed along the run-time path, but the in-formation which it contains is never accessed or used. An EJB specific variationon this antipattern is called aggressive loading of entity beans, whereby an en-tity bean is loaded but only a small proportion of the persistent data it containsis ever accessed 2.

5. Pooling Antipatterns: Antipatterns in this category can be detected byanalysing the related pool size and queue information. For example, to deter-mine if an inefficient pool size has been specified we need to consider the poolsize, the number of object instances being used, on average, and the averagequeue length. Examples of these antipatterns include inefficient configuration ofany of the container pools (thread pools, session bean pools, entity bean pools,database connection pools)2.

6. Intra-Component Antipatterns: Intra-component antipatterns can be detectedby analysing internal component details. Examples include the SimultaneousRemote and Local Interfaces Antipattern [161] or the Rusty Keys antipattern[161]. In both these cases the antipattern can be identified by analysing the com-ponent meta data.

6.2 A Rule Engine Approach for Antipattern Detection

The detection module is responsible for identifying performance design and deploy-ment antipatterns in the extracted design model. Detection can be achieved throughthe use of a knowledge base or rule engine. The PAD tool makes use of the JESS ruleengine [69] for this purpose. JESS is a Java based rule engine, whereby informationcan be loaded into the rule engine in the form of JESS facts. Rules can be defined tomatch a number of facts in the knowledge base. If a rule matches the required facts,the rule fires. When a rule fires, particular tasks can be performed. Such tasks are out-lined in the rule definition. The information captured in the extracted design modelcan be easily loaded into the rule engine by converting the entities in the model to JESSfacts. JESS will accept Java objects as input and automatically convert them to JESSfacts. During analysis the model entities are created in the form of Java objects. Theobjects created are loaded into JESS by the detection module. To detect antipatternsin the design, rules that describe the antipatterns must be written and loaded into therule engine.

Jess provides for both forward and backward-chaining rules. Rules are generally inthe form of ”if-then” statements, where ”if” a number of conditions (specified in therule) match facts in the database, ”then” the rule fires. Forward-chaining rules firewhen the conditions fully match the facts in the database. Backward-chaining rules

2Precise Java Website, http://www.precisejava.com

114

however are goal seeking rules since, if a rule is only partially matched and the ruleengine can determine that firing some other rule would cause it to be fully matched,the engine tries to fire that second rule. With backward-chaining rules the engineactively tries to make the rules fire. Rule-based systems such as Prolog based systemssupport backward-chaining rules. Forward-chaining rules are often described as rulesthat discover the conclusions drawn from an existing set of facts, whereas backward-chaining rules are described as rules that search for the premises from which existingfacts can be derived.

The antipattern rules that we have implemented are all forward-chaining rules. Thisapproach was chosen as we wanted the rule library to be easily understandable andextensible. Forward-chaining rules are self contained and do not require other rulesto fire in order for the rule to match a set of facts. Thus rules in the library and thecorresponding antipatterns have a one to one relationship. This allows for users ofour approach to easily relate each rule in the library to the corresponding antipatternit describes.

(defrule detect-Local-and-Remote-Intrfs-Simultaneously-Antipattern

(?C<-(component (has_local_interface ?LI&:(eq? LI"true"))(has_remote_interface ?RI&:(eq ?RI "true"))))

=>

(printAPSolution "Local-and-Remote-Intrfs-Simultaneously" ?C))

Figure 6.1: Rule to Detect Simultaneous Interfaces Antipattern

6.3 Example Rules

Next we give examples of how we specified antipattern rules for a number of differ-ent antipatterns from the categories above. The rules given have been used to detectinstances of antipatterns in JEE applications as shown in section 7.4. JESS rules arewritten in a Lisp like syntax. A rule has two parts separated by the following sign:=>

The left hand side (LHS) of the rule consists of patterns that match facts. The righthand side (RHS) gives the functions to be executed when the pattern on the LHS ismatched. The RHS of the rules shown in this section consists of a function call tothe printAPSolution function. This function prints the antipattern description, solutionand corresponding contextual information for the particular antipattern detected.

The rule in figure 6.1 describes an antipattern from the intra-component antipatternscategory. The antipattern described is the Local and Remote Interfaces Simultane-

115

ously antipattern, whereby a component exposes its business methods through bothlocal and remote interfaces. The detection of this antipatterns is quite simple since itrequires the matching of only one fact i.e. is there a component fact that has the value”true” for both attributes ”has local interface” and ”has remote interface”.

(defrule detect-Needless-Session-Antipattern

?C1<-(component (name ?N) (type ?T&:(eq ?T "Session"))

(callees ?Callees)(callers ?Callers))

(component (name ?N2)(type ?T2&:(eq ?T2 "Entity"))|?T2&:(eq ?T2"DB"))

(not (test (existsInList ?N2 ?Callees)))

(not (test (existsInList ?N2 ?Callers)))

=>

(printAPSolution "Needless-Session" ?C1))

Figure 6.2: Rule to Detect Needless Session Antipattern

The rule shown in figure 6.2 is from the inter-component relationship antipatternscategory. It describes a situation where a session bean has been used but was not re-quired. In general a session bean is generally only required if there is interaction withthe persistent tier (e.g. entity beans or the database components) or if other containerservices are required. Otherwise a plain old java object (POJO), which is less resourceintensive, can be used. To identify this antipattern we try to identify session beansthat exist but do not have any relationships with entity or database tier components.Further checks can be made to identify the use of container services. However, wehave found that in many situations container services are used by sessions when theyare in fact not required (e.g. setting transaction attributes to ”Required” by default), soinstead in the rule below we check only for (persistent) component relationships. Therule in figure 6.2 (a) checks for a session bean component, C1 that has a list of callerand callee components, (b) checks for a second component (which we will call C2 forthe purpose of the example), that is either an entity bean or a database componentand (c) checks if C2 is a caller or callee of C1. JESS allows for the use of user definedfunctions which can be used to provide more complex functionality to the rules in aconcise manner. The existsInList function in figure 6.2 is a user defined function whichchecks a list (argument 2) for a particular value (agrument 1). Without the use of suchfunctions the rules can become overly complex and difficult to both write and com-prehend. The PAD tool provides a number of user defined JESS functions to allow forthe easy construction of rules.

116

(defrule detect-Bulky-Session-Entity-Comms-antipattern

(?C1<-(component (name ?N1)(type ?T1&:(eq ?T1 "Session")) (callees?Callees)))

(?C2<-(component (name ?N2)(type ?T2&:(eq ?T2 "Entity"))))

(?FS<-(frequentSequence(children ?Ch) (parents ?Ps) (methodIdLists?Ms) (support ?S)))

(test(existsInList ?N1 ?Ch))

(test(existsInList ?N2 ?Ps))

(test(flagHighResourceConsumption "frequentSequence" ?Ms ?S))

=>

(printAPSolution "Bulky-Session-Entity-Comms" ?FS ?C1 ?C2))

Figure 6.3: Rule to Detect Bulky or Unusual Levels of Database Communication

The final rule example given in this section (figure 6.3) is a rule from the antipat-tern category concerned with component communication. In this rule we identify arelationship between a session bean and an entity bean in the form of a frequently re-peating sequence. If this relationship exists, the average resource consumption of thefrequent sequence is calculated and is flagged if it is above a user defined threshold.The calculation of the resources consumed is performed by the flagHighResourceCon-sumption function, which is passed the list of methods in the sequence and the supportof the sequence. The function refers to a user defined configuration file which specifiesthe acceptable threshold values. Alternatively if performance metrics are unavailablethe frequency of the sequence alone can be used to identify the antipattern.

6.3.1 Antipattern Library

An antipattern library is provided by the PAD tool. The library provides rules to detectantipatterns from all categories outlined in section 6.1. Rules can be easily modifiedor added to the library such that domain specific or application specific antipatternscan be detected. Details in relation to the rule library are given in the thesis appendix.Here we also briefly describe how rules can be added. The rule library also makes useof a number of user defined Jess functions which are available as part of the PAD tool.Details on the Jess functions are also given in the thesis appendix.

117

6.3.2 Filtering Using Threshold Values

User defined threshold values can be used to either reduce or increase the levels ofantipatterns detected. This can be particularly useful if a high level of performanceantipatterns are being detected. A user may only be interested in the most significantantipatterns and can use the threshold values to filter out less significant antipatterninstances. User defined thresholds can be defined in a single configuration file.

Next we give a number of examples of how thresholds are used by the detection mod-ule to filter out uninteresting antipattern detections. In the antipattern outlined in fig-ure 6.3 a user may only be interested in sequences that consume over a certain amountof the system resources. Thus a user of the PAD tool can define the level of resourceusage that should not be exceeded. If sequences are identified which exceed this levelof resource consumption they will be flagged as antipatterns. The flagHighResource-Consumption function calculates the amount of resources consumed by the sequenceand compares it to the user defined threshold value. Similarly a threshold value canbe useful for filtering potential antipatterns from the pooling category. For examplethresholds can be used to set levels above which queue sizes, for the various pools,should not exceed.

Many such settings are very much application specific and thus it is essential thatthey are easily configurable. By modifying the configuration files users do not need tounderstand or edit the rule library. Examples of the types of settings that can be madein the configuration file are given in the thesis appendix.

Similarly, the FSM support count can be modified by a user during analysis to increaseor reduce the level of frequent sequences identified during the mining process. TheFSM support count can be considered an analysis module user configurable thresholdvalue.

6.4 PAD Tool User Modes

The PAD tool can be used in two different monitoring modes. Either single user modeor multi user mode. Single user means that there is only one user in the system duringmonitoring (e.g. a single developer testing the application). Multi user mode requiresa load on the system. Antipatterns from categories 1,2,3,4 and 6 can be detected insingle user mode. All antipatterns can be detected in multi-user mode and in fact thismode is required to detect the antipatterns in category 5. An added advantage of usingmulti user mode is that accurate performance metrics can be given to the tool user ondetection of an antipattern. Such metrics can be used by the tool user to quickly assessthe impact of the detected pattern. Performance metrics can also be collected duringsingle user mode, however they are less reliable since the system is being used in aless realistic manner.

118

There are two main drawbacks of using multi user mode however. Firstly, it requiresa load to be generated on the system. In most cases this requires the creation of an au-tomated test environment (e.g. using load generation tools). Secondly, a large amountof data is produced when monitoring enterprise systems under load. In particular,our monitoring module produces a large amount of run-time path information. Thisissue can be addressed however by applying the clustering and statistical analysistechniques outlined in section 5.5.

6.5 Related Work

In this section we discuss and compare related work to the research that has been pre-sented in this chapter and to our overall antipattern detection approach. We discussin particular alternative antipattern categorisations that have been presented, as wellas performance testing techniques, detection approaches for pattern and antipatternidentification and finally work in the area of antipattern detection closely related toour approach.

6.5.1 Antipattern Categorisation

Antipatterns have been categorised in the various literature in which they have beendocumented. For example, Brown et. al [36], distinguish between development, ar-chitecture and project management antipatterns. The literature [161] and [61] bothcategorise their antipatterns according to the related component types which are af-fected. For example, antipatterns related to entity beans, antipatterns related to ses-sion beans. In contrast we have taken technology specific (JEE) performance antipat-terns and categorised them according to the data required to detect them. A similarapproach has previously been taken by Reimer et al., who have categorised program-ming errors based on the algorithms used to detect them [145]. Our antipattern cate-gorisation also differentiates between performance design antipatterns, performancedeployment antipatterns and performance programming errors. Similarly, Moha andGueheneuc [116] provide a taxonomy in which they attempt to clarify the differencebetween errors, antipatterns, design defects and code smells. In their analysis theydefine code smells as intra-class defects and design defects as inter-class defects. Ourantipattern categorisation is at a higher more abstract component level. Hallal et. al.[86] provide a library of antipatterns for multi-threaded Java applications. They alsodistinguish between errors and design antipatterns. They classify the multi-threadedantipatterns that they present following a categorization, which reflects the effect ofthe antipatterns on the outcome of the tested program.

119

6.5.2 Performance Testing

Early analysis of systems from a performance perspective has mainly focussed onmodelling techniques whereby models of the system are created at an early stage us-ing system artifacts and estimates of resource consumption, user load etc. In the re-search community modelling has been applied to enterprise systems [168] [76], how-ever in reality they are rarely used for performance analysis. This is due to the fact thatthey can be inaccurate, and time consuming to build and maintain for large enterpriseapplications.

Performance testing of running systems is generally the preferred choice of industrytoday. Often, however, testing is left until late in the development cycle and as a resultissues identified can be expensive to fix and can lead to major project delays. Ideallyperformance testing should be carried out as early as possible in the development cy-cle. Recent research efforts have focused on early performance testing through thecreation of component stubs [58] or through the automatic generation of componenttest-beds 3. This allows for a running system to be created, deployed and analysedearly in the development cycle such that performance tests can be run. These ap-proaches allow for the system design to be analysed at an early stage in developmentwhereby performance issues can be identified even before much of the coding takesplace. Our work complements these initiatives since design level antipatterns can beidentified in a real running system or in test bed or stub applications. Our antipat-tern detection approach can be applied early in the development process as soon asa running system is in place and should be carried out iteratively as the applicationevolves during development. The approach we present is suitable for any iterativedevelopment processes whereby a running prototype of the application is producedearly in development (e.g extreme programming [24]).

Current tools for performance testing generally fall into the categories of load genera-tion tools or performance profilers (see section 2.8.2). Current tools merely preset thedata collected to the user unanalysed. Thus tool users must perform analysis of thedata manually which can be a difficult task. Our work builds on current performanceprofilers. The PAD tool performs automatic analysis of the data produced duringmonitoring to automatically identify design issues.

6.5.3 Detection Techniques

The detection technique presented in this chapter makes use of a knowledge base/ruleengine based approach. A number of previous pattern detection approaches havemade use of the logic programming language Prolog for software analysis [103] [49].Prolog allows for the creation of a knowledge base of facts and for the creation ofrules that can be used to query the information in the knowledge base. In general a

3AdaptiveCells/J, http://adaptivecellsj.sourceforge.net/

120

software system’s design is represented by Prolog facts and rules are used to expressknown patterns. Prolog works using backward-chaining. A disadvantage of usingProlog for pattern detection is that the creation of new rules to detect new patternsrequires an understanding of the Prolog language. The literature [29] introduces theRelational Manipulation Language (RML) which like Prolog is based on predicate cal-culus. The authors claim that RML is simpler to use than Prolog and can be used fordifferent program analysis tasks such as pattern detection, antipattern detection, cal-culation of design metrics etc. The authors have implemented the CrocoPat tool whichinterprets programs written in RML. The tool was designed for the analysis of graphmodels of software systems and it has been shown to scale well on large systems. Incomparison our detection approach makes use of very simple forward-chaining rulesin the form of ”if-then” statements. We do not require backward-chaining or the use oflogic programming. The analysis we perform prior to detection (see chapter 5) allowsfor simple rules to be written. If analysis was not performed, our rules would needto be more complex in order to identify the relationships which are extracted duringanalysis.

A number of techniques have been applied to identify structural patterns in software.For example, graph matching [138], graph rewrite systems [123], or meta program-ming [11] [98]. The representations used to model the software structure includegraph models [138], abstract syntax graphs [123], or design description languages [11][98]. However such approaches are less suitable for the detection of design and de-ployment antipatterns. This results from the fact that the design model we extractfrom the running application contains information other than structural informationthat can not be represented easily when applying these techniques. For example, be-havioural information (e.g. repeating sequences of events), performance metrics, re-constructed container services and object pool information. Instead we have made useof a knowledge based approach which allows for this information to be represented inthe form of knowledge base facts. An alterative way of representing this informationmight be to store it using a relational database. Vokac [172] has previously made useof a relational database for the purpose of structural design pattern detection.

Formal concept analysis (FCA) has also been used for the purpose of pattern detection[39] [164]. FCA does not require a library of predefined rules. Instead FCA works byidentifying groups of classes that share common relations. FCA approaches try toinfer patterns/antipatterns from the code or system design. In contrast, our approachtries to identify antipatterns based on a library of defined antipatterns to be matched.

6.5.4 Antipattern Detection

Software pattern and antipattern detection techniques have been discussed in section2.10. Many of the pattern detection techniques could potentially be applied to detectinstances of antipatterns. For example, with most of the pattern detection approaches

121

discussed, the patterns and application are described in terms of structural (and insome instances behavioural) representations which are analysed to identify instancesof the pattern. Since design flaws or antipatterns can also be described in this way,the same approaches can be used to identify them. Some work however has focusedmore specifically on antipattern detection [83] [49] [117] [114] (see section 2.10). Theaforementioned approaches all rely on extracting information from the system sourcecode.

A run-time antipattern detection approach very closely related to our work, that ex-tracts information from a running system, has recently been presented. Eologic 4 havedeveloped an antipattern detection tool for JEE applications. This tool identifies anumber of general JEE antipatterns and presents the user with possible solutions.This tool, similar to our approach, extracts a model from a running system 5. Thetool makes use of JMX and AOP to monitor a running system and to extract an Entity-Relationship-Attribute model. A major drawback of the tool is that it is currently notportable across different application servers. Also it does not detect antipatterns basedon communication patterns or object tracking. Furthermore it is designed to be used insingle user mode and does not perform data reduction when monitoring applicationsunder load.

Other work closely related to our antipattern detection work has been presented byBischofberger et al. [31] in the form of an architecture conformance checking toolcalled Sotograph. Sotograph makes use of cycle analysis and metrics based analy-sis to identify architectural level problems in the application. This is performed byanalysing the system source code. Many of the issues identified are similar to the an-tipatterns that we try to detect. For example the author’s list architectural issues thatcan be identified such as large artifacts, small artifacts and bottlenecks. These issuesare closely related to a number of antipatterns from the inter-component relationshipcategory that we detect using the PAD tool.

Similarly, IBM’s Structural Analysis for Java (SA4J) tool 6 analyses structural depen-dencies of Java applications in order to identify structural antipatterns within the ap-plication. SA4J measures the quality of the application structure by evaluating theweb of dependencies between packages, classes, and interfaces of a Java application.

The Smart Analysis Based Error Reduction Tool (SABER) [145] is also closely relatedto our work. SABER uses a combination of domain specific knowledge and staticanalysis to identify high impact coding errors in large commercial (JEE) applications.This tool has been successfully applied to detect hundreds of errors in these appli-cations. The SABER tool identifies lower level issues (e.g. improper clean up of re-sources, concurrency issues) than detected by the PAD tool. However similar to our

4Eologic’s Eosense for JEE, http://www.eologic.com/eosense.shtml5Derived Model Analysis,Detecting J2EE Problems Before They Happen, Alan West and Gordon

Cruickshank, December, 2006, http://dev2dev.bea.com/pub/a/2007/07/derived-model-analysis.html,6IBM’s Structural Analysis for Java tool, http://www.alphaworks.ibm.com/tech/sa4j

122

approach problems detected are annotated with contextual information such that theissues identified can be better understood. For example the SABER tool provides con-textual path and data flow information which can explain how the defect occurred.

123

CHAPTER

SEVEN

Results and Evaluation

In this chapter we present a range of different tests, and their corresponding results,that we have carried out to validate our work. Results are presented from tests onCOMPAS JEEM that show it is a low-overhead and portable monitoring approach.We also show how the output from COMPAS JEEM can be used to manually reverseengineer and identify antipatterns in JEE applications. Furthermore results are pre-sented in relation to tests carried out on the analysis module components. We showhow the analysis of the monitored data can be carried out in an efficient and timelymanner. The analysis module test results show that the analysis module scales wellin relation to data from real enterprise systems in terms of both the output producedand its run-time. In addition the usefulness of component communication patternsextracted during analysis has been highlighted. Our test results show the output fromthe analysis module can be used to (manually) identify serious performance designflaws in sample and real JEE applications. We also provide results that show how theanalysis module can apply data reduction techniques to reduce the amount of dataproduced during monitoring when the PAD tool is used in multi-user mode. In thepenultimate section we present results from applying the PAD tool to two JEE appli-cations. The results show how using our tool we automatically identified a number ofperformance design and deployment antipatterns in each application under test. Fi-nally we revisit our claimed contributions and discuss how they have been evaluated.

124

Main Points

• Test have been performed using COMPAS JEEM which show:

– COMPAS JEEM is portable across a number of different application servers.Application servers do not spawn new threads for local calls to remotebeans. Thus the ThreadLocal approach can be used to tag requests locally.

– COMPAS JEEM is a low overhead monitoring approach.

– COMPAS JEEM can be used to reverse engineer JEE applications and tomanually identify design flaws.

• Test have been performed in relation to the Analysis Module components whichshow:

– FSM can be applied to run-time paths in an efficient manner if preprocess-ing is applied.

– The output from applying FSM to run-time paths can be used to identifyreal issues. The FSM output was in fact used to manually identify perfor-mance design antipatterns in two JEE applications, i.e. a sample applicationand a real enterprise application.

– Clustering can be successfully applied to reduce data collected when mon-itoring the system under load.

• Test have been performed using the PAD tool:

– Three antipattern types were automatically identified in a widely availableJEE sample application. Numerous instances of two of the antipattern typeswere identified.

– Antipatterns were successfully identified in a modified version of a JEEsample application. Antipatterns were added to the application for detec-tion purposes. All antipatterns added were identified.

– Test results show how six antipattern types were identified in a real largescale JEE enterprise application. Thirty four instances of these antipatterntypes were identified.

– Antipatterns from all categories outlined in chapter 6 have been identified.

• The validation criteria outlined in chapter 3 have been met.

125

7.1 Chapter Overview

In this chapter we present a wide range of results that have been carried out in order tovalidate our research. Section 7.2 gives results from a number of different tests carriedout on the COMPAS JEEM run-time path tracing tool (see section 4.2 for a descriptionof this tool). In subsection 7.2.1 we show how we applied COMPAS JEEM to a numberof JEE applications and how we could quickly deduce the system structure of theapplications by analysing the run-time paths collected. For one of the applications weshow that we could easily identify a number of performance design antipatterns fromthe system structure that was deduced. We also show the portability of our approach(in subsection 7.2.2) by applying it to applications running on application servers froma number of different vendors. In subsection 7.2.3 we give details on the performanceoverhead incurred by applying COMPAS JEEM to a JEE application.

Section 7.3 presents tests carried out on the analysis module components. In partic-ular in section 7.3.1 results are given which show the performance run-time of thedifferent FSM approaches, which have been presented in section 5.4, with respect to anumber of different databases. In section 7.3.2 we also show how FSM was applied todata collected from two JEE applications and how we used the results to successfully(manually) identify performance design flaws within them. In particular using ourtechnique we quickly identified a real design flaw in an IBM application (beta ver-sion). We also show (in section 7.3.3) how information collected during monitoringwas reduced by our analysis module’s data reduction mechanism, when informationwas collected under load.

In section 7.4 we present results from applying our PAD tool to two JEE applications.More precisely, we detail how the PAD tool was applied to two JEE applications i.e. asample JEE application (see section 7.4.1) and a real large scale enterprise system (seesection 7.4.2). Antipatterns from all the categories outlined in section 6.1 have beendetected. For each antipattern detected we give a brief description of the antipatternand the antipattern category. We also give the related information (PAD output) whichis presented to the tool user upon detection. Using this information the tool usercan easily determine the severity of the problem and a decision can be made as towhether refactoring is required or not. We do not show performance improvementsthat can be attained by refactoring the antipatterns detected since these improvementshave already been well documented [161] [61]. Also performance improvements canbe very much application specific and vary greatly depending on the severity of theantipattern.

Finally in section 7.5 we revisit our contributions claimed and the validation criteriawhich we previously outlined. In this section we discuss how our validation criteriahave been met.

126

7.2 COMPAS JEEM Results

In this section we show how we applied COMPAS JEEM to a number of JEE applica-tions. Three subsections follow. The first subsection shows how COMPAS JEEM canbe applied to reverse engineer JEE applications, the second provides a portability as-sessment of COMPAS JEEEM and the final subsection give details on the performanceoverhead incurred by applying COMPAS JEEM to a JEE application.

7.2.1 Deducing System structure

We applied COMPAS JEEM to two JEE applications: (1) a sample online banking ap-plication, called Duke’s Bank 1 and (2) a sample e-commerce application, called Plants-ByWebsphere 2. Duke’s Bank is a sample application provided by Sun Microsystemsas a showcase for the JEE technology. PlantsByWebsphere is a sample e-commerceapplication provided by IBM. We monitored both applications using COMPAS JEEMand performed a number of user actions. The run-time paths that corresponded toeach user action were obtained. A diagramatic representation of each system wasmanually constructed by analysing the run-time paths.

Duke’s Bank is an online banking application. Modifications were required to fix anumber of bugs in this application such that tests could be carried out 3. When auser logs in to the Duke’s Bank application he/she can perform the following actions:view a list of accounts, view an individual account’s details, withdraw or lodge cash,transfer cash from one account to another or finally log off. Each of the different useractions was performed and the run-time paths recorded. Figure 7.1 shows the run-time path associated with the account list user action. It shows the interactions thatoccur between components to satisfy a user request (for the list of accounts that be-long to a particular customer). The run-time paths collected by COMPAS JEEM alsocontain related performance metrics (e.g. method execution time as shown in figure7.1). Such information can be useful for identifying bottlenecks or performance designissues that exist in the application. Five different components make up the run-timepath: accountList (web component), AccountControllerBean (session bean), Account-Bean (entity bean), PreparedStatement (JDBC component) and ResultSet (JDBC com-ponent). The run-time path can be easily traversed to identify the relationships thatexist between the different components.

By analysing six run-time paths we manually produced the diagram in figure 7.2. Thepaths analysed were produced as a result of a user logging in and performing the

1Duke’s Bank Sample Application, Sun Microsystems, http://java.sun.com/j2ee/tutorial/2How to Run WebSphere Application Server Version 5 Samples on

DB2, Tom Krieger, April 2003, IBM Developerworks, http://www-128.ibm.com/developerworks/websphere/library/techarticles/0304 krieger/krieger.html

3Duke’s Bank J2EE Application - Exceptions and Troubleshooting, Ada Diaconescu,http://adadiaconescu.there-you-are.com/dukes-bank/dukesBank-troubleshooting.htm

127

Figure 7.1: AccountList Run-Time Path and UML sequence diagram

128

available banking options. After studying the system documentation we expect thatthe system would be used in this manner. Figure 7.2 shows the design of the sectionof the application which is responsible for handling the different requests that can bemade by banking customers that log in to the system. The diagram was constructedmanually by traversing the run-time paths. The testing environment consisted of theJBoss application server (version 3.2.7) and a MySql database (version 4.0.2). Becausethe web server that comes as part of JBoss 3.2.7 implements the servlet 2.3 specificationthe web tier is monitored as one software component (see section 4.2.5.1 for furtherdetails on why this occurs) as shown in figure 7.2.

Figure 7.2: Diagram Showing Components in Duke’s Bank

129

The diagrams that we produced can be used for identifying performance design is-sues within enterprise applications. For example, by analysing figure 7.2 we quicklyidentified the existence of two EJB performance antipatterns in the Duke’s Bank ap-plication. The diagram consists of four EJBs two stateful session beans and two entitybeans.

• Antipattern 1: The first antipattern we identified is known as the Conversa-tional Baggage antipattern [161]. It identifies situations where a stateful sessionbean is used but not required. This can be wasteful since there is a performanceoverhead required to manage stateful sessions within the EJB container. Often a(more lightweight) stateless session bean would suffice. Figure 7.2 shows that astateful session bean is invoked for each user request. However there is no ob-vious need for a stateful session bean in any of the user actions we performed,as state is not maintained across the different web pages returned for the relateduser requests. The stateful session beans in figure 7.2 could easily be replaced bymore lightweight stateless session beans which is the suggested solution to thisantipattern.

• Antipattern 2: The second issue we identified with the Duke’s Bank applicationwas that the entity beans exposed remote interfaces. The entity beans in thisapplication (unlike the session beans) are only accessed by EJBs which wouldmost likely be deployed in the same container. Thus there is no real need for theentity beans to expose remote interfaces. The beans can be refactored to exposelocal interfaces and thus reduce the overhead associated with remote methodcalls.

PlantsByWebsphere is a sample e-commerce application that comes installed withWebsphere 6.1. Websphere 6.1 contains a websever that conforms to the servlet 2.4specification and thus can monitor inter web component calls. As above we per-formed a number of user actions in the PlantsByWebsphere application and con-structed a diagram by analysing the corresponding run-time paths. In total 19 useractions were invoked. Figure 7.3 shows the diagram that was constructed. The dia-gram shows the different web and business tier components that made up the applica-tion (for simplicity we did not analyse the JDBC components for PlantsByWebsphere).No antipatterns were identified from the diagrams for this application.

As shown in figures 7.2 and 7.3 COMPAS JEEM monitors only the main components(JSPs, Servlets, EJBs and JDBC calls) that make up the J2EE system. Smaller utilityclasses and the underlying middleware calls are omitted. This allows for run-timepaths (and thus diagrams) that contain only the main components of the applica-tion. As a result our run-time paths and diagrams are not cluttered with unimportantmethod calls that play only a minor role in the overall application design structure.

130

Figure 7.3: Diagram Showing Components in PlantsByWebsphere

131

7.2.2 Portability Assessment

In order to show that our approach is truly portable we applied it to a number ofapplication servers running JEE applications. The table in figure 7.4 shows resultsfrom applying our tool to three different application servers, JBoss, Websphere andOC4J. The table gives the server vendor, the server version, results from monitoringthe web tier and results from monitoring the business tier.

Server Vendor Version Web Tier Business TierJBoss Group JBoss 3.2.7 As one comp. Remote + Local Comps.

IBM Websphere 6.1 Inter comp. comms. Remote + Local Comps.Oracle OC4J 10.1.2.0.2 As one comp. Remote + Local Comps.

Figure 7.4: Portability Test Results

In relation to web tier monitoring we tested to see if the interactions between webcomponents could be monitored. From the results table it can be seen that only theWebsphere application server could achieve monitoring of the individual web tiercomponents i.e. it monitored inter component communications in the web tier. InJBoss and OC4J the web tier was monitored as one software component. This canbe explained by the fact that both the JBoss and OC4J versions that we used duringtesting contain servlet containers that conform to the servlet 2.3 specification. Laterversions of the application servers will conform to the servlet 2.4 specification andwill allow for complete web tier monitoring as is the case with Websphere 6.1.

The testing of the business tier was designed to determine if calls made to a compo-nent through a remote interface from a local component (i.e. within the same JVM)could be traced using our technique based on the thread preservation in EJB remotecalls, outlined in section 4.2.5.2. For the three applications servers that were tested, itwas found that in situations where components were located within the same JVM allcalls made between components could be traced regardless of whether the calls werethrough local or remote interfaces. Even though the RMI specification stipulates that,”a method dispatched by the RMI run-time to a remote object implementation may ormay not execute in a separate thread”, our results suggest that in the case of currentapplication servers a new thread is not spawned for such calls when the objects areco-located. Since this is the case the approach we outline for tracing run-time pathsabove can be applied.

The monitoring of the EIS tier is achieved be extending the P6Spy open source moni-toring framework. P6Spy can be used to monitor database statements of any applica-tion that uses JDBC, thus the EIS tier monitoring is portable across different databaseimplementations and is independent of the web and application server tiers.

132

7.2.3 Performance Overhead

To assess the performance overhead incurred by COMPAS JEEM we instrumentedthe web and business tiers of the PlantsByWebsphere application described in section7.2.1 and ran a number of performance tests. The test environment was made up ofthree machines. The first machine (Pentium 4, 2.25GHz, 1GB RAM) was used for loadgeneration. Apache Jmeter was used as a load generation tool. PlantsByWebspherewas installed on WebSphere 6.1 which ran on a second machine (Pentium M, 1.7GHz,1GB RAM). Finally a third machine (Pentium 4, 1.6 GHz, 768 MB RAM) was employedto log the data produced by COMPAS JEEM remotely such that the performance over-head on the machine running the JEE application could be kept to a minimum. This iscommon practice for production environments. To assess the performance overheadwe ran a number of test cases with different user loads for two versions of the sampleapplication (i.e. one version included COMPAS JEEM instrumentation and the otherversion did not). The test cases consisted of a warm up period (to warm up the JVM assuggested in the literature [38]), followed by a measurement period. During the differ-ent test cases we observed the response time of the application for the following loads:20 users, 50 users, 100 users, 150 users, 200 users. Users entered the system at a rate ofone user per second with a random delay of between 1-2 seconds. Each measurementperiod lasted 30 minutes. For each of the given loads three test cases were run and anaverage value obtained. To assess the performance overhead we analysed the aver-age response time of the instrumented application versus the average response timeof the non-instrumented application for the same user load. The results are shown infigure 7.5. Note that the response time for each run is an average value that has beennormalised based on the average response time for 20 users (non-instrumented).

Application Version 20 Users 50 Users 100 Users 150 Users 200 UsersNon-Instr. resp. time 1 1 1.1 4.12 X

Instr. resp. time 1 1 1.1 4.9 XOverhead 0 0 0 19% X

Figure 7.5: Performance Overhead Test Results

At 20 and 50 users there was no noticeable difference (on average) in response timefor either the instrumented or non-instrumented version of the application. Thus therewas no noticeable overhead for these loads. At 100 users there was a 10% increase inresponse time for both the instrumented and non-instrumented versions comparedwith 20 users. Again there was no noticeable overhead that could be attributed toCOMPAS JEEM. At 150 users, however the system was heavily saturated and responsetime increased significantly. For the non-instrumented application response time in-creased by 310%. The instrumented version at 150 users showed an increase of 390%.Therefore there was a 19% overhead that could be attributed to COMPAS JEEM at 150users. At 200 users (for the default application server settings) the system under testhad reached its capacity for both the instrumented and non-instrumented application.

133

There were a large number of errors at this user load and the results were ignored.

As can be seen from the results above there is no obvious overhead when COMPASJEEM is applied to a non-saturated system. This can be explained by the fact thatCOMPAS JEEM only monitors at the component level and does not instrument lowerlevel utility classes (unlike most of today’s JVMPI based monitoring tools). Adding asmall number of monitoring probes is insignificant in the context of large enterpriseapplications that generally contain very large numbers of classes. The sudden increasein overhead at 150 users can be explained by the fact that at this point the system seemsto be already heavily saturated, with the response time over 4 times slower than atlower loads. Any further increase in load at this point adds to the bottleneck andincreases response time significantly. Further testing is required to assess if COMPASJEEM will scale for increased loads when more hardware is available.

7.3 Analysis Module Results

In this section we provide results on the performance of the FSM implementationsmodified with the different support counting approaches and preprocessing tech-niques. We also provide results on how we applied FSM to two enterprise applica-tions to identify design flaws within them. Our test environment consisted of a singlemachine (Pentium M, 1.7GHz, 1GB RAM) running Windows XP. The FSM implemen-tation was run through the Cygwin environment 4. Run-times were recorded usingthe GNU time tool 5. Gprof was also used to inspect how the time was being spentwithin the FSM implementation 6.

The FSM implementation used was a modified version of Ferenc Bodon’s open sourceapriori implementation for mining frequent sequences of items 7. The main modifi-cation made to this code involved the adding of non-overlapping support counting.This code is currently being reviewed for submission into the FIM template library8. We also modified the original version of the FSM implementation to output thecorresponding transaction IDs in which a frequent sequence occurred.

The final subsection in section 7.3 shows how we applied clustering for data reductionpurposes. Clustering is applied to data collected from a JEE application under loadand shows how the number of paths collected can effectively reduced.

4Cygwin, http://www.cygwin.com/5GNU time command, http://www.gnu.org/software/time/time.html6GNU Gprof, http://www.gnu.org/software/binutils/manual/gprof-2.9.1/gprof.html7Apriori implementation for mining frequent sequence of items,

http://www.cs.bme.hu/ bodon/en/index.html8Frequent Itemset Mining Template library, http://www.cs.bme.hu/ bodon/en/index.html

134

7.3.1 FSM Performance Tests

This section details a number of tests that were run on different databases to show therun-times of the various FSM approaches discussed above. For the purpose of the testswe make use of a number of transactional databases; two publicly available databasesfrom the Open Source Data Mining (OSDM) repository 9 and two databases that weconstructed from run-time paths collected from monitoring distributed enterprise ap-plications (also publicly available 10). The databases from the OSDM repository weregenerated from weblogs of a large web news portal. The reason we chose to makeuse of these two databases was to show how the different techniques performed ondatabases with significantly different transaction lengths. The database kosarak2-10-2 (238209 transactions) for instance has a maximum transaction length of 10 items,whereas kosarak2-100-2 (604280 transactions) has a maximum transaction length of100. The databases generated from run-time paths were used to show how the dif-ferent implementations performed in relation to data obtained from real enterpriseapplications of different sizes. The first of these databases, Sun-Microsystems-sample-app (72 transactions, max length 83), was generated from monitoring a sample enter-prise application that is available from Sun Microsystems (see section 7.4). This is asmall enterprise application. The second of these, IBM-enterprise-app (233 transactions,max length 1617),was generated from monitoring a beta version of the IBM Workplacecollaborative enterprise application 11.

The different FSM implementations that we tested on the OSDM databases were FSMwith non-weighted support counting (FSM-NW), FSM with weighted support count-ing (FSM-W), FSM with non-overlapping weighted support counting (FSM-NO) andFSM with non-overlapping weighted support counting using Consecutive Item Re-placement (FSM-NO-CIR). For each test we measured the run-time of the algorithm.Figure 7.6 gives results when we ran the different implementations on the kosarak2-10-2 database. As expected FSM-NW performed best for all support counts. The FSM-NW run-times ranged from 10.66 seconds (for a min. support of 5) to 1.71 seconds(for a min. support of 50). This was expected since this approach looks for only thefirst instance of each item sequence per transaction whereas all the other approachestested on this database continue searching (in different ways, see sections 5.4.2 and5.4.4) after the first instance has been identified. Also the output produced by FSM ismuch smaller than with the other approaches since it only counts the first sequence inthe transaction. For the remaining three FSM approaches the FSM-NO approach wasslowest in all cases. The reason why FSM-NO is slowest for the kosarak2-10-2 databaseis because for this database a large number of candidates are generated (e.g. for min.support of 5, 80347 frequent sequences were generated for FSM-NO). The FSM imple-

9Open Source Data Mining repository, http://www.ilab.sztaki.hu/∼bodon/osdm/10Frequent Sequence Mining Implementation and Databases, Trevor Parsons,

http://pel.ucd.ie/tparsons/fsm11IBM Workplace, http://www.ibm.com/software/workplace

135

Figure 7.6: Test Results on K2 10 2 Database

mentations store the candidates in a trie data structure. The transactions are analysedone by one to count the support of the candidates. For each candidate that exists in thetransaction the trie is traversed and the support of the candidate is incremented. FSM-NO makes use of flags in the trie to prevent the counting of overlapping sequences.When we begin comparing a new transaction to the trie the flags in the trie must bereset. When the trie is big (i.e. a large number of candidate sequences exist) and thereare a large number of transactions in the database, this (resetting) process can be time-consuming and can significantly affect the performance of the FSM implementation.In fact for the tests run on the kosarak2-10-2 database 86-97% of time of the FSM-NOapproach was spent performing this task. For the kosarak2-10-2 database FSM-W wassignificantly quicker than FSM-NO.

To reduce the FSM-NO run-time we applied CIR to reduce the length of the trans-actions in the database. CIR was performed by initially running the FSM-NO imple-mentation with a high min. support threshold (50). The results from this initial run areused to identify consecutive sequences that can be replaced for the subsequent runsof lower min. support threshold. In total, after the first run, the database was reducedin size by 4449 items by making 771 replacements of 12 consecutive item sequencesvarying in size from 6 consecutive items to 9 consecutive items (50 item sequences of

136

size 9 were replaced, 104 of size 8, 236 of size 7 and 381 of size 6). This process resultedin a new database kosarak2-10-2*. Reducing the database by 4449 items, however, hadno noticeable improvement for FSM-NO when we applied it to kosarak2-10-2* with amin. support set to 40. This can be explained by the fact that replacing 4449 items inthe kosarak2-10-2 database is insignificant when we consider the database size (over230,000 transactions varying in size from 1 to 10). After the second run of the algo-rithm (at min. support 40) the output was analysed and CIR was again performed byreplacing 14 frequent item sequences. This time kosarak2-10-2* was reduced by 17020items resulting in a new database kosarak2-10-2**. On running FSM-NO with a min.support of 10 a noticeable run-time improvement of 30 seconds (4.3%) was recorded.The CIR process was repeated after the 3rd run and kosarak2-10-2** was reduced in sizeby a 10204 items resulting in a new database kosarak2-10-2***. Using kosarak2-10-2***for FSM-NO with a min. support of 5 gave a run-time improvement of 148 seconds(6.9%). When analysing the results above, however, it must be considered that for theCIR runs other than the first FSM-NO-CIR run (min. support 50 in this case) previousiterations of the CIR process were required. So in fact for FSM-NO-CIR with a min.support of 5, 3 previous runs of FSM-NO (with min. support of 10, 20,50) were re-quired. Thus, the 6.9% performance improvements with FSM-NO-CIR (min. support5) over FSM-NO is in fact out-weighted by the fact that three previous runs of theFSM-NO algorithm are required. When considering the performance of FSM-NO-CIRwe must take into account the time taken for the previous runs. Also the modificationof the databases during the CIR process is also time consuming since this is currentlysemi-manual. Since this process could be easily automated, we will not consider thiswhen comparing FSM-NO-CIR to other approaches.

Figure 7.7 gives our results when we ran the different implementations on thekosarak2-100-2 database. For all kosarak2-100-2 tests the max. candidate size was setto 5. Again FSM-NW outperformed the other implementations, its run-times rang-ing between 1.5 and 17.4 seconds. However for kosarak2-100-2 FSM-W was by farthe slowest approach in all cases. FSM-NO was significantly quicker than FSM-W.In fact for support count of 10,000 FSM-W ran almost 10 times slower with a differ-ence of 8071 seconds (2.24 hrs). This can be explained by the fact that the FSM-Wapproach produces a much larger number of frequent sequences than the FSM-NOapproach for kosarak2-100-2, which results in a large trie data structure (e.g. for min.support of 50000 the number of frequent sequences resulting from FSM-W and FSM-NO is 8225 and 108 respectively). As a result the number of generated candidatesis much larger which leads to more time being spent counting the supports of thecandidates. Counting the weighted support of a given candidate sequence is timeconsuming when transactions are long. To understand why this is we must under-stand how weighted support counting works. For each candidate sequence the firstitem must be checked at each point along the transaction for a match. If the first itemmatches an item in the transaction, then the second item in the sequence is checked

137

Figure 7.7: Test Results on K2 100 2 Database

against all items in the transaction that occur after the matched item. This process con-tinues for any remaining items in the candidate sequence until all complete matchesof the sequence are found. When transactions are long this process can be very timeconsuming. In fact for kosarak2-100-2 over 99% of FSM-W was spent counting the sup-ports of the generated candidates for all support counts tested. In comparison withkosarak2-10-2, while the number of sequences output for FSM-W and FSM-NO is veryhigh, since the transactions are much shorter, the time spent counting supports is lesssignificant. The impact of applying CIR to the kosarak2-100-2 database for FSM-NOwas as follows: after the first iteration of the FSM-NO implementation at min. sup-port of 100,000, the database was reduced by 75,801 items by replacing 6 frequentconsecutive sequences. The run-time for the second run (min. support 50,000) wasreduced by 16.3 seconds (16.3%). After the second run the database was reduced insize by a further 390,250 items by replacing 10 frequent consecutive sequences. Therun-time improvement of the third run was 170.2 seconds (47.2%). After the third runthe database was reduced by a further 318,612 items by replacing 8 frequent consec-utive sequences. The run-time improvement for FSM-NO with min. support 10,000was 475.1 seconds (52.8%).

For both theSun-Microsystems-sample-app and the IBM-enterprise-app databases we ran

138

Figure 7.8: Test Results on Sun Database

the same FSM approaches as with the previous two databases. However since boththese databases were created from run-time paths, we could also make use of parentalinformation to split the transactions and thus reduce their length (see section 5.4.4.1).Applying the FSM-NO approach to the split transactions is referred to as the SPLITapproach for the remainder of this section. Figure 7.8 gives results when we ran thedifferent implementations on the Sun-Microsystems-sample-app database. For all casesthe SPLIT approach was quickest. After splitting the database the maximum transac-tion length had been reduced from 84 to 37 items. This approach produced a smallernumber of frequent sequences than the other approaches (e.g. for a min. support of10 the FSM, FSM-NO and SPLIT produced 32234, 55906 and 174 frequent sequencesrespectively). FSM-NO-CIR was the next quickest. After the initial run (min. sup-port 50) the database was reduced by 126 items. After the second run (min, support40) the database was reduced by a further 40 items and finally after the 4th run (min.support 20) the database was reduced by a further 105 items. The FSM-W approachwas slowest running at least 50 times slower than FSM-NO at min. support count20. For min. support 20 the FSM-W implementation failed to complete within 4300seconds and the test was stopped. Figure 7.9 gives results when we ran the differentimplementations on the IBM-enterprise-app database. For this database the SPLIT ap-

139

proach did not have as significant an effect as with the Sun-Microsystems-sample-app.This was because even after splitting the transactions the transaction lengths were stillvery long (max transaction length had been reduced from approximately 1600 to 900items). However there was still a speed up of 14701.49 seconds (58.8%) over the FSM-NO approach. The most significant run-time improvement however for this databasewas with the FSM-NO-CIR approach. After the first FSM-NO-CIR run (min. support400) the database was reduced by 1044 items by replacing 348 occurrences of a fre-quent consecutive sequence of size 4. The run-time improvement for a min. supportof 300 was 24983.04 seconds (99.9%). The run-time for the FSM-NO-CIR approach re-mained low up to a min. support of 25. The database was reduced again by 654 itemsafter the min. support 100 run and 207 items after the min. support 50 run. For minsupport of 300 the FSM-W implementation failed to complete within 35000 secondsand was stopped.

Figure 7.9: Test Results on IBM Database

7.3.2 Applying FSM to Identify Design Flaws

In this subsection we show how we applied FSM to identify design flaws in two enter-prise applications. The first application is a sample Java enterprise application from

140

Sun Microsystems called Dukes Bank discussed previously in section 7.2.1. DukesBank is a sample e-commerce application that Sun provided to showcase the JEE tech-nology. We modified the application and introduced a number of delays in certainsections of the code to simulate different performance design issues. In particular weintroduced a delay in the business tier of the application to simulate a performance de-sign issue in this part of the application (a typical example for distributed enterprisesystems might be network latency as a result of making a number of remote calls).Next we introduced another delay into the JDBC wrapper to simulate a database is-sue. In fact we introduced a delay to give the impression of looping through databasecalls in a resource intensive fashion. This is a common problem in enterprise systemswhereby a large amount of information is often retrieved from the database resultingin significant performance degradation. Often it is the case that the information couldbe retrieved in a more efficient manner. As a result it is important to be able to identifywhen this occurs.

To identify performance issues in the application we monitored the running appli-cation and collected run-time paths corresponding to the different user actions thatcould be executed in the system. The run-time paths also contained performance in-formation. The performance data corresponded to the time spent in each method thatmade up the run-time path. The run-time paths were converted to flat transactions(augmented with the performance data). This resulted in the Sun-Microsystems-sample-app-performance database. Since we are interested in finding performance issues in theapplication, it was more interesting to identify sequences of resource intensive meth-ods rather than simply identifying frequent sequences. To identify resource intensivesequences we modified the FSM-NO implementation to use an alternative criteria ofinterest other than frequency, i.e. time spent in a sequence of methods (see section5.4.3).

Applying the modified FSM-NO implementation to the Sun-Microsystems-sample-app-performance database we quickly identified the most resource intensive methods in thesystem and also the most resource intensive method sequences. For the test we set themin. support to 100 milliseconds and the max. sequence length to 9 (i.e. the algorithmwould complete after candidates of size 9 were generated and their support counted).The algorithm run-time was 177 seconds. Although a large number of sequences weregenerated (3324) the most resource intensive methods and sequences could be easilyidentified by sorting by resources consumed (i.e. support). We correctly identified thetwo design flaws that had been introduced into the application. In fact from the outputwe could see that out of a total time the application was running (831.2 seconds), 95%(787 seconds) of the time was being spent in the database tier delay (in method Result-Set.next()) that we introduced. 4% of the time was spent in the business tier delay (inmethod AccountControllerBean.getDetails()). However the real advantage of usingFSM is that not only does it highlight the most resource intensive methods, but it alsogives the sequences in which they occurred (i.e. the run-time context) which is invalu-

141

able information for developers trying to solve design problems. From the output wecould see that the ResultSet.next() method was repeatedly being called across a num-ber of transactions in sequences of various sizes (i.e. loops in the run-time paths). OurFSM implementation was modified to record and output the transaction number inwhich the sequences occurred. Using this information we could easily identify theseresource intensive loops and the particular run-time paths in which they existed. Wecould also see from the output that the AccountControllerBean.getDetails() methodwas not executed in a looping sequence and occurred at most once per transaction,except in one instance where it was called twice.

The second application was an early beta version of the collaborative Workplace (en-terprise) application from IBM. Again we collected run-time paths from the applica-tion by performing different user actions. In total we collected 233 run-time paths fromthe application. We did not collect performance information for these tests (due to ourlimited access to the system) and thus had to rely on identifying the most frequentsequences of methods to find design flaws in the application. We applied the FSM-NO-CIR approach to the database (IBM-enterprise-app). Initially (min. support 400)we identified that the most frequent method occurred 1464 times across the 233 trans-actions. On closer inspection (min. support 30) we identified that the same methodmade up a consecutive sequence of size 24 occurring 58 times across the transactions(making up a total of 1392 calls). We made modifications to the original FSM imple-mentation such that it listed the transactions in which the sequence occurs and howoften for each particular transaction. From this we could see that for one particulartransaction the sequence of size 24 occurred 33 times. The transaction length was1617 (method calls). We identified 792 (24x33) of these calls were to one method. Themethod being called was an EJB Entity bean method. Entity beans are used in JEE toaccess persistent data in the database from the application tier. There are a numberof common design flaws that occur in these systems related to this kind of databaseactivity i.e. looping through the database from the application tier which is inefficient[161]. On reporting that we had identified these potential loops in the applicationtier to the IBM development team they confirmed that we had indeed identified a de-sign issue which they subsequently rectified for the release. Our approach allowedan individual unfamiliar with the internals of the application (e.g. a system tester) toquickly identify potential problems in the application and the (run-time) context inwhich these problems occurred. This information can be quickly used by developersto refactor the application.

7.3.3 Data Reduction Results

To show the effectiveness of the data reduction techniques discussed in section 5.5 wehave applied them to data collected from a JEE application under load. For this testwe loaded the Duke’s Bank sample e-commerce application with 40 users for a five

142

Figure 7.10: Class Diagram of a Modified Version of Duke’s Bank with Communica-tion Patterns Highlighted

minute period. Each user logged onto the system, browsed their account information,and deposited funds onto different accounts. In total each user performed 8 differ-ent user actions. A total of 1081 run-time paths were collected during this period. Toreduce the data produced we clustered the paths (a) by the component methods thatwere invoked in each path and (b) by the different components that were invokedin each path. After applying clustering criteria (a) we grouped the paths into 11 dif-ferent path clusters. That is, our cluster analysis reduced the 1081 paths recorded to11 (component-method level) paths through the system. In this instance statisticalanalysis can be applied to the component methods contained in each cluster to give asummary of the performance metrics associated with the run-time paths in each clus-ter. Applying the single user mode approach to the same user actions results in 11distinct call paths. Our results show that (in this instance) applying clustering analy-sis to data collected in multi-user mode can effectively reduce the number of distinctpath clusters to the number of different paths observed in single user mode. The pathclusters in multi-user mode in fact contain more useful information than the pathscollected in single user mode, since they give more realistic performance metrics foreach method that is invoked in the path. Applying clustering criteria (b) to the 1081paths resulted in 8 path clusters. That is, at the more abstract component level, therewere 8 different paths through the system.

143

7.4 PAD Tool Results

In this section we present results attained from applying the PAD tool to two JEEapplications i.e. a sample JEE application and a real large scale enterprise system.Section 7.4.1 details the test environment for our tests on the Duke’s Bank sampleapplication, modifications that we made to the application and our test results. Anumber of modifications were made to Duke’s Bank in order to introduce a numberof antipatterns for detection purposes. Section 7.4.2 gives the test environment for ourtests on a real JEE application from IBM called Workplace and the results from thesetests. Antipatterns from all categories outlined in section 6.1 have been identified. Foreach antipattern identified we describe the corresponding PAD tool output.

7.4.1 Antipatterns Detected in the Duke’s Bank Application

The first application we applied the PAD tool to, in order to identify performancedesign and deployment antipatterns, was Duke’s Bank. Duke’s Bank is an onlinebanking application. When a user logs in to the Duke’s Bank application he/she canperform the following actions: log on, view a list of accounts, view an individual ac-count’s details, withdraw or lodge cash, transfer cash from one account to another orfinally log off. For our tests each of the different (8) user actions was performed. COM-PAS JEEM was used to collect run-time paths. We instrumented the application sourcecode manually to perform object tracking, since COMPAS BCI could not run on the1.4.1 JVM that was used. Duke’s Bank was deployed on the JBoss application server(version 3.2.7) with a MySQL database (version 4.0.2) as the backend. Our MEJB mon-itoring tool was used to interface with the application server to collect informationon the server resources. For multi user mode (which was required to identify the In-correct Pool Size antipattern) the open source Apache JMeter load generation tool wasused to load the application. Two versions of dukes bank were tested, the original ver-sion and a modified version with a number of antipatterns added. The original dukesbank application consists of 6 EJBs (4 of these were invoked during the tests, see figure7.2 for a class diagram). We also modified the original version of duke’s bank to adda number of antipatterns to be detected by the PAD tool such that antipatterns fromall categories discussed in section 6.1 were present (see figure 7.10 for a class diagramof the modified version of dukes bank). The antipatterns introduced are describedin detail below. For each antipattern detected we give its corresponding category asdescribed in section 6.1. In total 3 antipatterns were detected in the original version ofDuke’s Bank by the PAD tool:

• Conversational Baggage Antipattern [161] (category 1): This antipattern de-scribes a situation where stateful sessions beans are being used but are not nec-essarily required. Stateful session beans maintain state on the application serverbetween client requests and should be used only when there is a clear need to

144

do so. Stateless session beans will scale better than stateful session beans andshould be used when state does not need to be maintained. Detection of thisantipattern involves flagging the creation of unusually high numbers of statefulsession beans across the run-time paths. A potential instance of this antipatternwas flagged by the PAD tool when Dukes Bank was analysed as the numberof stateful session beans was above the user defined threshold. This thresholdwas set to zero for the dukes bank application since there is no noticeable statemaintained from one user action to the next. When the potential antipattern wasdetected the PAD tool showed us that stateful session beans were used in 100%of the run-time paths where the business tier is invoked. On closer inspectionof the application (source code) we saw that indeed the stateful sessions couldhave been replaced with stateless sessions to improve scalability. In total 2 state-ful sessions could have been replaced.

• Fine Grained Remote Calls (also known as the Face Off antipattern [161]) (cat-egory 3): This antipattern describes a situation where a number fine grained callsare consistently made from a client to the same remote bean. This results in ahigh number of expensive remote calls. A better approach is (if possible) to makea more coarse grained call that performs the combined tasks of the fine-grainedcalls. Performing a single coarse grained call instead of a number of fine grainedcalls will reduce network latency and thus the overall time required for the useraction to execute. An instance of this antipattern was identified by the PAD tool.The tool identified a frequent sequence which contained fine grained remotecalls. In fact it identified that the remote methods AccountBean.getType and Ac-countBean.getBalance appeared together 100% of the time they were called (i.e.the sequence had a confidence value of 100%). Thus it would be more efficientto combine these calls into a single coarse grained remote call. This antipatternis far from severe in this instance, however the identification of this antipatternshows that the tool can indeed identify antipatterns in this category. The rule toidentify this antipattern can in fact be modified with a user defined threshold toonly flag more severe instances.

• Remote Calls Locally (also known as Ubiquitous Distribution [161]) (category2): This antipattern describes a situation where beans that run in the same JVMas the client are called through their remote interfaces. In this situation the clienthas to perform an expensive remote call even though the bean is local. Whilesome containers can optimize in this situation, this optimization is not a stan-dard feature of JEE. A better approach would be to write the beans with localinterfaces (instead of remote, provided the beans are not also called from remoteclients) such that they can be accessed in a more efficient manner. The PAD toolidentified this relationship between components. In fact all (4) beans invoked inthe Duke’s Bank application are accessed through remote interfaces even thoughtheir clients are within the same JVM.

145

Next we describe the antipatterns that were added to Duke’s Bank and the informa-tion given by the PAD tool when they were detected:

• Accessing Entities Directly [61](also known as Customers In The Kitchen [161]Antipattern) (category 2): The application was modified such that componentsin the web tier were directly accessing entity bean components (see figure 7.10).This antipattern can cause a number of different performance issues as docu-mented in the antipattern description (e.g. problems with transaction manage-ment). Furthermore it can create maintainability issues since it mixes presenta-tion and business logic. The PAD tool identifies the inappropriate componentrelationships and flags them as antipattern instances. The component identifiedwas the accountList.jsp which was modified to call the AccountBean (entity) di-rectly.

• Needless Session Antipattern (category 2): Another antipattern added was theneedless session antipattern. This antipattern is described in section 6.3 and out-lines a situation where a session bean is used but not required. In the Duke’sBank application we added a session bean that was being used to calculated thecurrent time and date. This function could have been easily carried out by aPOJO which would have been a more efficient solution. The antipattern wasdetected by the PAD tool which reported that the session bean (CalculateDate-Bean) had no relationships with any persistent components (e.g. entity beans ordatabase components) and thus was a potential needless session bean.

• Application Filter Antipattern [161] (category 3): The application filter antipat-tern describes a situation where large amounts of data are retrieved from thedatabase tier and filtered in the application tier. However, databases are de-signed to process filters a lot faster than can be done in the application tier. Thusfilters should, where possible, be implemented in the database tier. We modifiedthe Duke’s Bank application to include an application filter. The dukes bankapplication can retrieve account information for a customer that has logged in.When this is performed the application performs a search for the accounts thatare owned by the particular user. We created an application filter in this situa-tion by modifying the SQL statement which filtered the information (using thefollowing SQL statement ”select account id from customer database where cus-tomer id = ?”) to instead retrieve all accounts and send them to the applicationtier for filtering. In the application tier, filtering of the data was performed bychecking the details of each account to see if it matched the id of the customer.We applied the PAD tool to the modified version of Dukes Bank. The tool suc-cessfully identified the application filter that had been added to the application.The tool identified two frequent sequences of communication: (1) between a ses-sion bean (AccountControllerBean) and an entity bean (AccountBean) (see figure6.3 on page 117) and (2) between the same entity bean (AccountBean) and the

146

database in these paths. The first sequence of size 1 (AccountBean.getDetails)occurred 1180 times across the different run-time paths. The second sequence ofsize 2 (ResultSet.next, ResultSet.getString) occurred 6336 times across the run-time paths. It was evident from this information that an application filter existedin the application, especially when considering that this the amount of activitywas occurring in single user mode. In fact the PAD tool identified that by mod-ifying this filtering 7/11 of run-time paths collected during the test run wereaffected.

• Unused Data Object/Aggressive Loading (category 4) 12: A second antipatternwas identified when the PAD tool was applied to the dukes bank applicationmodified with an application filter antipattern. The antipattern detected was theunused data object antipattern/aggressive loading antipattern. This antipatterndescribes a situation whereby information is loaded from the database, but thedata (or at least a high percentage of the data) is never used. A solution to thisproblem can be to refactor the application such that information is not retrievedif it is never actually required. Often however the information may be required asmall percentage of the time. In such circumstances the information can be lazyloaded when it is required. This antipattern occurred in the application filterabove. During filtering a check is performed to check each account to see if itsowner id matched that of the (logged in) customer’s. To obtain each account’sowner id, the application loads the account information from the database intoan entity bean. An AccountDetails Data Transfer Object (DTO) [7] is then cre-ated by the entity bean with the account information. Finally during the filter-ing the DTO is accessed to obtain the account owner id. Rather than loading allthe account information into the entity bean, and subsequently into the DTO, alazy loading approach can be used to load only the information that it generallyneeded (i.e. the account owner id). If the remaining (unloaded) information isrequired it can be loaded later. This antipattern will in fact be removed if theapplication filter is pushed into the database tier as suggested in the antipat-tern solution for the application filter. However there may be situations where itmay not be possible to easily remove the application filter and where lazy load-ing can be applied. Lazy loading can in fact be applied in any situation whereonly a small proportion of the entity bean fields are ever accessed. The PAD toolprovided the following information when this antipattern was identified: TheAccountDetails Object was created on average 1886 times across the 7 run-timepaths with a maximum value per path of 2400 times and a minimum value of1200 times. The object has 8 accessor methods. Each method is given below withthe corresponding average number of times it was accessed per run-time path:getCustomerID’s 1886 times, getType 1.4 times, getDescription 3 times, getBal-ance 2.6 times, getAccountId 8.1 times, getCreditLine 2.3 times, getBeginBalance

12Precise Java Website, http://www.precisejava.com

147

0 times and getBeginBalanceCreditLine 0 times. From the PAD tool output it isevident that (if the application filter could not be removed) it would be benefi-cial from a performance perspective to apply lazy loading for the account beanand DTO in this application to all fields except for the CustomerIds field.

• Incorrect Stateful Session Instance Cache Size (category 5) 13: The final an-tipattern added to the dukes bank application was a deployment antipattern.We redeployed the application and modified the size of the stateful session in-stance cache to 10 (from the default 1000000). To detect this antipattern we ranthe tests in multi user mode. We loaded the application with 40 users (over 10seconds) and monitored the system for a 5 minute period. We specified the max-imimum passivation level as 4 (10% of user load) in (the user configuration filesassociated with) our antipattern rules. The passivation level is the number ofbeans that are passivated at any one time. Since passivation can be an expensivetask, it should ideally be kept to a minimum. The PAD tool detected the statefulsession beans instance cache size antipattern and presented the following infor-mation to the tool user: The AccountControllerBean instance cache passivationlevels exceeded the specified user threshold of 4. The average passivation levelduring the measured period was 15. In this situation increasing the size of theinstance cache is recommended.

Next we discuss the issue of false positives and negatives detected by the PAD toolwhen applied to Duke’s Bank. In the strictest sense no false positives were found dur-ing the tests i.e. the tool did not identify antipatterns instances that were not present inthe system. However with performance related antipatterns we are more concernedwith identifying antipatterns that have an impact on the overall system performance.It is likely that the fine grained remote calls antipattern instance would not have asignificant impact on the system performance and thus might be considered a falsepositive in this instance. However, by modifying the user defined threshold asso-ciated with the rule to detect this antipattern we can filter out instances with a lowperformance impact. Our aim was to show that instances of this antipattern can beidentified by our tool and thus we set the threshold value such that even insignificantinstances (from a performance perspective) were also identified. Similarly the remotecalls locally antipattern may not have a performance impact in application servers thatcan optimize remote calls that are made to local components. However, again our aimwas to show that this antipattern can be identified using our tool.

By studying the Duke’s Bank documentation 14 and source code we were confidentthat our tests did not produce false negatives i.e. there were no antipatterns in theapplication, which were defined in our antipattern library that we did not detect.

13Precise Java Website, http://www.precisejava.com14Duke’s Bank Documentation, http://java.sun.com/javaee/5/docs/tutorial/doc/

148

7.4.2 Antipatterns Detected in the IBM Workplace Application - BetaVersion

The second system tested was an early beta version of the IBM Workplace Applica-tion15. IBM Workplace is a collaborative enterprise application built on the JEE tech-nology. In total 76 EJBs were instrumented, 38 of these were only ever invoked duringthe test runs (17 entity beans with Container Managed Persistence and 21 Sessionbeans). The test run consisted of invoking 25 different user actions. Monitoring wasperformed using the COMPAS JEEM tool (COMPAS BCI was unavailable at the timeof testing and could not have been applied to the JVM version we were using). Objecttracking was not performed and performance metrics were not collected during thesetests due to our limited access to the system. All tests were carried out in single usermode. The IBM Workplace application was running on the IBM WebSphere applica-tion server (version 5.x). The database used was IBM’s DB2. Using the PAD tool weidentified antipatterns from four of the different categories outlined in section 6.1:

• Local and Remote Interfaces Simultaneously (category 6): This antipattern oc-curs when a bean exposes both local and remote interfaces. There are a numberof issues associated with this antipattern including exception handling issues,security issues and performance problems. From the 76 beans instrumented thePAD tool identified 30 (session) beans that exposed both local and remote in-terfaces. Many of the beans identified were not invoked during the test run,however the required information for antipattern detection in this instance wasavailable in the beans meta data. On identification of this antipattern we con-tacted the development team who acknowledged that this was indeed an an-tipattern that had been removed in a later release due to security concerns.

• Unusual/Bulky Session-Entity Communication (category 3): The second an-tipattern type identified by the PAD tool was in relation to database commu-nication. The tool identified unusually high communications between sessionand (persistent) entity beans (see rule in figure 6.3 on page 117). In fact as aresult of the 25 user actions, a frequent sequence of entity bean calls (from a ses-sion bean) of size 24 occurred 58 times. In fact in one particular run-time paththis sequence occurred 33 times. The 24 calls in the sequence were all to thesame method which suggested that the sequence was in fact a loop of size 24.From this data it seemed that there was potentially an application filter causingthis issue. Again we contacted the development team responsible for this code.The development team had identified this antipattern in a later release and hadrectified it by pushing the filtering into the database tier as suggested by theapplication filter antipattern solution.

• Transactions-A-Plenty/Incorrect Transaction size [61](category 1): Another an-

15IBM Workplace, http://www.ibm.com/software/workplace

149

tipattern detected by the PAD tool was the transactions-a-plenty antipattern.The tool identified that for one particular use case a high number of transac-tions were being created. In fact the tool identified that 131 transactions werecreated in a single run-time path. In fact, for every session bean method call,a transaction was being initiated. On further inspection we discovered that theissue was that the session beans methods’ transactional settings were being setto ”Requires New” by default. The development team addressed this issue byediting the transactional settings for the beans in the deployment descriptors toinitiate transactions only where new transactions were actually required.

• Bloated Session Bean Antipattern [61](category 2): The final antipattern typedetected was the bloated session bean antipattern. This antipattern is similar tothe well known God class antipattern [36]. It describes a situation in EJB sys-tems where a session bean has become too bulky. Such session beans generallyimplement methods that operate against a large number of abstractions. Forexample one method may check the balance of an account while another mayapprove an order, and yet another may apply a payment to an invoice. Suchsessions can create memory and latency issues since their creation and manage-ment can come with much overhead due to their bulky nature. The PAD toolidentified two potential bloated session beans in the IBM Workplace applica-tion. The first potential instance of this antipattern was a session bean whichhad 8 entity bean relationships, 6 session bean relationships and was invoked in11 of the 25 user actions executed. It also contained a high number of businessmethods (47). The second instance of this antipattern was a session bean whichhad 7 entity relationships 1 session relationship and was invoked in 6 of the 25user actions. This session had 14 business methods. In general the rule of thumbis that there should be a 1 to 1 relationship between session and entity beans.From the information (above), presented by the PAD tool upon identification ofthese potential antipattern instances, it seemed that these session beans were infact bloated session beans. Unfortunately we were unable to contact the devel-opers originally responsible for this code. However we did discover that thiscode was removed from later releases of the application which indicated that itwas indeed problematic.

No false positives were identified when we applied the PAD tool to the IBM work-place application and in fact all antipatterns identified were addressed in the sub-sequent versions of the application which suggested they were indeed problematicpieces of code. Unfortunately we could not assess whether false negatives were iden-tified in the application as we did not have access to the complete system source codeor documentation.

150

7.5 Validation

In this section we revisit the validation criteria outlined in section 3.1.1 as well as thecontributions which we have claimed. We discuss how the contributions have beenvalidated in respect to these criteria either through the tests that have been carried outin this chapter or by other means.

7.5.1 Overview of Contributions and Evaluation Criteria

We have claimed the following contributions in this thesis:

• An approach for the automatic detection of design and deployment antipatternsin systems built using component based enterprise frameworks.

• A portable, low overhead, non-intrusive, end-to-end run-time path tracer forJEE systems.

• A technique for the identification of interesting component communication pat-terns in a collection of run-time paths.

In chapter 3 we outline a number of validation criteria to evaluate our work against.These criteria are attributes which we believe are required in order for our approachto be considered useful. The criteria are as follows:

• The monitoring approach should be portable.

• The monitoring approach should be non-intrusive.

• The monitoring approach should exert a low-overhead on the system under test.

• The monitoring approach should be end-to-end.

• The monitoring approach should be applicable in distributed environments.

• Analysis and detection should be efficient.

• The number of false positives detected should be low.

• The approach should be user configurable and extensible.

• The approach should scale when applied to large real world enterprise applica-tions.

The above criteria are discussed in more detail in chapter 3. Next we discuss howthese criteria have been met.

151

7.5.2 Validation of Contributions

The first contribution outlined above has been realised in the form of the PAD tool.We have evaluated the tool by applying it to two JEE applications (see section 7.4).Our results show how it has been used to correctly identify a number of performancedesign and deployment antipatterns in these systems. We have shown how the num-ber of false positives was low when applied to both applications. The fact that thistool has been applied to identify real problems, in a real large scale JEE application,shows that our approach scales when applied to real systems. Applying our tool toreal systems also shows that the problems we try to detect do indeed exist in realityand that they can be automatically detected using this approach. The PAD tool is ex-tensible and user configurable, since it allows for custom antipatterns to be addedand user specified thresholds to be set.

Our tests on the analysis module (see section 7.3) show that the approach is efficientin terms of the time taken for data analysis. The most time consuming component ofthe analysis is the extraction of communication patterns (the third contribution fromabove). Our test results show that this can be performed in a timely manner usingour preprocessing techniques (see section 7.3.1). While the extraction of communi-cation patterns is currently semi-manual (using CIR), this process is not particularlytime consuming or arduous. We also believe it can be automated in the future. Thisapproach has been shown to scale well in relation to data from real enterprise systemsin terms of both the output produced and its run-time. We have also shown in ourtests that the amount of data output has been significantly reduced using our non-overlapping support counting technique, in comparison to weighted support count-ing (see section 7.3.1). Finally the usefulness of extracting component communicationpatterns has also been highlighted. Our results show the output from FSM applied torun-time paths can be used to (manually) identify serious performance design flawsin sample and real JEE applications (see section 7.3.2).

The second contribution outlined above has also been evaluated on a number of lev-els. Firstly our results show that it is truly portable across different middleware im-plementations (see section 7.2.2). Secondly it has been shown to be a low-overheadmonitoring approach (see section 7.2.3). Performance tests show that it has no signif-icant impact on a loaded system running under normal conditions. The monitoringtool has been applied to all server side tiers of a number of JEE applications and iscompletely end-to-end. A solution for the tracking of physically distributed calls hasalso been presented in section 4.2.5.3. Finally the output of the monitoring tool hasbeen shown to be particularly effective for reverse engineering purposes and has beenapplied to reserve engineer a number of JEE applications such that design flaws canbe (manually) identified (see section 7.2.1).

152

CHAPTER

EIGHT

Conclusions

In this chapter we give our conclusions. We also outline the limitations of this ap-proach. Finally we give some ideas on directions for future work in this area.

153

8.1 Thesis Conclusions

This thesis aims to address an issue in relation to today’s enterprise applications: de-velopers very often suffer from a lack of understanding in relation to these systemsand as a result make suboptimal design choices which can lead to major performanceissues. To address this issue we present an approach to automatically detect performancedesign and deployment antipatterns in component based enterprise applications. The thesisoutlines the various components that are required in order to be able to achieve per-formance antipattern detection in such systems. Namely, a monitoring module, ananalysis module and a detection module.

The thesis outlines why dynamic analysis is required for detecting performance re-lated antipatterns in such systems and how it can be achieved in a portable, non-intrusive and low overhead manner. We discuss the various techniques that can beused to collect information from the system such that a complete representation of thesystem’s run-time design can be reconstructed. In particular we present an approach andimplementation for portable, low-overhead and non-intrusive tracing of run-time paths in JEEapplications. We also show how information, such as server resource information andcomponent meta-data, can be obtained.

The thesis further outlines how, from the monitoring data, the run-time design of theapplication can be automatically extracted. A reconstructed design model of the sys-tem is required such that the system design can be further analysed for antipatterndetection. We show how to extract information from the monitoring data, that formsthe basis of the run-time design model. In particular we have shown how interesting com-ponent communication patterns can be identified in run-time paths using frequent sequencemining. The design model is constructed by extracting the following information andadding it to the model: component structural and functional information, componentrelationships, component communication patterns and component object usage pat-terns. Reconstructed run-time container services that are utilised by the componentsare also added. Furthermore we show how large volumes of data recorded at run-timecan be summarised and how this summarised data can be used to enhance the designmodel.

In addition, this thesis shows how the design model can be further analysed such thatperformance design and deployment antipatterns can be detected within it. Detec-tion can be performed using a rule-engine based approach. An antipattern library isprovided where by antipatterns are described in the form of rule-engine specific rules.

We have outlined a number of validation criteria that we have used to evaluate ourwork against. We have shown through empirical studies (i.e. through case studiesand experiments) how our work meets these criteria.

154

8.2 Limitations

In this section we detail the limitations of this approach and how we feel they can beaddressed:

• Dynamic Analysis: Since the approach outlined in this thesis relies heavily ondynamic information it is not possible to identify antipatterns in the parts ofthe system that are never actually executed. Thus we assume a realistic testingscenario is available.

• Run-Time Path Tracing Tool: we found during our tests that the main limitationof this approach was the fact that it the instrumentation process requires the sys-tem to be redeployed. During our initial tests on smaller sample applications,which were usually packaged as a single archive file, redeployment was a verysimple task. On real larger systems however (e.g. IBM Workplace) the systemwas made up of a large number of different components and deployment wasmore complex. Thus, instrumenting the application was more time consumingthan had been initially imagined. We believe this can be solved by modifyingthe monitoring approach used by the run-time path tracing tool. The moni-toring approach can be modified to make use of JVMTI and dynamic bytecodeinstrumentation to allow for instrumentation without redeployment. Howeverthis technique will only be applicable to newer JVMs (1.5+) and thus could nothave been applied to the applications we have tested in this work. An initiativeis already underway in this regard, i.e. COMPAS BCI (see section 4.2.5.8).

• Applying FSM to Run-time Paths: The main limitation to this approach is factthat it is currently semi-manual. That is, a user of the approach is required toidentify consecutive items and to replace them. We found during our tests thatthis can be easily performed, making use of the results from the initial FSMrun and by using simple search and replace functions of a text editor. Thetask is not particularly time consuming or difficult. However we believe thatthe task could be easily automated to allow for complete automation. In factafter the initial FSM run, the results could be easily analysed and any consec-utive items in the database could be automatically replaced. A record of thechanges/replacements would be maintained and a reversal process could becarried out after the CIR process has finished. This work will form part of ourplans for future work. Another issue with this approach is that it can only be ap-plied when the output from the initial FSM run contains consecutive sequences.While, from our results, its seems that this will be the case for typical JEE sys-tems, if a systems is highly dynamic in terms of its run-time bindings this maynot necessarily occur. An alternative approach in this situation might be to useparental splitting of the transactions (see section 5.4.4.1). However, as shownin our results section (7.3.1), this approach may not be effective for long trans-

155

actions. Further research is required in this area to identify how FSM can beapplied to highly dynamic enterprise systems.

• PAD Tool Output: The PAD tool currently presents all antipatterns identifiedto the tool user with corresponding contextual information. The informationpresented (e.g. antipattern description, performance metrics) can be used by thetool user to easily determine the impact of the antipatterns detected. Howeverit would be more desirable that the performance impact assessment would beautomated. This would allow for the PAD tool to rank antipatterns instancesof the same type according to their performance impact and developers couldconcentrate their efforts on addressing the most significant issues first. This willform part of our ideas for future work.

• PAD Tool: There are a number of steps in the PAD tool detection process thathave not been fully automated and which require human intervention. A fullyautomated tool is currently being developed.

• Message Driven Beans: In this work we do not address issues related to messagedriven beans. We consider only session and entity beans. A number of antipat-terns however exist for message driven beans 1 [161] [61] and research in thisarea could form part of future work.

8.3 Future Work

In this section we detail interesting areas for future work in this area:

• Monitoring: We plan to address the limitations detailed above in relation to ourmonitoring module. In particular we plan to remove the need for system re-deployment during the instrumentation process. This can be achieved throughdynamic bytecode instrumentation. Dynamic bytecode instrumentation can beperformed for newer versions of the JVM (1.5+) through the JVMTI. Work inthis area is already underway and forms part of the COMPAS BCI tool. Anotherlimitation of our monitoring approach is that it has not been applied to asyn-chronous components (such as message driven beans). We plan to investigatehow the run-time path tracing approach in particular can be applied to messagedriven beans. This could most likely be achieved by modifying our proxy com-ponents such that they intercept calls to the onMessage method of the messagedriven beans.

• Integration with Real Time Correlation Engine: A proof of concept, real-timecorrelation engine has recently been implemented with the IBM Dublin Soft-

1Precise Java, http://www.precisejava.com/

156

ware Lab [46]. The tool correlates server logs from the different servers on a dis-tributed system such that a single log is available to system testers at run-time.The solves the problem of the system testers having to correlate a range of logsfrom different machines in order to understand the results of their tests. The toolcan allow for correlation of log files by various criteria, (e.g. time stamp, user id,error type) and allows for easier and quicker problem determination. Currentlyan industry research project is underway to integrating this tool with COMPASJEEM, such that system testers can analyse both system level information (e.g.the server logs) and application level information (e.g. run-time paths). The aimis to give the system tester a complete picture of the system under test, i.e. bothat the system level and application level.

• Analysis: Future work can be carried out to address the limitations of the FSMCIR approach by fully automating the replacement approach. The results fromapplying FSM to run-time paths show that this information is indeed useful onceit can be extracted. An interesting area for future work would be to investigatesliding window approaches for finding episodes in sequences [112] to assesswhether they can be modified such that they can be applied to a transactionaldatabase (e.g. a database of run-time paths). To date this approach has generallybeen applied to event sequence data. The benefit of this approach over the FSMapproach using weighted support is that it does not suffer from combinatorialexplosion when transactions grow in length and can not be reduced in size usingpreprocessing techniques.

• Antipattern documentation: Technology specific antipatterns have been welldocumented for the Java technology covering pitfalls for both base java and thejava enterprise technology. Enterprise applications are often built using othertechnologies/enterprise frameworks (e.g. .Net, CCM, Spring 2). However, weare unaware of any literature that details instances of antipatterns in the othercommon enterprise technologies. We firmly believe that similar problems con-sistently occur in the other enterprise technologies. Documenting these prob-lems in the form of antipatterns would bring understanding to developers build-ing systems using these technologies.

• Antipattern detection for other technologies: The creation of a catalog of antipat-terns for alternative enterprise technologies would allow for the automatic an-tipattern detection using the approach we have outlined in this thesis. Researchwould be required to develop or identify a monitoring framework that couldperform call path tracing. Much of the call path tracing logic and mechanismswe present could be reused. The analysis and detection modules are portableacross different technologies.

2The Spring Framework, http://www.springframework.org/

157

• Automatic Antipattern Assessment: Assessment of detected antipatterns wouldallow for automatic ranking of antipatterns in terms of the antipattern perfor-mance impact. This would allow developers to address the most significantantipatterns first. This can be achieved using predefined cost functions. Costfunctions can be potentially created for each antipattern described in the an-tipattern library. Cost functions could be created to rank instances of the sameantipattern. Cost functions describe antipatterns performance characteristics,and using the performance metrics collected by the monitoring module wouldallow for automatic assessment of the antipatterns performance impact.

158

REFERENCES

[1] M. K. Agarwal, M. Gupta , G. Kar, A. Neogi and A. Sailer, Mining ActivityData for Dynamic Dependency Discovery in e-Business Systems, IEEE eTransac-tionson Network and Service Management Journal, Vol.1 No.2, September,2004

[2] R. Agrawal, H. Mannila, R. Srikant, H. Toivonen and A.I. Verkamo. Fastdiscovery of association rules. In Advances in Knowledge Discovery and DataMining, 1996.

[3] R. Agrawal and R. Srikant. Mining sequential patterns. In P. S. Yu and A. L.P. Chen, editors, In Proceedings 11th International Confonference on DataEngineering, 1995

[4] R. Agrawal, T. Imielinski, and A.N. Swami. Mining association rules betweensets of items in large databases. In P. Buneman and S. Jajodia, editors, In Pro-ceedings of the 1993 ACM SIGMOD International Conference on Manage-ment of Data, volume 22(2) of SIGMOD Record, 1993.

[5] H. Albin-Amiot, P. Cointe, Y. G. Gueheneuc, N. Jussien. Instantiating andDetecting Design Patterns: Putting Bits and Pieces Together, 16th IEEE Interna-tional Conference on Automated Software Engineering (ASE’01), 2001.

[6] C. Alexander. A Timless Way of Building. Oxford University Press, 1979.

[7] D. Alur, J. Crupi and D. Malks. Core J2EE Patterns: Best Practices and DesignStrategies. Prentice Hall, Sun Microsystems Press, 2001

[8] G. Ammons, T. Ball and J.R. Larus, Exploiting hardware performance counterswith flow and context sensitive profiling. ACM Conference on ProgrammingLanguage Design and Implementation. ACM Press, New York, 1997.

[9] G. Ammons, J.D. Choi, M. Gupta, and N. Swamy. Finding and RemovingPerformance Bottlenecks in Large Systems., In Proceedings of ECOOP, 2004.

[10] J.M. Anderson, L.M. Berc, J. Dean, S. Ghemawat, M.R. Henzinger, S.A. Le-ung, R.L. Sites, M.T. Vandervoorde, C.A. Waldspurger and W.E. Weihl. Con-tinuous profiling: Where have all the cycles gone? Proceedings of the Sympo-sium on Operating Systems Principles. ACM Press, New York, 1997.

[11] G. Antoniol, G. Casazza, M. Di Penta, and R. Fiutem. Object-oriented designpatterns recovery. Journal of Systems and Software, 2001.

159

[12] M. Arnold and D. Grove. Collecting and Exploiting High-Accuracy Call GraphProfiles in Virtual Machines. International Symposium on Code Generationand Optimization, San Jose, California, March, 2005.

[13] Z. Balanyi, R. Ferenc. Mining Design Patterns from C++ Source Code. 19th IEEEInternational Conference on Software Maintenance, 2003.

[14] T. Ball and S. K. Rajamani. The SLAM project: debugging system software viastatic analysis, In Proceedings of the 29th POPL, January 2002.

[15] S. Balsamo, A. Di Marco, P. Inverardi, M. Simeoni. Model-Based PerformancePrediction in Software Development: A Survey. IEEE Transactions on SoftwareEngineering, , pp. 295-310, Vol. 30, No. 5, MAY 2004.

[16] J. Bansiya. Automating Design-Pattern Identification. Dr. Dobb’s Journal, vol.23, no. 6, June, 1998.

[17] P. Barford and M. Crovella, Generating representative web workloads for networkand server performance evaluation. In ACM SIGMETRICS, 1998.

[18] P. Barham, A. Donnelly, R. Isaacs and R. Mortier. Using Magpie for requestextraction and workload modelling, Symposium on Operating Systems Designand Implementation, pp 259–272, San Francisco, CA, USA, December 2004.

[19] P. Barham, R. Isaacs, R. Mortier and D. Narayanan. Magpie: online modellingand performance-aware systems. In the 9th Workshop on Hot Topics in Oper-ating Systems, May, 2003.

[20] V.R. Basili. The Experimental Paradigm in Software Engineering. Experimen-tal Software Engineering Issues: Cirtical Assessment and Future Directives,Springer-Verlang, Lecture Notes in Computer Science, 1993.

[21] L. Bass, P. Clements and R Kazman. Software Architecture in Practice (2ndEdition). Addison-Wesley, 2003.

[22] I. D. Baxter, C. Pidgeon, M. Mehlich. DMS: Program transformations for prac-tical scalable software evolution. Proceedings of the IEEE International Confer-ence on Software Engineering. IEEE Computer Society Press: Los Alamitos,CA, 2004.

[23] J. Beck, D. Eichmann, Program and Interface Slicing for Reverse Engineering. InProceedings of the International Conference on Software Engineering, 1993.

[24] K. Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley,1999.

[25] K. Beck. Test Driven-Development by Example, Addison-Wesley, Boston, MA,2003.

[26] K. Beck and W. Cunningham. Using Pattern Languages for Object-OrientedPrograms, Technical Report No. CR-87-43 submitted to OOPSLA-87 work-shop on the Specification and Design for Object Oriented Programming,September, 1987.

160

[27] F. Bergenti and A. Poggi. Improving UML Designs Using Automatic DesignPattern Detection. Proceedings of the 12th International Conference on Soft-ware Enginneering and Knowledge Engineering, July, 2000.

[28] P. Berkhin. Survey of clustering data mining techniques. Technical report, Ac-crue Software, San Jose, CA, 2002.

[29] D. Beyer, A. Noack and C. Lewerentz. Efficient Relational Calculation for Soft-ware Analysis. IEEE Transactions on Software Engineering, vol. 31, no. 2,February, 2005.

[30] W. Binder, Portable and accurate sampling profiling for Java. Software Practiceand Experience, Vol. 36, Issue 6, John Wiley and Sons, Inc., New York, NY,USA, February, 2006.

[31] W. Bischofberger, J. Kuhl and S. Loffler. Sotogrpah - a Pragmatic Approach toSource Code Architecture Conformance Checking. Proceedings First EuropeanWorkshop on Software Architecture, Lecture Notes in Computer Science3047, Springer, 2004.

[32] F. Bodon, A Trie-based APRIORI Implementation for Mining Frequent Item Se-quences. ACM SIGKDD Workshop on Open Source Data Mining Workshop(OSDM’05), pages 56 - 65, Chicago, IL, USA. 2005

[33] M. D. Bond and K. S. McKinley. Continuous Path and Edge Profiling. 38th In-ternational Symposium on Microarchitecture, Barcelona, November, 2005.

[34] L.C. Briand, Y. Labiche, and J. Leduc, Towards the Reverse Engineering of UMLSequence Diagrams for Distributed, Multithreaded Java Software. Technical Re-port SCE-04-04, Carleton Univ., http://www.sce.carleton.ca/Squall, Sept.2004.

[35] L.C. Briand, Y. Labiche and J. Leduc. Toward the Reverse Engineering of UMLSequence Diagrams for Distributed Java Software. IEEE Transactions on Soft-ware Engineering, vol. 32, no. 9, September, 2006.

[36] W. J. Brown, R. C. Malveau, and T. J. Mowbray. AntiPatterns: RefactoringSoftware, Architectures, and Projects in Crisis. Wiley, 1998.

[37] E. Bruneton, T. Coupaye, M. Leclercq, V. Quema, and J.-B. Stefani. The Frac-tal Component Model and Its Support in Java. Software Practice and Experi-ence, special issue on Experiences with Auto-adaptive and ReconfigurableSystems. 2006.

[38] A. Buble, L. Bulej and P. Tuma. CORBA Benchmarking: A Course with HiddenObstacles. Proceedings of IPDPS Workshop on Performance Modeling, Eval-uation and Optimization of Parallel and Distributed Systems, Nice, France,2003.

[39] F. Buchli. Detecting Software Patterns using Formal Concept Analysis, DiplomaThesis, University of Bern, 2003.

[40] M. Burch, S. Diehl and P. Weissgerber. Visual data mining in software archives.Proceedings of the 2005 ACM symposium on Software visualization, St.Louis, Missouri, 2005.

161

[41] D. Burdick, M. Calimlim, J. Flannick, J. Gehrke and T. Yiu, MAFIA: A Max-imal Frequent Itemset Algorithm. IEEE Transactions on Knowledge and DataEngineering, vol. 17, no. 11, pp. 1490-1504, Nov., 2005.

[42] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad and M. Stal. Pattern-Oriented Software Architecture, A System of Patterns, Volume 1. Wiley, 1996.

[43] E. Cecchet, J. Marguerite and W. Zwaenepoel. Performance and Scalability ofEJB Applications. Proc. 17th ACM SIGPLAN Conference on Object-OrientedProgramming, Systems, Languages, and Applications, pp 246-261, Seattle,Washington, USA, 2002.

[44] M. Chen, E. Kiciman, A. Accardi, A. Fox and E. Brewer. Using runtime pathsfor macro analysis. In Proceedings 9th Workshop on Hot Topics in OperatingSystems, Lihue, HI, USA, May 2003

[45] M. Chen, E. Kiciman, E. Fratkin, A. Fox and E. Brewer: Pinpoint: Problem De-termination in Large, Dynamic, Internet Services. In Proceedings InternationalConference on Dependable Systems and Networks (IPDS Track), Washing-ton, D.C., June 2002

[46] L. Chen, P. O’Sullivan, L. P. Bergman, V. Castelli, E. Labadie, P. Sohn, T. Par-sons. Problem Determination in Large Enterprise Systems. At the Software En-gineering Tools For Tomorrow (SWEFT) 2006 conference, T.J. Watson, NewYork, Oct 17 - 19, 2006. (Abstract only available)

[47] H. Chen and D. Wagner. MOPS: an infrastructure for examining security prop-erties of software. In Proceedings of the Ninth ACM Conference on Computerand Communications Security, Washington, DC, November 2002.

[48] E. J. Chikofsky and J. H. Cross II. Reverse Engineering and Design Recovery: ATaxonomy. IEEE Software, 7(1):13– 17, January/February 1990.

[49] O. Ciupke. Automatic detection of design problems in object-oriented reengineer-ing, In proceedings Technology of Object-Oriented Languages and Systems,1999.

[50] P. Clements. A Survey of Architecture Description Languages. In Proceedings ofthe 8th International Workshop on Software Specification and Design, 1996.

[51] P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, R. Nordand J. Stafford. Documenting Software Architectures: Views and Beyond. Boston:Addison-Wesley, 2003.

[52] P. Clemeents, R. Kazman and M. Klein. Evaluating Software Architectures,Methods and Case Studies Addison-Wesley, 2001.

[53] J. Coplien. Advanced C++ Programming Styles and Idioms. Addison-Wesley,1991.

[54] J. Coplien and N. Harrison. Organizational Patterns of Agile Software Develop-ment. Prentice-Hall, 2004.

[55] J. Corbett, M. Dwyer, J. Hatcliff, C. Pasareanu, Robby, S. Laubach, H. Zheng.Bandera: Extracting Finite-state Models from Java Source Code, In Proc. of ICSE,June 2000.

162

[56] G. Costagliola, A. De Lucia, V. Deufemia, C. Gravino, and M. Risi. DesignPattern Recovery by Visual Language Parsing, Proceedings of the Ninth Eu-ropean Conference on Software Maintainance and Reengineering, March,2005.

[57] M. Dahm. Byte code engineering with the BCEL API. Technical Report B-17-98, Freie Universit¨at Berlin, 2001.

[58] G. Denaro, A. Polini, and W. Emmerich, 2004. Early performance testing ofdistributed software applications. In Proceedings of the 4th international Work-shop on Software and Performance, Redwood Shores, California, 2004.

[59] M. Dmitriev, Profiling Java applications using code hotswapping and dynamiccall graph revelation. In Proceedings of the 4th international workshop onSoftware and performance, Redwood Shores, California, 2004.

[60] M. Dodani: Patterns of Anti-Patterns, in Journal of Object Technology, vol. 5,no. 6, pp. 29-33, July - August 2006.

[61] B. Dudney, S. Asbury, J. K. Krozak and K. Wittkopf. J2EE Antipatterns, Wiley,2003.

[62] A. H. Eden and R. Kazman. Architecture, Design, Implementation. In pro-ceesings of the International Conference on Software Engineering, Portland,USA, May, 2003.

[63] M. El-Attar and J. Miller. Matching Antipatterns to Improve the Quality of UseCase Models. 14th IEEE International Requirements Engineering Conference,Minnesota, USA, 2006.

[64] S. Elbaum and M. Diep,Profiling Deployed Software: Assessing Strategies andTesting Opportunities. IEEE Transactions on Software Engineering, Vol. 31,No. 4, April 2005.

[65] G. Florijn, M. Meijers, and P. van Winsen. Tool support for object-oriented pat-terns. In Ecoop 1997: Object-Oriented Programming, volume 1241 of LectureNotes in Computer Science, 1997.

[66] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. The KDD process for extract-ing useful knowledge from volumes of data. Communications of the ACM, vol-ume 39, issue 11, pp27-34, November 1996

[67] M. Fowler. Patterns of Enterprise Application Architecture. Addison-WesleyProfessional, 2002

[68] M. Fowler. Refactoring: improving the design of existing code. Addison-Wesley,1999.

[69] E. Friedman-Hill. Jess in Action. Manning Publications, July, 2003.

[70] L. Fulop, T. Gyovai, and R. Ferenc. Evaluating C++ Design Pattern MinerTools. In Sixth IEEE International Workshop on Source Code Analysis andManipulation, Sep. 2006.

163

[71] E. Gamma , R. Helm, R. Johnson and J. Vlissides. Design Patterns: Abstrac-tion and Reuse of Object-Oriented Design. European Conference on Object Ori-ented Programming, 1993

[72] E. Gamma , R. Helm, R. Johnson and J. Vlissides: Design Patterns: Elementsof Reusable Object-Oriented Software, Addison-Wesley, 1995

[73] D. Garlan and M. Shaw. An introduction to software architecture. In Advances inSoftware Engineering and Knowledge Engineering. Volume 1. World ScientificPublishing Co., 1993.

[74] R. Glass. The Software Research Crisis. IEEE Software, November, 1994.

[75] B. Goethals, and M. J. Zaki , ”Advances in Frequent Itemset Mining Implemen-tations: Report of FIMI 2003”, ACM SIGKDD Explorations; Vol. 6, nr. 1, 2003.

[76] I. Gorton and A. Liu, Software Component Quality Assessment in Practice: Suc-cesses and Practical Impediments, the Proceeding of the International Confer-ence on Software Engineering, U.S., May 2002

[77] I. Gorton and L. Zhu. Tool Support for Just-In-Time Architecture Reconstructionand Evaluation: An Experience Report, Proc. 27th International Conference onSoftware Engineering, St. Louis, Mousiri, USA, 2005

[78] K. Govindraj, S. Narayanan, B. Thomas, P Nair, and S. Peeru. On Using AOPfor Application Performance Management. In proceedings Fifth InternationalConference on Aspect-Oriented Software Development (Industry Track),Bonn, Germany, March 2006.

[79] S. Graham and M. McKusick, Gprof: A Call Graph Execution Profiler. ACMSIGPLAN, vol. 17, no. 6, pp. 120-126, June 1982.

[80] M. Grand. Patterns in Java. vol. 2. New York, New York: Wiley: 1999.

[81] J. Grass. Object-oriented design archeology with CIA. Computing Systems,1992.

[82] T. Gschwind, K. Eshghi, P. K. Garg and K. Wurster. WebMon: A Perfor-mance Profiler for Web Transactions, Proc. Fourth IEEE International Work-shop on Advanced Issues of E-Commerce and Web-Based Information Sys-tems, Newport Beach, California, USA, June, 2002

[83] Y.G. Gueheneuc, H. Albin-Amiot. Using Design Patterns and Constraints toAutomate the Detection and Correction of Inter-Class Design Defects. 39th Inter-national Conference and Exhibition on Technology of Object-Oriented Lan-guages and Systems (TOOLS39), 2001.

[84] R.J. Hall. Call Path Refinement Profiles. IEEE Transactions on Software Engi-neering, vol. 21, no. 6, June 1995.

[85] R.J. Hall. CPPROFJ: Aspect-capable Call Path Profiling of Multi-threaded JavaApplications. Proceedings of the 17 th IEEE International Conference on Au-tomated Software Engineering, 2002.

164

[86] H. H. Hallal, E. Alikacem, W. P. Tunney, S. Boroday, and A. Petrenko.Antipattern-Based Detection of Deficiencies in Java Multithreaded Software, Pro-ceedings of the Quality Software, Fourth International Conference on(QSIC’04), IEEE Computer Society, Washington, DC, USA, 2004.

[87] D. Hand, H. Mannila, P. Smyth. Principles of Data Mining, MIT Press, 2001.

[88] M. Hauswirth, P.F. Sweeney, A. Diwan and M. Hind. Vertical profiling: Un-derstanding the behavior of object-oriented applications. Proceedings of the 19thAnnual ACM SIGPLAN Conference on Object-oriented Programming, Sys-tems, Languages, and Applications, 2004.

[89] D. Heuzeroth, T. Holl, G. Hogstrom, and W. Lowe. Automatic design pat-tern detection. In 11th International Workshop on Program Comprehension(IWPC), USA, May 2003. IEEE.

[90] D. Heuzeroth, T. Holl, and W. Lowe. Combining static and dynamic analysesto detect interaction patterns. In Sixth International Conference on IntegratedDesign and Process Technology (IDPT), Jun 2002.

[91] D. Heuzeroth, S. Mandel, and W. Lowe. Generating design pattern detectorsfrom pattern specifications. In 18th International Conference on AutomatedSoftware Engineering (ASE). IEEE, Oct. 2003.

[92] C.W. Ho, M.J. Johnson, L. Williams and E.M. Maximilien. On Agile Perfor-mance Requirements Specification and Testing. Proceedings of AGILE 2006.

[93] G. Hohpe, and B. Woolf: Enterprise Integration Patterns. Addison-WesleyProfessional, 2003

[94] D.F. Jerding, J.T. Stasko and T. Ball. Visualizing Interactions in Program Execu-tions. In the proceedings of the International Conference on Software Engi-neering, 1997.

[95] S.C. Johnson. Lint, a C program checker. Unix Programmer’s Manual, 4.2Berkeley Software Distribution Supplementary Docs; U.C. Berkeley, 1984.

[96] M.J. Johnson, E.M. Maximilien, C.W. Ho and L. Williams. Incorporating Per-formance Testing in Test-Driven Development. IEEE Software, May/June, 2007.

[97] S. Kamin and D. Hyatt. A Special Purpose Language for Picture Drawing, InProceedings of the Conference of Domain Specific Languages, Santa Bar-bara, CA, USA, 1997

[98] R.K. Keller, R. Schauer, S. Robitaille, and P. Page. Pattern-based reverse engi-neering of design components. In Software Engineering, 1999. Proceedings ofthe 1999 International Conference on, 1999.

[99] M. Kis. Information Security Antipatterns in Software Requirements Engineering.In proceedings 9th Conference on Pattern language of Programs, 2002.

[100] B. Kitchenham, L. Pickard and S.L. Pfleeger. Case Studies for Method and ToolEvaluation. IEEE Software, July, 1995.

[101] A. Koenig. Patterns and Antipatterns, Journal of Object Oriented Program-ming, volume 8, number 1, 1995.

165

[102] P. Kovari, D. Cerecedo Diaz ,F. C. H. Fernandes ,D. Hassan , K. Kawamura, D. Leigh, N. Lin, D. Masicand, G. Wadley and P. Xu. WebSphere ApplicationServer Enterprise V5 and Programming Model Extensions WebSphere HandbookSeries, September, August, 2003

[103] C. Kramer and L. Prechelt. Design recovery by automated search for structuraldesign patterns in object-oriented software. In Reverse Engineering, 1996, Pro-ceedings of the Third Working Conference on, 1996.

[104] P. Kruchten. The 4+1 View Model of Architecture. IEEE Software, volume 12,issue 6, November, 1995.

[105] M. Kunnumpurath. JBoss Administration and Development Third Edition (3.2.xSeries, Apress, October 2003

[106] J. R. Larus and T. Ball. Rewriting executable files to measure program behavior.Technical Report Computer Sciences Technical Report 1083, University ofWisconsin-Madison, 1992.

[107] C.H. Lee, J.C. Ou and M.S. Chen. Progressive Weighted Miner: An EfficientMethod for Time-Constraint Mining. PAKDD, 2003.

[108] M. Leleu, C. Rigotti , J.F. Boulicaut and G. Euvrard. GO-SPADE: Mining Se-quential Patterns over Datasets with Consecutive Repetitions. Proc. 3rd Int. Conf.Machine Learning and Data Mining in Pattern Recognition, Leipzig, Ger-many, 2003.

[109] S. Lo. Binary Prediction Based on Weighted Sequential Mining Method. Proc. Int.Conf. on Web Intelligence, 2005.

[110] M. Mahemoff. Ajax Design Patterns. Sebastopol, California: O’Reilly, 2006.

[111] H. Mannila, H. Toivonen and A.I. Verkamo. Discovering Frequent Episodes inSequences, Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining,Montreal, Canada, 1995

[112] H. Mannila, H. Toivonen and A.I. Verkamo. ”Discovery of frequent episodesin event sequences”, Data Mining and Knowledge Discovery 1(3): 259 - 289,November 1997.

[113] F. Marinescu. EJB Design Patterns. Wiley, 2002.

[114] R. Marinescu. Measurement and Quality in Object-Oriented Design. PhD thesis,Politehnica University of Timisoara, 2002.

[115] N. Medvidovic and R. N. Taylor. A Classification and Comparison Frameworkfor Software Architecture Description Languages. IEEE Transactions on Soft-ware Engineering, vol. 26, no. 1, January, 2000.

[116] N. Moha and Y.G. Gueheneuc. On the Automatic Detection and Correction ofDesign Defects. In Serge Demeyer, Kim Mens, Roel Wuyts, and StephaneDucasse, editors, Proceedings of the 6th ECOOP workshop on Object-Oriented Reengineering, July 2005.

166

[117] N. Moha, J. Rezgui, Y.G. Gueheneuc, P. Valtchev and G. El Boussaidi. UsingFCA to Suggest Refactorings to Correct Design Defects. Proceedings of the 4thInternational Conference On Concept Lattices and Their Applications (CLA2006),In S. Ben Yahia and E. Mephu Nguifo Eds, Hammamet, Tunisia, 2006.

[118] A. Mos. A Framework for Adaptive Monitoring and Performance Management ofComponent-Based Enterprise Applications. PhD thesis, Dublin City University,Ireland, 2004.

[119] A. Mos and J. Murphy. COMPAS: Adaptive Performance Monitoring ofComponent-Based Systems. Proc. Workshop on Remote Analysis and Mea-surement of Software Systems at 26th International Conference on SoftwareEngineering, Edinburgh, Scotland, May 2004.

[120] A. Mos and J. Murphy. Performance Management in Component-Oriented Sys-tems using a Model Driven Architecture Approach, Proc. 6th IEEE InternationalEnterprise Distributed Object Computing Conference, Lausanne, Switzer-land, September 2002.

[121] M.J. Munro. Product metrics for automatic identifcation of bad smell design prob-lems in java source-code. In Lanubile, F., Seaman, C., eds.: proceedings of the11th International Software Metrics Symposium, IEEE Computer SocietyPress, 2005.

[122] G.C. Murphy, D. Notkin, and K. Sullivan. Software Reflexion Models: Bridgingthe Gap between Source and High-Level Models. Proc. SIGSOFT Symp. Founda-tions of Software Eng., ACM Press, New York, 1995.

[123] J. Niere, J. P. Wedsack, and L. Wendehals. Handling Large Search Space inPattern-based Reverse Engineering. In 11th IEEE International Workshop onProgram Comprehension, May 2003.

[124] G. Papadimitriou, A. Vakali, G. Pallis, S. Petridou and A. Pomportsis, Sim-ulation in Web data management, Applied system simulation: methodologiesand applications, Kluwer Academic Press, Norwell, MA, USA, 2003.

[125] T. Parsons and J. Murphy. Detecting Performance Antipatterns in ComponentBased Enterprise Systems. Accepted for Publication in the Journal of ObjectTechnology.

[126] T. Parsons, J. Murphy, P. O’Sullivan, Applying Frequent Sequence Mining toIdentify Design Flaws in Enterprise Software Systems. In Proceedings 5th Intl.Conference on Machine Learning and Data Mining, Leipzig, Germany, July18-20, 2007.

[127] T. Parsons, J. Murphy, S. Pizzoli, P. O’Sullivan, A. Mos, Reverse EngineeringDistributed Enterprise Applications to Identify Common Design Flaws. At theSoftware Engineering Tools For Tomorrow (SWEFT) 2006 Conference, T.J.Watson, New York, Oct 17 - 19, 2006.

[128] T. Parsons, A. Mos and J. Murphy. Non-Intrusive End to End Run-time PathTracing for J2EE Systems. IEE Proc. Software, August, 2006.

167

[129] T. Parsons and J. Murphy.. The 2nd International Middleware Doctoral Sympo-sium: Detecting Performance Antipatterns in Component-Based Enterprise Sys-tems. IEEE Distributed Systems Online, vol. 7, no. 3, March, 2006.

[130] T. Parsons. A Framework for Detecting Performance Design and Deployment An-tipatterns in Component Based Enterprise Systems. Proc. 2nd Int’l MiddlewareDoctoral Symp., ACM Press, art. no. 7, Grenoble, France, 2005.

[131] T. Parsons. A Framework for Detecting, Assessing and Visualizing PerformanceAntipatterns in Component Based Systems. First Place at ACM SIGPLAN Stu-dent Research Competition Graduate Division, In OOPSLA’04: Companionto the 19th annual ACM SIGPLAN conference on Object-oriented program-ming systems, languages, and applications, Vancouver, BC, Canada, 2004.

[132] T. Parsons and J. Murphy.. A Framework for Automatically Detecting and As-sessing Performance Antipatterns in Component Based Systems using Run-TimeAnalysis. The 9th International Workshop on Component Oriented Pro-gramming, part of the 18th European Conference on Object Oriented Pro-gramming. Oslo, Norway, June, 2004.

[133] T. Parsons and J. Murphy. Data Mining for Performance Antipatterns in Compo-nent Based Systems Using Run-Time and Static Analysis. Transactions on Au-tomatic Control and Control Science, Vol. 49 (63), No. 3, May 2004.

[134] D.J. Pearce, P. H. J. Kelly, T. Field, U. Harder. Gilk: A dynamic instrumentationtool for the Linux kernel. Proceedings of the International TOOLS Conference.Springer: Berlin, 2002.

[135] D. J. Pearce, M. Webster, R. Berry and Paul H. J. Kelly, Profiling with AspectJ.Software Practice and Experience Vol. 37, Issue 7, John Wiley and Sons, Inc.New York, NY, USA, June 2007.

[136] R. Pena-Ortiz, J. Sahuquillo, A. Pont and Jose A. Gil, Modeling ContinuousChanges of the User’s Dynamic Behavior in the WWW, Proceedings of the 5thinternational workshop on Software and performance, Palma, Illes Balears,Spain, 2005.

[137] N. Pettersson. Measuring precision for static and dynamic design pattern recog-nition as a function of coverage. In ICSE Workshop on Dynamic Analysis, St.Louis, USA, May 2005.

[138] N. Pettersson and W. Lowe. Efficient and Accurate Software Pattern Detec-tion. 13th Asia Pacific Software Engineering Conference (APSEC), Decem-ber 2006.

[139] N. Pettersson, W. Lowe and J. Nivre. On Evaluation of Accuracy in PatternDetection. First International Workshop on Design Pattern Detection for Re-verse Engineering, October 2006.

[140] F. Plasil, D. Balek and R. Janecek. SOFA/DCUP: Architecture for Compo-nent Trading and Dynamic Updating. In proceedings of ICCDS.98, Annapolis,Maryland, USA, IEEE CS Press, May 1998

[141] C. Ponder and R.J. Fateman. Inaccuracies in Program Profilers. Software Prac-tice and Experience, May, 1998.

168

[142] M. Pont. Patterns for Time-Triggered Embedded Systems. Harlow, England:Addison-Wesley: 2001.

[143] A. Postma: A method for module architecture verification and its applica-tion on a large component based system, Information and Software Tech-nology 45(4), 2003, 171-194

[144] C. Potts. Software Engineering Research Revisited. IEEE Software, September,1993.

[145] D. Reimer et al. SABER: Smart Analysis Based Error Reduction, Proceedings ofthe ACM SIGSOFT international symposium on Software testing and anal-ysis, 2004.

[146] T. Richner and S. Ducasse. Recovering High-Level Views of Object-OrientedApplications from Static and Dynamic Information. Proceedings InternationalConference on Software Maintenance, 1999.

[147] H. Ritch and H. M. Sneed. Reverse engineering programs via dynamic analysis.In Proceedings of WCRE 1993. IEEE, May 1993.

[148] E. Roman, Scott W. Ambler and Tyler Jewell, Mastering Enterprise JavaBeans,second edition, J.Wiley and Sons, USA and Canada, 2002.

[149] B. Schmerl, J. Aldrich, D. Garlan, R. Kazman, and H. Yan. Discovering Archi-tectures from Running Systems. IEEE Transactions on Software Engineering,July, 2006.

[150] M. Schumacher, E. Fernandez-Buglioni, D. Hypertson, F. Buschmann, andP. Sommerlad, Security Patterns. Chichester, England: Wiley, 2006.

[151] J. Seemann and J. W. von Gudenberg. Pattern-based design recovery of Javasoftware. SIGSOFT 1998/FSE-6: Proceedings of the 6th ACM SIGSOFT in-ternational symposium on Foundations of software engineering, 1998

[152] M. Shaw and D. Garlan. Software Architecture, Perspectives on an EmergingDiscipline. Prentice Hall, 1996.

[153] N. Shi and R. A. Olsson. Reverse Engineering of Design Patterns from JavaSource Code. ASE ’06: Proceedings of the 21st IEEE International Conferenceon Automated Software Engineering, 2006.

[154] C. U. Smith, and L. Williams. Performance Solutions. Addison Wesley, 2002.

[155] A. Srivastava and A. Eustace. ATOM: A system for building customized pro-gram analysis tools. Proceedings of the ACM Conference on ProgrammingLanguage Design and Implementation. ACM Press: New York, 1994.

[156] P.F. Sweeney, M. Hauswirth, B. Cahoon, P. Cheng, A. Diwan, D. Groveand M. Hind. Using hardware performance monitors to understand the behaviorof Java applications. Virtual Machine Research and Technology Symposium.USENIX Association: Berkeley, CA, USA, 2004.

[157] C. Szyperski. Component Software - Beyond Object-Oriented Programming,Addison-Wesley, 1999

169

[158] TakeFive Software GmbH. SNiFF+, 1996.

[159] F. Tao, F. Murtagh and M. Farid. Weighted Association Rule Mining usingweighted support and significance framework. KDD, Washington, D.C., 2003.

[160] B. Tate. Bitter Java. Manning Publications Co., Greenwich, CT, USA, 2002.

[161] B. Tate, M. Clarke, B. Lee and P. Linskey. Bitter EJB. Manning, 2003

[162] W.F. Tichy. Should Computer Scientists Experiment More?. IEEE Computer,1998.

[163] P. Tonella, M. Torchiano, B. Du Bois and T Systa. Empirical studies in reverseengineering: state of the art and future trends. Journal on Empirical SoftwareEngineering, 2007.

[164] P. Tonella and G. Antoniol. Object Oriented Design Pattern Inference. Proceed-ings of IEEE Conference on Software Maintenance, 1999.

[165] Call Graph-Directed Boundary Condition Analysis in Contextual Composi-tion Frameworks. PhD Thesis, University College Dublin, April, 2007.

[166] M. Trofin and J. Murphy. Removing Redundant Boundary Checks in ContextualComposition Frameworks. Journal of Object Technology, vol. 5, no. 6, July-August, 2006.

[167] N. Tsantalis, A. Chatzigeorgiou, G. Stephanides, and S. T. Halkidis. DesignPattern Detection Using Similarity Scoring. IEEE Transactions on Software En-gineering, November 2006.

[168] A. Ufimtsev and L. Murphy. Performance modeling of a JavaEE componentapplication using layered queuing networks: revised approach and a case study.In Proceedings of the 2006 Conference on Specification and Verification ofComponent-Based Systems, Portland, Oregon, 2006.

[169] D. Viswanathan and S. Liang, Java Virtual Machine Profiler Interface. IBM Sys-tems Journal, Java Performance ,vol. 39, no. 1, 2000.

[170] E. Visser. Stratego: A language for program transformation based on rewritingstrategies. Proceedings of the International Conference on Rewriting Tech-niques and Applications. Springer: Berlin, 2001.

[171] J. M. Vlissides, J. O. Coplien and N. L. Kerth. Pattern Languages of ProgramDesign 2 (Software Patterns Series). Addison-Wesley Professional, 1996.

[172] M. Vokac: An efficient tool for recovering Design Patterns from C++ Code. Jour-nal of Object Technology, vol. 5, no. 1, 2006.

[173] J. Wang and J. Han, BIDE: efficient mining of frequent closed sequences. Proc.20th Int. Conf. on Data Engineering, 2004

[174] L. Wendehals. Improving design pattern instance recognition by dynamic analy-sis. In ICSE 2003 Workshop on Dynamic Analysis (WODA), Portland, USA,May 2003.

[175] L.Wendehals and A. Orso. Recognizing Behavioral Patterns at Runtime usingFinite Automata. In Workshop on Dynamic Analysis (WODA), May 2006.

170

[176] E. Weyuker and F. Vokolos. Experience with performance testring of softwaresystems: issues, an approach, and case study. IEEE Transactions on SoftwareEngineering, 2000.

[177] N. Wilde and M. C. Scully. Software reconnaisance: Mapping program featuresto code. Software Maintenance: Research and Practice, 1992.

[178] C. Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell and A. Wesslen.Experimentation in Software Engineering, An Introduction. Kluwer AcademicPublishers, 2000.

[179] E. Wohlstadter, S. Jackson and P. Devanbu, DADO: Enhancing Middleware toSupport Crosscutting Features in Distributed, Heterogeneous Systems, Proceed-ings of the 25th International Conference on Software Engineering, May,2003

[180] X. Yan, J. Han, and R. Afshar, CloSpan: Mining Closed Sequential Patterns inLarge Databases. In SIAM Int. Conf. on Data Mining , San Francisco, CA, May2003.

[181] M.J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed associa-tion rule mining. Technical Report 99-10, Department of Computer Science,Rensselaer Polytechnic Institute, Troy, NY, USA, 1999.

[182] M. J. Zaki and C. Hsiao. Charm: An efficient algorithm for closed itemset mining.In 2nd SIAM Int’l Conf. on Data Mining, 2002.

171

APPENDIX

A

Antipattern Rule Library

This appendix chapter details the different rules that are contained in the PAD toolrule library. Examples of rules are given in section 6.3 which show how rules arestructured in a Lisp like syntax. This section gives all the rules currently provided bythe PAD tool.

A.1 Rules

In section we list the rules currently available as part of the PAD Tool rule library.Rules are documented in their respective categories as outlined in section 6. The rulesthat form the antipattern library also make use of a number of user defined functionsand configuration settings which are discussed in sections A.2 and A.3 respectively.Full descriptions of these antipatterns are available in the different JEE antipatternliterature1 2 3.

A.1.1 Category1: Antipatterns Across or Within Run-Time Paths

Transactions-A-Plenty

(defrule detect-Transactions-A-Plenty-Antipattern

(?rtp<-(runTimePath (numTrans ?nt)))

(test (checkRTSLevel "Transactions" ?nt "THRESHOLD"))

=>

(printAPSolution "Transactions-A-Plenty" ?rtp))

Figure A.1: The Transactions-A-Plenty Rule

1B. Tate, M. Clarke, B. Lee and P. Linskey. Bitter EJB. Manning, 20032B. Dudney, S. Asbury, J. K. Krozak and K. Wittkopf. J2EE Antipatterns, Wiley, 20033Precise Java, http://www.precisejava.com

172

This rule in figure A.1 identifies a situation where the number of transactions in aparticular run-time path are above a user defined threshold value. The rule makesuse of the checkRTSLevel user defined function which checks if number of transactionsis this run-time path is too high, i.e. above a user defined threshold. If this rule fires theprintAPSolution function is called. This function prints out the antipattern descriptionand solution and related contextual information. Threshold values are specified in aconfiguration file.

Conversational-Baggage

(defrule detect-Conversational-Baggage-Antipattern

(?rtp<-(runTimePath (compQuickList ?cql)(componentTypeList ?ctl)))

(test (checkForComponent ?cql ?ctl "Stateful" "THRESHOLD"))

=>

(printAPSolution "Conversational-Baggage" ?rtp))

Figure A.2: The Conversational-Baggage Rule

This rule in figure A.2 identifies a situation where the number of stateful sessions in aparticular run-time path are above a user defined threshold value. The user definedfunction checkForComponent is used by the rule to check the component types that existin the run-time path and to see if the number of stateful beans is above the thresholdlevel.

Sessions-A-Plenty

(defrule detect-Sessions-A-Plenty-Antipattern

?rtp<-(runTimePath (ID ?id)(compQuickList ?cql) (componentTypeList?ctl))

(test (checkForComponent ?cql ?ctl "Stateless-Entity-ratio""THRESHOLD"))

=>

(printAPSolution "Sessions-A-Plenty" ?rtp))

Figure A.3: The Sessions-A-Plenty Rule

This rule in figure A.3 identifies a situation where the number of stateless sessionsin a particular run-time path are above a user defined threshold value or if the ratioof sessions to entities is too high. The user defined function checkForComponent isused to check the session/entity ratio and to see if it is above the threshold level fora particular run-time path. Similarly this rule can also use the function to check if thenumber of stateless sessions in the run-time path is above the threshold level. If eitherof these facts match the rule can fire. In this example rule we check the session/entity

173

ratio only.

A.1.2 Category2: Inter-Component Relationship Antipatterns

Needless-Session

(defrule detect-Needless-Session-Antipattern

?C1<-(component(name ?N)(type ?T&:(eq ?T "Session"))

(callees ?Callees)(callers ?Callers))

(component(name ?N2)(type ?T2&:(eq ?T2 "Entity")|?T2&:(eq ?T2"DB")))

(not (test (existsInList ?N2 ?Callees)))

(not (test (existsInList ?N2 ?Callers)))

=>

(printAPSolution "Needless-Session" ?C1))

Figure A.4: The Needless-Session Rule

The rule in figure A.4 identifies a situation where a session bean does not have anyrelationships with either the database or any entity beans. The rule makes use of theexistsInList user defined function which is used to check the components list of callerand callee components, for entity or database components. Further checks could bemade in this rule to see if the session bean is making use of the container services (seesection 6.3).

Remote-Calls-Locally

The rule in figure A.5 identifies a situation where two components are running withinthe same JVM, and one of the components calls the other component through its re-mote interface. The rule use the sameJVM function to compare the JVM details. Italso uses the existsInList function to check the component’s (C1’s) list of callers for thecomponent C2.

Accessing-Entities-Directly

The rule in figure A.6 identifies a situation where a web component directly accessesan entity or database component. The existsInList function is used to identify the entityor DB components in the web components callee list.

Bloated-Session

The rule in figure A.7 identifies a situation where the number of relationships a ses-sion bean has with other beans is above a user specified threshold. The checkRelation-shipLevel function is used to check whether the number of relationships is above thethreshold level.

174

(defrule detect-Remote-Calls-Locally-Antipattern

?C1<-(component (isLocal ?L&:(eq ?L "false")) (isRemote ?R&:(eq ?R"true")) (JVMDetails ?jvm1) (Callers ?callers))

?C2<-(component (name ?N2) (JVMDetails ?jvm2))

(test (sameJVM ?jvm1 ?jvm2))

(test (existsInList ?N2 ?Callers))

=>

(printAPSolution "Remote-Calls-Local" ?C1 ?C2))

Figure A.5: The Remote-Calls-Locally Rule

(defrule detect-Accessing-Entities-DB-Directly-Antipattern

?C1<-(component (name ?N) (type ?T&:(eq ?T "Web"))

(callees ?Callees))

?C2<-(component (name ?N2)

(type ?T2&:(eq ?T2 "Entity"))|?T2&:(eq ?T2 "DB"))

(test (existsInList ?N2 ?Callees))

=>

(printAPSolution "Accessing-Entities-DB-Directly" ?C1 ?C2))

Figure A.6: The Accessing-Entities-Directly Rule

(defrule detect-Bloated-Session-Antipattern

?C1<-(component (name ?N) (type ?T&:(eq ?T "Session"))

(callees ?Callees) (callers ?Callers))

test(checkRelationshipLevel ?Callees ?Callers "THRESHOLD")

=>

(printAPSolution "Bloated-Session" ?C1))

Figure A.7: The Bloated-Session Rule

175

A.1.3 Category3: Antipatterns Related to Component CommunicationPatterns

Unusual-or-Bulky-Session-Entity-Communication

(defrule detect-Bulky-Session-Entity-Comms-Antipattern

(?C1<-(component (name ?N1)(type ?T1&:(eq ?T1 "Session")) (callees?Callees)))

(?C2<-(component (name ?N2)(type ?T2&:(eq ?T2 "Entity"))))

(?FS<-(frequentSequence(children ?Ch) (parents ?Ps) (methodIdLists?Ms) (support ?S)))

(test(existsInList ?N1 ?Ch))

(test(existsInList ?N2 ?Ps))

(test(flagHighResourceConsumption ?Ms ?S "THRESHOLD"))

=>

(printAPSolution "Bulky-Session-Entity-Comms" ?FS ?C1 ?C2))

Figure A.8: The Unusual-or-Bulky-Session-Entity-Communication Rule

The rule in figure A.8 identifies a situation where a frequent sequence exists that hasa session as parent and an entity component as its child. For the rule to match thefrequent sequence must also have a resource consumption (or frequency) level abovea user defined threshold. The flagHighResourceConsumption function is used to identifyif the threshold has been surpassed.

Fine-Grained-Remote-Calls

(defrule detect-Fine-Grained-Remote-Calls-Antipattern

(?FS<-(frequentSequence (methodList ?Ms) (support ?S) (conf ?C)))

(test (confidenceAboveThreshold ?C))

(test (supportAboveTheshold ?S))

(test (remoteMethods ?Ms))

=>

(printAPSolution "Fine-Grained-Remote-Call" ?FS))

Figure A.9: The Fine-Grained-Remote-Calls Rule

176

The rule in figure A.9 identifies a situation where a frequent sequence exists that hasa support level above the user defined threshold (supportAboveTheshold), a rule confi-dence above the threshold level (confidenceAboveThreshold) and which contains two ormore remote method calls.

A.1.4 Category4: Data Tracking Antipatterns

Unused-Data-Object

(defrule detect-Unused-Data-Object-Antipattern

?TO<-(TrackedObject (name ?Name) (stats ?Stats) (type ?Type))

(test (lazyLoadable ?Stats ?Type))

=>

(printAPSolution "Unused-Data-Object" ?TO))

Figure A.10: The Unused-Data-Object Rule

The rule in figure A.10 identifies a situation where the data in a tracked object is notbeing accessed more than a user specified percentage level. The lazyLoadable user de-fined function determines if a particular tracked object is suitable for lazy loadingusing the tracked object statistics and user defined thresholds.

A.1.5 Category5: Pooling Antipatterns

Incorrect-Pool-Size

(defrule Incorrect-Pool-Size

?P<-(Pool (name ?N)(stats ?S) (type ?T))

test (poolThresholdExceeded ?S ?T)

=>

(printAPSolution "Incorrect-Pool-Size" ?P))

Figure A.11: The Incorrect-Pool-Size Rule

The rule in figure A.11 identifies a situation where the pool queues have exceeded auser specified threshold. The poolThresholdExceeded function looks up the user speci-fied threshold and compares it to the pool statistics.

177

A.1.6 Category6: Intra-Component Antipatterns

Local-and-Remote-Simultaneously

(defrule detect-Local-and-Remote-Intrfs-Simultaneously-Antipattern

(?C<-(component (has_local_interface ?LI&:(eq? LI"true"))(has_remote_interface ?RI&:(eq ?RI "true"))))

=>

(printAPSolution "Local-and-Remote-Intrfs-Simultaneously" ?C))

Figure A.12: The Local-and-Remote-Intefaces-Simultaneously Rule

The rule in figure A.12 identifies a situation where a component exposes both localand remote interfaces.

A.1.7 Adding Rules to The Rule Library

The PAD tool’s rules are stored in a single rules.clp file. Adding rules to the library canbe achieved by simply editing and appending to the rule library. Rules can also beeasily modified by editing this file.

A.2 Jess User Defined Functions provided by the PAD Tool

The PAD tool library makes use of a number of Jess user defined functions. Thesefunctions are packaged as part of the PAD tool and can assist in representing rules ina more concise manner. Rules that use the functions can be less complicated and arethus easier to both write and read. By using these functions users can easily extendthe antipattern library with their own antipattern rules.

Jess has a number of built in functions that can be used when writing Jess rules. Insome cases however Jess does not provide the capabilities to perform the tasks re-quired. To overcome this issue Jess provides a way to extend the language throughthe Userfunction interface4. The Userfunction interface can be implemented by a Javaclass. The PAD tool has made use of this capability to provide a number of functionsthat can be used when writing antipattern detection rules. Next we give an overviewof the functions that have been developed as part of the PAD tool.

printAPSolution: This function takes the antipattern name and any relevant data asarguments. The function then prints out the antipattern description, correspondingsolution and contextual information that can be used by the tool user to reason aboutthe problem identified.

existsInList: This function takes a list and a string as input. It searches the list for theparticular string and returns true if it exists.

4Ernest Friedman-Hill, Jess In Action, Manning, 2003.

178

sameJVM: This function takes two values (JVMDetails) as input and compares them.It returns true if they are the same.

poolThresholdExcceded: This function takes the pool statistics and type as input andevaluates (i.e. returns true) if the user defined threshold has been exceeded. The userdefined threshold is modifiable by editing a threshold configuration file.

remoteMethods: This function takes a list of methods as input and returns true if twoor more of the methods are remote calls.

lazyLoadable: This function take the statistics associated with a tracked object andthe tracked object type as input and evaluates (i.e. returns true) if the object is suitablefor lazy loading using user defined thresholds.

supportAboveThreshold: This function takes the support of a frequent sequence asinput and returns true if it is above a user defined threshold.

confidenceAboveThreshold: This function takes the confidence of a frequent se-quence as input and returns true if it is above a user defined threshold.

flagHighResourceConsumption: This function takes the frequent sequence methodlist and the support (which can represent frequency or resources used) of a frequentsequence. It calculates if the resource consumption of the sequence is above a userdefined threshold. If resource information is not available frequency is used.

checkRelationshipLevel: This function take a list of a components callers and a list ofcomponent callees as input. The function evaluates if the number of bean relationshipsis above a user defined threshold.

checkForComponents: This function takes a list of components as input and a com-ponent type. It checks if the number, of a particular component type is above a userdefined threshold. It can also check for component type ratios (e.g. entity to sesseion).

checkRTSLevel: This function takes a runtime service type as input, and the num-ber of run-time services. If the number of run-time services exceeds a user definedthreshold level it returns true.

A.3 Configuration Settings

The PAD tool user can set threshold levels for a number of the antipatterns outlinedabove. The threshold values can be made by modifying a single configuration file.The user functions outlined above make use of this configuration file to determine ifthe specified user thresholds have been exceeded.

179

APPENDIX

B

JEEM and FSM ImplementationSource Code

The COMPAS JEEM source code and the FSM algorthim implementation source codehave both been made freely available.

COMPAS JEEM is part of the COMPAS open source project and is available to down-load from the COMPAS web site1.

The FSM implementation used in this work is a modified version of Ferenc Bodon’sTRIE based FSM implementation. Bodon’s version is part of the open source Fre-quent Itemset Mining Template Library2. Our code has been made available3 and iscurrently being reviewed such that it can be added to the Frequent Itemset MiningTemplate Library. The data sets used as part of the tests in this work are also freelyavailable3 4.

1COMPAS Open Source Project, http://compas.sourceforge.net/2Frequent Itemset Mining Template Library, http://www.cs.bme.hu/ bodon/en/index.html3Trevor Parsons FSM implementation, http://pel.ucd.ie/tparsons/fsm4Open Source Data Mining repository, http://www.ilab.sztaki.hu/∼bodon/osdm/

180