Platform Support for Developing Analysis and Testing Plugins
-
Upload
holly-gilbert -
Category
Documents
-
view
21 -
download
0
description
Transcript of Platform Support for Developing Analysis and Testing Plugins
![Page 1: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/1.jpg)
Platform Support for Developing
Analysis and Testing Plugins
Shauvik Roy Choudharywith
Jeremy Duvall, Wei Jin, Dan Zhao, Alessandro Orso
School of Computer Science – College of Computing
Georgia Institute of Technology
![Page 2: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/2.jpg)
Development Environment
![Page 3: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/3.jpg)
IDE features
Source CodeEditor
RuntimeBuilders
SCM
Debugger
Views
Testing
IDE
… … …
Plug-in
Plug-in
Plug-in
Note: Plug-ins are called as Add-ins in Visual Studio
![Page 4: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/4.jpg)
IDE features
Source CodeEditor
RuntimeBuilders
SCM
Debugger
Views
Testing
IDE
… … …
Plug-in
Plug-in
Plug-in
Note: Plug-ins are called as Add-ins in Visual Studio
Focus of this talk:IDE Features for developing plug-ins that
support“Program Analysis and Software
Testing”Case Study: BERT
(BEhavioral Regression Testing)[WODA’08][ICST’10][FSE-demo’10]
![Page 5: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/5.jpg)
Regression TestingProcess and Issues
?
![Page 6: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/6.jpg)
Regression Testing Process and Issues
![Page 7: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/7.jpg)
Existing test suites typically target a small subset of the program behavior
Tests focus on core functionality
Oracles often approximated
Traditional Regression
Testing
![Page 8: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/8.jpg)
Traditional Regression
Testing
BERT
![Page 9: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/9.jpg)
Phase I: Generation of test cases for changed code
BERT
![Page 10: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/10.jpg)
BERT
Phase II: Behavioral comparison
![Page 11: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/11.jpg)
BERT
Phase III: Differential behavior analysis and reporting
![Page 12: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/12.jpg)
BERTTraditional Regression
Testing
![Page 13: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/13.jpg)
Our Experience
Programming language Java C#
Intercepting save and build events ✔ ✔
Triggering and checking build process ✔ ✔
Computing program differences ✔ ✖ (custom)
Generating test cases ✔ (Randoop, JPF) ✔ (PEX, Randoop.NET)
Instrumenting code ✔ (Javassist) ✔ (CCI)
Running test cases and code ✔ ✔
Visualizing information for user ✔ ✔
![Page 14: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/14.jpg)
IDE Support for Program Analysis and Testing Plug-ins
Dynamic InformationProject Artifacts and Events
Static Information
Plug-in Development Support
Graphical Features
![Page 15: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/15.jpg)
1. Project Artifacts and Events
![Page 16: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/16.jpg)
2. Static Information
Intermediate representations
Program differencing
Instrumentation
Build parameters
AST
![Page 17: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/17.jpg)
3. Dynamic Information
Test Execution
Runtime Metrics
Debugging information
RunTest
![Page 18: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/18.jpg)
4. Graphical Features
Provide GUI hooks to access plug-in functionality
Obtain user input or tool preferences
Present data to user and allow them to interact with it
![Page 19: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/19.jpg)
5. Plug-in Development Support
DevelopmentAPI Documentation
Sample Code and Informal Documentation
Testing, Debugging, and MaintenanceLaunch and monitor IDE instance
![Page 20: Platform Support for Developing Analysis and Testing Plugins](https://reader035.fdocuments.net/reader035/viewer/2022070400/56813552550346895d9cb2ea/html5/thumbnails/20.jpg)
Summary