Scaleable Computing Jim Gray Microsoft Corporation [email protected].
Unit Testing for Domain-Specific Languages 1 Hui Wu, 1 Jeff Gray and 2 Marjan Mernik 1 University of...
-
Upload
jody-garrison -
Category
Documents
-
view
214 -
download
0
Transcript of Unit Testing for Domain-Specific Languages 1 Hui Wu, 1 Jeff Gray and 2 Marjan Mernik 1 University of...
Unit Testing for Domain-Specific Languages
1Hui Wu, 1Jeff Gray and 2Marjan Mernik
1University of Alabama at Birmingham, USA{wuh,gray}@cis.uab.edu
2University of Maribor, [email protected]
DSL 2009 Oxford, UK
July 15, 2009
Programming Methodologies Laboratory
Emperors New DSLs
• As we show off our new DSLs and approaches for creating them, are there some obvious limitations that we may be ignoring?
2
But where are the tools I can use?
OOPSLA 2008 DSL Panel:The Good, the Bad, and the Ugly
• Four of the five participants listed lack of tool support as a top challenge for DSLs– Utility of a new DSL is severely diminished if
supporting tools needed by a software developer and/or end-user are not available
– Poor interoperability and reuse of such tools 3
44
What Kinds of Tools are Needed?
Domain Experts program at DSL level
DSL translated into General Purpose Language (GPL)
Domain Experts deal with DSL
Integrated Development Environment (IDE)
subsel ect mebegin l eft ri ght up downend
Transl ater
subsel ect mepubl i c cl ass Robot{ publ i c stati c voidmain(Stri ng[] args) { Robot robot =new Robot(0,0,0); / /move l eft robot.move_l eft(); / /move down robot.move_down();robot.x = 5; robot.y = 6;
EditorEditor
CompilerCompiler
Visualizer
Visualizer
DebuggerDebugger
Domain Experts deal with GPL
Test EngineTest Engine
ProfilerProfiler
55
Mismatch Between Abstraction Levels
….commands : ( c:command cs:commands | );command : ( RIGHT { fileio.print("//move right"); fileio.print("x=x+1;"); fileio.print("time=time+1;"); fileio.print(" "); } |LEFT { fileio.print("//move left"); fileio.print("x=x-1;"); fileio.print("time=time+1;"); fileio.print(" ");…
….public final void commands() throws RecognitionException, TokenStreamException {
try { // for error handling {
switch ( LA(1)) { case CALL: case INIT: case SET: case PRINT: { command(); commands(); break; } case END: { break; } default: { throw new NoViableAltException(LT(1), getFilename());
…public final void function_name() throws RecognitionException, TokenStreamException { try { // for error handling { switch ( LA(1)) { case RIGHT: { match(RIGHT); fileio.print(" //move right"); fileio.print(" move_right();"); fileio.print(" "); break; } case LEFT: { match(LEFT); sllinenumber=dsllinenumber+1; fileio.print(" //move left"); fileio.print(" move_left();"); fileio.print(" "); break; }…
ANTLR GrammarGenerated Java Parser Code
Why is basic DSL tool support lacking?
• Why are such tools scarce?– Building new DSL tools from scratch for each new DSL can be
time consuming, error prone and costly– Tooling implementation effort generally not reusable for a
different context (even in same domain)6
• Example– ANTLRStudio
http://www.placidsystems.com/antlrstudio.aspx
– Fixed tool support customized for specific DSL
Idea of this paper
• Introduce the idea of a framework for building DSL tools that reuse existing GPL tools
• Assumptions/Limitations: – Only applicable to source-to-source translation (DSL
reuses GPL compiler and other tools of GPL context)– DSL tool implementation dependent on available GPL
mappings; requires adaptation to existing grammars – Does not consider case when one DSL statement is
mapped to non-contiguous GPL code (e.g., an aspect language)
– Paper uses simple examples for illustration; scalability of approach not certain
7
88
DSL to GPL Source Mapping…
3 knight:
4 position(+0,+1);
5 position(+0,+1);
6 position(+1,+0);
7 knight:
8 …
9 Init position(0,0);
10 left;
11 down;
12 knight;
13 Set position(5,6);
14 up;
15 right;
16 Print position;
…
…
6 public static void move_knight(){
7 x=x+0;
8 y=y+1;
9 x=x+0;
10 y=y+1;
11 x=x+1;
12 y=y+0;}
13 public static void main(String[] args) {
14 x=0;
15 y=0;
…
18 move_knight();
…
20 x = 5;
21 y = 6;
…
26 System.out.println("x coordinate="+x+""+
27 "y coordinate= " + y);}
…
{13, "Robot.java", 20, 21, "main", "none"}
99
Reusing GPL Tools in DSL Context
Re-interpreter GPL Tool
Source CodeMapping
Tool Results Mapping
Tool Methods Mapping
ANTLR Translator
GPL
DSL Tool Actions
DSL Variable View
DSL Level
GPL Level
End-User
GPL Tool Commands
DSL
Background Context: DSL Debuggers
• Our initial work for DSL tool support focused on debuggers
• The DSL Debugging Framework (DDF) provides a context for understanding the source mapping necessary to reuse GPL tools in a DSL context
10
1111
Debugging Methods Mapping
Mapping DSL GPL
Source Coden_i maps to m_i to m_j
Line Number:n_1n_2n_...n_in_i+1n_...n_jn_j+1n_...
Line Number: m_1m_2m_...m_im_i+1m_...m_jm_j+1m_...
Breakpoint Set breakpoint at n_i Set breakpoint at m_i
Step Over Step over line at n_i Step Over algorithm
Step Into Step into line at n_i Step Into algorithm
Terminate Terminate at line n_i Terminate at line m_i
Resume Resume at line n_i Resume at line m_i
1212
Debugging Results Mapping:Tangled Concerns within Grammar
1 | INIT var:VARIABLES LPAREN init_num1:NUMBER COMMA init_num2:NUMBER RPAREN
2 {
3 dsllinenumber=dsllinenumber+1;
4 fileio.print("x="+init_num1.getText()+";");
5 fileresult.print("x_coordinate=print x");
6 gplbeginline=fileio.getLinenumber();
7 fileio.print("y="+init_num2.getText()+";");
8 fileresult.print("y_coordinate=print y");
9 fileio.print("time=0"+";");
10 gplendline=fileio.getLinenumber();
11 filemap.print("mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+
12 gplbeginline +","+gplendline+","+"\""+funcname+"\""+","+"\""+funcall+"\""+"));");
13 fileresult.print(variable=var.getText()+"(x_coordinate,y_coordinate)");
14 }
Black: Basic functionalityBlue: Source code mappingRed: Debugging results mapping
1313
Imperative DSL Debugger
User has access to traditional debugging
operations
Variables defined in terms of DSL, not GPL
1414
Declarative DSL Debugger (FDL)
1 Car : all (Carbody, Transmission, Engine, Horsepower, opt(pullsTrailer))2 Transmission : oneof (automatic, manual)3 Engine : moreof (electric, gasoline)4 Horsepower : oneof (lowPower, mediumPower, highPower)5 include pullsTrailer6 pullsTrailer requires highPower
18
Current Focus: DSL Testing
• Before locating software errors how do we know there are bugs inside a DSL program?
1919
• Complement to the DSL Debug Framework (DDF) – the DUTF assists in identifying the presence of errors and the DDF assists in isolating the specific location of the error
• Architecture and process of construction is similar to the DDF architecture (Figure 1 in paper); source code mapping similar
DSL Unit Testing Framework (DUTF)
2020
DSL Unit Testing Framework (DUTF)
5
...21 Init position(0,0);22 down;23 knight;24 Set position(5,6);25 right;26 Print position; ...
public class Robot{ public static void main(String[] args) { …… //move left x=x-1; time=time+1;
//move down y=y-1; time=time+1; …… } }
import java.util.ArrayList;
public class Mapping { ArrayList mapping; public Mapping(){ mapping=new ArrayList(); mapping.add(new Map(1, "Robot.java",2,8)); mapping.add(new Map(2, "Robot.java",10,14)); …… } }
Test Results Mapping
Robot DSL Grammar Source Code Mapping
Robot DSL
Generated Lexer, and Parser by ANTLR
Robot.java and Mapping.java
Robot DSL Unit Test Perspective in Eclipse
Robot DSL Unit Test Script GrammarTest Case Mapping
...2 Init position(0,0);3 Expectedposition (1,2);4 knight;5 AssertEqual(Expectedposition, position);...
Robot DSL Unit Test Script public class TestRobot extends TestCase{ ... public void testknight() { i=1; j=2; robot.knight(); assertEquals(i, robot.x); assertEquals(j, robot.y); } ...}
TestRobot.java
Test Results View
Unit Test Script Editor
1 2
3
4
2121
Robot DSL Unit Test Case1 TestCase testknight {2 Init position(0,0);3 Expectedposition(1,2);4 knight;5 AssertEqual (Expectedposition, position);6 }…
GPL Unit Test Case (JUnit)11 public void testknight() {12 robot.x = 0;13 robot.y =0;14 int x=1;15 int y=2;16 robot.move_knight();17 assertEquals(x, robot.x);18 assertEquals(y, robot.y);19 }…
{1, “TestRobot.java”,11,“testknight”}
• A testing assertion at the DSL level may result in multiple assertions at the GPL level depending on variable mapping
DSL Unit Testing Framework (DUTF):Test Cases Mapping
2222
Car FDL Unit Test Case1 TestCase testFeatures {2 Expectedfeature:(carbody, manual, highPower);3 use Car.FDL(All);4 Constraint C1: include pullsTrailer;5 AssertTrue(contain(Expectedfeature, feature));6 AssertEqual(6, numberof feature);7 }
GPL Unit Test Case (JUnit)11 public void testFeatures () {12 testFeatures.add("carbody");13 testFeatures.add("manual");14 testFeatures.add("highPower");…27 assertTrue(compareFeatures(testFeatures,parse(fc,root,cons)));28 assertEquals(6,getFeatureListNumber(parse(fc,root,cons)));…
DSL Unit Testing Framework (DUTF):Test Cases Mapping
2323
DSL Unit Testing Framework (DUTF)Robot Language Unit Test Engine
Correct knight method
1 begin knight:
2 position (+0,+1);
3 position (+0,+1);
4 position (+1,+0);
5 end knight:
Incorrect knight method
1 begin knight:
2 position (+0,+1);
3 position (+1,+1);
4 position (+1,+0);
5 end knight:
User sees results as typical JUnit test report, but with DSL test cases
2626
Adapting Existing DSL Grammars for DSL Tool Support
• Crosscutting concerns emerge in different software artifacts (e.g., model, grammar, and source code)
• AspectG: A domain-specific aspect language for language grammars written in ANTLR– Program transformation rules are generated from a
domain-specific aspect language focused on language specification in ANTLR
– Generated rules modify grammars, not source code
2727
dsllinenumber=dsllinenumber+1;
gplbeginline=fileio.getLinenumber();
gplendline=fileio.getLinenumber();
filemap.print(" mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+gplbeginline+","+gplendline+"));");
Addition of DSL Tool Support: A Crosscutting Grammar Concern
dsllinenumber=dsllinenumber+1;
gplbeginline=fileio.getLinenumber();
gplendline=fileio.getLinenumber();
filemap.print(" mapping.add(new Map("+dsllinenumber+",\"Robot.java\","+gplbeginline+","+gplendline+"));");
…command :( RIGHT {
fileio.print(" //move right");fileio.print(" x=x+1;");
fileio.print(" time=time+1;");
fileio.print(" ");
} |LEFT {
fileio.print(" //move left");fileio.print(" x=x-1;");
fileio.print(" time=time+1;");
fileio.print(" ");
}|…
Base Grammar
Duplicate Tool Aspect Code
What if this line changes?Change here
Change here
Change ……
2828
pointcut productions(): within(command.*);
pointcut count_gpllinenumber(): within(command.*) && match (fileio.print("time=time+1;"));
before(): productions() {dsllinenumber=dsllinenumber+1;}
before(): count_gpllinenumber() {gplbeginline=fileio.getLinenumber();}
after(): count_gpllinenumber() {gplendline=fileio.getLinenumber();}
after(): productions() {filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); }
…command :( RIGHT { dsllinenumber=dsllinenumber+1; fileio.print("x=x+1;"); gplbeginline=fileio.getLinenumber(); fileio.print("time=time+1;"); gplendline=fileio.getLinenumber(); fileio.print(" "); filemap.print(" mapping.add(new Map("+ dsllinenumber+", \"Robot.java\","+gplbeginline+ ","+gplendline+"));"); }…
Join Point Model for AspectGANTLR Grammar AspectG
2929
Summary and Lessons Learned
• Tool development requires deep understanding of DSL, our framework, and the required mappings
• JUnit tests for equality in assertion checks may not match semantics of domain and need to be adapted
• Among 22 software components in DUTF, there are 3,001 lines of code that are generalized and can be reused to generate different DSL unit test engines
DSL Category DSL Name Number of Specific Functions or Classes
Customized Lines of Code
Imperative DSL Robot Language 2 239
Declarative DSL FDL 4 482
3030
Future Work
• An extension of the current framework that enables DSL profiling
• Adaptation of DUTF to address more complex DSLs
• Investigation into extending DUTF to other platforms– Application of different IDE platforms
(Microsoft Visual Studio .Net) and GPLs (C# and C++) using Nunit and other testing tools