ANTLR in SSPXingzhong XuHong ManAug. 11

OutlineANTLRAbstract Syntax TreeCode Equivalence (Code Re-hosting)Future Work

What is ANTLR?ANTLR, ANother Tool for Language Recognition, is a language tool that provides a framework for constructing recognizers, interpreters, compilers, and translators from grammatical description containing actions in variety of target languages.-- antlr.org

Why use ANTLR?SSPLooking for a framework to understanding the signal processing source code semantically.Classical analysis method in CSCode Recognizer: Lexer & Parser Interpreter: Cognitive Linguistic Modeling & other syntax treeTranslator: code re-hosting, different target

How ANTLR Work? - I LexerConverting a sequence of characters into a sequence of tokens. ParserConverting a sequence of tokens which generated from the Lexer to determine its grammatical structure.Abstract Syntax TreeTree representation of the abstract syntactic structure of source code.The syntax is abstract which means it does not represent every detail of the real syntax.

Example

How ANTLR Work? - IIIn order to generate the Lexer, Parser and AST. We need analyze the structure of the target code and write related ANTLR grammar.Example: Matrix Declaration in MatlabM1 = [1 2 3; 4 5 6];M2 = [1,2,3;4,5,6];M3 = [M1;M2];M4 = 1;

ANTLR Grammar - IM1 = [1 2 3; 4 5 6];Statement[Variable] [Equal] [Expression] [Semicolon] (optional)Expression [Left Square Bracket] [Matrix] [Right Square Bracket] or [one digit]Matrix[Line] [Semicolon] [Line] [Semicolon] .Line[digit] [comma] (optional) [digit] [comma] (optional)

ANTLR Grammar - II

Abstract Syntax TreeM1 = [1 2 3; 4 5 6];

Abstract Syntax Tree*M2 = [1,2,3;4,5,6];

Abstract Syntax TreeM3 = [M1;M2];

Abstract Syntax TreeM4 = 1;

AST Example from GNU-RadioUsing ANTLR, some example from GNU-Radio code has been tested.http://sites.google.com/site/stevensxingzhong/

Code Equivalence In order to re-hosting the codeThe proper rule to abstract the code.The functionality of the code segment.MethodologyAbstraction Code Segmentation Functionality Analysis Replace the segment by equivalence code.

Current Method in CS Syntax Tree based ComparisonGenerate AST or other related abstract tree, perform tree-matching algorithm.Use hash function to mapping the tree structure and simplify the algorithm. Radom Test ComparisonCode Chopper, segment the code.Randomly test the Input/Output behavior.Schwartz-Zippel lemma, enough time of the test can derive the functionality.

Simplest Filter ExampleTake the simplest filter as an example, following code segments have exactly same functionality.

for (i = 0; i < n; i++) acc0 += d_taps[i] * input[i];

for (i = 0; i < n ; ) acc0 += d_taps[i] * input[i++];

i = 0;while ( i < n ) acc0 += d_taps[i] * input[i++];

i = 0;for ( ; i < n ; ) acc0 += d_taps[i] * input[i++];

Ordinary ASTfor (i = 0; i < n; i++) acc0 += d_taps[i] * input[i];

Modified ASTThe ordinary AST is derived from the programming grammar level.Following the idea of the semantic signal processing. For example, in signal processing domain abstraction:For, While, do while -> LOOP+=, VAR = VAR + whatever -> ACCUMLATE

Simplest Filter Examplefor (i = 0; i < n; i++) acc0 += d_taps[i] * input[i];

Simplest Filter Examplefor (i = 0; i < n; ) acc0 += d_taps[i] * input[i++];

Simplest Filter Examplei = 0;while ( i < n ) acc0 += d_taps[i] * input[i++];

i = 0;for ( ; i < n ; ) acc0 += d_taps[i] * input[i++];

Code Equivalence Objection: From the syntax tree to determine the code segments are equivalence. AbstractionTree matching.Perform code re-hosting.

Simplest Filter Examplefor (int i = 0; i < noutput_items; i++) { gr_complex sum(0.0, 0.0); for (k = 0; k < l; k++) sum += d_taps[l-k-1]*in[j+k]; out[i] = sum;}From gr_adaptive_fir_ccf.cc

AbstractionThe basic element for the simplest filter include:LOOPACCUMLATION MULTIPLYARRAYMOVING INDEX

Similarity Tree PatternNo abstraction can guarantee the same functional code have precisely same abstraction form. Therefore, we need perform a similarity tree pattern recognition.Similar enough to determine the equivalence

Future WorkUsing ANTLR generate other language Lexer and Parser for language recognition.Abstract the language into Cognitive Linguistic Modeling. Find proper method to perform a similarity tree pattern recognition.

ReferenceTerence Parr, The Definitive Antlr Reference: Building Domain-Specific Language (Pragmatic Programmers), 2007http://www.antlr.orghttp://www.stringtemplate.orgJiang L. and Su, Z. 2009. Automatic Mining of functionally equivalent code fragments via random testing. In Proceedings of the Eighteenth international Symposium on Software Testing and AnalysisGabel, M., Jiang, L., and Su, Z. 2008. Scalable detection of semantic clones. In Proceedings of the 30th international Conference on Software EngineeringC.K. Roy, J.R. Cordy and R. Koschke B. 2009. Comparison and Evaluation of code Clone Detection Techniques and Tools: A Qualitative Approach. Science of Computer ProgrammingBertran, M., Babot, F., and Climent, A. 2005. An Input/Output Semantics for Distributed Program Equivalence Reasoning. Electron. Notes Theor. Comput. Sci. 137, 1 (Jul. 2005)