GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families

PosterPrintSize:ThispostertemplateissetupforA0interna7onalpapersizeof1189mmx841mm(46.8”highby33.1”wide).Itcanbeprintedat70.6%foranA1posterof841mmx594mm.

Placeholders:ThevariouselementsincludedinthisposterareonesweoNenseeinmedical,research,andscien7ficposters.Feelfreetoedit,move,add,anddeleteitems,orchangethelayouttosuityourneeds.Alwayscheckwithyourconferenceorganizerforspecificrequirements.

ImageQuality:Youcanplacedigitalphotosorlogoartinyourposterfilebyselec7ngtheInsert,Picturecommand,orbyusingstandardcopy&paste.Forbestresults,allgraphicelementsshouldbeatleast150-200pixelsperinchintheirfinalprintedsize.Forinstance,a1600x1200pixelphotowillusuallylookfineupto8“-10”wideonyourprintedposter.

Topreviewtheprintqualityofimages,selectamagnifica7onof100%whenpreviewingyourposter.Thiswillgiveyouagoodideaofwhatitwilllooklikeinprint.Ifyouarelayingoutalargeposterandusinghalf-scaledimensions,besuretopreviewyourgraphicsat200%toseethemattheirfinalprintedsize.

Pleasenotethatgraphicsfromwebsites(suchasthelogoonyourhospital'soruniversity'shomepage)willonlybe72dpiandnotsuitableforprin7ng.

[Thissidebarareadoesnotprint.]

ChangeColorTheme:Thistemplateisdesignedtousethebuilt-incolorthemesinthenewerversionsofPowerPoint.

Tochangethecolortheme,selecttheDesigntab,thenselecttheColorsdrop-downlist.

Thedefaultcolorthemeforthistemplateis“Office”,soyoucanalwaysreturntothataNertryingsomeofthealterna7ves.

Prin7ngYourPoster:Onceyourposterfileisready,visitwww.genigraphics.comtoorderahigh-quality,affordableposterprint.EveryorderreceivesafreedesignreviewandwecandeliveryasfastasnextbusinessdaywithintheUSandCanada.

Genigraphics®hasbeenproducingoutputfromPowerPoint®longerthananyoneintheindustry;da7ngbacktowhenwehelpedMicrosoN®designthePowerPointsoNware.USandCanada:1-800-790-4001Interna7onal:+(1)913-441-1410Email:[email protected]

[Thissidebarareadoesnotprint.]

Automatic Non-functional Testing of Code Generators Families

Mohamed Boussaa, Olivier Barais, Gerson Sunye and Benoit Baudry INRIA Rennes, France

Mohamed BOUSSAA INRIA Rennes, France Presentation date: GPCE 2016 - Tue 1 Nov: 16:40 at Z1 Email: [email protected] Personal webpage: [email protected] Tool webpage: testingcodegenerators.wordpress.com Phone: +33626492436

Contact

CodegeneratorsareusedeverywhereTheyautoma7callytransformhigh-level system specications (Models,DSLs,

GUIs,etc.)intogeneral-purposelanguagesTargetdiverseandheterogeneous soNwareplakorms(JAVA,C++,C#,etc.)

ThegeneratedsoNwarear7factsworkbut…

Howaboutthenon-funcDonalproperDes(quality)

ofgeneratedcode?

Context ¤ Definition (Code generator family): We define a code generator family

as a set of code generators that takes as input the same language/model and generate code for different target platforms

¤ Non-functional testing of code generators: The classical way

•  Analyze the non-functional properties of generated code using platform-specific tools, profilers, etc.

•  Report inconsistencies, bugs, performance issues, etc.

Lack of tools for automatic non-functional testing of code generators

15th International Conference on Generative Programming: Concepts & Experiences (GPCE 2016) Amsterdam, Netherlands, October 31, 2016

Footprint C

Footprint A

DSL(Model)

SUT

SUT

SUT

Design

Generate

Generate

Generate

CodeGenerator A

CodeGenerator B

CodeGenerator C

Execute

Execute

Execute

C++

Platform C

Platform B

Platform A

JAVA

C#

Profiler A

Profiler B

Profiler C

Bugs Finding

Report

Report

Report

Footprint B

Code Generation Non-functional TestingCode ExecutionSoftware Design

Software Platform Diversity

Software Design Automatic Code Generation

Software Designer

DSL

Model

GPL

Specs

GUICode Generator

creators/maintainers

Code generators

Generated code

Container C

Container B

Container A

DSL(Model)

SUT

SUT

SUT

Design

Generate

Generate

Generate

CodeGenerator A

CodeGenerator B

CodeGenerator C

Code Generation Runtime monitoring engineCode ExecutionSoftware Design

Container A’

C#

Container B’

Container C’

MonitoringContainer

Back-end Data Base Container

Front-endVisualization

Container

JAVA

C++

Footprint C’

Footprint A’

RESTCalls

Footprint B’ Request

Bugs Finding

Testing Infrastructure

GPCE’16

Research Team diverse.irisa.fr Research Lab inria.fr/centre/rennes EU Funding Project heads-project.eu

Running…

Haxe library + Test suites

Component Under Test

Back-end Database

Component

Cgroup file systems

Running…

Monitoring records

Front-end: Visualization Component

Time-series database

HTTP Requests

CPU

Memory

…

8086:

Back-end: Monitoring Component Target

language … Code

Generation

Software Tester

Motivation

We propose: •  A black-box approach to automatically check the potential inefficient

code generators •  A runtime monitoring infrastructure, based on system containers

(Docker) as execution platforms, that allow code-generator developers to evaluate the non-functional properties of generated code

Differential Testing: Compare equivalent implementations of the same program written in different languages Standard deviation (std_dev): Quantify the amount of variation among the execution traces in terms of memory usage and execution time

Testing Method

Approach Overview

Running…

¤  Test suites with the highest variation in terms of execution time (k=60) ¤ Test suites with the highest variation in terms of memory usage (k=400) We can identify a singular behavior of the PHP code regarding the memory usage and exectution time

Validation …

Test suites with Std_dev > threshold value are interpreted as code generator inconsistencies

…

Memory usage Memory usage Memory usage

Compare Std_dev > kStd_dev < kBugNo Bug

GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families

Engineering

Transcript of GPCE16 Poster: Automatic Non-functional Testing of Code Generators Families