Designing an Unobtrusive Analytics Framework

for Monitoring Java Applications Sampo Suonsyrjä and Tommi Mikkonen

Dept. of Pervasive Computing Tampere University of Technology

Contents •  Context

–  Paradigm change in software development

–  Web apps vs. installable programs

•  Motivation –  Why the

unobtrusiveness?

•  Case –  Vaadin framework as the

platform of a target program –  The designed analytics

framework

•  Discussion •  Conclusions

17.10.15 2

Paradigm Change in Software Development

•  Waterfall method and such –  User involvement only before

development –  Requirements defined before

any use –  Systems remained the same

throughout their life –  Changes in user

requirements? No more use! Throw the cd away

•  Agile methods –  Requirements defined as late

as possible –  Users involved iteratively –  More knowledge of what is

possible and needed –  Developing less, but with a

greater accuracy –  Changes in user requirements?

Order an extension to the sw. dev. project

17.10.15 3

Paradigm Change and the First Frontier

•  Feedback with agile methods –  User surveys, thinking aloud,

observations –  Manually collected

qualitative data –  Stops when development

stops •  Continuous Delivery &

Deployment systems –  New versions are delivered

faster than ever

–  Developing experiments, not just reacting to requirements

–  Usage Data = results from these experiments

–  Automatic collecting needed •  Web as a platform for analytics

–  Channel is clear –  Tools are available –  A/B testing is easier than ever –  Continuous experimentation

17.10.15 4

Why the Unobtrusiveness?

•  Collecting usage data leads to changes in the target program •  Target program is continuously under change •  Target program can be built by others •  Data should remain

–  Collectable –  Comparable between different versions

•  If data collecting is independent of the target.. •  ..then data collecting mechanism could be reused

17.10.15 5

Vaadin Framework •  Open source framework

for developing Rich Internet Applications

•  Applications are written in Java..

•  ..and then transformed into AJAX applications

17.10.15 6

Target Program •  Demo app of the Vaadin

Framework •  QuickTickets Dashboard

Demo •  Source code available •  Built by someone else

than the researchers

17.10.15 7

Target Program public DashboardView() {

HorizontalLayout top = new HorizontalLayout();

addComponent(top);Button notify = new Button('2'); Notify.addClickListener(

new ClickListener(){...

});top.addComponent(notify);

};

17.10.15 8

The Designed Framework for Unobtrusive Analytics

•  AspectJ for usage monitoring

•  Fluentd for collecting •  ElasticSearch for storing •  Kibana for visualizing

17.10.15 9

AspectJ public aspect AddComponentListener {pointcut addComponentCall(Button b):

call(* *.addComponent(*))&& args(b);

after(final Button b):addComponentCall(b) {

b.addClickListener(new Button.ClickListener()

{ public void click(ClickEvent e) {

dataCollector.logEvent(b, e);}});

}}

public DashboardView() {HorizontalLayout top = new HorizontalLayout();addComponent(top);Button notify = new Button('2'); Notify.addClickListener(new ClickListener(){

...

});top.addComponent(notify);

};

17.10.15 10

AspectJ

•  Aspect-oriented programming is unobtrusive by nature

•  No alteration of the original source code •  Tooling is affected however

–  AspectJ dependency inserted –  Aspect class file inserted –  Normal compiling + aspect weaving

17.10.15 11

Fluentd

•  Implemented similarly as aspects –  Fluentd dependency added –  A logging class added

•  Parses data into JSON •  Sends data for storing •  In this case, Fluentd was

installed and run on the same machine as the target program

17.10.15 12

Fluentd

public class DataLogger {private static FluentLogger LOG = FluentLogger.getLogger("button.click");

public void logButtonClick(Button b, ClickEvent event){ Map<String, Object> data = new HashMap<String, Object>();

data.put("Button Caption", b.getCaption());data.put("Button ID", b.getId());...

LOG.log("click", data);}}

17.10.15 13

ElasticSearch and Kibana •  ElasticSearch is document

oriented •  Data was already in JSON

àName fields automatically •  Real-time access with Kibana •  Different visualizations readily

available

17.10.15 14

Discussion

•  RQ1: To what extent can a data collecting feature be implemented without compromising the evolution of the target program?

•  Usage monitoring inserted without changing the source code •  Aspect and logging class files and dependencies inserted •  Target application’s evolution was not compromised

–  No additional effort needed if new version has different buttons –  However, if the new version introduces new ways of implementing

buttons aspects need changing as well

17.10.15 15

Discussion

•  RQ2: What types of data can be collected with the given approach?

•  Aspect-oriented monitoring is flexible –  Pointcuts could be made on a vast variety of different points –  Advices can include almost arbitrary code –  Access to the target program in source code level

•  Vaadin Framework provided an abundant platform for collecting data •  Usage data collected in this case •  Perhaps end-user feedback in the future?

17.10.15 16

Discussion

•  RQ3: How to connect the data collecting feature with an analysis framework?

•  Aspect-code can be put to produce data in various formats àUnified logging layer (Fluentd: JSON) àPossibility to use standardized storing and visualization tools àUseful when combining different kinds of data (access, error, application logs etc.)

17.10.15 17

Conclusions •  New means of getting feedback

from end-users are needed à Analytics with usage data

•  New versions continuously à Need for unobtrusiveness à Aspect-oriented collecting

Thank you!

17.10.15 18