Alyvix: Synthetic Monitoring @ Icinga Camp Berlin 2017

PowerPoint Presentation

1 more than software Wrth Phoenix 2016Francesco MelchioriDeveloperAlyvix

Good afternoon everybody,

my name is Francesco Melchiori and I work for Wuerth Phoenix. We are actively developing a monitoring solution for assessing the end user experience, for which we are devoting more and more effort.

--

Monitoraggio della End User Experience di Applicazioni Cloud1

IT and Consulting Company of the Wrth-GroupHeadquarters in Italy, European-wide presence, more than 130 highly skilled employeesInternational experience in Business Software and IT Management Core competencies in trading processes, wholesale distribution and logisticsMicrosoft Gold Certified Partner, ITIL certified, OTRS Preferred Partner2About Wrth PhoenixFacts & FiguresMore than 1.000 customers worldwideOver 500.000 service checks with NetEye25.000 monitored hosts4 offices in 3 countriesHQ in ItalyWe create the right balance between technology and services for our customers to support their IT operations and deliver in that way a better business result

Wrth Phoenix 2016 more than software

3 more than software Wrth Phoenix 2016Francesco MelchioriDeveloperAlyvixEnd user experience monitoringof network applications

Good afternoon everybody,

my name is Francesco Melchiori and I am a product manager in Wuerth Phoenix.

Today it is a pleasure to me to talk to you about a solution we are developing, for which we are devoting more and more effort..

--

Monitoraggio della End User Experience di Applicazioni Cloud3

4 more than software Wrth Phoenix 2016

*aaS

APPLICATIONTRANSPORTNETWORKMEDIUM

End User

performancequality of servicefinalSLA

Questo un tipico scenario cloud, dove App/Service/Platform/Infrastructure/Desktop as a Service in cloud vengono forniti ad unazienda, la quale li mette a disposizione ai propri utenti finali (dipendenti o clienti).

Il servizio influenzato dai molti fattori che interessano una rete di telecomunicazioni:il numero di richieste simultanee al serviziolottimizzazione del softwareil pool di risorse hardware a disposizionemezzo fisico di comunicazione e topologia della reteprotocolli applicativi, di trasporto, di retevpn, firewall, proxy, etc.

Tutto ci determina la performance del servizio presso lutente finale.

Un IT manager si prende cura di mantenere un certo livello di qualit del servizio: garantire un SLA con lutente finale.

Naturalmente, ancor prima del controllo viene la misura: si vuole monitorare il servizio cloud.4


vAPPvideo i/oRDPRDWA HDXCitrix

API ?!

APPdata closed client

API ?!

APPdata RESTJSONinterface HTMLCSS

API calls

La questione questa: esistono scenari che un monitoraggio attivo classico non puo risolvere.

Ho riportato tre scenari classici:

Un browser che consuma un servizio cloud, di cui e nota per esempio la RESTful API.Un client closed source di cui non si conosce il middleware.Ed infine, caso pi estremo, un browser che renderizza unapplicazione virtualizzata in cloud: ad esempio tramite protocollo RDP per RDWebAccess o HDX per Citrix.

Chiamare classicamente le API per monitorare il servizio cloud possibile solo nel primo caso. Nel secondo caso non possibile: non dato sapere come e implementata la comunicazione client-server. Magari potrebbe non usare nemmeno protocolli noti tipo HTTP. Nel terzo caso ci che abbiamo equivale ad avere un canale video, tastiera, mouse e stampante: impossibile raggiungere le API del servizio cloud.

La soluzione che WP ha sviluppato risolve tutti e tre gli scenari.

5

What is Alyvix ?6 more than software Wrth Phoenix 2016

I am talking about Alyvix.

Maybe some of you could remember Alexa, which was the original project. But Alyvix is a brand new piece of software and the goal of my keynote is to give you an overview and just some insights about it.

So, the question is: what is Alyvix?

[Actually, for months I have updated you about the state of its development, but maybe most of you did not even know what Alyvix is.]

6

7 more than software Wrth Phoenix 2016Visual Synthetic Monitoring

Alyvix is a Visual Synthetic Monitoring system.

Alyvix is an open source Python software, which is easily deployable on Windows 64bit machines, thanks to Anaconda.

OK, but now, what does Visual Synthetic Monitoring mean?7

8 more than software Wrth Phoenix 2016VisualSyntheticMonitoringAlyvix looks at graphic interfaces Alyvix behaves like human usersAlyvix tracks transaction performances

It is Visual because Alyvix looks at graphic interfaces. If you can see something on your screen Alyvix can do that too.

It is Synthetic because Alyvix behaves like human users. If you can synthetize something (e.g. a music instrument, a vitamin), that is because you can reproduce it artificially. And thats exactly what you can do with Alyvix synthetizing graphical application states and the way to interact with.

And finally, it is a Monitoring system because Alyvix (with a proper integration in Icinga) keeps track of the performance measures about each application transactions in a given user interaction flow.8


APM ApplicationPerformance Management

End UserExperience

ActiveMonitoring

VisualSyntheticMonitoring

IT Galaxy

Before to proceed discovering the features of Alyvix, I would like to localize this kind of product in the IT galaxy.

The Alyvix system (I mean together with Icinga) is a so-called APM, because it is able to monitor and manage the performance measures ofsoftware applications. You can diagnose application performance issues in order to maintain an expected level of service.

More specifically, Alyvix assesses the End User Experience, that is the point of view from which the Alyvix performance measurement works.

Moreover, as I said, Alyvix simulates interaction flows that end users would take on applications, that is why Alyvix belongs to the Active Monitoring subset.

But, finally, the most accurate label as product category for Alyvix is Visual Synthetic Monitoring.9

10 more than software Wrth Phoenix 2016Alyvixautomatesany applicationinteracting with GUIs exactly as a human would do

Point number ONE.

Alyvix can automate any application, simply because it interacts with GUIs exactly as a human would do.

How it is possible to achieve that?10

11 more than software Wrth Phoenix 2016Alyvix provides GUI toolsto design any apptransactions

Alyvix provides GUI tools to design any application transactions, from the point of view of their graphical aspects and interaction modes.

By the way, at its core, Alyvix relies on

- Python (|Anaconda)- RobotFramework- OpenCV- TesseractOCR

11

12 more than software Wrth Phoenix 20161 transaction1 Alyvix keyword

citrix_storefront_ready1. Define a unique application graphics in a keyword2. Plan interactions on an application state in a keyword3. Measure the time that a keyword takes to appear

So, what wed like to achieve is, in some sense, to translate 1 transaction in 1 callable Alyvix keyword.

Then, if Ill call that keyword, Alyvix acts the defined interactions on the defined graphical elements.

It is possible to define transactions for 3 types of graphics:ImagesRectanglesText strings12

13 more than software Wrth Phoenix 2016Alyvix Image Finder keyworddetects and interacts withimage components

1 Alyvix keyword can be made by 1 so called Image FinderIn Image Finders are defined 1 or more image componentsImage components can be for example:any kind of icon: tiny or bigI mean a picture that can be big as you wantgroup of icons in toolbarsarrow icons, that are really importantlogotypes and bannersand so on: were talking about any kind of pixel matrix, between a single point to, eventually, an entire screen

13

14 more than software Wrth Phoenix 2016Alyvix Rect Finder keyworddetects and interacts withrectangle components

1 Alyvix keyword can be made by 1 Rect Finder

In Rect Finders are defined 1 or more rectangle components

Rect components can be for example:Check boxesButtonsSpin boxesCombo boxesText boxesTabsProgress barsList boxesPicture framesand so on: were talking about any kind of rectangular shape, between a small check box to an entire window frame

And of course, they are seen as rectangles.14

15 more than software Wrth Phoenix 2016Alyvix Text Finder keyworddetects and interacts withtext string components

LA DIVINA COMMEDIA di Dante Alighieri INFERNO CANTO I [Incomincia la Comedia di Dante Alleghieri di Fiorenza, ne la quale tratta de le pene e punimenti de' vizi e de' meriti e premi de le virt. Comincia il canto primo de la prima parte la quale si chiama Inferno, nel qual l'auttore fa proemio a tutta l'opera.] Nel mezzo del cammin di nostra vita mi ritrovai per una selva oscura, ch la diritta via era smarrita. Ahi quanto a dir qual era cosa dura esta selva selvaggia e aspra e forte che nel pensier rinova la paura! Tant' amara che poco pi morte; ma per trattar del ben ch'i' vi trovai, dir de l'altre cose ch'i' v'ho scorte. Io non so ben ridir com' i' v'intrai, tant' era pien di sonno a quel punto che la verace via abbandonai. Ma poi ch'i' fui al pi d'un colle giunto, l dove terminava quella valle che m'avea di paura il cor compunto, guardai in alto e vidi le sue spalle vestite gi de' raggi del pianeta che mena dritto altrui per ogne calle. Allor fu la paura un poco queta, che nel lago del cor m'era durata la notte ch'i' passai con tanta pieta. E come quei che con lena affannata, uscito fuor del pelago a la riva, si volge a l'acqua perigliosa e guata, cos l'animo mio, ch'ancor fuggiva, si volse a retro a rimirar lo passo che non lasci gi mai persona viva. Poi ch'i posato un poco il corpo lasso, ripresi via per la piaggia diserta, s che 'l pi fermo sempre era 'l pi basso. Ed ecco, quasi al cominciar de l'erta, una lonza leggiera e presta molto, che di pel macolato era coverta; e non mi si partia dinanzi al volto, anzi 'mpediva tanto il mio cammino, ch'i' fui per ritornar pi volte vlto. Temp' era dal principio del mattino, e 'l sol montava 'n s con quelle stelle ch'eran con lui quando l'amor divino mosse di prima quelle cose belle; s ch'a bene sperar m'era cagione di quella fiera a la gaetta pelle l'ora del tempo e la dolce stagione; ma non s che paura non mi desse la vista che m'apparve d'un leone. Questi parea che contra me venisse con la test' alta e con rabbiosa fame, s che parea che l'aere ne tremesse. Ed una lupa, che di tutte brame sembiava carca ne la sua magrezza, e molte genti f gi viver grame, questa mi porse tanto di gravezza con la paura ch'uscia di sua vista, ch'io perdei la speranza de l'altezza. E qual quei che volontieri acquista, e giugne 'l tempo che perder lo face, che 'n tutti suoi pensier piange e s'attrista; tal mi fece la bestia sanza pace, che, venendomi 'ncontro, a poco a poco mi ripigneva l dove 'l sol tace. Mentre ch'i' rovinava in basso loco, dinanzi a li occhi mi si fu offerto chi per lungo silenzio parea fioco. Quando vidi costui nel gran diserto, Miserere di me, gridai a lui, qual che tu sii, od ombra od omo certo!. Rispuosemi: Non omo, omo gi fui, e li parenti miei furon lombardi, mantoani per patra ambedui. Nacqui sub Iulio, ancor che fosse tardi, e vissi a Roma sotto 'l buono Augusto nel tempo de li di falsi e bugiardi. Poeta fui, e cantai di quel giusto figliuol d'Anchise che venne di Troia, poi che 'l superbo Iln fu combusto. Ma tu perch ritorni a tanta noia? perch non sali il dilettoso monte ch' principio e cagion di tutta gioia?. Or se' tu quel Virgilio e quella fonte che spandi di parlar s largo fiume?, rispuos' io lui con vergognosa fronte. O de li altri poeti onore e lume, vagliami 'l lungo studio e 'l grande amore che m'ha fatto cercar lo tuo volume. Tu se' lo mio maestro e 'l mio autore, tu se' solo colui da cu' io tolsi lo bello stilo che m'ha fatto onore. Vedi la bestia per cu' io mi volsi; aiutami da lei, famoso saggio, ch'ella mi fa tremar le vene e i polsi. A te convien tenere altro vaggio, rispuose, poi che lagrimar mi vide, se vuo' campar d'esto loco selvaggio; ch questa bestia, per la qual tu gride, non lascia altrui passar per la sua via, ma tanto lo 'mpedisce che l'uccide; e ha natura s malvagia e ria, che mai non empie la bramosa voglia, e dopo 'l pasto ha pi fame che pria. Molti son li animali a cui s'ammoglia, e pi saranno ancora, infin che 'l veltro verr, che la far morir con doglia. Questi non ciber terra n peltro, ma sapenza, amore e virtute, e sua nazion sar tra feltro e feltro. Di quella umile Italia fia salute per cui mor la vergine Cammilla, Eurialo e Turno e Niso di ferute. Questi la caccer per ogne villa, fin che l'avr rimessa ne lo 'nferno, l onde 'nvidia prima dipartilla. Ond' io per lo tuo me' penso e discerno che tu mi segui, e io sar tua guida, e trarrotti di qui per loco etterno; ove udirai le disperate strida, vedrai li antichi spiriti dolenti, ch'a la seconda morte ciascun grida; e vederai color che son contenti nel foco, perch speran di venire quando che sia a le beate genti. A le quai poi se tu vorrai salire, anima fia a ci pi di me degna: con lei ti lascer nel mio partire; ch quello imperador che l s regna, perch' i' fu' ribellante a la sua legge, non vuol che 'n sua citt per me si vegna. In tutte parti impera e quivi regge; quivi la sua citt e l'alto seggio: oh felice colui cu' ivi elegge!. E io a lui: Poeta, io ti richeggio per quello Dio che tu non conoscesti, a ci ch'io fugga questo male e peggio, che tu mi meni l dov' or dicesti, s ch'io veggia la porta di san Pietro e color cui tu fai cotanto mesti. Allor si mosse, e io li tenni dietro.

1 Alyvix keyword can be made by 1 Text Finder

In Text Finders are defined 1 or more text string components

Text string components can be for example:any text string into GUIs, so text that can be typically found closed to images or rectanglesbut also, any text string immersed in a text blockand so on: were talking about any kind of text starting from the length of 2 characters15

16 more than software Wrth Phoenix 2016ON/OFF measurement toggleWarning and critical thresholdsTimeout limit

TIMEOUTCRITICALWARNINGNORMALongoing detection6. Set performance settingsHooveringSingle or double left clickingSingle right clickingText string typingShortcut pressing

keyboardinputsmouseinputs

Esc

Tab

Caps

Shift

Ctrl

~

`!

1@

2#

3$

4%

5^

6&

7*

8(

9)

0_

-+

=Bksp

Q

W

E

R

T

Y

U

I

O

P

{

[}

]|

\Del

A

S

D

F

G

H

J

K

L

:

;

ENTERZ

X

C

V

B

N

M

.?

/Fn

Shift

Alt

Alt

Ctrl

Tools

EN

5. Set interaction modesIF similarity thresholdRF size thresholds TF regular expression

4. Set recognition settingscitrix_login_form3. Type keyword name

2. Select graphic components

1. Select an Alyvix Finder typeTransaction definitionin an Alyvix keyword

Now Im going to tell you more in depth about the procedure to create one Alyvix keyword.

First of all, you need to select one of the Alyvix Finder type as we previously took into account.

Second, you have to drag and drop selection areas on the screen in order to highlight one or more components and ROIs they must lie. Here, Ive put a Rect Finder example.

Third, type a unique keyword name.

Fourth, tune recognition settings for every components:similarity threshold for IFsize thresholds for RF (here youve got the example)regular expression for TF

Fifth step: set interaction modes for every components; you can choose to do nothing, just to hoover on the components, to left, double or right click on them.

Sixth and final step: set performance settings for the whole keyword; you could enable or not the Alyvix performance measurement and setup its thresholds: warning, critical and timeout. Thats in order to properly label the related transaction because of its time consumption or to break the keyword after a while reporting a failure.16

17 more than software Wrth Phoenix 2016Alyvix Object Finder keyworddetects and interacts withmultitype components

IFRFTFOF IF similarity threshold RF size thresholds TF regular expression IFRFTF regions of interest

One step further.

You can bind together several Alyvix Finders of different types under a so called Object Finder.

You can basically create an Alyvix keyword with multitype components in order to define a really unique and unambiguous transaction.

Here weve got the Citrix login form as an example with IF, TF, RF, ROI in between to link everything together and of course all the interactions are inherited.

Everything together in a powerful keyword Object Finder based.17


OK, now, think about this real scenario: Alyvix should access the virtualized application menu through Citrix.

But, the Citrix login form is in a different position on the screen, the rendering of images and text is not good, it is pixelized, the edit boxes and the button are slightly shifted.

Thanks to all the good settings of our keyword, it does break and it can solve the situation accessing the StoreFront.18


End user interaction flowList of transactionsTest case

So, at the end of the day what wed like to do it to translate entire user interaction flows synthetizing transaction flows which are keyword flows.

Practically, what we get is a list of keywords in the Alyvix editor, so we get a executable test case. Keywords are, by the way, Python methods within a Python module.19

20 more than software Wrth Phoenix 2016Alyvix basic keywordProcessesCreate ProcessKill Process

WindowsWait WindowMaximize WindowShow WindowCheck WindowClose WindowWait Window Close

I/OsSend KeysMouse ScrollMouse MovePerformancesAdd PerfdataPrint PerfdataRename PerfdataSum PerfdataDelete PerfdataStore PerfdataPublish Perfdata

DebuggingAlyvix Config

SettingsSet Alyvix Info

ScreenshotsAlyvix Screenshot

Non capiscono perch si possa scrivere lowercased underscored invece che Camel Cased con spazi in mezzo

Dove si usano le keyword allinterno del test case, quali sono gli argomenti opzionali e in che formato vanno

Setup e teardown

20

21 more than software Wrth Phoenix 2016 Alyvix deployment1. Install Anacondabrowse https://www.continuum.io/downloadsdownload the Anaconda2 installerPython 2.7 for Windows 64bit versionrun the Anaconda2 installer as administratorAnaconda2--Windows-x86_64.exeinstalling Anaconda for all users and under C:\Anaconda2

2. Install Alyvixrun the command prompt as administratorexecute the following command:conda install -c https://conda.anaconda.org/alyvix alyvixcheck the Alyvix deployment with the following command:conda list alyvix|pillow|pyqt

22 more than software Wrth Phoenix 2016Alyvix measuresapp transactionsdefined and then listed in test cases

23 more than software Wrth Phoenix 2016Transaction performance

Frame grabber250ms

Object detector1s

Measurement systema little under the hood

23


1. detects object2. takes its time

3. interacts with it

1. detects object2. takes its time

3. interacts with it

Here youve got an example on a web service: getting results for Google search.

Alyvix runs a browser addressing Google, then Alyvix tries continuously to detect that object. When itll appear on the screen Alyvix takes that time passed and interacts with that object.

Thats what happens on and on until the end of the test case.

The important thing to stress here is that Alyvix engine is design to really output net performances, without image processing time, detection and interaction times.

So, precise and accurate measures.24


Word (virtualized)

unavailable

or

RDWebAccessCheckAVAILABILITY

RDWebAccess2. MeasureRESPONIVENESS

Word (virtualized)

1s

Word (virtualized)

or

2s

Word (virtualized)

or

3s

So, what do we want to know about every transaction?

Well, couple of things:

First, its AVAILABILITY: a transaction, so a keyword, can tell us if an certain application state is available or not.

Second, its RESPONSIVENESS: if a given application state is available, well, I want to know how fast is that availability. I mean the amount of milliseconds.

Absence of availability causes downtime services. Poor responsiveness causes latencies spikes.25

26 more than software Wrth Phoenix 2016Alyvix reports test case transactions in HTML pages


First of all, Alyvix test cases are easily scriptable thanks to an Alyvix command line utility. And you can get test case performances at the command line, also in an Icinga format.

But mostly, Alyvix prints out rich HTML reports, where you can find all the list of executed keywords, together with their performances and screenshots, that are grabbed when defined graphics were detected. Actually, every single graphic component of a keyword is highlighted on its screenshot. It good to have a certification of the availability of an application state at that time from that place.

Finally, if a keyword breaks (because it reaches its timeout limit) I can know why, because an animation is provided in that case and a ROI is blinking, where Alyvix cant detect a component.27

28 more than software Wrth Phoenix 2016Alyvix visualizes test case performancestracking them thanks to Icinga

28

29 more than software Wrth Phoenix 20161. Icinga can schedule one o more test cases on the same Alyvix probe

Alyvix probe WinOS 64bit

APP

2. NSClient++ receives NRPE commands from Icinga on Alyvix probes3. NSClient++ runs Alyvix test case scripts and returns back their outputs4. Alyvix test case performances are collected and visualized in Icinga

Icinga server

Im going to show you the big picture about Alyvix test case integration in Icinga.

We have an Alyvix probe which is a Windows 64bit machine where a cloud application will run in a browser.

As I explained during this keynote we can build an Alyvix test case that could be automate and measure that app.

Now, I can run that test from a prompt, but I can even run it remotely thanks to NSClient++ easily defining a run command in its config file.

And finally, I can set commands, services and host in Icinga to monitor that probe.29


latency spikesservice downtimesEnd user experience monitoringof business-critical transactions throughapplications in the cloud or on-premises

This is an example on the final achievement.

Service downtime and latency spikes detectable.

We can assess the level of the end user experience.30


Alyvix 2.4 | RELEASED March 3, 2017Store test case performances in a SQLite database and publish all the data in CSV files and in Windows Performance MonitorDisappeared performance measurement as a detection mode for all the findersCustomizable frequencies of frame grabber, object detector, interaction

Interaction Selection and Hold'n'Release as new interaction modesBackground Mode for locked and multiple sessions

31

32 more than software Wrth Phoenix 2016Demo of AlyvixTest case on MS Outlook through Citrixmonitored by Icinga

32


34 more than software Wrth Phoenix 2016Alyvix report


36 more than software Wrth Phoenix 2016Alyvix tracker


- navigazione in icinga per visualizzare impostazioni e grafici di un test

37

38 more than software Wrth Phoenix 2016Alyvix certifies the ongoingquality of IT services

39 more than software Wrth Phoenix 2016IT ops teams can modulateinfrastructure resourcesIT clients can checktheir SLA with providers

40Our Services Training and Consulting more than software Wrth Phoenix 2016

2 Days of Training2 Days of ConsultingAlyvix is deployedFirst test cases on business-critical applications are runningYou are able to create new test cases independently+=

40

41Our Services Training and Consulting more than software Wrth Phoenix 2016DATES & LOCATIONS

ItalyJune 13-14, 2017Wrth Phoenix GmbH, Bolzanothe training will be held in Italian

GermanyJune 20-21, 2017Wrth IT GmbH, Niedernhallthe training will be held in German

Use the promotion code ICINGA_BERLIN to get a 20% discount

alyvix.com/training

41

42 more than software Wrth Phoenix 2016call for actiongetting startedalyvix.comcontributegithub.com/alyvix/alyvix

So, what do we want to know about every transaction?

Well, couple of things:

First, its AVAILABILITY: a transaction, so a keyword, can tell us if an certain application state is available or not.

Second, its RESPONSIVENESS: if a given application state is available, well, I want to know how fast is that availability. I mean the amount of milliseconds.

Absence of availability causes downtime services. Poor responsiveness causes latencies spikes.42

43Our References more than software Wrth Phoenix 2016

43

44 more than software Wrth Phoenix 2016With a stopwatch.Eyes and hands like a human.

44

45 more than software Wrth Phoenix 2016Thanks for your attentionalyvix.comgithub.com/alyvix/alyvix

it.surveymonkey.com/r/alyvix_berlin

Alyvix: Synthetic Monitoring @ Icinga Camp Berlin 2017

Software

Transcript of Alyvix: Synthetic Monitoring @ Icinga Camp Berlin 2017