MAC-05:Maconomy Performance Testing & Functional Test Automation

Greg Kozy & Chris Challenor, Deltek

» Increased Exposure

» Unicode Migration

» Accelerated introduction of new functionality

» Limits to ‘riding the wave’ of Hardware Acceleration

» Increased interest in MS Windows & SQL Server

» Maconomy Application Server Migration

» Application Degradation

» Identification of Recursive DB Lookups

Maconomy Performance | Progress

INSIGHT 2019

» New Scalable Server

» Returned to pre-2.2 Performance Levels

» New focus on Performance QA

» Database Optimizations

» Background Task Framework

» Hardware Recomendations & Scalability Tool

» New Extensions SDLC & Rules of Engagement

» Improved Integration Performance including iAccess

» Introduced Application Performance Monitoring

Maconomy Performance | Progress

INSIGHT 2019

Greg Kozy

» Vice President, Engineering

» Quality Background

» Automated Testing

» Performance Testing

» Metrics / Productivity

Special Guest Presenter | Introduction

Background:

» What are Customers missing/requesting?

1. Load Testing to gain confidence that

Maconomy can cope with ongoing

deployment or planned growth demands

2. Reducing the overhead (Cost, Resource

Reqs, Errors, etc.) of End User Testing

via Functional Test Automation

Performance Defined:

» /per-for-mance tes-ting/ noun

» Executing procedures to evaluate the

response time, throughput and utilization

of system to execute its required

functions in comparison with different

versions of the same product or a

different competitive product

Why Performance Testing & Test Automation?

Performance Testing

» Upgrades to major Maconomy Releases

» Hardware updates to Maconomy

Application & Database Servers

» Infrastructure & Resource changes to the

environment (IO, Memory, CPU, etc.)

Test Automation

» Upgrades to major & minor Maconomy

Releases

» Security & Statutory updates

» Browser and 3rd-party updates to

relevant Maconomy components

When Performance Testing & Test Automation is Effective

Performance Testing

» Combination of Microfocus’ Load Runner

http-protocol (iAccess) and Citrix VMs (for

Maconomy WSC)

Functional Test Automation

» Workspace Client via Deltek JTR (Java

Test Runner)

» iAccess via Deltek Test Runner

Deltek’s Solution Offering

Performance Testing WSC using Load Runner / Citrix

Tester Maconomy App Cluster

LR Controller

Citrix virtual users

HTTP virtual users

Citrix Hosts WSC Client App

iAccess Web App

• Citrix vUser scripts emulate Citrix ICA protocol communication

between a Citrix client and server. VuGen records all activity

during the communication and creates a Vuser script.

• Before creating scripts for the Citrix protocol, Citrix

environment must be configured to work with VuGen.

• VuGen generates Citrix functions when user performs actions

on the remote server. Each function begins with a ctrx prefix.

These functions emulate the analog movements of the mouse

(Co-ordinate based) and keyboard strokes.

Results: Max’d out around

2800 timesheets over 40

minutes. Load limited due to

Citrix licenses & instability

Macono

my

Industry Leading Microfocus LoadRunner Tool

Straight http posts

and gets directly to

application/coupling

server

Results: 10,000 timesheets

created without max’ing out

Load Runner vusers

Performance Testing iAccess using Load Runner

A MS Windows, GUI-Based

Test Automation Tool. The

Test Runner client is installed

on laptops or VMs, and the

scripts can be created and

executed from there.

Front-End developed by

Deltek for ease of use, but

calls Selenium Web Drivers

(APIs) and uses industry-

leading Selenium Test

Automation Tool as the

engine.

Mapping of iAccess objects

stored in Test Runner’s Object

Store: a collection of files for

each application that identifies

every text box, radio button,

list box, label, message box,

and toolbar item.

75+ Costpoint customers have

downloaded the tool (Deltek

Costpoint, mostly) and have

the capability to use it for their

own testing purposes. Most

have only received one hour

of training.

Automatic Email Notification

of results. Displays number

of tests executed, status

(Pass/Fail), and summary

information regarding

execution time.

Supports customized objects

through simple updating of the

Object Store.

Functional Test Automation for iAccessDeltek Test Runner

Test Runner Main Screen

» Left Panel: Simple Directory Structure

to identify available test scripts and test

suites

» Top Right Panel: Execution portion of

the your Test Suite Execution

» Bottom Right Panel: Test Step –by-

Test Step execution of each script

iAccess

Test Editor

» Keyword Driven framework. Easily

readable spreadsheet-style script.

» Data-Driven Framework. Variables that

can be used to add Data variations to

the same test script (Test Script =

Business Use Case) Exportable and

Importable to MS Excel.iAccess

Test Scheduler

» Ability to schedule test scripts to run

during off-peak hours.

» Flexibility to schedule by specific time

(i.e. 2200 hours) or based on order (i.e.

run “Submit Timesheet” script first; then

run “Approve Timesheet” script second;

etc.)

» Capability to schedule on Virtual

Machines or Desktops

iAccess

Test Capture

» Record/Playback application that

records mouseclicks and keystrokes as

you use the product.

» Currently only available for Costpoint,

but working on iAccess

» Speeds up script writing in fractions of

time, as compared to writing scripts from

scratch

iAccess

JTR for Maconomy WSCFunctional test automation

framework which is used to

automate the testing of the

Maconomy Workspace

Client (WSC).

Embeds SWTBot libraries in

the Maconomy code & the

JTR tool to interact with the

WSC

SWTBot is an open-source

Java-based UI/functional

testing tool for testing SWT,

Eclipse and GEF- based

applications.

JTR and JTE can be run in a

cloud based architecture. So

setting up a test environment

can be quick and end-users

don’t need to install JTR

specific software on their

machines.

JTE is a web application

that can be accessed using

any web browser.

Test cases are ‘keyword’

driven and are written in xml

using a UI front end called

Java Test Editor (JTE).

JTR ComponentsJava Test Runner (JTR)

Executes the test cases.

Sends requests to Maconomy’s McSWTBot*

libs to interact with WSC

Code is built into Maconomy Application

Java Test Editor (JTE)

Web app runs inside a Tomcat container

End Users create Test Scripts

Keyword-Driven & Data-Driven Functions

JTR Common Library (commonlib)

Library used by the 3 components above

Contains mainly JAXB readers/writers to read

and update test cases, test suites, test results

and widget maps.

Messenger

Used to pass data to back and forth between

JTR and JTE

Admin interface used to manage server

JTR Test Editor» JTE provides a forms-based interface

to create test cases

» Each step starts with a keyword

» JTE will display required fields based

on keyword

» A test step can contain Validation

steps

» [?] offers help

» Template button extracts specified

steps into a template

Workspace Client

JTR Main Screen» The Test Cases tab shows the result of

the test from the last time that it was

run.

» Historical test execution results will be

available soon.

» For a failed test, the result shows the

step that was executing when the failure

occurred. This tells you which step to

investigate and fix the test case if

needed.

» A screen capture is made when a test

fails and it can be viewed by clicking on

the icon shown in red

Workspace Client

Long-Term Vision for JTR

Aggressively create internal tests to increase Quality Q4 / 2019 (ongoing)

Expand availability to “beta” customersQ4 / 2020

(Maconomy v2.5)

Share scripts (not) data between Deltek & Customers Q2 / 2021

Improve Quality & Time-to-Market Always

Delivered Benefits

Performance Test Framework

» Satisfied Users…

» Improved Performance…

» Reliable Results…

» Capacity Management…

» Measured Speed, Accuracy, and Stability of

your system…

» Eliminating Bottlenecks in your Infrastructure

» Higher Quality

Partnering with Deltek through Services

» You can leverage our Load Runner

infrastructure, within our network, to run the

Performance Tools. We’ll develop scripts in-

house and point to your test environment.

» Free up your staff by leveraging our expertise &

knowledge with Load Runner to write the scripts

Delivered Benefits

Functional Test Framework & Tools

» Faster Feedback. Automated testing comes as a

relief for validation during phases of a software

project…

» Accelerated Results…

» Reduced Business Expenses…

» Testing Efficiency Improvement…

» Higher Overall Test Coverage…

» Reusability of Automated Tests…

» Earlier Detection of Defects…

» Thoroughness in Testing…

Partnering with Deltek through Services

» Your scripts (not necessarily data) can be shared with

Deltek to help us run the applications the way you do

before we release code…

» We can share our library of scripts to help you get

started…

» Free up your staff by using our knowledge, expertise,

and staff to quickly get your Test Automation

Framework off and running, without negatively

impacting your business

Best Practices – Performance TestingOverview and Capacity Planning

» Collect baseline performance metrics

on memory, CPU, and IO during peak

performance periods from all app tiers

» Compare baselines month-to-month,

as well as app execution timers

» Enables quicker reaction to data

growth or pattern changes

Virtual Environments

» For best testing, reserve CPU and

memory for all Maconomy Servers

» IO and Application performance for

long running jobs scale linearly

» IO is usually the first bottlenecks to

check

Organize Test Appropriately

» Understand your company’s end

user processing schedule

» Don’t overlap system processes

(Backups) with end-user critical

jobs

Resource Utilization

» Never allow all CPU or memory

resources of an app or DB server

to be used by the applications

» Always reserve 20-25% for the

O/S

DB Monitoring Tools

» For Oracle, AWR &

Performance/Tuning Pack

» For SQL Server, 3rd parties like

Idera, Quest, and Solar Winds

Test Environment

» Equal importance as production

» Ad Hoc reports can be longest

running jobs in need of

performance management

Best Practices – Functional Test AutomationPreparation & Planning

» Focus on most effective test cases

that will save time during test cycle

and find critical, high-severity

defects.

» Have business users write or

review test scripts for accuracy.

Modularize Test Scripts

» Write short scripts that perform

single functions that can be used

in different scripts.

» Save time when business use

cases or functionality change

(update the modular script, not

every script)

Use Conditional Settings Checks

» These are run before the script to

ensure the data and configuration is

correct before the main script runs

(time-saver)

Backup/Restore DB before Runs

» Backup your product database and

restore it to its “base state” every

time before the scripts are run to

ensure they will run correctly.

Take Advantage of Data-Driven

Capability

» Parameterize your test scripts so

that the same script can be run with

different variations of data

elements

» Turns 1 test in many tests in matter

of seconds.

Organize Test Suites

» Especially when capability exists to

mass create scripts, organize by

function / area to ensure you can

run them as needed (don’t assume

you’ll always run them all)

Performance Testing Functional Test Automation

Availability & Engagement

» Infrastructure and Toolsets are

available now

» Contact Sales Rep or Services

Contact to discuss

» iAccess solution ready, available, and free

now

» WSC solution being used internally at

Deltek. Should be available to customers

in 2H 2020

» Contact GregKozy@Deltek.com to get

started

Q&A

Day/Time Session Title

Tuesday: 10:00 – 11:00 AM MACKO-01 Maconomy Kickoff

Tuesday: 11:10 – 12:10 PM MAC-02 Customer Spotlight for Maconomy: Mott MacDonald

Tuesday: 2:10 – 3:10 PM MAC-03 What’s New in Maconomy?

Tuesday: 3:20 – 4:20 PM MAC-04 iAccess for Maconomy

Tuesday: 4:30 – 5:30 PM MAC-05 Performance Assurance in Maconomy

Wednesday: 10:00 – 11:00 AM MAC-06 Making Moves - Taking Maconomy to the Deltek Cloud

Wednesday: 11:10 – 12:10 PM MAC-07: Maconomy Customer Advisory Council

Wednesday: 2:10 – 3:10 PM MAC-08 Maconomy Tips & Tricks

Wednesday: 3:20 – 4:20 PM MAC-09 Maconomy Customer Care

Thursday: 8:30 – 9:30 AM MAC-10 Maconomy Finance

Insight 2019 | Maconomy Track (Emerald 8)