Deliver Fast, On Time & With High Quality

Description

Karim Fanadka QA Manager HPE Software

About me: Karim Fanadka

Job: QA Manager at HPE Software, 7+

Product: StormRunner Load

Favorite activates: Traveling, Ski, Blogging

Ski – France 2013

Road Trip – Iceland 2015

We are going to talk about:

AgileModern QA

SaaS

Automation

Environments

Hotfixes Continuous Deployment

SaaS – Software as a Service

Before SaaS – On Premise

SoftwareUsers

SaaS – Software as a Service

Data Base

Servers

File System

Firewall

Users SaaS

journal note

I have been asked to build the first DevTest team that combines:

• Modern technologies and techniques

• Agile Methodologies

• SaaS product testing

• Deliver fast

StormRunner Load - SaaS product

Which product my team is testing :

Performance testing solution that makes it easy to plan, run, and scale web and mobile testing over the cloud.

10

1 Create and run a load test using your favorite scripting tool

2 Load Generators spin up in AWS EC2 or MS Azure

3 Results and app insights are available in real time

The product my team is testing: StormRunner Load

New

Soon

Performance testing solution that makes it easy to plan, run, and scale web and mobile testing over the cloud.

StormRunner Load as a Service

Data Base

Servers

File System

Firewall

Users SaaS Cloud

Agile – Do It Fast

Before Agile - Delivery Timeline

1 or 2 Major releases to production in 1 year

Agile StormRunner Load Delivery Timeline

March April AugustJune Nov Dec

Whats the CodeName of the next release?If you know the answer, I will invite you to a

AlleyCat BubbleBobble Elevator Action Frogger Golden Axe Hang-On

FebruaryDec

CommanderKeen Digger

6 Major releases to production in 1 yearYear: 2015

Agile – The Challenges

• Test new features

• Test Regression

• Automation

• Environments

• Continues Integration / Continues Delivery

• Hotfixes

Do it in 1.5 months

• Data Migration (When update SaaS with the newer version)

• Performance Testing

• Security Testing

• Globalization

• Documentation

• And more …

journal note

As the first DevTest engineer at HPE I have been asked to build the first DevTest Team to handle:

• SaaS product testing

• Deliver fast

Modern QA Team (DevTest)

What is DevTest?Using top end QA/DEV practices combined with DevOps techniques

QA & DEV Working very close, on the same IDE, code base, and in united process

Automation is a MUST

Manual is for new features / exception flows that are difficult to automate

NO MORE “Deliveries for QA” YES for “Push to Production”

Training“we are testing, and breaking code constantly but we help rebuild it better…….”

Continues Testing

Push new code

Release branch (1.5 months)

Test new codeShort Regression

Push to production

Automation

Developer Engineer Test Engineer DevTest Team

Builds in 1 Release (1.5 month): 1,400• Full: 236• Commit: 1,164

Continuous Testing

Push new code

Automation – Full package

Builds in 1 Release (1.5 month): 1,400

Automation – Commit package

Feature branch

Release branch

Full: 236

Commit: 1,164

Developer Engineer

Modern QA

DevTest

DevOpsSaaS

Dev

Product

Customer

Automation

journal note

I was aware that automation must find real defects and not just false alarm, DevTest team has a new challenge, and it’s to build:

• Reliable frameworks

• Fast results (not more than 7 min)

Frameworks:

Frameworks Tests

Client Testing Framework

API Testing Framework

Cloud Framework

Results

Example for Automation challenge17 Regions in the Cloud 6 different images for each region

15 different metrics

102 Tests

(~1500 metrics)

Automation – Cloud Framework - Report

Grunt

Protractor

TestResults

WebDriver

Angular

SeleniumAPI’s

UIControls(Storm)

Test

DBLayer ProtractorWrapper

JasmineStyle JS

AutomationLibrary

Executer

Client automation framework

Grunt

MochaTestFrameworkTests

DBWrapper

APIWrapper

AutomationLibrary

StromRunnerAPIs

Repo

rts

TestsResults

Backend automation framework

API discovery tool

Cost cloud tool

030

80 100 120150 170

200

0 0

75120

147

220

390

600

0 0 0 0

100

200

270 270

0

100

200

300

400

500

600

700

1.1 - AlleyCat 1.2 - Bubble 1.3 - CK 1.4 - Digger 1.5 - Elevator 1.6 - Frogger 1.7 - Golden Axe

1.8 - Hang-On

Automation Active Tests - Trend

Client (UI) Backend Cloud

85% Coverage

100% Coverage

60% Coverage

Next Generation: Application Lifecycle Management

Test Environments

Test Environments Local

Lab – Simulate SaaS Env

• Each tester have his own local machine for fast testing

• Lab Env = 7 Servers• We have 6 Lab Env:

• QA1• QA2• Security• Performance• Integration

Staging

• SaaS responsible for staging:• Maintenance • Deployment• Sanity

Test Environments Farm

QA Security Performance IntegrationLocal (Linux)

Staging

Continuous Deployment

Continuous Deployment

Push new code

Automation – Full package

Builds in 1 Release (1.5 month): 1,400

Automation – Commit package

Feature branch

Release branch

Full: 236

Commit: 1,164

Developer Engineer

Continuous Deployment

Test Engineer

Developer Engineer

Hey Tester, How do you manage to handle this amount of builds and deployments on daily basis

I’m a Modern QA

Continuous DeliveryLab

Nexus

Tel Aviv

PIT

Jenkins

QA2

Security

PCoE

QA2 (Portal)

PIT

PIT

PIT

Hotfixes in 1 day

Hotfixes Lifecycle

07:00 AMProduction bug

08:00 AMResearch by QA and Dev

10:00 AMFixing bug

16:00 AMTesting (Staging)

14:00 AMTesting (Local)

18:00 Push to production

Articles • 10 best practices for QA teams to deliver quality software, fast

• Blurring the lines: How the development tester role changes on agile teams

• 3 steps to building test databases for the real world

Tip

“Changing our mindset takes precedence over technology.

When we work together, we will succeed.”

Thank you

@kim0z85

Question?

https://www.facebook.com/karim.f.as

https://il.linkedin.com/in/kareemfakareem9k@gmail.com

Karim Fanadka QA Manager HPE Software