Automated Testing

for the ATM ChannelJames Tomany,

Barclays Bank, UK

Europe’s Premier Software Testing Event

Stockholmsmässan, Sweden

WWW.EUROSTARCONFERENCES.COM

“Testing For Real, Testing For Now”

ATM Refresh

Embedding QA,

Improving Quality & Reducing Costs

EuroSTAR 2009 December 2009

Or

How to reinvent your ATM lifecycle and

save £millions

3

The ATM Refresh Programme - Some Background

• The current system

• Bespoke, host-based application providing ATM acquiring, debit card authorisation and

connectivity to national debit switch (LINK)

• Bespoke, proprietary ATM application for transaction processing and alerting running on

Windows/NT

• SNA Communications

• No direct connections to VISA or Mastercard

• IBM crypto-processors

• The new system

• BASE24-atm, Release 6 Version 9 on HP NonStop Blades

• Wincor Nixdorf ProCash/NDC ATM Application for transaction processing running on

Windows/XP

• Wincor Nixdorf ProView for ATM monitoring

• Wincor Nixdorf Platform Security Agent for ATM lockdown

• IP Communications throughout

• Thales HSMS

• VISA EU and BankNet connectivity

• New PC Cores inside ATMS, along with some EPP upgrades

• New debit card host application

• Wincor Nixdorf ProView Analysis for ATM Channel Business Intelligence

4

The Public Face Of ATM Failure

Trade Press Reports

26 March 2008 - 14:17

Customers cash in on 'double your money' ATM

Hundreds of people flocked to a faulty Payzone ATM in the UK city of Hull last week after it

started dispensing twice the amount of cash keyed in for withdrawals.

29 February 2008 - 14:55

Nationwide admits ATM blunder

Nationwide Building Society has been forced to apologise to thousands of customers after a

technical glitch led to accounts not being debited when cash was withdrawn from some

ATMs in Northern Ireland.

10 April 2008 - 10:14

Danske glitch wipes out Northern Bank, National Irish and Sampo ATMs

Danske Bank says a glitch with its IBM-managed payments network resulted in customers of

its subsidiaries in three countries being unable to use their cards to withdraw money from

cash machines.

The financial impacts of ATM failure?

06 November 2008 - 13:09

RBS ATM dishes out 'free' cash

Residents of the English market town of Chorley flocked to a Royal Bank of Scotland

(RBS) ATM last week after word spread that the unit was dispensing double the amount of

cash requested.

6

So Where Does The Money Go?

Come common assertions:

– ATM systems are difficult and expensive to to change and

maintain

– The underlying technology is what drives expense.

– Manage the cost of change by not changing

My view:

– The cost software is small compared to the cost of the

lifecycle

– The technology is largely irrelevant in the cost of change

– The best way to manage the cost of change is to plan for

constant change.

An explanation:

– Software pricing is elastic

– Human behaviour is the biggest driver in the cost of systems

maintenance

– Everything gets better with practice!

7

Sc = Tt * Ct * I * Rc

Sc = 5 * 8 * 5 * 20

Sc = 4000

Sc -> Test Scenarios

Tt -> Transaction Types

Ct -> Card Types

I -> Issuer End Points

Rc -> Response Codes

Maths Disclaimer:

Not all card types support all transaction types and all response codes

But the order of magnitude is relevant

000s not 00s

Previous Manual Testing Regime

800 test Scripts

23 Man-days to execute….

….. AT LEAST

... none of which address Windows/XP

environment failures from the earlier slide

What was the scale of the problem at Barclays?

8

What was our test philosophy?

Accept the scale of the problem!

Embed QA throughout the life cycle – not just at the end

Simulate AND Automate the testing

Focus specialist resources on defect resolution and

change management instead of on repetitive test execution

Accelerate components into integration testing

9

Current State of Play

Streamlined and semi-automated software management

model implemented

“Change Anything – Test Everything” Philosophy

4500 Transaction Test Scenarios in regression suite so far,

which are executed in a 12 hour time window

“Smoke test” of ~450 transactions executed every night,

overnight,

Same test artefacts used by the Business in UAT

10

What does it look like?

• Simulated ATM hardware on desk top

• Drives REAL ATM software in virtual ATM

• Transactions processed by BASE24

• Alerts processed by ProView

• Issuer Systems simulated by VersaTest

• All simulators programmatically compare

results and record outcomes in HP Quality

Centre

11

BRIDGE:Test Environment(Windows Blade Server)

Versatest Environment(Windows Blade Server)

BASE24

HP NonStop Blade

Cards Host

IBM zSeries

VISA DFS

BNET DFS

LIS5 DFS

BICI DFS

HISOI DFS

HISOA DFS

VATM1

VATM2

VATM16

BRIDGE Sim1

BRIDGE Sim2

BRIDGE Sim16

(Via SNA proxy on HP)

HP Quality Centre – Enterprise Test Management

ATM Alerting SubSystem

System Diagram

Ve

rsa

tes

t Au

tom

atio

n S

erv

er

12

How can we exploit this investment?

Faster Time To Market for new changes, while reducing risk

Demonstrate the multi-vendor capability of any ATM

application

Extended beyond the UK ATM network to support testing

across Barclays Group.

The ATM test tool is also being used for developing training

material for branch staff

13

Any Questions?

Questions?

Enquiries Tojames.tomaney@barclays.com

Barclays ATMR

Testing PartnersVersaTest Issuer Simulator BRIDGE:Test ATM Test Tool

And test automation experts And test automation experts

ATM Domain Testing Skills