Stress Testing & Scenario Analysis · The key components of an effective stress testing & scenario...

© 2011 Deloitte LLP. Private and confidential.

Stress Testing & Scenario Analysis

October 2011

Mick CampbellStephen Boyd

© 2011 Deloitte LLP. Private and confidential.2

Stress Testing & Scenario AnalysisThe Agenda

1 Setting the Scene

2 The Fundamentals

3 An approach to stress testing and scenario analysis

4 Using loss data to challenge stress testing and scenario analysis

5 Working Scenario


Setting the scene - Stress, what stress?

Presentation title3


Recovery in the US…?

4


…in Europe…?

5


…How about Asia…?

6


Thoughts from the other side...

“Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do

so.”

Douglas Adams

The credit risk view:

“Credit is a system whereby a person who can’t pay gets another person who can’t pay to guarantee that he can pay”

Charles Dickens

7


• The FSA has challenged the both the design and use of stress testing and indicated that often stress testing models and results are not understood by the Board.

• Key recommendations for Firms include:• Establish suitable stress scenarios (both macro-economic and firm specific) which

equate to 1 in 25 year events (referencing the FSA anchor scenario where needed);

• Provide an appropriate level of challenge regarding the key assumptions underpinning the stress scenarios;

• Ensure the integrity of the models;• Obtain a complete and full analysis of the stress testing results, including the

validity of management actions.

• Some Firms are developing formal stress testing policies and scenarios which can be used for a range of purposes including the effect of stress scenarios across funding shortages, capital reductions, reduced profitability and operational risk issues.

On a more serious note......key findings from a recent Deloitte survey of ARROW and SREP visits relating to stress testing and scenarios:

8


The Fundamentals

Presentation title9


What is stress testing and scenario analysis?

We will use these terms interchangeably today for simplicity!

• “Identifying how risk profiles respond to shifts in economic variables or risk parameters”

Stress testing

• “Assessing the resilience of financial institutions and the financial system to severe but plausible scenarios”

Scenario analysis

Stress Testing & Scenario AnalysisThe Fundamentals

10


“Operational risk is subjective!”

“How can we base our capital on something like this?”

“Will the capital numbers ever become stable?”

“Whenever you ask a business questions about their Risk SELF Assessment, the next time you see them, they

have changed it…”

“Management want the lowest number possible – we have the answer before we start and have to retro fit the

scenario!”

Operational risk common challenges – familiar?


11


• Complements other risk management techniques

• Adds a broader perspective to the risk profile

• Helps management gain a view of the complete risk profile

• The regulators – favour it, and want it to be used to inform business decisions

How can stress testing and scenario analysis help?


......but – very little prescription

12


• Applying another subjective process to a subjective process!

• Articulating the value for stakeholders in participating in the exercise can be difficult!

• People have a perception that they know what the key sources of risk is and can find it hard to change their view without evidence

Stress testing and scenario analysis for operational risk – the challenges!


13


Risk & Control Assessment

What are the top risks?What are the key controls?

Is coverage complete?Would the controls be effective?

Loss Data Collection

Key Risk Indicators

Operational Risk Appetite

Operational Risk Management FrameworkScenario Design Scenario Execution

Are the any themes?Where has exposure been deteriorating

Is coverage complete? Are triggers set appropriately?Is there adequate predictive KRIs

Where have the material losses been occurring? What is happening in the industry?

What is the loss history?How much has this cost competitors?

What are the key measures and limits?What is outside of the Boards Appetite?

Is there appetite calibrated correctly?Are the limit complete & accurate?

Stress Testing & Scenario AnalysisA Typical Process

Using the existing Operational Risk Management Framework

14


An approach to stress testing and scenario analysis

Presentation title15


Base Scenario Creation

Scenario Approval

Stress Numbers

Mgmt Action Review

Mgmt Action

NumbersAnalysis Reporting Comms Review

Work Shop 1 Work Shop 2 Modelling & Structure

Communication Plan

• Business plan review;

• Risk Register Review;

• External Loss Data Review;

• Large Exposure identification;

• Reverse stress;• FSA requirement review; and

• Concentration review.

• Balance sheet, P&L and Capital Plan review;

• KRI triggers review; and• Mgmt Action validity and impact

• Model creation / integration

• Key financials analysis;

• Capital Analysis;• Risk Appetite and Capacity Review;

• KRI review; and• Report structuring

• Internal Communication strategy & plan;

• External communication strategy & plan; and

• Ongoing Review requirements

Snr Mgmt / Board Sign Off

Stress Testing & Scenario AnalysisA potential approach

Operational risk – an approach to developing stress testing and scenario analysis

16

Presenter

Presentation Notes

Below is a typical process for identifying scenarios and running the associated stress testing. The workshop methodology ensures business engagement and awareness throughout the process, which encourages ownships and accountability for the results. Ownership and accountability is key with respect to the numbers for the stress (pre) and management action (post) views as the viability of the process is dependent upon their accuracy. The degree of modelling varies from firm to firm however it is a critical phase that will materially enhance the analysis of the scenarios and project a forward looking view of the scenario on key metrics, KRIs, Risk Appetite and Risk Capacity.


Top Down Objectives

Stress Testing Toolkit

Scenario Design

Scenarios Description

Models

Mgmt Actions

Templates

Governance

Reporting

Comm’s and Training

Use Test

Regulatory Compliance

All objectives and outputs agreed and documented with appropriate methodology.

Toolkit comprises MYST, RST and sensitivity tests. MYST results used to inform RST scenarios.

Multiple scenarios are considered, covering a range of likelihoods; account for current and future business context.

includes description regulative and competitive environments. Clear list of variables quantified.

Outputs from models are cross checked from alternative sources; user guidelines have been documented

Proactive and reactive mgmt are fully documented; Actions aligned to Recovery and Resolution Plan

Guidelines are in place for collection of quant and qual data, along with documented process and procedures

Governance process is fully documented with clear responsibilities allocated throughout the process

Stress testing reporting used to inform potential future performance plans; and supports management actions.

All relevant staff trained, all staff educated about stress testing process; outputs are shared across firm.

Management actions and future plans rely on stress test output

Regulatory requirements understood and met. Internal experts are identified who keep abreast of reg changes.

Stress Testing & Scenario AnalysisWhat Does Good Look Like?

The key components of an effective stress testing & scenario analysis framework

17

Presenter

Presentation Notes

Top down objectives: Stress testing only seen as a risk management tool for minimum regulatory compliance To: All objectives and outputs agreed and documented with appropriate methodology and communicated to stakeholders. Stress testing tool kit: Sensitivity tests are carried independently across business To: Toolkit comprises MYST, RST and sensitivity tests. MYST results used to inform RST scenarios. Scenario design: No central scenario set – sensitivity tests conducted to ensure regulatory compliance. To: Multiple scenarios are considered, covering a range of likelihoods; account for current and future business context. Scenario description: No scenario description – sensitivity tests conducted to ensure regulatory compliance. To: Scenario description includes description regulative and competitive environments. Comprehensive list of variables quantified. Models: No models in place: scenarios are translated using management judgement, which is not necessarily validated. To: Output from models are cross checked from alternative sources; user guidelines have been documented. Management actions: No management actions from stress test process identified. To: Proactive and reactive management plans are fully documented; Actions aligned to Recovery and Resolution Plan Templates: No templates are in place for either the stress testing outputs or inputs. To: User guidelines are in place for collection of quantitative and qualitative data, along with documented process and procedures. Governance inc. review and challenge: No central policy. Stress testing conducted inconsistently throughout firm. To: Governance process – including timeline – is fully documented with clear responsibilities allocated throughout the process. Deadlines are clear and are met. No Review and Challenge conducted. To: Review and Challenge integral to process; guidance documentation of process in place. Central top down model in place to support Review and Challenge. Reporting: No formal reporting of stress testing To: Stress testing reporting used to inform potential future performance plans; and supports management actions. Communication and training: No training in place – purpose of stress testing only understood by Risk function. To: All relevant staff trained, all staff educated about stress testing process; outputs are shared across firm. Use test: No formalised inclusion of stress testing framework within firm’s risk management framework; only performed for regulatory compliance. To: Management actions and future plans rely on stress test output. Regulatory compliance: Basic understanding of regulatory requirements. To: Regulatory requirements of all relevant jurisdictions fully understood and met. Internal experts are identified who keep abreast of regulatory changes.


Reverse Stress TestingCompanies require to have thought through what different scenarios could make their business model unviable –including from a reputational perspective – and what mgmt actions they could put in place to try and mitigate the impact of the scenario.

Multi-year stress testsFirms should consider scenarios that evolve over a few years for example, 3-5years. This can be achieved through alignment with the Long Term Strategic Plan.

Use TestThe FSA want to see that stress testing is being used by the firm to think about its business strategy; validate the Risk Management Framework; set limits and thresholds for Risk Appetite and KRIs; and integrate with existing models.

Management ActionsFirms need to provide evidence that all management actions, and their impacts, are credible. There should also be evidence of how the management action would be triggered e.g. MI review, KRI breach etc.


The stress testing toolkit

18

Loss of confidence by customers

Loss of appetite from shareholders

Loss of confidence by auditors and regulators

Exhaustion of capital and liquidity resources

Collapse of a particular sector

Counterparties are unwilling to transact

with the firm or seek to terminate existing

contracts

Shareholders no longer have the

appetite to provide capital to the firm

Auditors and regulators no longer

recognise the firm as a going concern

Breach of regulatory ratios and unable to raise new funding

required

Outcome = Business Failure

Any of these events, stand-alone or in combination, could lead to the overall failure of a business

The customer base no longer exists or is

diminished resulting in a key failure of the overall business

strategy

19

Other terminology you might have heard of...

Test-to-destruction Stress-to-failure Break-the-Bank Business model stress test


Reverse Stress Testing – The Challenge

• The 'breaking' point of a firm may also be reached before a firm’s regulatory capital and liquidity resources are exhausted and therefore a breach in capital or liquidity regulatory limits is not necessarily the only 'fail' point of a firm.

• Some examples of indicators of a business model failure include:



Using loss data to challenge stress tests and scenarios– establishing fact from fiction



The shape of the body of the distribution gives insight in the variability of the yearly volume

of operational losses

The tail of the distributions is determined by low frequency

– high impact losses=>

The larger percentiles can be benchmarked against large

industry loss cases

Operational risk capital must cover the unexpected or tail risks


21


Enron went bankrupt in 2001 and Barclays was accused of contributing to Enron's bankruptcy by helping it hide its true financial condition through

financial structures. To settle the litigation, Barclays agreed to pay Enron $144M and in

return Enron would allow the bank's $310M of claims to go forward in the bankruptcy case.

In March 2005, ABN AMRO Inc, a US investment bank and subsidiary of ABN

AMRO Holding NV, reported that it agreed to pay $278.4M to settle a class action suit

related to the company's role in underwriting telecommunications company WorldCom Inc

issue of corporate bondsIn June 2004, BNP Paribas, a French bank, reported that was ordered to pay a $46.13M (38.11M EUR) fine in a ruling

upheld by France's Cour de cassation to settle charges filed by the Conseil de la

concurrence (French Competition Council) that it had participated in an anti-

competitive pact with other banks involved in the property loan sector.


Analysis of external loss data is the starting point for identifying and assessing tail risks through scenario analysis

22


Cards business is all about managing fraud risk…that is what we do!


Example 1: Card servicesA risk manager’s perception of the operational risks she is facing

23



Example 1: Card Services…factual data shows an entirely different risk profile

24


The risk manager’s perception Factual data


Example 1: Card ServicesA view on the tail risk

25


A Working Scenario


Stress Testing & Scenario AnalysisA Working Scenario

Cyber Security – Recent Press Coverage

27 © 2011 Deloitte LLP. Private and confidential.

• 8.33 AM – A trader’s computer was running slowly and called the IT help desk to see if they could help.

• 8.45 AM - The IT helpdesk checked the traders machine and noticed a series of unusual and suspicious background processes that were utilising a large amount of CPU:

• “covert_data_collect”, “covert_data_send”, “cover_tracks”.

• A ticket was left open and the IT helpdesk started to investigate further.......


Cyber security incident details


1) Identification and activation:

• If the user hadn’t identified the initial issue how would it have been identified?• Once identified, how would the incident be escalated?• How and by whom would the incident be investigated?• How widespread might this issue be and how do we know?• What response plans would be initiated?• How could the malicious program have been installed?

2) Access and assessment

• What methods/access points could be used to facilitate an external security breach? • How likely is it that someone would break in this way?• Who is responsible for assessment of the risk?• How do we differentiate between ‘normal’ levels of attack and concerted attack?• Without considering specific risks, would some methods give greater access than

others? Would detection and escalation paths be any different?


Questions for consideration


• 9.27 AM – Internet investigation suggested that several other trading organisations had been targeted by a sophisticated attack that used a ‘cover’ software program (“financial news desktop tool”) to install a series of malicious components that were designed to harvest sensitive data and send the information to a remote location.

• 9.37 AM – The IT helpdesk checked the traders computer and found the ‘financial news desktop tool’ was installed. After checking other traders machines it was found that 80% of machines where infected.

• 10.44 AM – Head of trading reports to Compliance that sensitive competitor trading information has suddenly appeared in their inbox.....


Cyber security incident details


3) Risks and vulnerability:

• What information is at risk?• What could they find?• How secure is this data, and what controls

might stop them?• What is the likelihood of successful

access?• What is the immediate impact of this

issue?• What might the longer term impacts be?• Do the regulators need to be informed and

if so what is the process for this?• Could other operational risks be

heightened because of this action?• Can we estimate the overall impact to us?• Could other operational risks be

heightened because of this action?

4) Motivation:• What other motivations could lead to

someone breaching our information security and how would that change the risks and impact?

5) Actions:• How would we react to such an event /

what plans are in place / who is responsible?

• How would we know what had been done?

• Who undertakes the risk assessment?• Who within the organisation needs to be

told?• Who will manage the internal/external

communications?• How would we try and mitigate the impact

at the time?


Impact Analysis – risks, motivations, and actions


1. Recap of the various intrusion methods we discussed

•What mitigating actions can be taken now to reduce the likelihood of intrusion?•Are we confident and practiced in the steps to take should the intrusion occur?

2. Recap of the individual risks/sub-scenarios and impacts

•What mitigating actions can be taken now to reduce the impact of an intrusion?•Are we confident that our sensitive information is adequately protected?

3. General Questions

•Any other IT risks / scenarios that have emerged that should be assessed?•Any further connected issues to this scenario which need addressing?


Mitigation & wrap-up


Speaker details

33

Mick CampbellDirector Deloitte LLPEmail: [email protected]: 0141 314 5899Mobile: 07900 607 601

Stephen BoydManagerDeloitte LLPEmail: [email protected]: 0141 304 5613Mobile: 07827 843444


mailto:[email protected]

mailto:[email protected]


Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited (“DTTL”), a UK private company limited by guarantee, and its network of member firms, each of which is a legally separate and independent entity. Please see www.deloitte.co.uk/about for a detailed description of the legal structure of DTTL and its member firms.

Deloitte LLP is the United Kingdom member firm of DTTL.

This publication has been written in general terms and therefore cannot be relied on to cover specific situations; application of the principles set out will depend upon the particular circumstances involved and we recommend that you obtain professional advice before acting or refraining from acting on any of the contents of this publication. Deloitte LLP would be pleased to advise readers on how to apply the principles set out in this publication to their specific circumstances. Deloitte LLP accepts no duty of care or liability for any loss occasioned to any person acting or refraining from action as a result of any material in this publication.

© 2011 Deloitte LLP. All rights reserved.

Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 2 New Street Square, London EC4A 3BZ, United Kingdom. Tel: +44 (0) 20 7936 3000 Fax: +44 (0) 20 7583 1198.

Member of Deloitte Touche Tohmatsu Limited

Stress Testing & Scenario Analysis · The key components of an effective stress testing & scenario...

Documents

Transcript of Stress Testing & Scenario Analysis · The key components of an effective stress testing & scenario...