Meaningfull security metrics
13
Meaningful and useful Security metrics Vladimir Jirasek About.me/jirasek 5 st Oct 2011
-
Upload
vladimir-jirasek -
Category
Business
-
view
2.766 -
download
1
description
A presentation slides given at NetFocus 2011 in Bournemouth.
Transcript of Meaningfull security metrics
Meaningful and useful Security metrics
Vladimir Jirasek
About.me/jirasek
5st Oct 2011
About me
Security professional (11 years), current work at Nokia as Enterprise Security architect
Founding member and steering group member of (Common Assurance Maturity Model) CAMM (common-assurance.com)
Director of Research, CSA UK & Ireland
I love reading books: thrillers (Clive Cusler) and business management (Jo Owen)
I will cover three topics today
Information Security Model
Metrics for CIO
Metrics for Operations manager
Metrics for CISO
Metrics for CEO and the Board
Security model – business drives security
Information Security policies
Input
Business objectives
Compliance
requirements
Laws & Regulation
s
Business impact
Business & information
risks
Define
Define
Define
Security threats
International security standards
Information Security
standards
Information Security
guidelines
Security intelligence
Input
Line Manageme
nt
Auditors
Security manageme
nt
Risk & Complianc
e
Governance
Product Manageme
nt
Program Manageme
nt
Assurance
Security Services
Security Professiona
ls
IT GRC
Inform
Information Security Processes
Tech
nolo
gy
Policy framework
Security management
People
Serv
ices
Define security controls
Execute security controls
Information Security
Metrics objectives
Metrics framework
Measure security maturity
External security metrics
Mandate
Measured by
Input
Correction of security processes
Feedback: update business requirements
Process framework
Security metrics characteristics
Measurable
Objective
Quantitative (ideally)
Meaningful
With KPIs attached – know what is good and bad
Linked to business objectives – money speaks
Metrics for CIO – (1) Policy compliance and control maturityPolicy statement
IT Unit A IT Unit B IT Unit C Overall IT
Governance 3 3.5 2 3
Awareness 3 4 3 3.5
Development N/A 2 1 1.5
Hardening 4 N/A 2 3
Network N/A N/A 3 3
End devices 2 2 3 2
Overall 3 (£3m) 3 (100k)
2 (£10m)
3 (£13.1m)
Metrics for CIO – (2) Value at risk*
InputAsset valuesMaturity of controlsSystem weaknessesThreat information
Output – most likely (probability distribution) £ value of total exposure that IT organisation is exposed to
Work in progress
Inspiration in
BASEL II
* Eq most likely Total Exposure
Metrics for Ops manager
The morning dilema: “Can I have a coffee or is there something urgent to fix?”
Suggested metrics: A number/percentage of systems outside SLA for fixing
security weaknesses (both patches and configuration errors) – details of highly critical offenders – sorted by value at risk
Security incidents that resulted in breached SLA (SLA is both time and £ value)
And of course: Value at Risk
Quiz: Is “A number of critical vulnerabilities good metric?”
Answer: Not on its own!
Metrics for CISO
Gartner: by 2014 IT GRC and eGRC will merge in 70% of organisations. Likely head: CISO
Relevant metrics:Value at Risk – includes IT and other departmentsCompliance matrix ( same as for CIO)Annual risk reduction - Difference between VaR now
and last year compared to money spent
Showing value for money
Year VaR beginning VaR endTCO
security Net value
2009 £2,000,000 £1,100,000 £300,000 £600,000
2010 £1,100,000 £1,000,000 £150,000 -£50,000
2011 £1,000,000 £400,000 £200,000 £400,000
End year review: We have spent more than the risk reduction but
there were no incidents!
VaR can also increase with new business processes and changes in regulatory and threat landscape.
Metrics for CEO and board
Total exposure (£) = Value at Risk indicator
Unmanaged risk = likelihood there are risks that we do not know about = inverse of eGRC maturity
How do I know I have good metrics – metrics of metrics Decision effectiveness approach
% of important management decisions that can be or have been influenced by double learning (i.e. revision and refinement of targets, measures, criteria, etc.)
Investment approach
% of security metrics costs for “exploratory/testing” vs. total metrics cost
Speed
Cycle time from “Sense” to “Respond” for changing security metrics and management procedures.
% of metrics that are collected and calculated automatically
Cost
Cost of changing security metrics and management procedures as % of total security management costs.
Error
% of security metrics that do not tie to any decisions or decision processes (over-shoot)
% of decisions that have inadequate metrics support (under-shoot)
% of metrics which have significant number of false signals
Summary
Metrics need to include monetary value otherwise the business leaders will not understand why the metrics are collected and presented
Security (and GRC in general) are here to keep the company risk at acceptable level – that needs to be measured
Link security metrics to policy which is linked to business objectives
Boards do not like “un-managed risk”
Measure the metrics
