1

SDLC Gap Analysis and

Remediation Techniques

Jason Taylor

CTO

Security Innovation

About Security Innovation

• Software and Crypto Security Experts– 10+ years research on vulnerabilities and cryptography

– Hundreds of assessments on world’s most dominant software

• Products, Services & Training

– Software Assurance • white and black box assessments

• secure development lifecycle and crypto consulting

– Training & Guidance• eLearning, instructor-led, and secure coding standards

– Encryption

• fast, lightweight, patented, and future-proof

• Helping organizations:

– Build internal software security competency

– Protect data in transit and while applications are accessing it

– Develop secure software applications and products

2

Agenda

Brief overview of key security engineering activities

• Identifying goals and objectives

• Assessing your existing process relative to industry best practices

– Conducting the Gap Analysis

• Planning the remediation roadmap

• Implementing the Roadmap: Introducing security activities, tools and training

– Case Study: Sony Corporation

– Pointers to helpful resources (free and commercial)

Secure Software Development

Requires Process Improvement

• Key Concepts

– Simply “looking for bugs” doesn’t make software secure

– Must reduce the chance vulnerabilities enter into design and code

– Requires executive commitment

– Requires ongoing process improvement

– Requires education & training

– Requires tools and automation

– Requires incentives and consequences

Break the “Pen Test” cycle of testing as a catch-all

3

Repeatable, Secure Development WorksA look at the Microsoft SDL

11966

400

242

157

Windows®

XP

Windows

Vista®

OS I OS II OS III

Total Vulnerabilities Disclosed 12 Months After Release

34

3

187

SQL Server® 2000 SQL Server 2005 Competing

commercial DB

Total Vulnerabilities Disclosed 36 Months After Release

Before SDL After SDL

45% reduction in Vulnerabilities

Before SDL After SDL

91% reduction in Vulnerabilities

Consistent application of sound security practices during all phases

of a development project will result in fewer vulnerabilities

Security Engineering: What it is and what it entails

• Integrating security into your lifecycle

– Upfront security design, secure coding practices, and testing

for security must all be an integral part of your application

development processes

• Identifying your objectives

– Understanding early what the security objectives are for your application

– Will play a critical role in shaping threat modeling, code reviews, and testing

• Knowing your threats

– Analyzing your application in a structured and systematic way to recognize its threats and vulnerabilities

• Using an iterative approach

– Some activities should be performed multiple times during the development process in order to maximize application security

4

Security Engineering: What it is and what it entails

Key Security Activities

• Identify Security Objectives understand key security objectives and scenarios

• Apply Security Design Guidelines don’t make common security design mistakes, learn from past vulnerabilities

• Conduct Security Architecture and Design Reviews identify security problems that can have multiplier effect in later phases

• Create Threat Models identify threats, attacks, vulnerabilities and countermeasures

• Perform Assessments: Security Code Reviews & Penetration Testinguncover vulnerabilities during development and in deployment

• Conduct Security Deployment Reviewsensure configuration/deployment problems are found before app is in production

5

Agenda

• Brief overview of key security engineering activities

Identifying goals and objectives

• Assessing your existing process relative to industry best practices

– Conducting the Gap Analysis

• Planning the remediation roadmap

• Implementing the Roadmap: Introducing security activities, tools and training

– Case Study: Sony Corporation

– Pointers to helpful resources (free and commercial)

Identifying Goals & Gaps

• What is driving the improvement?

– Regulatory compliance

– Customer requirement

– Standards compliance

– Reduce risk

• Where are the biggest problem areas?

– Where do you fall short

– What are the technical and business risks associated with each gap

• The result of this phase is a customized set of goals

– Used to drive a remediation plan

– Leveraged to improve your security development policies

– Basis for new procedures and security activities

6

Can you Define Measurable Goals?

343

187

SQL

Server®

2000

SQL Server

2005

Competing

commercial

DB

Before SDL After SDL

91% reduction in Vulnerabilities

• Recall Microsoft SDL Study

– Activity: adopt secure SDLC following best practices

– Result: 91% reduction in vulnerabilities

• Results drove cost and reputation savings

– Reduction of vulnerability count alone not great metric

– For a software vendor like Microsoft, this means

• Less time ($$) finding same mistakes

• Less time developing fixes for vulnerabilities

• Less time issuing and maintaining patches

• Less support burden to end users

– For Enterprise IT Security/Risk team, this may means

• Meeting key compliance objective

• More efficient use of internal resources

• Less support burden and risk to end users

• Less out-of-pocket expense with outsourced vendors

Match metrics to

objectives for higher

chance of success

Agenda

• Brief overview of key security engineering activities

• Identifying goals and objectives

Assessing your existing process relative to industry best practices

– Conducting the Gap Analysis

• Planning the remediation roadmap

• Implementing the Roadmap: Introducing security activities, tools and training

– Case Study: Sony Corporation

– Pointers to helpful resources (free and commercial)

7

Assessing your Existing Development Process

• Relative to industry best practices, standards or internal mandates

– ISO 27002, NIST-800, ITIL frameworks, the Microsoft SDL, internally-defined

• Determine organizational capabilities related to security

• Start with Policies/Standards, then look at procedures at each phase– Iterate with team leads to analyze tools, process, and staff skill

• Assess your Security training program, too– Training ensures tools and other activities are executed in the right manner

– Understand gaps in your training program

• Is your team regularly trained?

• Do architects know how to choose secure design components?

• Do developers know best practices for secure coding?

• Have testers had training on attack techniques?

The goal is to understand the development standards & processes, including

everything that is currently being done with respect to software security

1.) Review Org Structure and Team Roles

2.) Analyze

Policies &

Standards

Reqts.

3.) Analyze &

Aggregate Data4.) Refine via focused

Interviews (usually team leads)

5.) Create Gap Analysis Report

with recommendations

SDLC Process Assessment – Graphical View

Best Practices

8

Assessing your Existing Development ProcessSecurity Policies

• Security policies

– Are the backbone of your development process

– Without them, many efforts are wasted

• i.e. What good is a code scanning tool if it’s use is not required

• Questions to ask yourself

– Do you have a formal development process with well-defined phases and activities?

– Do you have a dedicated security team?

– Do you have corporate security and compliance policies?

– How is the development team made aware of security policies?

– How does the development team access security policies?

– How does your development team interact with company security policies (governance, compliance, etc)?

Assessing your Existing Development ProcessRequirements & Design Phase

• Requirements and design phase security activities– security requirements objectives

– threat modeling

– design best practices & design reviews

• Questions to ask yourself:– Do you gather security objectives?

• How are they stored? How are they mapped to the rest of the design process?

– Do you have a set of design best practices that you employ for security?• How are they stored? How do you ensure architects are using them?

• How do you revise and improve them over time?

– Does your team conduct security architecture and design reviews?• How often? Is it done before implementation?

• Do you use checklists to drive the process?

• How are the results tracked and used to improve the design?

– Does your team create threat models for your application’s architecture & design?• When? Where is it stored? Is it updated over time?

• How is it used to improve the design, implementation and testing?

9

Assessing your Existing Development ProcessImplementation Phase

• Implementation phase security activities

– development best practices

– security code reviews

• Questions to Ask– Does your team use a formalized set of security coding best practices?

– What type of code scanning tools do you use?

– Do you perform code reviews against security best practices?

• How often? What is the process?

• Do you have a set of checklists that can use drive the review process?

• How are the results tracked and used to improve the implementation?

Assessing your Existing Development ProcessVerification Phase

• Verification phase security activities

– abuse case definition

– penetration testing

• Questions to ask:

– Does your team conduct 3rd party or internal penetration tests?

• How often do you perform internal and 3rd party penetration tests

• Do you prioritize attack paths based on a threat model?

• Do you have a set of vulnerabilities, unique to your system, that you test against?

• How are the results tracked and used to improve the implementation?

– Are your testers & QA trained on the latest attack trends and test techniques

– Do you use security testing tool

• Web scanners such as AppScan or WebInspect

• File and network fuzzers

• etc

10

Assessing your Existing Development ProcessRelease & Response Phase

• Release and response phase security activities and preparedness

– security deployment review

– security attack response

– patching processes

• Questions– Does your team use a formalized set of security deployment best practices?

– Do you have a security incident response plan?

– Do you use network scanning tools such as Nessus?

– Do you have a set of deployment best practices that you employ for security?

• How are they stored? Do you ensure your developers are using these?

• How do you revise and improve these best practices over time?

– Do you review your deployment for security best practices before deployment?

• How often are inspections performed?

• What is the process? Do you have a set of checklists to drive the review process?

• How are the results tracked and used to improve the deployment?

Agenda

• Brief overview of key security engineering activities

• Identifying goals and objectives

• Assessing your existing process relative to industry best practices

– Conducting the Gap Analysis

Planning the remediation roadmap

• Implementing the Roadmap: Introducing security activities, tools and training

– Case Study: Sony Corporation

– Pointers to helpful resources (free and commercial)

11

Planning the Remediation Roadmap

• Use your goals and key risks to analyze the results of your analysis and prioritize the areas most in need of augmentation

– based on practical and proven IT risk and cost/benefit considerations.

• Consider a stakeholder strategy and planning workshop

– designed to review the major software risk management strategies

(avoid, transfer, accept, remediate) and attach the appropriate control

options to each identified threat or risk category

• Create your software risk remediation roadmap– This will become the basis of specific subsequent

security improvement initiatives

Assessing your Existing Development ProcessActivity Matrix

Product A Product B Product C

Define Security Objectives X X

Apply Security Design Guidelines X X

Threat Model X X

Security Architecture and Design Review X X

Apply Security Implementation Guidelines X

Security Code Review X X X

Security Penetration Testing X X X

Apply Security Deployment Guidelines X

Security Deployment Review X

3rd party Security Penetration Test X X X

Security Incident Response Plan X X X

12

Technical SolutionsExample

• Update IDE to latest version

• Use Visual Studio Code Analysis (free)

• Use compiler options to improve security (free)

• Deploy Fortify for static analysis (additional cost)

• Deploy PC Lint for static analysis (free)

• Improve access control and monitoring for source code access (free)

Training/Skills TransferExample

• Security 101 Training for all staff

• Application Security Fundamentals training for development staff

• Architecture and risk analysis training for architects

• Creating Secure Code Java training for developers

• Penetration test training for the QA team

13

Training Roadmap

Product A Product B Product C

How to Define Security Objectives PM, SC PM, SC

Application Security Fundamentals E E

Attacker Techniques Exposed O O O

Architecting Secure Solutions O O O

Security Architecture and Design Review A, SC A, SC A, SC

Threat Modeling A, D, SC A, D, SC

Creating Secure Code Java D

Creating Secure C++ Code D D

Conducting a Security Code Review D, SC D, SC D, SC

Classes of Security Defects D, T D, T D, T

Buffer Overflows D D D

Security Testing T T O

Security Champions for Each TeamExample

• Each application development team should appoint a security champion or “representative” that will:

– drive security and ensure compliance with application security best practices within team and when interacting with other teams

• The CSO will call regular meetings to discuss security issues encountered by each team and review issues that have been logged during the SDLC

• Each team will start to analyze security statistics such as:

– the number of security issues dealt with

– the number of times the Incident Response Plan has been used

– how issues have been resolved.

14

Agenda

• Brief overview of key security engineering activities

• Identifying goals and objectives

• Assessing your existing process relative to industry best practices

– Conducting the Gap Analysis

• Planning the remediation roadmap

Implementing the Roadmap: Introducing security activities, tools and training

– Case Study: Sony Corporation

– Pointers to helpful resources (free and commercial)

Implementing the Roadmap:

• Should be designed based on your findings and determination of where you need the most help

• Typical implementations:

– Training courses that cover security design, development and testing best practices; or a specific tool

– Threat Modeling conducted earlier in the SDLC

– More frequent, iterative code reviews

– Rolling our secure development best practices

• Sequencing is critical

– Introduce baseline guidance for all first

– Work with security champions; develop them as mentors for intermediate and advanced topics that will be rolled out at later stages

– Beware not to invest in new tools too soon, e.g., before baseline domain training

15

SDL Case Study: Sony Corporation

Sony requested an SDLC business proposal, with several phases, that will help Sony:

• Build and maintain internal software security expertise

• Become more proficient developing secure, high-quality web applications

• Implement a recurring security assessment program

• Rollout a repeatable, easily-adoptable development process that includes security activities & check points at each phase of the SDLC

• Distinguish themselves as the premier provider of integrated and collaborative computing solutions in Europe

End goal was nothing short of making Sony significantly more self-reliant

for security expertise via tailored processes, practices, and technology.

Sony SDL Case Study: Challenges

• Had high-throughput, near shore development team of roughly 100, but limited expertise in secure development and security testing

• A critical marketing site that is regularly updated and needs frequent security assessments with short turn-around/delivery timelines

• Lack of a “Security Champion” in each software development team

• Limited time that developers and testers can be taken “off the bench”

• Danger of vulnerabilities in their applications exploited

– could mean loss of customer base, reputation, and share price

• The risk of operating in increasingly open environments (web, ESA, et al) with no foreknowledge of operating environments or user intent

– translates to drastically accelerated risk

16

Define

Security Requirements

Use Case and Abuse Case –

Definition and Review

Design

Threat Modeling

Security Design Review

Architecture Risk Analysis

Security Test Planning

Code

Security Code Analysis

Metrics Gathering and

Reporting

Test

Penetration Testing

Metrics Gathering

and Reporting

Deploy

- Online ApplicationSecurity Monitoringportal

- Recurring Assessments (Penetration Testing)

- Reporting

Software Security Risk Management Solution encompassing : Process Improvement (services), Education (training) and Tools to greatly improve both efficiency,

reliability, and accuracy during the phases of the SDLC

Sony SDL Case Study: SDLC long-term vision

Sony SDL Case Study: Solution

• 3-phase, 18-month program

• Define a recurring security assessment program

• Customized training program for the development team

• Adopt best-practices and standards

– Customized development best practice knowledge base

• Optimize their SDLC with:

– appropriate team activities at each phase

– appropriate phase transition gates

– introduction of the role of security champion

• Define assessment metrics so effectiveness can be measured– trend reports for the recurring web security assessments

– exam questions to gauge evolution of the team pre- and post-training sessions

17

RoadmapTe

amM

en

tor Baseline Guidance

- Guidelines & Principles

- Language Independent

Team

Me

nto

r Integrated Guidance

-Checklists & How-To’s

- Web & Java technology

- Collaboration Team

Me

nto

r Advanced Guidance

- Detailed How-To’s

-New technology content

First 6 months “Basic Training”

6-12 months “Intermediate”

12-18 months“Self-Sufficiency”

Tra

inin

g

Introductory and Baseline

- Application security Fundamentals

- Fundamentals of security testing

Tra

inin

g

Intermediate

- Creating Secure J2EE applications

- Breaking Software Security

Tra

inin

g Advanced

- New technology training

- Architecting Secure Solutions

SD

LC

E

nh

an

ce Assess & Introduce

Activities

-Review existing “gates”

-Health Checks

-Identify Champions

SD

LC

E

nh

an

ce Proactive Activities

-Improve “gates” in use

-Pre-deployment testing

-Champions contribute to SDLC optimization

SD

LC

E

nh

an

ce

Optimized SDLC

- AppScan to validate TeamMentor guidance

-Security Champions mentoring rest of team

Recurring Web Security Assessments

How Security Innovation can Help

• eKnowledge products

– eLearning

• For each phase of the SDLC

– Secure Development Process Product

• Aligns corporate standards and compliances withdevelopment implementation

• Source Software Development Services

– SDLC Assessment & Optimizations

– Design & Requirements Review

– Code Review

– Security Testing

18

eKnowledge Solutions for Secure Development &

Code Review

Software Security eLearning:

– Creating Secure Code

– How to Break Software Security

– Fundamentals of Application Security

– Introduction to Threat Modeling

– Intro to Cryptography

TeamMentor:Secure Development Guidance System

– Out of the box secure development standards and best practices (maps to several compliance reqt’s)

– How-to’s, how not-to’s, code snippets, attacks, checklists

– Targeted, on-demand, context specific application security training

– Dedicated section for software security engineering

Try eLearning for free

http://elearning.securityinnovation.com

Free eLearning Course for AttendingIntroduction to Threat Modeling

Fundamentals of Application Security

Introduction to the Microsoft SDL

“Security Engineering Explained” Whitepaper

getsecure@securityinnovation.com