Measuring the Structural Quality of Software€¦ · CWE-252 Unchecked Return Parameter of Control...

2/16/2017

1

Measuring the Structural Quality of Software

Measuring the Structural Quality of Software

Paul C. BentzDirector of Government and Industry Programs, CISQ

What is CISQ?

OMG Special Interest Group

CISQ is chartered to define automatable measures of software size and quality that can be measured in the source code, and promote them to become Approved Specifications of the OMG®


OMG Special Interest Group


CISQ Sponsors

CISQ

Co-founders

IT Executives

Technical Experts

Copyright © 2016 CISQ. All rights reserved.

2/16/2017

2

CISQExec

Forum

CISQ/OMG Standards Process

Automated Function Points

Reliability

PerformanceEfficiency

Security

Maintainability

Approved MeasureSpecifications

ISO Fasttrack

Deployment Workshops

OMG


• OMG Supported Specification for Automated Function Points

• Mirrors IFPUG counting guidelines, but automatable

• Specification developed by international team led by David Herron of David Consulting Group

Automated Function Points

2/16/2017

3

Content of CISQ Measures

CISQ Quality Characteristic Measures Example architectural and

coding violations composing the measures

Security 22 violations(Top 25 CWEs)

• SQL injection• Cross-site scripting• Buffer overflow

Reliability 29 violations• Empty exception block• Unreleased resources• Circular dependency

Performance Efficiency

15 violations• Expensive loop operation• Un-indexed data access• Unreleased memory

Maintainability 20 violations• Excessive coupling• Dead code• Hard-coded literals


� CWE-22 Path Traversal Improper Input Neutralization� CWE-78 OS Command Injection Improper Input Neutraliza tion

� CWE-79 Cross-site Scripting Improper Input Neutraliza tion

� CWE-89 SQL Injection Improper Input Neutralization� CWE-120 Buffer Copy without Checking Size of Input

� CWE-129 Array Index Improper Input Neutralization

� CWE-134 Format String Improper Input Neutralization� CWE-252 Unchecked Return Parameter of Control Elemen t Accessing Resource

� CWE-327 Broken or Risky Cryptographic Algorithm Usag e

� CWE-396 Declaration of Catch for Generic Exception� CWE-397 Declaration of Throws for Generic Exception

� CWE-434 File Upload Improper Input Neutralization

� CWE-456 Storable and Member Data Element Missing Ini tialization� CWE-606 Unchecked Input for Loop Condition

� CWE-667 Shared Resource Improper Locking

� CWE-672 Expired or Released Resource Usage� CWE-681 Numeric Types Incorrect Conversion

� CWE-706 Name or Reference Resolution Improper Input Neutralization

� CWE-772 Missing Release of Resource after Effective Lifetime� CWE-789 Uncontrolled Memory Allocation

� CWE-798 Hard-Coded Credentials Usage for Remote Auth entication

� CWE-835 Loop with Unreachable Exit Condition ('Infin ite Loop')

Common

Weakness

Enumeration

cwe.mitre.org

Robert MartinMITRE

The 22 CWEs in the Security Measure


2/16/2017

4

Confidential

Mul

ti-la

ngua

ge,

mul

ti-la

yer

Arc

hite

ctur

e

EJB

PL/SQL

Oracle

SQL Server

DB2

T/SQL

Hibernate

Spring

Struts.NET

COBOL

IMS

Messaging

Sybase

� Integration quality� Architectural compliance� Risk propagation� Application security � Resiliency checks� Transaction integrity � Function point,� Effort estimation

� Data access control� SDK versioning� Calibration across

technologies� IT organization level

System Level

• Code style & layout • Expression complexity• Code documentation• Class or program design• Basic coding standards• Developer level

Unit Level1

Java

Java

JavaJava

Web Services • Single language/technology layer

• Intra-technology architecture• Intra-layer dependencies• Inter-program invocation• Security vulnerabilities• Development team level

Technology Level

JSP ASP.NETAPIs

Technology Stack

CISQ Measures the Technology Stack

2

3


Multi-language, Multi-layer Architecture

EJB

PL/SQL

Oracle

SQL Server

DB2

T/SQL

Hibernate

Spring

Struts.NET

COBOL

IMS

Messaging

Sybase

JavaJava

Java

Web Services

JSP ASP.NETAPIs

Technology Stack

Entry

Filtering,Authentication

Access

Analyzing transactions and data flows across languages and layers

Challenges in the Technology Stack


2/16/2017

5

How Do CISQ Measures Relate to ISO?

• ISO 25000 series replaces ISO/IEC 9126 (Parts 1-4)• ISO 25010 defines quality characteristics and sub-c haracteristics• CISQ conforms to ISO 25010 quality characteristic definitions• ISO 25023 defines measures, but not at the source c ode level• CISQ supplements ISO 25023 with source code level measures

CISQ defined automatable measures for quality characteristics highlighted in blue


CISQ in Service Level Agreements


Monitor and Track Product Quality against Targets i n Service Level Agreements

Monitor and Manage Service Provider Performance Ove r Time

Automated Function

Points245

628

931

86

1047

659

2/16/2017

6

App Certification Using CISQ

CISQ measuresCISQ-conformant

technology CISQ-

conformance assessment

Technology vendors

used in

CISQ service process

CISQ-conformant service process

CISQ-conformance assessment

Service providers

to provide

ApplicationCertification

Security XσReliability XσPerformance XσMaintainability Xσ

� CISQ/OMG− only assess conformance− do not certify applications− program initiates 2016

� Service providers− use CISQ-conformant technology− in a CISQ-conformant service process− to provide application certifications

Copyright © 2016 CISQ. All rights reserved. 11

Copyright © 2016 CISQ. All rights reserved. 12

Join CISQ ! www.it-cisq.org

2/16/2017

7

CISQ-Standards in der Praxis für Test & Entwicklung

Steffen Ritter | CAST: Leader in Software Analysis and Measurement

CAST Application Intelligence Platform

Unit Leader Team Cost Date

1 MODEL

49+LANGUAGES

12+DATABASES

2 ANALYZE 3 MEASURE

ANALYSIS ENGINE

SOURCE CODE DEMOGRAPHICS

Analyses Driven Industry

Standards

• Single-technology

• Cross-technology

• Cross-language

• Cross-component

• Dataflow Analysis

• Transactional analysis

Automated

meta-model

Reverse engineered

through the

interactions of

components

through

heterogeneous

layers.

Curated Quality Assessment Model

Identified

violations are

evaluated for

risk

Violations are

categorized in

the context of

Health Factors

RESEARCH HOTSPOTS EXPLORE ARCHITECTURE

AED ADP & AAC

SEE THE BIG PICTURE

SYNTHESIS & ANALYTICS

INPUT PROCESSING

Automated

completeness

check

Code extraction

& formatting

Standards-Based Sizing

Rooted in

Industry

Standards

Application size

is measured in

Function Points

2/16/2017

8

Beispiel: Automatisierte Transaktionserkennung

Frontend HTML 5

Client-side Java Script

Java Code

Java Persistence API (JPA)

Datenbank

Wo Rauch ist… Brennpunkte objektiv priorisieren

Ausfallrisiko vs. Wichtigkeit der Anwendung für das Unternehmen

Applikations-EbeneObjektiver Portfolio-Gesamtüberblick

Heatmap aller Komponenten nach Risiken, Performance und Qualität

2/16/2017

9

CAST-Dashboard für Test Manager & Architekten

(1) Kritische Verstöße schnell identifizieren

(2) Risikoreichste Komponenten erkennen

(3) Regelverstöße verstehen

(4) Priorisieren und Aktionsplan entwickeln

(1) Kritische Verstöße schnell identifizieren

(2) Risikoreichste Komponenten erkennen

(3) Regelverstöße verstehen

(4) Priorisieren und Aktionsplan entwickeln

11

4433

22

Deep Dive auf Code-Level zu konkreten Transaktionen

2/16/2017

10

CAST-Dashboard für CIOs und IT-Management

CAST: 25 Jahre Erfahrung & 150 Millionen F&E

Hunderte internationale Konzerne bauen weltweit auf CAST

Nahezu alle großen ADM-Dienstleister nutzen CAST intern

Management-Beratungen nutzen und empfehlen CAST

Globale Systemintegratoren bieten Services auf Basis von CAST an

CGI

Kontakt: Steffen Ritter | Director Enterprise Sales

Herzog-Wilhelm-Str. 26, 80331 München, Tel: +49 89 215 89 441, E-Mail: [email protected]

Measuring the Structural Quality of Software€¦ · CWE-252 Unchecked Return Parameter of Control...

Documents

Transcript of Measuring the Structural Quality of Software€¦ · CWE-252 Unchecked Return Parameter of Control...