1 HiPEAC Workshop on Building Partnerships September 25, 2014 Ljubljana, Slovenia.
© Programming Research Brief Overview: Company, Software Products & Methods Dr. Evgueni Kolossov,...
-
Upload
bryson-sharps -
Category
Documents
-
view
213 -
download
0
Transcript of © Programming Research Brief Overview: Company, Software Products & Methods Dr. Evgueni Kolossov,...
© Programming Research
www.programmingresearch.com
Brief Overview: Company, Software Products & Methods
Dr. Evgueni Kolossov, R&D Director
Second HiPEAC Industry Partner ProgramTallinn, 8 October 2013
•ISO 26262 (automotive) up to ASIL level D•IEC 61508 (general industrial) up to SIL 4•EN 50128 (railways) up to SW-SIL 4•IEC 62304 (medical devices) up to level C•IEC 60880 (nuclear power)
© Programming Research
PRQA Overview (www.programmingresearch.com)
Code auditsCoding standards developmentTraining - languages, standards,
productsCustom integrations – compiler,
IDE, VCS
UK: Hersham (HQ) US: Boston + San Jose India: Bangalore Ireland: Dublin Netherlands: Zeist Ukraine: Lviv Romania: Bucharest
Distributors: Germany Japan China Korea
Technical:
Products: Locations Partners
Services:
Over 25 years track record with focus on static analysis C / C++, defect prevention, coding standards compliance and effective code reviews
© Programming Research
Standing in the Software Community
• Committee (BSI Sector) Member
ISO C
• Committee voting Members
ISO C++
• Committee Founding Member
MISRA C (Motor Industry Software Reliability Association)
• Committee Founding Member
MISRA C++
© Programming Research
Multiple Stakeholders
Stakeholders extend through the organisation:o Developers & Project Leads, Test operations, QA managers,
Senior VP management, Customers, Suppliers
Many organisations handle this in an ad-hoc manner today...
Project Devs/LeadsIs my project
meeting compliance on each release?
Corporate VPsWhich projects
present the greatest risk?
CustomersHow does this
release compare to the last one?
QA Manager Is our overall software
quality improving over time?
© Programming Research
PRQA Global Solution
Build Server
Su
mm
ary
An
aly
sis
re
su
lts
Build Server
PRQA ServerCentral Analysis
Su
pe
rvis
ory
an
d
ma
na
ge
me
nt
co
ntr
ol
PRQA ServerCentral Analysis
PRQA ServerCentral
Analysis
PRQA ServerCentral Analysis
PRQA ServerCentral Analysis
Distributed development /outsourcing
In-house Development
OEM/Customer Summary Information
and Reports
Management Policy Input and control
Lo
cal An
alysis L
ocal A
nalysis
© Programming Research
PRQA Enterprise Solution
Build Server
Build Server
Build Server
L
oca
l An
alys
is
L
oca
l An
alys
is
Lo
cal An
alysis
Lo
cal An
alysis
PRQA ServerCentral
Analysis
© Programming Research
PRQA Professional Solution
Build ServerBuild Server
Build Server
PRQA ServerCentral
Analysis
© Programming Research
Components
© Programming Research
Static Analysis Types
Lint-Like• Inexpensive• Limited Analysis
Capability• High False Positive
and False Negative rates
Bug Catchers• Strong on Simulation• Strong on whole program
test verification• Multi-Language support• Often part of Swiss Army
knife solution: bundled with testing tools
• High false negative rates.• Poor Language Usage,
Portability and Preventative analysis
Automatic Code Inspection• Strong on 4 technology types:
pattern-based, simulation, metrics, and comprehension.
• Facilitates code review and pretest checking with code collaboration, sophisticated suppression management and measurement analysis
• Low False Positive and False Negative rates.
• Weak on multi-language support
• Weak on whole program static test
© Programming Research
What & How we are Analyzing?
- Pattern-Based Analysis – This is the scanning of source code and checking for patterns that indicate issues correlating (within various degrees of severity) to defects in software. This involves checks for quality characteristics (or lack thereof) in Portability, Style, Language Usage, and Preventative Practices
- Simulation (Deep Data Flow) is a technique in identifying likely Run-time Defects (otherwise known as “Bugs” during execution). It is a form of testing without actually executing the code
- Metrics – It has often been stated that if you can’t measure it, you can’t improve it. For example, since complex code has been highly correlated to buggy code and impacts the testability and maintainability of software, metrics that measure these aspects can provide great insight into the quality of the code and act as an indicator to where concentrated improvement needs to be made.
- Comprehension – Static Analysis of code can also provide many different views upon the actual structure of the code in forms of graphs and diagrams that help in comprehension, helping with architectural decisions using tools that provide meaningful abstractions
Simulation
StructureMetrics
Pattern-Based
© Programming Research
Language Misuse
PRQA is industry leader in analysis of defensive coding practices JSF++ (Joint Strike Fighter - Bjarne Stroustrup) MISRA C++ MISRA C HICPP (our company standard from 2003, new version 3 October 2013)
Not all bugs are dataflow or resource usage based Incorrect language usage can result in hard (expensive) to detect bugs
Our software extends the defensive language analysis to provide for language based bug checking:
Calling an implicitly defined member function, where sister function has been explicitly declared.
Heap object of derived type undergoing derived to base conversion without a virtual destructor.
© Programming Research
Resource Misuse
Acquired resources are tracked to ensure that they are released.
Checking is not limited to memory create, open, close, fopen, fclose, strdup, dup
Analysis uses Syntax Usage Engine; tracking is performed by inter-function analysis within the translation unit.
Special handling of constructors and destructors allows for checking that resources allocated in a constructor are freed in a destructor.
© Programming Research
Deep-flow Dataflow
Dataflow analysis provides a mechanism to detect serious runtime behaviour problems:
Buffer overflows (security) NULL pointer dereference Undefined mathematical operations Use of unset variables much more...
Results are accurate and precise due to in-depth modelling of the language combined with a state of the art Satisfiability Modulo Theories (SMT Yices 2) solver.
(Dutertre, B., de Moura, L.: A fast Linear-Arithmetic solver for DPLL(T). In: Ball, T., Jones, R. B. (eds.) Computer Aided Verification. LNCS, vol. 4144, pp. 81--94. Springer, Heidelberg (2006))
© Programming Research
Deep-flow Dataflow (Continue)
Analysis is performed across function boundaries within a translation unit.
Software highlights obvious defects where cause and effect are localized, or where project wide knowledge is required to determine there are no issues.
Inter variable dependencies tracked ensuring low false positives/negatives
Tracking of values referred to by pointers increases both depth of analysis as well as improving modelling through function boundaries.
© Programming Research
Screenshots
© Programming Research
Screenshots
© Programming Research
Screenshots
© Programming Research
General Description
First and foremost we must parse the code correctly: Requires a fully functional C and C++ pre-processor and parser.
Initial analysis takes place during parsing where code is checked for conformance to the respective language standard. Both parsers are written to conform to the standards, and deviations from "legal code" is highlighted with a message and in some cases it is then controlled under a configuration option. The default behaviour is to comply to the standard.
The parser builds an internal Abstract Syntax Tree for the source code and then the rest of the analysis takes place.
Most of the analysis in QA C++ (and some in QA C) is performed in the reverse order of the call tree, ie. 'leaf functions' are analysed before their callers. This allows QA C++ to use information about a called function during the analysis of the caller. This is especially important for 'dataflow' and is a core requirement for inter function analysis within the translation unit.
© Programming Research
General Description (Continue)
A benefit of using our own parser technology is that we are not limited in the information that can be used for analysis. Macro history, instantiation history etc. that can normally be dropped by the compiler when generating code can be kept around. This aspect will again be used for some new C++ '11 checking, for example of the 'auto' keyword.
A common dataflow engine is used by QA C and QA C++. QA C++ translates C++ constructs into an equivalent C representation and this is then passed into the dataflow engine.
The resulting flow graph is further simplified and SMT solver is then used to search for defects.
As part of analysis, a semantic representation is also produced and this is then checked during Cross Module Analysis (CMA). A significant amount of undefined behaviour goes undetected by most of the available linkers, for example different function declarations etc. CMA performs this checking.
© Programming Research
General Description (Continue)
Areas with limited support:• The output format used for the semantic representation is verbose,
and in the case of C++ can result in huge amounts of information being written (and therefore being read). As projects have increased in size, and with libraries such as boost, this is becoming more of an issue. We're working to change the output format to improve this situation.
• Once this change is made we should be able to widen the scope of our existing analysis and provide new richer analysis for the entire program.
© Programming Research
What we are looking for?
• Collaboration in areas:– Architectural analysis of our dataflow with the target for
implementation:• Interprocedural Dataflow Analysis• Security Issues Analysis• Multi-threading Issues• Parallel Processing Issues
– Information about compilers new features & switches– Timing Analysis (executable, run-time)– New methods in code parsing, etc.
• Types of Collaboration: different types are available – subject for discussions
© Programming Research
© Programming Research
Customer Case study
“QA·C is above other tools when it comes to coding standard compliance”Site Software Director
© Programming Research
“Since we began using the MISRA C Compliance Module, the quality and consistency of our first generation code has skyrocketed, and our final products have been virtually error-free” Stuart Jobbins, Head of Software Development
Customer case study
© Programming Research
Customer Case study
“QA·C++ provides an efficient, robust, fully automated environment to introduce and enforce coding standards”Benjamin Pitzer, Senior Research Engineer
© Programming Research
Customer Case study
“With QA·C our prototypes retain much of their integrity, because most defects are caught early... even as our goals evolve and the code changes, our product quality remains consistently high.”Dana Sawyer, Senior Software Designer
© Programming Research
Customer Case study
“For us, it’s critical to focus on detecting issues early and tools like QA·C will help there”Samir Kulkarni, Head of Productivity
© Programming Research
No FalsePositives
All False Positives
Reportsevery
violation
Fails to report true
violation
Dr Marijn Temmerman from TERA-Labs observed, “On paper all the selected tools claimed to provide comprehensive MISRA C
compliance checking - but the reality was different!”
Independent research on Static Analysis Tools / MISRA Compliance by TERA-Labs