TEST I VIDAREUTVECKLINGEN AV GRIPENS AVIONIK- OCH...
Transcript of TEST I VIDAREUTVECKLINGEN AV GRIPENS AVIONIK- OCH...
Saab Propriety
TEST I VIDAREUTVECKLINGEN AV GRIPENS AVIONIK- OCH MARKSTÖDSYSTEM
Håkan Anderwall, Saab AB Jacob Gustafsson, Combitech 2011-11-17
Saab Propriety
AGENDA
! The Gripen weapon system – an overview ! Part 1: The Gripen Aircraft Computer System
• Scope – what is done, what needs to be done, and why? • Aircraft Computer System, IMA • Architecture and System functions • Verification and Validation methods and goals • Test environments and Test architecture • Automated Verification and Validation flow
! Part 2: The Mission Support System (MSS) • Description of the MSS system • The test process, steps & content • The build process • Test methods and experience
Saab Propriety
A NEW AIRCRAFT COMPUTER SYSTEM IN GRIPEN ! Why?
• Obsolescence issues for several equipments. • Old development environment/tools, language. • Software is not cost effective to maintain and add
functionality in.
! A new IMA architecture has been engineered • More effective, allocate functions more freely among
common computational resources. • Focus on model based development, from models on
entire aircraft and its hardware and software components and their interfaces, to design of applications and library functions. (Using Rhapsody, BridgePoint, Matlab/Simulink)
! Re-implemented system functions requires re-verification/validation!
Saab Propriety
INTEGRATED MODULAR AVIONICS (IMA)
! IMA is a shared set of flexible, reusable, and interoperable hardware and software resources that, when integrated, form a platform that provides services, designed and verified to a defined set of safety and performance requirements, to host applications performing aircraft functions.
! Apart from Gripen, there are IMA systems also in Lockheed Martin F-22 and F-35, Dassault Aviation Rafale, and Airbus A350 and A380.
! Guidance and certification aspects for the development of an IMA system is described in RTCA DO-297.
Saab Propriety
Flight Critical Functions
IOP APP
Flight Critical Functions
IOP APP
Flight Critical Functions
IOP APP
Flight Critical Functions
IOP APP
GRIPEN - ARCHITECTURE
Flight Critical Functions
IOP APP
Tactical Functions
IOP APP
Displays
IOP APP
Head-up Display
IOP APP
Stores Functions
IOP APP
General Functions
IOP APP
Video Processing IOP APP
Data Transfer
IOP APP
Nav
igat
ion,
Flig
ht c
ontro
l, C
omm
unic
atio
n sy
stem
s et
c.
! Many new DCM’s (Digital Computing Modules), each hosting two or more processors, using one of several new RTOSes supporting partitioning.
! Re-implemented functions in APP (APlication Processor)
! Kept interfaces to unchanged systems
! A large amount of reverification/validation
Nav
igat
ion,
Flig
ht c
ontro
l, C
omm
unic
atio
n sy
stem
s et
c.
Saab Propriety
SYSTEM FUNCTIONS, SOFTWARE APPLICATIONS
! A set of DCM software applications and libraries performs a system function using the IMA platform.
! Needs to verify all CSCIs (software components), system requirements and validate system functions.
Symbolfile BIN
Symbolfile BIN
AGS (A661 Server)
Software Application Display_Server
Display
APP Partition
DCM
APP Partition
Software Application A Library Service
(e.g. A661 Adapter)
Sensor
Actuator
Saab Propriety
THE OVERALL GOAL FOR VERIFICATION AND VALIDATION
Software – 95% host/target breadboard test All tests fully automated Software – 5 % Equipment test Hardware – 100% Equipment vendor
< 5% Aircraft Ground and Flight test ~15-20% System simulation, some automated ~10-30% Analysis ~50-70% Reuse SW and HW level verification
~100% ref to SSS, SRS, and EqS verification results with analysis and justification
Stakeholder Requirements / contract
~100% ref to SSS, SRS, and EqS verification results with analysis and justification
Acceptance Test
Product requirements and functions
Product VoV
Subsystem requirements and functions
System integration tests
Software requirements (several levels)
Application/ Library/ Unit tests
Code & Models Static code & model analysis
Automated rules checking ~70% Manual rules checking ~30%
VoV C
ost
Saab Propriety
VERIFICATION METHODS - SOFTWARE
! SW/SW Integration Tests • Black-box verification of CSCI (Application, Library)
! HW/SW Integration Tests • Integration with hardware interfaces (mostly the platform CSCI:s).
! Unit Tests • Verification of Computer Software Units.
! Analysis/Inspection • Visual inspection of code or design. • Static analysis of models and code where possible.
Saab Propriety
VERIFICATION & VALIDATION METHODS - SYSTEM ! Analysis
• Analysis of design, specifications etc. of a systems ability to perform its intended functions.
! Inspection • Inspection of a system items fulfillment of required characteristics (e.g.
emission, conformity to standards and regulations)
! Test & Demonstration • MySim • SW verification (reused) • Simulation (models) • Flight Test / Ground Test • Avionics Rig / Total System Rig
! Service Experience • Previous usage of subsystem/system
Saab Propriety
MySim ! Desktop simulator for system development, and software
development. ! Validation of concepts. ! Background mode:
Automated execution of system integration test cases.
Saab Propriety
VERIFICATION AND VALIDATION – AUTOMATION GOAL ! Compile and link all tests for different test environments:
• MySim Automated system integration tests
• Solaris/Linux Host software integration and unit test, code coverage measured • Target breadboard Target software integration and unit test, object code analysis • Testrig with DCM Hardware/software integration test (incl. WCET)
! Execute all possible tests automatically and evaluate: • MySim, Host, Target breadboard • Daily VoV result presentation – how many test failed etc.?
! Static code analysis against C/C++/Ada95 ”Coding Standards” ! Model consistency check of models against each tool ”Modeling
Standard” ! Automated model code generation ! DOORS checks – Untested / changed requirements, traceability etc.?
Saab Propriety
Test environments (Applications and libraries)
Test in Models
Test on Host Test on Target
Processor (breadboard)
Collect evidence of the CSCIs requirements fulfillment
No big surprises!
Daily test of generated code
Code coverage
Quick feedback
Automated tests enables quick regression testing!
Test in System Simulator (MySim)
Test in DCM (Rig)
CSCI Level
System Level
Application/ Application integration
Application/ Platform integration
Manual Automated
Partly automated
Saab Propriety
TEST ARCHITECTURE: BREADBOARD TEST CLUSTER ! A set of breadboards for all DCM CPU architectures ! Managed by a continuous integration server deploying test binaries
per dedicated CPU architecture Flight
Critical Functions
APP
Tactical Functions
APP
Tactical Functions
APP
Tactical Functions
APP
Flight Critical
Functions APP
Tactical Functions
APP
Tactical Functions
APP Displays
APP
Head-up Display APP
Tactical Functions
APP
Tactical Functions
APP
Video Processing
APP
Breadboard test server
Use
r net
wor
k
Continuous integration server
<<Send test binaries>>
<<Send test results>>
Flight Critical
Functions APP
<<Deploy and Execute>>
<<Get test result>>
Saab Propriety
AUTOMATED VERIFICATION AND VALIDATION FLOW
Check-out, compile, link, execute
CI server: sw test
Host test
report
*.atr *.ctr *.log
Post process report files
transform
xml test
summary
transform & present
Test Page Summary CI X Pass
transform
*.csv import (manual) DOORS
SVC module
Check-out, cross compile, link
report *.atr *.ctr *.log
Target test
target breadboard Execute
CI server: system integration test
Check-out, compile, link, execute
MySim test
report
*.log
DOORS SSTD
module
Accumulated
code coverage
Saab Propriety
Content
! MSS and its vital role in the tactical loop ! The different test steps and phases ! The build & installation process ! Experience from different test methods
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
MISSION SUPPORT SYSTEM (MSS) Scope of Functionality
! Planning missions objectives and routes for Gripen missions
! Loading planned data to the aircraft ! Loading flown data from the aircraft ! Analysis and replay of the flown missions ! Functionality for mission simulations
! The functionality is vital to achieve mission success
Saab Propriety
Data loading overview
Digital Maps
Library data
Communication Data
MSS Mission Support System
DTU-P
MMC
Order, Intelligence, Weather & Reports
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Data Support
Map info
Libraries
Rehearsal Scenario
Simulation
3D View
Preparation Data Libraries
Navigation data Other data Settings
Evaluation
Mission Replay including • Flight path
• Events
• Cockpit
• Audio/Video
Mission Analysis
Planning
Navigation
Sensors
Communication
Configuration Intelligence Updates
DTU-P MMC
Mission Report
Intelligence Data
Recorded Data
Flight Path
Events
Cockpit panels
Audio & Video
Image Analysis
TACTICAL INFORMATION HANDLING
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Test phases (performed weekly)
Automatic build of the code
Load on host for development
Development tests (unit & functionality)
Early feedback
Load on target for further testing System test
CSCI Level
System Level
Sanity & integration testing
Early feedback Validation test
Performed in steps throughout the development Requirement tests
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Type of test Responsible Definition
Code analysis Development team Code analysis to find initialized variables, false pointers etc.
Unit test Development team Development test in parallel with the implementation
Functional test Development team Functional tests of the implemented code
Integration test Development team Tests in the teams to early assure correct integration between components
Sanity test Common test team Short “up & running” test of the complete product
Integration test of complete product
Common test team Integration test performed weekly & every sprint (3 weeks) to assure that the integration of the complete product is correct
System test Development team Contractual requirement tests with validation perspective
Dry Run & Cust Demos Common test team Customer demos to present the system
Validation test Common test team Final test with validation perspective
TEST ACTIVITIES
Saab Propriety
Testing characteristics
! The amount of automatic unit testing is increasing. Time consuming for old code as it requires much effort
! Still a large amount of manual testing ! Integration testing on target is tested from a user point of
view. Uses several simulation tools to verify interfaces ! GUI testing of the integrated product has not been
successful. Tricky to develop and requires a lot of under haul during development
! System & final testing is performed with a validation perspective in order to assure customer satisfaction according to contractual requirements
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Build overview MISSION SUPPORT SYSTEM (MSS)
.h .cpp svn
code repository
gcc
installations
buildscripts jenkins
unit tests code analysis
.iso .rpm .exe
test lab
test results installation files
Saab Propriety
Build process & installation
The build process is automatic and serves many advantages: ! Enables quick response & support to developers. Host system
environment is similar to the target environment ! Easy to switch between customer configurations on host machines ! The code is built automatically once checked in ! Build results are distributed to the development teams ! Several customer configurations are built over night ! Unit tests are run automatically as regression tests with distributed
results ! Installation requires little manual effort and will be performed
automatically
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Test methods & experience ! Team organized versus functional organisation ! Used to have separate teams for integration & system testing ! Intended to shorten development time and increase quality ! Team organisation:
• Resulting in many faults discovered late in the development process • “Somebody else” tested the development teams implemented functionality • Development teams relied on the test teams to find faults • Unless good communication, the test teams were not fully updated on “what’s new”
! Functional organisation: • The teams are responsible for a wider functional scope • The testers are located in the development teams • The testers know exactly which functionality that is under development • Integration tests are performed earlier in the development phase • Test cases are developed with a widened content
MISSION SUPPORT SYSTEM (MSS)
Saab Propriety
Contact
Jacob Gustafsson E-mail: [email protected] Phone: +46709-891744
MSS
Contact
Håkan Anderwall E-mail: [email protected] Phone: +46734184234
GRIPEN