Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer...
Transcript of Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer...
![Page 1: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/1.jpg)
Copyright © 2004 Computer Sciences Corporation. All rights reserved. 3/16/2007 9:17:48 AM 1
NDIA National Test and Evaluation ConferenceMarch 12-15, 2007
Early Application of Computer Program Systems Integration, Test and Performance MeasurementTom SobieralskiComputer Sciences Corporation
![Page 2: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/2.jpg)
3/16/2007 9:17:48 AM 2
Agenda
• Background
• Systems Integration, Test and Performance Measurement
• Summary
![Page 3: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/3.jpg)
3/16/2007 9:17:48 AM 3
Background
![Page 4: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/4.jpg)
3/16/2007 9:17:48 AM 4
What is Computer Program Systems Integration, Test and Performance Measurement?
• The ability to verify the interfaces, functions and measure performance of two or more computer programs on the target hardware suite and operating environment.
![Page 5: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/5.jpg)
3/16/2007 9:17:48 AM 5
Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement
• Introduction of COTS hardware• Conversion of legacy software to new languages• System complexity with multiple hardware and operating
environments
![Page 6: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/6.jpg)
3/16/2007 9:17:48 AM 6
PB
System IntegrationTesting
Computer ProgramDefinition & Design
Computer ProgramImplementation
System Definition& Design
Computer ProgramTesting
Code
Unit Test
Element CPIntegration & Test
OperationalTests
System Demo &System Qual Test
System T&E,MEIT
EngineeringTest & Evaluation
Comp ProgramDetailed Design
ComputerProgram Design
CP PerformanceReqts Definition
OperationalReqts Definition
SystemReqts Definition
ElementReqts Definition
FB AB Development PhasesSystem Requirements Analysis/DesignSoftware Requirements AnalysisPreliminary DesignDetailed DesignCoding & CSU TestingCSC Integration & TestingCSCI TestingSystem Integration & Testing
DDR
IPR
CR
SDR
PDR
CDR
TPR
MRP
CPAP
PRIMEResponsibility
CSCResponsibility
HDR
Software Development Process
![Page 7: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/7.jpg)
3/16/2007 9:17:48 AM 7
Multi-Processor Environments
• Legacy– AN/UYKs
• COTS Processors– Single Board Computers (SBCs)– Symetric Multi-Processors (SMPs)
AN/UYK-43
COTS ProcessorCabinet
OperatorConsole
UserDisplayConsole
![Page 8: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/8.jpg)
3/16/2007 9:17:48 AM 8
Multi-Operating Environments• Legacy
– Aegis Tactical Executive System (ATES)• COTS
– Concurrent Powermax– Sun Solaris– Red Hawk Linux– LynxOS
![Page 9: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/9.jpg)
3/16/2007 9:17:48 AM 9
Multi-Computer Programming Languages
• Legacy– Compiler Monitor System -2 (CMS-2)
• New– Ada– C/C++– Java
C/C++
![Page 10: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/10.jpg)
3/16/2007 9:17:48 AM 10
Systems Integration, Test and Performance Measurement
![Page 11: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/11.jpg)
3/16/2007 9:17:48 AM 11
Process Improvement
• Incremental Systems Integration, Test and Performance Measurement during the software development process
– Improve System Stability– Early Identification and Resolution of issues and defects– Significantly reduce Engineering Test & Evaluation failure rates– Processes and Procedures QA reviewed and approved
CSC DMEI DE CMMI Level 5
![Page 12: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/12.jpg)
3/16/2007 9:17:48 AM 12
PB
System IntegrationTesting
Computer ProgramDefinition & Design
Computer ProgramImplementation
System Definition& Design
Computer ProgramTesting
Code
Unit Test
Element CPIntegration & Test
OperationalTests
System Demo &System Qual Test
System T&E,MEIT
EngineeringTest & Evaluation
Comp ProgramDetailed Design
ComputerProgram Design
CP PerformanceReqts Definition
OperationalReqts Definition
SystemReqts Definition
ElementReqts Definition
FB AB Development PhasesSystem Requirements Analysis/DesignSoftware Requirements AnalysisPreliminary DesignDetailed DesignCoding & CSU TestingCSC Integration & TestingCSCI TestingSystem Integration & Testing
DDR
IPR
CR
SDR
PDR
CDR
TPR
MRP
CPAP
PRIMEResponsibility
CSCResponsibility
Software Development Process
HDR
Systems Integration,Test and Performance Measurement
![Page 13: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/13.jpg)
3/16/2007 9:17:48 AM 13
Systems Integration
• Validate and maintain operability of system hardware and operating environments
• Verify Computer Program Interfaces
Hardware, OE, Computer Program Interface issues resolved during the computer program development phase
![Page 14: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/14.jpg)
3/16/2007 9:17:48 AM 14
Systems Integration continued• Integration issues tracking and reporting
– Integration Issue resolution prior to computer program delivery
0195271Totals
0163753Build 5
031518Build 4
BUILDS
ACTIVE ISSUES BEING
ADDRESSED
DEFECTS WHICH
BECAME TORs
ISSUES CLOSED
DURING BUILD
BUILD ISSUES FOUND
SYSTEMS INTEGRATION & TEST
![Page 15: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/15.jpg)
3/16/2007 9:17:48 AM 15
Systems Test
• Development and Regression Functional Testing– Documentation
• Plans• Procedures
– Execution• Utilizing two or more computer programs
– Multiple hardware and operating environments
5% to 10% Improvement of Engineering Test & Evaluation
![Page 16: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/16.jpg)
3/16/2007 9:17:48 AM 16
Systems Test continued• Test defect reporting and tracking
– Test defect resolution prior to computer program delivery
S y s t e m I n t e g r a t i o n & T e s t
0
2
4
6
8
10
12
14
C a p a b i l i t y
L O W
M E D I U M
H I G H
![Page 17: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/17.jpg)
3/16/2007 9:17:48 AM 17
Systems Performance Measurement
• CPU and Memory Utilization• Thread and Response Timing• Measurement Tools
– Legacy• ATES Data Recording
– COTS• UNIX TOP and Kernel Trace• Concurrent Nightview• LynxOS Spyker
• Standard scenario– Function and information loading– Repeatable
![Page 18: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/18.jpg)
3/16/2007 9:17:48 AM 18
Systems Performance Measurement continued• Performance Measurement Abnormalities
– CPU and Memory Utilization Increase– CPU Utilization Spikes– Memory Leaks– Slow Thread or Response
![Page 19: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/19.jpg)
3/16/2007 9:17:48 AM 19
Systems Performance Measurement – CPU UtilizationU s e r / S y s e m C P U U t i l i z a t i o n
0 . 0 0
0 . 5 0
1 . 0 0
1 . 5 0
2 . 0 0
2 . 5 0
3 . 0 0
3 . 5 0
T i m e ( A u g - 2 9 - R u n )
S y s
U s e r
U s e r + S y s
![Page 20: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/20.jpg)
3/16/2007 9:17:48 AM 20
Systems Performance Measurement – Memory UtilizationU s e r P r o c e s s M e m o r y
0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 0
1 0 0
9 :4 6 : 3 2
9 :4 8 : 3 7
9 :5 0 : 4 2
9 :5 2 : 4 8
9 :5 4 : 5 3
9 :5 6 : 5 8
9 :5 9 : 0 31 0 : 0 1 :0 91 0 : 0 3 :1 4
1 0 : 0 5 :1 91 0 : 0 7 :2 4
1 0 : 0 9 :3 01 0 : 1 1 :3 5
T im e ( A u g - 2 9 - R u n )
Mem
ory
(Mem
) j a v a
t i m e s e r v i c e s
s y s te m c o n t r o l
![Page 21: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/21.jpg)
3/16/2007 9:17:48 AM 21
Systems Performance Measurement – Memory LeakF r e e M e m o r y
1 7 0
1 8 0
1 9 0
2 0 0
2 1 0
2 2 0
2 3 0
2 4 0
2 5 0
2 6 0
2 7 0
9 :4 6 : 3 2
9 :4 8 : 3 7
9 :5 0 : 4 2
9 :5 2 : 4 8
9 :5 4 : 5 3
9 :5 6 : 5 8
9 :5 9 : 0 3
1 0 : 0 1 :0 9
1 0 : 0 3 :1 4
1 0 : 0 5 :1 9
1 0 : 0 7 :2 4
1 0 : 0 9 :3 0
1 0 : 1 1 :3 5
T im e ( A u g - 2 9 - R u n )
Mem
ory
(Meg
)
P 1
![Page 22: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/22.jpg)
3/16/2007 9:17:48 AM 22
Systems Performance Measurement continued
• Performance Measurement issues resolved before computer program delivery
10% to 20% CPU Utilization Improvement
![Page 23: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/23.jpg)
3/16/2007 9:17:48 AM 23
Summary
• Incremental Systems Integration, Test and Performance Measurement
• Integration and Test issue and defect resolution during the computer program development phase
• Improved Computer Systems Stability and Performance
Cost Effective by identifying and resolving systems issues and defects during the computer program development phase
![Page 24: Early Application of Computer Program Systems Integration ... · Catalyst for Early Computer Program Systems Integration, Test and Performance Measurement • Introduction of COTS](https://reader036.fdocuments.net/reader036/viewer/2022070709/5ebabf9791a9cd23f65f81a6/html5/thumbnails/24.jpg)
3/16/2007 9:17:48 AM 24
Tom SobieralskiProject ManagerComputer Sciences Corporation304 West Route 38Moorestown, New Jersey 08057Voice: (856) 252-5052Email: [email protected]