Software Engineering Chapter 3 Critical systems Ku-Yaw Chang [email protected] Assistant...

Software EngineeringSoftware Engineering

Chapter 3 Chapter 3 Critical systemsCritical systems

Ku-Yaw ChangKu-Yaw [email protected]@mail.dyu.edu.tw

Assistant ProfessorAssistant ProfessorDepartment of Computer Science and Information EngineeringDepartment of Computer Science and Information Engineering

Da-Yeh UniversityDa-Yeh University

22Ku-Yaw ChangKu-Yaw Chang Software EngineeringSoftware Engineering

ObjectivesObjectives

Understand that in a critical system, system failure can Understand that in a critical system, system failure can have severe human or economic consequenceshave severe human or economic consequences

Understand four dimensions of system dependability: Understand four dimensions of system dependability: availability, reliability, safety and securityavailability, reliability, safety and security

Understand that to achieve dependability you need to Understand that to achieve dependability you need to avoid mistakes during the development of a system, to avoid mistakes during the development of a system, to detect and remove errors when the system is in use and detect and remove errors when the system is in use and to limit the damage caused by operational failuresto limit the damage caused by operational failures


PreamblePreamble

System failuresSystem failures Cause inconvenience but no serious, long-term damageCause inconvenience but no serious, long-term damage Result in significant economic losses, physical damage or Result in significant economic losses, physical damage or

threats to human lifethreats to human lifeCritical systemsCritical systems

Three main typesThree main types Safety-critical systemsSafety-critical systems

Injury, loss of life, serious environmental damageInjury, loss of life, serious environmental damagee.g. chemical manufacturing plante.g. chemical manufacturing plant

Mission-critical systemsMission-critical systemsFailure of goal-directed activityFailure of goal-directed activitye.g. navigational system for a spacecrafte.g. navigational system for a spacecraft

Business-critical systemsBusiness-critical systemsVery high costsVery high costse.g. customer accounting system in a banke.g. customer accounting system in a bank


PreamblePreamble

DependabilityDependability Cover related system attributesCover related system attributes

AvailabilityAvailability

ReliabilityReliability

The most important emergent property of a critical The most important emergent property of a critical systemsystem

Systems that are unreliable, unsafe or insecure are often Systems that are unreliable, unsafe or insecure are often rejected by their usersrejected by their users

System failures may be enormousSystem failures may be enormous

Untrustworthy systems may cause information lossUntrustworthy systems may cause information loss

SafetySafetySecuritySecurity


PreamblePreamble

Trusted methods and techniques must be usedTrusted methods and techniques must be used Well-tried techniques rather than new techniquesWell-tried techniques rather than new techniques Developers are naturally conservativeDevelopers are naturally conservative

Expensive software engineering techniques may Expensive software engineering techniques may sometimes be usedsometimes be used

Costs of verification and validation are usually very high – more Costs of verification and validation are usually very high – more than 50% of the total costthan 50% of the total cost

Most are socio-technical systemsMost are socio-technical systems System operators canSystem operators can

Help recover from problemsHelp recover from problemsCause problems if they make mistakesCause problems if they make mistakes


ContentsContents

3.1 A simple safety-critical system3.1 A simple safety-critical system

3.2 System dependability3.2 System dependability

3.3 Availability and reliability3.3 Availability and reliability

3.4 Safety3.4 Safety

3.5 Security3.5 Security

3.6 Exercises3.6 Exercises


DiabetesDiabetes

A common condition where the human pancreas is A common condition where the human pancreas is unable to produce sufficient quantities of a hormone unable to produce sufficient quantities of a hormone called called insulininsulinInsulin metabolizes glucose in the bloodInsulin metabolizes glucose in the blood

Low levels of blood glucose (too much insulin)Low levels of blood glucose (too much insulin)Temporary brain malfunctioning, unconsciousness and deathTemporary brain malfunctioning, unconsciousness and death

High levels of blood glucose (too little insulin)High levels of blood glucose (too little insulin)Eye damage, kidney damage and heart problemsEye damage, kidney damage and heart problems

Miniaturized sensorsMiniaturized sensors Automated insulin delivery systemsAutomated insulin delivery systems

Monitor blood sugar levelMonitor blood sugar levelDeliver appropriate dose of insulin when requiredDeliver appropriate dose of insulin when required


A software-controlled insulin pumpA software-controlled insulin pump

Needleassembly

Sensor

Display1 Display2

Alarm

Pump Clock

Controller

Power supply

Insulin reservoir


Data-flow model ofData-flow model ofthe insulin pumpthe insulin pump

Insulinrequirementcomputation

Blood sugaranalysis

Blood sugarsensor

Insulindeliverycontroller

Insulinpump

Blood

Bloodparameters

Blood sugarlevel

Insulin

Pump controlcommands Insulin

requirement


ContentsContents








System dependabilitySystem dependability

A property equating to its A property equating to its trustworthinesstrustworthiness The degree of user confidence that the systemThe degree of user confidence that the system

Will operate as they expectWill operate as they expectWill not ‘fail’ in normal useWill not ‘fail’ in normal use

Four principal dimensions to dependabilityFour principal dimensions to dependability AvailabilityAvailability

To deliver services when requestedTo deliver services when requested ReliabilityReliability

To delver services as specifiedTo delver services as specified SafetySafety

To operate without catastrophic failureTo operate without catastrophic failure SecuritySecurity

To protect itself against accidental or deliberate intrusionTo protect itself against accidental or deliberate intrusion


Dimensions of dependabilityDimensions of dependability

Dependability

Availability Reliability Security

The ability of the systemto deliver services when

requested

The ability of the systemto deliver services as

specified

The ability of the systemto operate withoutcatastrophic failure

The ability of the systemto protect itelf againstaccidental or deliberate

intrusion

Safety


Other dependability propertiesOther dependability properties

ReparabilityReparability Reflects the extent to which the system can be repaired in the Reflects the extent to which the system can be repaired in the

event of a failureevent of a failure

MaintainabilityMaintainability Reflects the extent to which the system can be adapted to new Reflects the extent to which the system can be adapted to new

requirementsrequirements

SurvivabilitySurvivability Reflects the extent to which the system can deliver services Reflects the extent to which the system can deliver services

while under hostile attackwhile under hostile attack

Error toleranceError tolerance Reflects the extent to which user input errors can be avoided Reflects the extent to which user input errors can be avoided

and toleratedand tolerated


System dependabilitySystem dependability

A trade-off between system A trade-off between system performanceperformance and and system system dependabilitydependability High dependability can only be achieved at the High dependability can only be achieved at the

expense of system performanceexpense of system performance Dependable software includes extra, often redundant, Dependable software includes extra, often redundant,

codecode

Increasing the dependability can significantly Increasing the dependability can significantly increase development increase development costscosts Additional design, implementation and validation Additional design, implementation and validation

costscosts


Cost/dependability curveCost/dependability curve

Low Medium High Veryhigh

Ultra-high

Dependability

Co

st


ContentsContents








Availability and reliabilityAvailability and reliability

ReliabilityReliability The probability that the system’s services will be correctly The probability that the system’s services will be correctly

delivered as specifieddelivered as specified

AvailabilityAvailability The probability that the system will be up and running to deliver The probability that the system will be up and running to deliver

these services to users when they request themthese services to users when they request them

Both properties are closely relatedBoth properties are closely related Availability is more critical than reliabilityAvailability is more critical than reliability

A telephone exchange switchA telephone exchange switch Availability depends onAvailability depends on

The system itselfThe system itselfThe time needed to repair the faultsThe time needed to repair the faults


Practical problems Practical problems

Different environmentsDifferent environments an office environment vs. a university environmentan office environment vs. a university environment

Human perceptions and patternsHuman perceptions and patterns Unreliable windscreen wipers in a car may be Unreliable windscreen wipers in a car may be

irrelevant in a dry climateirrelevant in a dry climate

Severity of failure or consequences of Severity of failure or consequences of unavailabilityunavailability A failure of initialization in the engine management A failure of initialization in the engine management

softwaresoftware An engine that cuts out while they are drivingAn engine that cuts out while they are driving


Reliability terminologyReliability terminology

Term Description

System failure An event that occurs at some point in time whenthe system does not deliver a service as expectedby its users

System error An erroneous system state that can lead to systembehaviour that is unexpected by system users.

System fault A characteristic of a software system that canlead to a system error. For example, failure toinitialise a variable could lead to that variablehaving the wrong value when it is used.

Human error ormistake

Human behaviour that results in the introductionof faults into a system.


Faults and failuresFaults and failures

FailuresFailures Usually a result of system errorsUsually a result of system errors

Derived from system faultsDerived from system faults

ErrorsErrors Do not necessarily lead to system failuresDo not necessarily lead to system failures Can be corrected by built-in error detection and recovery Can be corrected by built-in error detection and recovery

FaultsFaults Do not necessarily result in system errorsDo not necessarily result in system errors May be transient and ‘corrected’ before an error arisesMay be transient and ‘corrected’ before an error arises


Reliability achievementReliability achievement

Fault avoidanceFault avoidance Development techniques are used that either minimize the Development techniques are used that either minimize the

possibility of mistakes or trap mistakes before they result in the possibility of mistakes or trap mistakes before they result in the introduction of system faultsintroduction of system faults

Fault detection and removalFault detection and removal Verification and validation techniques that increase the Verification and validation techniques that increase the

probability of detecting and correcting errors before the system probability of detecting and correcting errors before the system goes into service are usedgoes into service are used

Fault toleranceFault tolerance Run-time techniques are used to ensure that system faults do Run-time techniques are used to ensure that system faults do

not result in system errors and/or that system errors do not lead not result in system errors and/or that system errors do not lead to system failuresto system failures


Reliability modelingReliability modeling

Model a software system as an input-output mappingModel a software system as an input-output mapping Some inputs will result in erroneous outputsSome inputs will result in erroneous outputs

The reliability of the systemThe reliability of the system The probability that a particular input will lie in the set of inputs The probability that a particular input will lie in the set of inputs

that cause erroneous outputsthat cause erroneous outputs

Different people will use the system in different waysDifferent people will use the system in different ways The probability is not a static system attribute but depends on The probability is not a static system attribute but depends on

the system’s environmentthe system’s environment


Input/output mappingInput/output mapping

IeInput set

OeOutput set

Program

Inputs causingerroneous outputs

Erroneousoutputs


Reliability perceptionReliability perception

Possibleinputs

User1

User3

User2

Erroneousinputs


Reliability improvementReliability improvement

Removing X% of the faults in a system will not Removing X% of the faults in a system will not necessarily improve the reliability by X%.necessarily improve the reliability by X%.

A study at IBM showed that removing 60% of product defects A study at IBM showed that removing 60% of product defects resulted in a 3% improvement in reliabilityresulted in a 3% improvement in reliability

Program defects may be in rarely executed sections of Program defects may be in rarely executed sections of the code so may never be encountered by users.the code so may never be encountered by users.

Removing these does not affect the perceived reliabilityRemoving these does not affect the perceived reliability

A program with known faults may therefore still be seen A program with known faults may therefore still be seen as reliable by its users.as reliable by its users.

Deliberately avoid using system features that can cause Deliberately avoid using system features that can cause problemsproblems


ContentsContents








SafetySafety

A property of a system A property of a system The ability to operate, normally or abnormally, without The ability to operate, normally or abnormally, without

danger of causing human injury or death and without danger of causing human injury or death and without damage to the system’s environmentdamage to the system’s environment

Increasingly important to consider software Increasingly important to consider software safetysafety More and more devices incorporate software-based More and more devices incorporate software-based

control systems control systems

Safety requirements are exclusive requirements Safety requirements are exclusive requirements They exclude undesirable situations rather than They exclude undesirable situations rather than

specify required system servicesspecify required system services


SafetySafety

Safety-critical systemsSafety-critical systems Systems where it is essential that system operation is Systems where it is essential that system operation is

always safealways safe

Two classesTwo classes Primary safety-critical softwarePrimary safety-critical software

Software embedded as a controller in a systemSoftware embedded as a controller in a systemMalfunctioning cause a hardware malfunctionMalfunctioning cause a hardware malfunction

Result in human injury or environmental damageResult in human injury or environmental damage Secondary safety-critical softwareSecondary safety-critical software

Indirectly result in injuryIndirectly result in injury e.g., a medical database holding details of drugs administered e.g., a medical database holding details of drugs administered

to patientsto patients


Safety and reliabilitySafety and reliability

Safety and reliability are related but distinctSafety and reliability are related but distinct In general, reliability and availability are necessary but In general, reliability and availability are necessary but

not sufficient conditions for system safety not sufficient conditions for system safety

ReliabilityReliability Bs concerned with conformance to a given Bs concerned with conformance to a given

specification and delivery of servicespecification and delivery of service

SafetySafety Be concerned with ensuring system cannot cause Be concerned with ensuring system cannot cause

damagedamageIrrespective of whether or not it conforms to its specificationIrrespective of whether or not it conforms to its specification


Unsafe reliable systemsUnsafe reliable systems

Specification errorsSpecification errors If the system specification is incorrect then the system If the system specification is incorrect then the system

can behave as specified but still cause an accidentcan behave as specified but still cause an accident

Hardware failures generating spurious inputsHardware failures generating spurious inputs Hard to anticipate in the specificationHard to anticipate in the specification

Context-sensitive commands i.e. issuing the Context-sensitive commands i.e. issuing the right command at the wrong timeright command at the wrong time Often the result of operator errorsOften the result of operator errors


Safety terminologySafety terminologyTerm Definition

Accident (ormishap)

An unplanned event or sequence of events which results in human death or injury,damage to property or to the environment. A computer-controlled machine injuring itsoperator is an example of an accident.

Hazard A condition with the potential for causing or contributing to an accident. A failure ofthe sensor that detects an obstacle in front of a machine is an example of a hazard.

Damage A measure of the loss resulting from a mishap. Damage can range from many peoplekilled as a result of an accident to minor injury or property damage.

Hazardseverity

An assessment of the worst possible damage that could result from a particularhazard. Hazard severity can range from catastrophic where many people are killed tominor where only minor damage results.

Hazardprobability

The probability of the events occurring which create a hazard. Probability values tendto be arbitrary but range from probable (say 1/100 chance of a hazard occurring) toimplausible (no conceivable situations are likely where the hazard could occur).

Risk This is a measure of the probability that the system will cause an accident. The risk isassessed by considering the hazard probability, the hazard severity and the probabilitythat a hazard will result in an accident.


Safety achievementSafety achievement

Hazard avoidanceHazard avoidance The system is designed so that some classes of The system is designed so that some classes of

hazard simply cannot arise. hazard simply cannot arise.

Hazard detection and removalHazard detection and removal The system is designed so that hazards are detected The system is designed so that hazards are detected

and removed before they result in an accidentand removed before they result in an accident

Damage limitationDamage limitation The system includes protection features that minimize The system includes protection features that minimize

the damage that may result from an accidentthe damage that may result from an accident


Normal accidentsNormal accidents

Accidents in complex systemsAccidents in complex systems Rarely have a single causeRarely have a single cause A fundamental principle of safe systems designA fundamental principle of safe systems design

Be resilient to a single point of failureBe resilient to a single point of failure

Impossible to anticipate all possible Impossible to anticipate all possible combinations of system malfunctioncombinations of system malfunction Accidents are an inevitable part of using complex Accidents are an inevitable part of using complex

systemssystems


ContentsContents








SecuritySecurity

A system attributeA system attribute The ability to protect itself from external attacks that The ability to protect itself from external attacks that

may be accident or deliberatemay be accident or deliberate

Becoming increasingly important as more and Becoming increasingly important as more and more systems are connected to the Internetmore systems are connected to the InternetAn essential pre-requisite for availability, An essential pre-requisite for availability, reliability and safetyreliability and safetyErrors can lead to security loopholesErrors can lead to security loopholes Not respond to unexpected inputsNot respond to unexpected inputs Array bounds are not checkedArray bounds are not checked

Programs in CPrograms in C


Three types of damageThree types of damage

Denial of serviceDenial of service Normal services become unavailableNormal services become unavailable

Corruption of programs or dataCorruption of programs or data Be altered in an unauthorized wayBe altered in an unauthorized way

Disclosure of confidential informationDisclosure of confidential information Confidential information may be exposed to Confidential information may be exposed to

unauthorized peopleunauthorized people


Security terminologySecurity terminology

Term Definition

Exposure Possible loss or harm in a computing system. This can be loss ordamage to data or can be a loss of time and effort if recovery isnecessary after a security breach.

Vulnerability A weakness in a computer-based system that may be exploited tocause loss or harm.

Attack An exploitation of a system vulnerability. Generally, this is fromoutside the system and is a deliberate attempt to cause some damage.

Threats Circumstances that have potential to cause loss or harm. You canthink of these as a system vulnerability that is subjected to an attack.

Control A protective measure that reduces a system vulnerability. Encryptionwould be an example of a control that reduced a vulnerability of aweak access control system.


Security assuranceSecurity assurance

Vulnerability avoidanceVulnerability avoidance The system is designed so that vulnerabilities do not occurThe system is designed so that vulnerabilities do not occur e.g. No external network connectione.g. No external network connection

Attack detection and neutralizationAttack detection and neutralization Attacks on vulnerabilities are detected and removed before they Attacks on vulnerabilities are detected and removed before they

result in an exposureresult in an exposure e.g. Find and remove viruses before they infect a systeme.g. Find and remove viruses before they infect a system

Exposure limitationExposure limitation Adverse consequences of a successful attack are minimizedAdverse consequences of a successful attack are minimized e.g. A backup policy allows damaged information to be restorede.g. A backup policy allows damaged information to be restored


SecuritySecurity

Vast majority of vulnerabilitiesVast majority of vulnerabilities Human failings (rather than technical problems)Human failings (rather than technical problems)

Easy-to-guess passwordsEasy-to-guess passwords

Write down passwords in places where they can be foundWrite down passwords in places where they can be found

System administrators make errorsSystem administrators make errors Setting up access control or configuration filesSetting up access control or configuration files Forget to install or use protection softwareForget to install or use protection software

Take a socio-technical perspective Take a socio-technical perspective Not just about their technical characteristicsNot just about their technical characteristics


ContentsContents








ExercisesExercises

3.13.1

3.83.8

3.103.10

The EndThe End

Software Engineering Chapter 3 Critical systems Ku-Yaw Chang [email protected] Assistant...

Documents

Transcript of Software Engineering Chapter 3 Critical systems Ku-Yaw Chang [email protected] Assistant...