Correctness Correctness. Quality Perceptions The perception of quality associated with your code is...
-
Upload
jasper-dawson -
Category
Documents
-
view
240 -
download
2
Transcript of Correctness Correctness. Quality Perceptions The perception of quality associated with your code is...
CorrectnessCorrectness
Quality PerceptionsQuality Perceptions
The perception of quality associated with your code is typically bound to: Correctness Efficiency (speed of execution usually) Cost (if it costs more, it must be better right?) Robustness Flexibility Functionality Maintainability Security Usability Whatever the user likes.
The perception of quality associated with your code is typically bound to: Correctness Efficiency (speed of execution usually) Cost (if it costs more, it must be better right?) Robustness Flexibility Functionality Maintainability Security Usability Whatever the user likes.
CorrectnessCorrectness
We will focus on correctness in this class. Often has an indirect impact on:
Cost Robustness Flexibility Functionality Maintainability Security
What is correctness?
We will focus on correctness in this class. Often has an indirect impact on:
Cost Robustness Flexibility Functionality Maintainability Security
What is correctness?
Definition of Software Correctness
Definition of Software Correctness
Correct software must accomplish the following: Compute accurate results Operate safely Implement the requirements and meet the
specifications Achieve the above for all possible inputs Recognize input from outside its domain
What can we do to help us achieve this?
Correct software must accomplish the following: Compute accurate results Operate safely Implement the requirements and meet the
specifications Achieve the above for all possible inputs Recognize input from outside its domain
What can we do to help us achieve this?
Standard TechniquesStandard Techniques
Use design patterns – don’t reinvent the wheel Use standard libraries Code re-use
If it we know something is correct from past experience, then why start from scratch to re-invent it? Need to be mindful of legal issues – are you
allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?
Use design patterns – don’t reinvent the wheel Use standard libraries Code re-use
If it we know something is correct from past experience, then why start from scratch to re-invent it? Need to be mindful of legal issues – are you
allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?
Choice of Programming Language
Choice of Programming Language
Choose a language with strict type checking. A type checker is like a theorem prover which
confirms your assertions about the functions/methods and variables you use in the program.
Integer f (float x) {return …
}
When the type checker checks the code, it confirms: f is called with one argument and it is a value of
type float If f produces a result, then that result belongs to
the class Integer.
Choose a language with strict type checking. A type checker is like a theorem prover which
confirms your assertions about the functions/methods and variables you use in the program.
Integer f (float x) {return …
}
When the type checker checks the code, it confirms: f is called with one argument and it is a value of
type float If f produces a result, then that result belongs to
the class Integer.
Choice of Programming Language
Choice of Programming Language
While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.
You no longer have to check that the function f maps a float onto an Integer.
Does not alleviate the need for testing. Only true for languages with sound type systems
(Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string) and interpret it as a float.
While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.
You no longer have to check that the function f maps a float onto an Integer.
Does not alleviate the need for testing. Only true for languages with sound type systems
(Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string) and interpret it as a float.
AssertionsAssertions
Type checking is limited in the support it provides. We want something more.
An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.
We are probably used to writing comments that states assertions:int p; // p is a prime numberint x;...x = x * x; // x is now positive
Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.
Type checking is limited in the support it provides. We want something more.
An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.
We are probably used to writing comments that states assertions:int p; // p is a prime numberint x;...x = x * x; // x is now positive
Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.
AssertionsAssertions
Assertions are often the result of the execution of statements.
Used to argue that code fragments execute as expected.
Often referred to as pre-conditions and post-conditions (of statements).
Use logic to express or assert facts that we believe or expect to be true:Sort (A);// for all i, j such that 1 i j p// A[i] A[j]
PRECONDITION{ Code fragment }POSTCONDITION
Assertions are often the result of the execution of statements.
Used to argue that code fragments execute as expected.
Often referred to as pre-conditions and post-conditions (of statements).
Use logic to express or assert facts that we believe or expect to be true:Sort (A);// for all i, j such that 1 i j p// A[i] A[j]
PRECONDITION{ Code fragment }POSTCONDITION
ExampleExample
Consider:interface Queue {
// is the queue empty?boolean empty();
// add Item x to the end of the QueueQueue push (Item x);
// return and remove the Item at the head of the QueueItem pop ();
// how many elements in the Queue?int size();
}
Consider:interface Queue {
// is the queue empty?boolean empty();
// add Item x to the end of the QueueQueue push (Item x);
// return and remove the Item at the head of the QueueItem pop ();
// how many elements in the Queue?int size();
}
Adding AssertionsAdding Assertions
Questions: Can the method be called in all situations. If not,
when can it be called? Is there anything special about the result value or
state? It is always possible to call empty. It always returns
a true|false value. It is always possible to call push. The argument must
be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.
Questions: Can the method be called in all situations. If not,
when can it be called? Is there anything special about the result value or
state? It is always possible to call empty. It always returns
a true|false value. It is always possible to call push. The argument must
be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.
Adding AssertionsAdding Assertions
It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.
It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.
How do we describe this?
It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.
It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.
How do we describe this?
ExampleExample
size´() is the value of the size function BEFORE the current method was called – the old value if you will.
interface Queue {// is the queue empty?// PRE: noneboolean empty();//POST: none
// add Item x to the end of the Queue//PRE: size() < MAX_LENGTHQueue push (Item x);// POST: !(empty()) && size() = size´() + 1
size´() is the value of the size function BEFORE the current method was called – the old value if you will.
interface Queue {// is the queue empty?// PRE: noneboolean empty();//POST: none
// add Item x to the end of the Queue//PRE: size() < MAX_LENGTHQueue push (Item x);// POST: !(empty()) && size() = size´() + 1
ExampleExample
// return and remove the Item at the head of the Queue// PRE: !(empty())Item pop ();// POST: size() = size´() - 1
// how many elements in the Queue?// PRE: noneint size();// POST: 0 RESULT MAX_LENGTH
}
// return and remove the Item at the head of the Queue// PRE: !(empty())Item pop ();// POST: size() = size´() - 1
// how many elements in the Queue?// PRE: noneint size();// POST: 0 RESULT MAX_LENGTH
}
AssertionsAssertions
Q: How do we automatically enforce these assertions?
A: Write a class to help you. See sample Correctness.java in ~michael/CS351 on
esus.class Correctness {
public static void pre(boolean expr, Error ex) {…}public static void post (boolean expr) {…}public static void beginRequire () {…}public static void endRequire () {…}public static void beginEnsure () {…}public static void endEnsure () {…}
}
Q: How do we automatically enforce these assertions?
A: Write a class to help you. See sample Correctness.java in ~michael/CS351 on
esus.class Correctness {
public static void pre(boolean expr, Error ex) {…}public static void post (boolean expr) {…}public static void beginRequire () {…}public static void endRequire () {…}public static void beginEnsure () {…}public static void endEnsure () {…}
}
AssertionsAssertions
Pre-conditions must be within a beginRequire() … endRequire() bracket.
Post-conditions must be within a beginEnsure() … endEnsure() bracket.
This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).
May have multiple pre- and post-conditions within appropriate bracket.
May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.
Pre-conditions must be within a beginRequire() … endRequire() bracket.
Post-conditions must be within a beginEnsure() … endEnsure() bracket.
This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).
May have multiple pre- and post-conditions within appropriate bracket.
May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.
Assertions - ExampleAssertions - Example
public final Edge getEdge(int id) {Edge Result; // variable declarations// pre-condition checkingCorrectness.beginRequire();Correctness.pre( id >= 0 && id < numEdges(),
new IdOutOfRange() );Correctness.endRequire();Result = impl.getEdge(id); // evaluate result to be returned// post-condition checkingCorrectness.beginEnsure();Correctness.post( hasEdge(Result() == true );Correctness.endEnsure();return Result; // finally return the result.
}
public final Edge getEdge(int id) {Edge Result; // variable declarations// pre-condition checkingCorrectness.beginRequire();Correctness.pre( id >= 0 && id < numEdges(),
new IdOutOfRange() );Correctness.endRequire();Result = impl.getEdge(id); // evaluate result to be returned// post-condition checkingCorrectness.beginEnsure();Correctness.post( hasEdge(Result() == true );Correctness.endEnsure();return Result; // finally return the result.
}
ExpectationExpectation
Use assertions when developing your code. When testing your code, if a pre- or post-condition
fails you will get an error message. It helps you identify the failure and correct it far more quickly.
Use them in your project.
Use assertions when developing your code. When testing your code, if a pre- or post-condition
fails you will get an error message. It helps you identify the failure and correct it far more quickly.
Use them in your project.
CorrectnessCorrectness
Two basic techniques for attempting to produce programs without bugs: Testing: run the program on various sets of data
and see if it behaves correctly in these cases. Proving correctness: show mathematically that
the program always does what it is supposed to do.
Both techniques have their particular problems: Testing is only as good as the test cases
selected. A proof of correctness may contain errors.
Two basic techniques for attempting to produce programs without bugs: Testing: run the program on various sets of data
and see if it behaves correctly in these cases. Proving correctness: show mathematically that
the program always does what it is supposed to do.
Both techniques have their particular problems: Testing is only as good as the test cases
selected. A proof of correctness may contain errors.
CorrectnessCorrectness
A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.
Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place Excellent program documentation!
A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.
Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place Excellent program documentation!
Program CorrectnessProgram Correctness
Before looking at program proving in detail, there is something else that must be pointed out: A program can only be judged correct in relation
to a set of specifications for what it is supposed to do.
All programs do something correctly; the question is: does it do what it is supposed to do?
A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.
Before looking at program proving in detail, there is something else that must be pointed out: A program can only be judged correct in relation
to a set of specifications for what it is supposed to do.
All programs do something correctly; the question is: does it do what it is supposed to do?
A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.
Program CorrectnessProgram Correctness
Aspects of a program's correctness include:(1) Partial correctness: whenever the program
terminates, it performs correctly.(2) Termination: the program always
terminates.(1) + (2) ⇒ Program is totally correct.
Aspects of a program's correctness include:(1) Partial correctness: whenever the program
terminates, it performs correctly.(2) Termination: the program always
terminates.(1) + (2) ⇒ Program is totally correct.
Program Correctness ProofsProgram Correctness Proofs
Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page.
function exponentiate (x: in integer) return integer is–– Evaluates 2**x, for x0 –– {1}
i, sum: integer;begin
sum := 1;–– sum = 2**0 ` –– {2}for i in 1 .. x loop
sum := sum + sum–– sum = 2**i, i>0 –– {3}
end loop;–– sum = 2**x, x 0
–– {4}return sum;
end exponentiate;
Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page.
function exponentiate (x: in integer) return integer is–– Evaluates 2**x, for x0 –– {1}
i, sum: integer;begin
sum := 1;–– sum = 2**0 ` –– {2}for i in 1 .. x loop
sum := sum + sum–– sum = 2**i, i>0 –– {3}
end loop;–– sum = 2**x, x 0
–– {4}return sum;
end exponentiate;
Program Correctness ProofsProgram Correctness Proofs
{1} lists the goals of the function{2} asserts the initial value of "sum"We can prove {3} by induction.
The first time {3} is reached we havei = 1sum = 1 + 1 = 2 = 20 = 2i
Assume that the nth time {3} is reachedsum = 2n
then the (n+1)th time setssum' = sum + sum
= 2n + 2n = 2n+1
therefore {3} always holds.
{1} lists the goals of the function{2} asserts the initial value of "sum"We can prove {3} by induction.
The first time {3} is reached we havei = 1sum = 1 + 1 = 2 = 20 = 2i
Assume that the nth time {3} is reachedsum = 2n
then the (n+1)th time setssum' = sum + sum
= 2n + 2n = 2n+1
therefore {3} always holds.
Program Correctness ProofsProgram Correctness Proofs
If {4} is ever reached, there are two possibilities:a)The loop was never executed, in which case x=0,
and sum remains unchanged from {2}, i.e., sum = 1 = 20.
b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2x.
See handout for further examples involving induction.
If {4} is ever reached, there are two possibilities:a)The loop was never executed, in which case x=0,
and sum remains unchanged from {2}, i.e., sum = 1 = 20.
b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2x.
See handout for further examples involving induction.
Program Correctness ProofsProgram Correctness Proofs
For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.
The remedy is modularization. As we can not write a large program without the aid of modularization and top-down design, we can not understand an algorithm and prove correctness unless it is modularized.
As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.
For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.
The remedy is modularization. As we can not write a large program without the aid of modularization and top-down design, we can not understand an algorithm and prove correctness unless it is modularized.
As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.
Program Correctness ProofsProgram Correctness Proofs
A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.
The specification of a module consists of two parts: specification of the range of inputs of the module. desired effect of the module.
In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.
The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.
Consider...
A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.
The specification of a module consists of two parts: specification of the range of inputs of the module. desired effect of the module.
In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.
The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.
Consider...
Program Correctness ProofsProgram Correctness Proofs
procedure binary is –– binary search algorithmN: constant ...; –– some number 1}x: array (1..N) of float;key: float; L, R, K: integer; found: boolean;
begin key := ...;–– (x[I]x[J] iff 1IJN) and (X[1]keyx[N}) {0}L := 1; R := N; found := false;-- 1LRN and x(L)keyx(R) {1}while (LR) and (not found) loopK := (L+R) div 2;
–– 1LKRN and (x(L)keyx(R)) {2}found := (x(K) = key);if not found then –– x(K)key {3}
if key<x(K)then R := K–1; –– keyx(R) {4}else L := K+1; –– x(L)keyend if; –– {5}
–– x(L)keyx(R) {6}end if;
end loop;–– found= and (x(K)=key) {7}
end binary;[ is “key is present in the array x”]
procedure binary is –– binary search algorithmN: constant ...; –– some number 1}x: array (1..N) of float;key: float; L, R, K: integer; found: boolean;
begin key := ...;–– (x[I]x[J] iff 1IJN) and (X[1]keyx[N}) {0}L := 1; R := N; found := false;-- 1LRN and x(L)keyx(R) {1}while (LR) and (not found) loopK := (L+R) div 2;
–– 1LKRN and (x(L)keyx(R)) {2}found := (x(K) = key);if not found then –– x(K)key {3}
if key<x(K)then R := K–1; –– keyx(R) {4}else L := K+1; –– x(L)keyend if; –– {5}
–– x(L)keyx(R) {6}end if;
end loop;–– found= and (x(K)=key) {7}
end binary;[ is “key is present in the array x”]
Program Correctness ProofsProgram Correctness Proofs
{0} is a pre-condition describing what this module expects of its input.
{1} is a pre-condition describing the initial conditions before entering the loop.
{2} is an assertion true at that point for each iteration of the loop.
{3} is an assertion true whenever the if condition evaluates to true.
{4} holds if the then clause is executed. {5} holds if the else clause is executed. {6} holds after the if statement. It is true irrespective
of whether the then or else clause was executed. {7} is the post-condition of the module.
{0} is a pre-condition describing what this module expects of its input.
{1} is a pre-condition describing the initial conditions before entering the loop.
{2} is an assertion true at that point for each iteration of the loop.
{3} is an assertion true whenever the if condition evaluates to true.
{4} holds if the then clause is executed. {5} holds if the else clause is executed. {6} holds after the if statement. It is true irrespective
of whether the then or else clause was executed. {7} is the post-condition of the module.
TerminationTermination
A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.
In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.
In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.
A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.
In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.
In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.
TerminationTermination
A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.
In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.
A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.
In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.
TerminationTermination
In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.
Consider the following function:
In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.
Consider the following function:
function Ackermann(x, y: in integer) return integer is–– x and y must be nonnegative integersbegin –– Ackermann
if x = 0 then return (y+1);elsif y = 0 then return Ackermann((x-1), 1);else return Ackermann((x-1), Ackermann(x, (y-1)));end if;
end Ackermann;
ReadingReading
Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.
Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.
Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.
Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.
Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.
Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.
SummarySummary
Correctness is an important aspect of software quality.
Use appropriate tools (including programming language choice) to help.
Use assertions. Reason about the code to prove correctness when
necessary. It is easier and quicker to use assertions etc than it
is to test the code to demonstrate compliance with the requirements and specifications.
Still need to test the code – could have flaws in your logic!
Correctness is an important aspect of software quality.
Use appropriate tools (including programming language choice) to help.
Use assertions. Reason about the code to prove correctness when
necessary. It is easier and quicker to use assertions etc than it
is to test the code to demonstrate compliance with the requirements and specifications.
Still need to test the code – could have flaws in your logic!