Tutorial at FM2005 The Spec# Programming System: An Overview Tutor: Bart Jacobs PhD student at...

Tutorial at FM2005

The Spec# The Spec# Programming Programming

System: An OverviewSystem: An Overview

The Spec# The Spec# Programming Programming

System: An OverviewSystem: An OverviewTutor: Bart JacobsTutor: Bart Jacobs

PhD student at K.U.Leuven, BelgiumPhD student at K.U.Leuven, BelgiumSpec# contributorSpec# contributor

Tutorial at FM2005

What is Spec#?

• A programming language that extends C# with specification constructs

• A compiler that emits run-time checks for the specification constructs

• A static verifier that modularly proves that the run-time checks never fail

• An environment with base class library contracts and a Visual Studio extension

Tutorial at FM2005

Spec# Goals

• Make it easier to record detailed design decisions

• Provide tools to enforce these decisions

• Help prevent and detect bugs

• Reduce cost of software lifecycle

Tutorial at FM2005

Demo: The Bag Class

Demonstrates• Non-null types• Method contracts (including out-of-

band contracts for system libraries)• Loop invariants• The object invariant methodology• The static verifier

Tutorial at FM2005

Tutorial

• What is Spec#?• Non-null types• Method contracts• Object invariants• Ownership

―Break―• Inheritance• State Abstraction• Multithreading

Tutorial at FM2005

Non-null types

• Each reference type T includes the value null

• Spec#’s type T! contains only references to objects of type T (not null).

Tutorial at FM2005

Types versus Assertions

• Without non-null types:Person(string name) requires name != null;

• With non-null types:Person(string/*^!^*/ name)

[Q] What is the difference?

Tutorial at FM2005

Non-null types are flow-sensitive

• The non-null type of an expression is flow-sensitive.

void Foo(T o) { if (o != null) T! p = o; // OK!}

That is, it does not follow uniquely from the declared types of the variables and members mentioned in the expression.

Tutorial at FM2005

Non-null Fields and Object Creation

class C { public C() { ((D)

this).f.Foo(); }}

class D : C { T! f; D(T! x) : base() { f = x; }}

[Q] Can this be unsound?

[A] Yes!

class D : C { T! f; D(T! x) { f = x; base(); }}

Tutorial at FM2005

Non-nullness of Fields

Common coding pattern:if (o.f != null) o.f.Foo();

[Q] How can this go wrong?

Tutorial at FM2005

Non-nullness of Properties

Common coding pattern:if (o.P != null) o.P.Foo();

[Q] How can this go wrong?

Tutorial at FM2005

Fields and Properties

• For the non-null dataflow analysis, we assume that non-nullness of fields and properties is preserved in the absence of intervening heap-modifying operations

• Property reads are not considered heap-modifying operations

• But we check this at run time because of the possibility of– Data races– Impure property getters

Tutorial at FM2005

Tutorial


―Break―• Inheritance• State abstraction• Multithreading

Tutorial at FM2005

Preconditions

static int Divide(int a, int b){ if (b==0) throw new ArgumentException(); return a / b; }

[Q] What’s wrong with this?

static int Divide(int a, int b) /*^requires b != 0;^*/{ return a / b; }

• Throws a RequiresException if false. But how can we get full backwards compatibility?

static int Divide(int a, int b) requires b != 0 otherwise ArgumentException; { return a / b; }

Tutorial at FM2005

Checked Exceptions

int Eval(Expr e) throws EvalErrorException;

• Alternative:

bool TryEval(Expr e, out int value, out string errorMessage)

+ explicit propagation through recursive calls

Tutorial at FM2005

Definedness of contracts

public static int BinarySearch (int[] a, int value) ensures result < a.Length; ensures a[result] == value;

[Q] What if result < 0 ?This contract is ill-defined. In Spec#, an ill-defined contract is

considered an error.If at run time, evaluation of a contract clause throws an

exception, the exception is wrapped in an InvalidContractException and propagated.

For static checking, the program verifier generates an error if it cannot prove that a contract is well-defined.

Tutorial at FM2005

Tutorial

• What is Spec#?• Non-null types• Method contracts• Intermezzo: Inside the Spec# Program Verifier• Object invariants• Ownership


Tutorial at FM2005

From Spec#...

static int Abs(int x) ensures 0 <= x ==> result == x; ensures x < 0 ==> result == -x;{ if (x < 0) x = -x; return x; }

Tutorial at FM2005

…via BoogiePL …

procedure Abs(x$in: int) returns ($result: int); ensures 0 <= x$in ==> $result == x$in; ensures x$in < 0 ==> $result == -x$in;{ var x1, x2: int, b: bool;

entry: x1 := x$in; b := x < 0; goto t, f; t: assume b; x := -x; goto end; f: assume !b; goto end; end: $result := x; return; }

Tutorial at FM2005

…via BoogiePL-DSA …


entry: x1 := x$in; b := x1 < 0; goto t, f; t: assume b; x2 := -x1; goto end; f: assume !b; x2 := x1; goto end; end: $result := x2; return; }

Tutorial at FM2005

…via Passive BoogiePL …


entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; return; }

Tutorial at FM2005

… without contracts …


entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; return; }

Tutorial at FM2005

… without contracts …

procedure Abs(x$in: int) returns ($result: int);{ var x1, x2: int, b: bool;

entry: assume x1 == x$in; assume b == x1 < 0; goto t, f; t: assume b; assume x2 == -x1; goto end; f: assume !b; assume x2 == x1; goto end; end: assume $result == x2; assert 0 <= x$in ==> $result == x$in; assert x$in < 0 ==> $result == -x$in;

return; }

Tutorial at FM2005

…to Logic

[M. Barnett, K. R. M. Leino, in preparation]

entry && (entry <== (x1 == x$in ==> b == x1 < 0 ==> t && f))

&& (t <== (b ==> x2 == -x1 ==> end))

&& (f <== (!b ==> x2 == x1 ==> end))

&& (end <== ($result == x2 ==> (0 <= x$in ==> $result == x$in) && (x$in < 0 ==> $result == -x$in) && true))

Tutorial at FM2005

Tutorial



Tutorial at FM2005

Object Invariants

class Word { private string! line; int start, length;

public string ToString() { return line.Substring(start, length); } }

[Q] How can we prove ToString?

Tutorial at FM2005

Object Invariants

class Word { private string! line; int start, length; invariant 0 <= start && 0 <= length; invariant start + length <= line.length; public string ToString() { return line.Substring(start, length); } }

[Q] How can we prove ToString?

Tutorial at FM2005

Re-entrancy

class Subject { … invariant …; Observer d; void Foo(…) { // invariant assumed … if (…) d.Notify(…); …

// invariant reestablished }}

class Observer { … Subject c; void Notify(…) { … if (…) c.Foo(…); … }}

[Q] What could go wrong here?

==> Let’s drop the assumption that invariants hold on method entry.

[Q] But then, when can we assume they hold?

Tutorial at FM2005

Re-entrancy

class Subject { … invariant I; Observer d; void Foo(…) requires I; { // invariant assumed … if (…) d.Notify(…); …

// invariant reestablished }}


[Q] Can we simply require that the invariant holds?

No! This breaks abstraction!

Tutorial at FM2005

Spec# Object Invariant Methodology

• Each object gets a boolean field inv• Regular fields may be modified only when inv

is false• inv field may be modified only using special

BeginExpose() and EndExpose() calls• BeginExpose() sets inv to false• EndExpose() checks invariant; if it holds, sets

inv to true; otherwise, throws ObjectInvariantException

• Therefore, if inv is true, the invariant holds

Tutorial at FM2005

Exposing objects

class Subject { … invariant …; Observer d; void Foo(…) requires inv; { … // uses invariant

(sound!) BeginExpose(); … // field updates if (…) d.Notify(…); … // restores invariant EndExpose(); }}

Tutorial at FM2005

Exposing objects

class Subject { … invariant …; Observer d; void Foo(…) requires inv; { … // uses invariant

(sound!) expose (this) { … // field updates if (…) d.Notify(…); … // restores invariant } }}


Yes! This call is not allowed;

precondition does not hold!

[Q] Did we solve the problem?

Tutorial at FM2005

Object Invariants: Example

class Word { private string! line; private int start; public int length; invariant 0 <= start && 0 <= length; invariant start + length <= line.length; public void SelectPart(int start, int length) requires inv; requires 0 <= start && 0 <= length; requires length <= this.length; ensures inv; { expose (this) { this.start += start; this.length = length; } }}

A method typically requires and ensures inv

Also, the body is typically wrapped in

anexpose (this) block

Tutorial at FM2005

Object Invariants and Object Creation

class Word { private string! line; private int start; public int length; invariant 0 <= start && 0 <= length; invariant start + length <= line.Length;

public Word(string! line, int start, int length) requires 0 <= start && 0 <= length; requires start + length <= line.Length; ensures inv; { this.line = line; this.start = start; this.length = length; base(); EndExpose(); }}

When an object is created, its inv field is false. The constructor typically initializes the fields, establishing the invariant, and then

calls EndExpose() to set the inv field.

Tutorial at FM2005

Object Invariants: Recap

• Spec# supports object invariants• To avoid reentrancy problems, we

introduce an inv field; it abstracts the invariant and may be used in contracts

• BeginExpose() and EndExpose() calls (or expose blocks) toggle inv; they delimit the regions where the invariant does not need to hold and where the object may be modified

Tutorial at FM2005

Invariants and Exceptions

class EOFException : CheckedException {}

byte ReadByte() throws EOFException { expose (this) { … if (…) throw EOFException(); … }}

[Q] Should the expose block perform an EndExpose() if the body terminates with a checked exception?

Yes – throwing a checked exception does not mean the object is broken. Future calls on the object need to be able to rely on the invariant.

Note that if the invariant does not hold, the EndExpose() call replaces the checked exception with an ObjectInvariantException.

Tutorial at FM2005

Invariants and Exceptions

void Foo(int a, int b) { expose (this) { … … a / b … … }}

[Q] Should the expose block perform an EndExpose() if the body terminates with an unchecked exception?

No --- the object is broken. Leaving it exposed will prevent future method calls on the object.

Also, performing an EndExpose() might replace the original exception with an ObjectInvariantException, masking the real error.

Tutorial at FM2005

Tutorial



Tutorial at FM2005

Inter-object Invariants

class Account { List<int> deposits; int balance; invariant balance == Math.Sum(deposits);

public List<int> GetDeposits() { return deposits; } public static List<int> GetAllDeposits(Account a1, Account a2) { List<int> ds = a1.GetDeposits(); ds.AddRange(a2.GetDeposits()); return ds; }}

[Q] Is this okay?

Oops! This modifies a1’s list of deposits.

Also, a1’s invariant is now broken.

Spec# solution: Allow the list of deposits to be modified only when the account object is

exposed.

Tutorial at FM2005


class Account { [Owned] List<int> deposits; int balance; invariant balance == Math.Sum(deposits);


[Q] Is this okay?




exposed.

This is achieved by having the account object own

the list of deposits whenever the former is

not exposed.

Tutorial at FM2005




[Q] Is this okay?




exposed.

This is achieved by having the account object own

the list of deposits whenever the former is

not exposed.

ds.BeginExpose() fails if ds is owned, so AddRange won’t succeed in modifying ds.

Tutorial at FM2005

Updating Owned Objects


void Deposit(int amount) { expose (this) { deposits.Add(amount); balance += amount; } }}

Objects cannot be modified while they’re owned.

But calling BeginExpose() on the owner releases ownership of the

owned objects.

Calling O.EndExpose() causes O to (re-)gain ownership of the

objects pointed to by its [Owned] fields.

Tutorial at FM2005

Object Lifecycle

// Account a = new Account();Account a = new Account;List<int> d = new List<int>;d.EndExpose();a.deposits = d;a.EndExpose();

// a.Deposit(…);a.BeginExpose();d.BeginExpose();d.EndExpose();a.EndExpose(); exposed

exposableowned

Tutorial at FM2005

Object Lifecycle


// a.Deposit(…);a.BeginExpose();d.BeginExpose();d.EndExpose();a.EndExpose(); a

inv == falseowner == null

exposedexposable

owned

Tutorial at FM2005

Object Lifecycle


// a.Deposit(…);a.BeginExpose();d.BeginExpose();d.EndExpose();a.EndExpose();

dinv == false

owner == null

ainv == false

owner == null

exposedexposable

owned

Tutorial at FM2005

Object Lifecycle



dinv == true

owner == null

ainv == false

owner == null

exposedexposable

owned

Tutorial at FM2005

Object Lifecycle



dinv == true

owner == null

ainv == false

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

Object Lifecycle



dinv == trueowner == a

ainv == true

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

Object Lifecycle



dinv == true

owner == null

ainv == false

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

Object Lifecycle



dinv == false

owner == null

ainv == false

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

Object Lifecycle



dinv == true

owner == null

ainv == false

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

Object Lifecycle



dinv == trueowner == a

ainv == true

owner == null

exposedexposable

owned

deposits

Tutorial at FM2005

IsExposable

o.IsExposablemeans

o.inv == true && o.owner == null

class List<T> { void Add(T value) requires IsExposable; ensures IsExposable; { expose (this) { … } }}

A method typically requires and ensures

this.IsExposable

Tutorial at FM2005

Ownership System

• Each object has an inv field and an owner field• o.owner == null means that o is currently not owned• o.owner == p means that p owns o• o.IsExposable means o.inv == true && o.owner == null• o.BeginExpose() requires o.IsExposable• An object points to its component objects using

[Owned] fields• o.EndExpose() assigns o as the owner of o’s component

objects• o.BeginExpose() sets the owner field of o’s component

objects to null

Tutorial at FM2005

Tutorial• What is Spec#?• Non-null types• Method contracts• Object invariants• Ownership


Tutorial at FM2005

Subclassing

class Account { invariant true; public int balance; public void Add(int n) requires IsExposable; { expose (this) { balance += n; } }}

class StudentAccount : Account{ invariant 0 <= balance;}

static void MyAdd(Account a, int n) requires a.IsExposable; { a.Add(n); // breaks StudentAccount invariant! }

StudentAccount s = new StudentAccount();MyAdd(s, -1000);

[Q] How can we fix this? How can we translate this into something that we already know?

Tutorial at FM2005

Subclassing as Containment

Account frame

StudentAccount frame

class Account { … }

class StudentAccount : Account {}

Account object

StudentAccount object

class Account { … }

class StudentAccount {

[Owned] Account base;

}

Tutorial at FM2005

Subclassing as Containment


class StudentAccount { invariant 0 <= base.balance;

[Owned] Account! base;}

static void MyAdd(StudentAccount s, int n) requires s.IsExposable; { s.base.Add(n); // requires fails: Account object is not

exposable }

StudentAccount s = new StudentAccount();MyAdd(s, -1000);

Tutorial at FM2005

Learning from Containment

Account frame

StudentAccount frame

Account object

StudentAccount object

• Each frame has its own inv field and owner field

• Therefore, each frame has its own non-virtual IsExposable property

• Each frame owns its base class frame

• Therefore, the ownership system enables a subclass to safely mention the state of its base class

Tutorial at FM2005

Subclassing


class StudentAccount : Account{ invariant 0 <= balance;}

static void MyAdd(Account a, int n) requires a.IsExposable; { a.Add(n); // Okay }

StudentAccount s = new StudentAccount();MyAdd(s, -1000); // But now here the requires clause fails

Statically bound: IsExposable property of Account frame

[Q] How to solve this?

Tutorial at FM2005

Virtual methods

class Account { public int balance; public void Add(int n) { expose (this) balance += n; } public virtual bool TryAdd(int n) { Add(n); return true; }}

class StudentAccount { invariant 0 <= balance; public override bool TryAdd(int n) { if (balance + n < 0) return false; expose (this) return

base.TryAdd(n); }}

static void YourAdd(Account a, int n) { a.TryAdd(n); // OK! dynamically bound call}

StudentAccount s = new StudentAccount();YourAdd(s, -1000);

[Q] How to verify modularily?

Tutorial at FM2005

Modular verification of inheritance

class Account { public int balance;

public void Add(int n) requires IsExposable; { expose (this) { balance += n; } }

public virtual bool TryAdd(int n) requires IsVirtualExposable; { Add(n); return true; }}

class StudentAccount { invariant 0 <= balance;

public override bool TryAdd(int n) { if (balance + n < 0) return false; expose (this) { return base.TryAdd(n); } }}

static void YourAdd(Account a, int n) requires a.IsDynamicExposable; { a.TryAdd(n); }

StudentAccount s = new StudentAccount();YourAdd(s, -1000);

IsDynamicExposable binds dynamically to the IsExposable property of the most derived

frame

At dynamic call sites, IsVirtualExposable means

IsDynamicExposable

At static call sites and when verifying the callee, IsVirtualExposable means

IsExposable

Tutorial at FM2005



Tutorial at FM2005

Pure Methods[Pure] static int Double(int x) ensures result == x + x;

static int Quadruple(int x) ensures result == Double(Double(x));{ return 4 * x; }

[Q] How does one verify this statically?

For each pure method, a function symbol is defined.For each ensures clause of a pure method, an axiom is added.

function Double(int) returns (int);axiom (forall x:int, heap: … :: Double(heap, x) == x + x);

Tutorial at FM2005

State Abstractionclass StudentAccount { private int balance; invariant 0 <= balance;

public int Balance { get { return balance; } }

public void Deposit(int amount) requires 0 <= Balance + amount; { expose (this) { balance += amount; } }}

[Q] How does the static verifier know here that the

invariant holds?

Tutorial at FM2005


public int Balance { get ensures result == balance; { return balance; } }

public void Deposit(int amount) requires 0 <= Balance + amount; { expose (this) { balance += amount; } }}

[Q] Adding an ensures clause to this pure property getter would solve the problem. But is

this okay?

Tutorial at FM2005


public int Balance { get private ensures result == balance; { return balance; } }

public void Deposit(int amount) requires 0 <= Balance + amount; ensures Balance == old(Balance) + amount; { expose (this) { balance += amount; } }}

It turns out that in order to support state abstraction, it is sufficient to allow the

declaration of private ensures clauses on pure methods.

By default, these are derived from the body if the

body is of the form return E;

Tutorial at FM2005

State Abstraction and Interfaces

interface IList<T> { int Count { get; } T this[int i] { get; }

public void Add(T x) ensures Count == old(Count) + 1; ensures forall{int i in (0:old(Count)); this[i] == old(this[i])}; ensures this[old(Count)] == x;}

Tutorial at FM2005



Tutorial at FM2005

MultithreadingAccount a = new Account();

new Thread(delegate { a.Deposit(1000); }).Start();new Thread(delegate { a.Deposit(1000); }).Start();


Solution: we need to ensure that a thread has exclusive access to an object when it accesses the object.

Tutorial at FM2005

MultithreadingWhat if we redefine expose (o) so that it additionally locks o?

Account a = new Account(); Account b = new Account();

new Thread(delegate { a.Deposit(-1000); b.Deposit(1000); }).Start();new Thread(delegate { a.Deposit(-1000); b.Deposit(1000); }).Start();


Conclusion: We need a separate construct for locking an object.

Tutorial at FM2005

MultithreadingAccount a = new Account();a.EndAcquire();

new Thread(delegate { acquire (a) a.Deposit(1000); }).Start();new Thread(delegate { acquire (a) a.Deposit(1000); }).Start();

We achieve mutual exclusion by extending the notion of ownership: an object may now be owned by another object or by a thread. It may also be free (i.e. not owned by any thread or object).

Formally, the owner field may be either null (which means that the object is free), an object, or a thread.

A o.BeginAcquire(); call performed by a thread t waits until o is free and then assigns ownership of o to t.

An o.EndAcquire(); call releases ownership of o; o becomes free.An o.IsExposable call performed by a thread t means: o.owner == t && o.inv == true.

Tutorial at FM2005

MultithreadingAccount a = new Account(); Account b = new Account();

new Thread(delegate { acquire (a) { acquire (b) { a.Deposit(-1000); b.Deposit(1000);}}}).Start();new Thread(delegate { acquire (a) { acquire (b) { a.Deposit(-1000); b.Deposit(1000);}}}).Start();

Tutorial at FM2005

Multithreadingclass BankServer { Account! account; public void Run() { while (true) { int amount = AcceptDepositRequest(); acquire (account) { account.Deposit(amount); } } } …}

Tutorial at FM2005

Readers-Writers

class StatementPrinter { Account! account; public void Run() { while (true) { Thread.Sleep(30*24*3600*1000); acquire readonly (account) { PrintStatement(account.Balance); } } } …}

Tutorial at FM2005

Wait Conditions

class BankServer { StudentAccount! account; public void Run() { int amount = AcceptDepositRequest(); acquire (account; 0 <= account.Balance + amount) { account.Deposit(amount); } } …}

Tutorial at FM2005

Thank You!

Download papers and alpha bits athttp://research.microsoft.com/specsharp/

Tutorial at FM2005 The Spec# Programming System: An Overview Tutor: Bart Jacobs PhD student at...

Documents

Transcript of Tutorial at FM2005 The Spec# Programming System: An Overview Tutor: Bart Jacobs PhD student at...