Data Model Property Inference and Repair

Data Model Property Inference and Repair

Jaideep Nijjar and Tevfik Bultan{jaideepnijjar, bultan}@cs.ucsb.edu

Verification LabDepartment of Computer Science

University of California, Santa Barbara

ISSTA 2013

Motivation

• Web applications influence every aspect of our lives

• Our dependence on web applications keep increasing

• It would be nice if we can improve the dependability of web applications

Acknowledgement: NSF Support

Three-Tier Architecture

Backend Database

Browser

Web Server

Three-Tier Arch. + MVC Pattern

Backend Database

Browser

Model

ViewsController

Web Server

User Response

Model-View-Controller Pattern

DB

• Benefits of the MVC pattern:• Separation of

concerns • Modularity• Abstraction

• MVC has become the standard way to structure web applications• Ruby on Rails• Zend for PHP• CakePHP• Struts for Java• Django for Python• …

Model View

Controller

User Request

Data Model

• Data model is the heart of the web application

• It specifies the set of objects and the associations (i.e., relations) between them

• Using an Object-Relational-Mapping the data model is mapped to the back-end datastore

• Any error in the data model can have serious consequences for the dependability of the application

Our Data Model Analysis Approach

Model Extraction

Verification

FailingProperties

Active Records

Data Model + Inferred Properties

Property Inference

Data Model(Schema +

Constraints)

Orphan Prevention

Transitive Relations

Delete Propagation

Data ModelSchema

InferredProperties

RepairGeneration

Outline

• Motivation• Overview of Our Approach• Rails Data Models

• Basic Relations• Options to Extend Relations• Formalization of Semantics

• Verification Techniques• Property Inference• Property Repair• Experiments• Conclusions and Future Work

A Rails Data Model Exampleclass User < ActiveRecord::Base

has_and_belongs_to_many :roleshas_one :profile, :dependent => :destroyhas_many :photos, :through => :profile

endclass Role < ActiveRecord::Base

has_and_belongs_to_many :usersendclass Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroy has_many :videos, :dependent => :destroy, :conditions => "format='mp4'"endclass Tag < ActiveRecord::Base

belongs_to :taggable, :polymorphic => trueendclass Video < ActiveRecord::Base belongs_to :profile has_many :tags, :as => :taggableendclass Photo < ActiveRecord::Base ...

Role

*

0..1

1User

Profile

*

1

Video

*

1

Taggable

*Tag

1

* 1Photo

*

1

format=.‘mp4’

Rails Data Models

• Data model analysis verification: Analyzing the relations between the data objects

• Specified in Rails using association declarations inside the ActiveRecord files• Three basic relations

• One-to-one• One-to-many• Many-to-many

• Extensions to the basic relationships using Options• :through, :conditions, :polymorphic, :dependent

Three Basic Relations in Rails• One-to-One

.

• One-to-Many

class User < ActiveRecord::Base has_one :profileend.

class Profile < ActiveRecord::Base belongs_to :userend

class Profile < ActiveRecord::Base has_many :videosend.

class Video < ActiveRecord::Base belongs_to :profileend

User

Profile

0..1

Profile

Video

*

1

1

Three Basic Relations in Rails

• Many-to-Manyclass User < ActiveRecord::Base has_and_belongs_to_many :usersend

class Role < ActiveRecord::Base has_and_belongs_to_many :rolesend

User

Role

*

*

Extensions to the Basic Relations• :through Option

• To express transitive relations

• :conditions Option• To relate a subset of objects to another class

• :polymorphic Option• To express polymorphic relationships

• :dependent Option• On delete, this option expresses whether to delete the associated

objects or not

The :through Optionclass User < ActiveRecord::Base

has_one :profilehas_many :photos, :through => :profile

endclass Profile < ActiveRecord::Base belongs_to :user has_many :photosendclass Photo < ActiveRecord::Base belongs_to :profileend

Profile

User Photo

**

0..1 1

1

1

The :dependent Option

• :delete directly delete the associated objects without looking at its dependencies

• :destroy first checks whether the associated objects themselves have associations with the :dependent option set and recursively propagates the delete to the associated objects

class User < ActiveRecord::Base has_one :profile, :dependent => :destroyend

class Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroyend

PhotoProfileUser *1 10..1

Formalizing Data Model Semantics

• S: Data model schema• The sets and relations of the data model, e.g., { Photo, Profile, Role, Tag,

Video, User} and the relations between them

• C: Constraints on the relations• Cardinality constraints, transitive relations, conditional relations,

polymorphic relations

• D: Dependency constraints about deletions • Express conditions on two consecutive instances of a relation such

that deletion of an object from the first instance leads to the other instance

Formal data model: M = <S, C, D>

Outline

• Motivation• Overview of Our Approach• Rails Data Models• Verification Techniques

• Bounded Verification • Unbounded Verification

• Property Inference• Property Repair• Experiments• Conclusions and Future Work

Verification Overview

Alloy Translator

instance or unsat

formula

Data model + properties

AlloyAnalyzer

Properties

Active Records

SMTSolver

instance or unsat or unknown

formulaSMT-LIB Translator

Property Failed +

Counterexample

Property Verified

Unknown

Model Extraction

Results Interpreter

Results Interpreter

BOUNDED VERIFICATION

UNBOUNDED VERIFICATION

Bound

bound

Sample Translation to Alloy

class User < ActiveRecord::Base has_one :profileend


sig Profile {}sig User {}one sig State {

profiles: set Profile,users: set User,relation: Profile lone -> one User

}

Sample Translation to SMT-LIB

class User < ActiveRecord::Base has_one :profileend


(declare-sort User)(declare-sort Profile)(declare-fun relation (Profile) User)(assert (forall ((p1 Profile)(p2 Profile))

(=> (not (= p1 p2)) (not (= (relation p1) (relation p2) ))

) ))

Property Inference: Motivation

• Verification techniques require properties as input• Effectiveness depends on quality of properties written• Manually writing properties is time-consuming,

error-prone, lacks thoroughness• Requires familiarity with the modeling language

• We propose techniques for automatically inferring properties about the data model of a web application

• Inference is based on the data model schema• A directed, annotated graph that represents the relations

Data Model Schema Example

Outline

• Motivation• Overview of Our Approach• Rails Data Models• Verification Techniques• Property Inference

• Orphan Prevention Pattern• Transitive Relation Pattern• Delete Propagation Pattern

• Property Repair• Experiments• Conclusions and Future Work

Property Inference: Overview

• Identify patterns in the data model schema graph that indicates that certain property should hold in the data model

• Extract the data model schema from the ActiveRecords declarations

• Search for the identified patterns in the data model schema graph

• If a match is found, report the corresponding property

Orphan Prevention Pattern

• For objects of a class that has only one relation: when the object it is related to is deleted but the object itself is not, such an object becomes orphaned

• Orphan chains can also occur• Heuristic looks at all one-to-many or one-to-one relations to

identify all potential orphans or orphan chains• Infers that deleting an object does not create orphans

0 1 n. . .

. . .

Transitive Relation Pattern

• Looks at one-to-one or one-to-many relations in schema• Finds paths of relations that are of length > 1• If there is a direct edge between the first and last node of the

path, infer that this edge should be transitive, i.e. the composition of the others

1 2 n. . .

Delete Propagation Pattern

• Look at schema with all relations removed that are many-to-many or transitive

• Remove cycles in graph by collapsing strongly connected components to a single node

• Assign levels to all nodes indicating its depth in the graph• Root node(s), those with no incoming edges, are at level 0• Remaining nodes are at level 1 more than the maximum of their

predecessors• Propagate deletes if levels between nodes is one

12

3

4

1

2

c

level=0

level=1

level=2

Repair Generation

• Check the inferred properties on the formal data model

• If a property fails we can point out the option that needs to be set in the data model to make sure that the inferred property holds

• For delete propagates and orphan prevention patterns: Set the dependency option accordingly to propagate the deletes

• For transitive property: Set the through option accordingly to make a relation composition of two other relations

1 class User < ActiveRecord::Base2 has_one :preference, :conditions => "is_active=true”,3 :dependent => :destroy4 has_many :contexts, :dependent => :destroy5 has_many :todos, :through => :contexts6 end7 class Preference < ActiveRecord::Base8 belongs_to :user9 end10 class Context < ActiveRecord::Base11 belongs_to :user12 has_many :todos, :dependent => :delete13 end14 class Todo < ActiveRecord::Base15 belongs_to :context16 # belongs_to: user17 has_and_belongs_to_many :tags18 end19 class Tag < ActiveRecord::Base20 has_and_belongs_to_many :todos21 end

Repair Examples

Outline

• Motivation• Overview of Our Approach• Rails Data Models• Verification Techniques• Property Inference• Property Repair• Experiments• Conclusions and Future Work

Experiments on Five Applications

Application LOC Classes Data Model Classes

LovdByLess 3787 61 13

Substruct 15639 85 17

Tracks 6062 44 13

FatFreeCRM 12069 54 20

OSR 4295 41 15

• LovdByLess: A social networking application• Users can write blog entries• Users can comment on a friend’s blog entry• Friend deletes blog entry

A Social Networking Application

• A friend writes a blog entry• User comments on the friend’s blog entry• Friend deletes the blog entry

A Social Networking Application

A Failing Inferred Property

• deletePropagates property inferred for LovdByLess

Blog Commentdelete should propagate

• Tracks: A todo list application• Todos can be organized by Contexts• Users can also create Recurring Todos• Delete the Context. Then edit the Recurring Todo.

A Todo List Application

A Failing Inferred Property

• Data Model and Application Error: deletePropagates property inferred for Tracks

Context RecurringTododelete should propagate

False Positives

• deletePropagates property inferred for FatFreeCRM

• But in FatFreeCRM it is valid to have a contact not associated with any account

Account Contactdelete should propagate

False Positives

• transitive property inferred for LovdByLess

• Just not a transitive relation due to semantics of the application

User

ForumTopic

ForumPost

Experiment ResultsApplication Property Type # Inferred # Timeout # Failed

LovdByLessdeletePropagates 13 0 10

noOrphans 0 0 0

transitive 1 0 1

SubstructdeletePropagates 27 0 16

noOrphans 2 0 1

transitive 4 0 4

TracksdeletePropagates 15 0 6

noOrphans 1 0 1

transitive 12 0 12

FatFreeCRMdeletePropagates 32 1 19

noOrphans 5 0 0

transitive 6 2 6

OSRdeletePropagates 19 0 12

noOrphans 1 0 1

transitive 7 0 7

Property Type# Data Model & Application

Errors# Data Model

Errors# Failures Due

to Rails Limitations

# False Positives

deletePropagates 1 9 0 0

noOrphans 0 0 0 0

transitive 0 0 0 1


noOrphans 0 1 0 0

transitive 0 1 0 3


noOrphans 0 0 0 5

transitive 0 7 0 5


noOrphans 0 0 0 0

transitive 0 0 0 6


noOrphans 0 1 0 0

transitive 0 7 0 0

Conclusions and Future Work

• It is possible to extract formal specifications from MVC-based data models and analyze them

• We can automatically infer properties and find errors in real-world web applications

• Most of the errors come from the fact that developers are not using the ActiveRecords extensions properly• This breaks the modularity, separation of concerns and

abstraction principles provided by the MVC pattern• We are working on analyzing actions that update the data

store• We are also investigating verifiable-model-driven development

for data models

Related Work

• Automated Discovery of Likely Program Invariants • Daikon [Ernst et al, ICSE 1999] discovers likely invariants by

observing the runtime behaviors of a program• [Guo et al, ISSTA 2006] extends this style of analysis and applies it

to the inference of abstract data types • We analyze the static structure of an extracted data model to

infer properties• Static Verification of Inferred Properties

• [Nimmer and Ernst, 2001] integrate Daikon with ESC/Java, a static verification tool for Java programs

• We focus on data model verification in web applications

Related Work

• Verification of Web Applications • [Krishnamurti et al, Springer 2006 ] focuses on correct handling of

the control flow given the unique characteristics of web apps • Works such as [Hallé et al, ASE 2010] and [Han et al, MoDELS

2007] use state machines to formally model navigation behavior• In contrast to these works, we focus on analyzing the data model

• Formal Modeling of Web Applications• WebAlloy [Chang, 2009]: user specifies the data model and

access control policies; implementation automatically synthesized• WebML [Ceri et al, Computer Networks 2000]: a modeling

language developed specifically for modeling web applications; no verification

• In contrast, we perform model extraction (reverse engineering)

Related Work• Verification of Ruby on Rails applications

• Rubicon [Near et al, FSE 2012] verifies the Controller whereas we verify the Data Model

• Requires manual specification of application behavior, whereas we verify manually written properties

• Limited to bounded verification• Data Model Verification using Alloy

• [Cunha and Pacheco, SEFM 2009] maps relational database schemas to Alloy; not automated

• [Wang et al, ASWEC 2006] translates ORA-SS specifications to Alloy, and uses the Analyzer to produces instances of the data model to show consistency

• [Borbar et al, Trends 2005] uses Alloy to discover bugs in browser and business logic interactions

Related Work

• Unbounded Verification of Alloy Specifications using SMT Solvers• [Ghazi et al, FM 2011], approach not implemented• More challenging domain since Alloy language contains

constructs such as transitive closures • Specification and Analysis of Conceptual Data Models

• [Smaragdakis et al ASE 2009, McGill et al ISSTA 2011] use Object Role Modeling to express data model and constraints

• Focus is on checking consistency and producing test cases efficiently

Questions?

Data Model Property Inference and Repair

Documents

Transcript of Data Model Property Inference and Repair