PROOF TRANSLATION AND SMT LIB CERTIFICATION

Yeting Ge Clark BarrettSMT 2008July 7 Princeton

SMT solvers are more complicated

CVC3 contains over 100,000 lines of code Are SMT solvers correct?

Quest for correct SMT solvers?

To verify a SMT solver is correct? To develop a correct SMT solver?

Good news: we have proofs Some SMT solvers could produce proofs Proof checking should be easier than

proving the correctness of a SMT solver A proof could be represented as a proof

tree

1| ba 1| b

0| a

Bad news: Proof checking for SMT solvers is not so easy

Theory proof rules require the proof checker to have theory reasoning ability a/2 = b

Choice of proof rules A small set of simple proof rules?

Good for proof checking Large set of complex proof rules?

Good for performance (CVC3 has 298 rules) The correctness of the proof checker becomes

questionable SMT solvers are in constant change

The ideaUse a second prover to check the proof Translate the proof into the second prover The benefits

Could easily handle both simple and complex proof rules Flexible

The challenges A suitable second prover

The correctness is reduced to the second prover Efficiency Translation

This is feasible!

SMT LIB certification SMT LIB

A collection of over 40,000 SMT benchmarks, most of which from industry applications

Each file contains a status field

Some files are incorrectly labeled The proof in the second prover is a certificate A certified SMT LIB will be beneficial to SMT

community Prove as many unsatisfiable cases as possible

(benchmark tmp:source {piVC} :status unsat :category { industrial } :difficulty { 0 } :logic AUFLIA :extrafuns ((V_6 Int))

CVC3 A proof is a tree A proof rule maps a set of proofs to a proof

Some proof rules are rather complex

The second prover: HOL Light

Simple The core:

430 lines of Ocaml, 10 inference rules, 3 axioms Definitional extension guarantees correctness

Except equality, all logic symbols are defined All proofs in HOL Light can be broken down

into the 10 rules and 3 axioms, if needed “it sets a very exacting standard of

correctness” Efforts to verify the correctness of the core

HOL Light Powerful

Capable of formalizing most mathematics (up to axiom of choice)

Flexible Programmable

Ocaml as meta-language A number of built-in theories

Reals, integers A lot of useful tools

Decision procedures for first-order logic, propositional logic

Decision procedures for reals, integers, …

Translation of terms HOL Light and CVC3 are connected

through C API functions of CVC3 distinct(x1,x2,…,xn)

Define a predicate on the fly Mixed integers and reals

Lift to reals Skolem constant

Choice operator (@x.P))()(. skoPxPx

Translation of proof rules An Ocaml function for each proof rule Naïve method

call HOL Light’s decision procedure Exploit HOL Light’s capability of higher

order reasoning Prove a meta-theorem off-line During the translation, instantiate the meta-

theorem Engineering the translation of a proof rule

Propositional reasoning SAT solvers can dump a resolution proof

Sequent representation

Definitional CNF and ITE

hole5 Time(s)Try 1 255Try 2 155Seq 37Sorted

2.8

Resultscatetory cases CVC3 Translation

proved Ave time proved Ave time

simplify1 833 833 0.98 833 19.51Simplify2 2329 2306 1.11 2164 8.85burns 14 14 0.02 14 1.38ricart 14 13 0.07 13 17.60piVc 41 41 0.12 41 1.45

Hard cases

CVC3 Translation

No Prep 5 47.25 5 41.49With Prep 4 48.91 4 64.27

Hard cases in simplify1: CVC3 spent more than 20 seconds

Results Found one proof rule that does not

preserve validity in CVC3 Found one faulty proof rule in CVC3 Found two mis-labled SMT LIB cases in

AUFLIA

Discussion Instantiating a meta-theorem in HOL Light is

almost like rewriting Most proof rules can be converted into some

meta-theorem Other methods to improve efficiency

Compiling HOL Light

Conclusion It is feasible to translate proofs from

CVC3 into HOL Light It is possible to certify many SMT LIB

cases in HOL Light

Future works Prove more SMT LIB cases Improve the translation of arithmetic

proof rules Support more proof rules Support more theories Improve the proof rules of CVC3

Thanks John Harrison for help with HOL Ligh Sean McLaughlin for writing the first

version of the translator

Reference C. Barrett and C. Tinelli. CVC3. In W. Damm and H. Hermanns, editors,

Proceedings of the 19th International Conference on Computer Aided Verification (CAV ’07), LNCS 4590, pages 298–302. Springer-Verlag, July 2007. Berlin, Germany.

J. Harrison. Hol light: A tutorial introduction. In M. K. Srivas and A. J.Camilleri, editors, FMCAD, LNCS 1166, pages 265–269. Springer, 1996.

S. McLaughlin, C. Barrett, and Y. Ge. Cooperating theorem provers: A case study combining HOL-Light and CVC Lite. In A. Armando and A. Cimatti, editors, Proceedings of the 3rd Workshop on Pragmatics of Decision Procedures in Automated Reasoning (PDPAR ’05), volume 144(2) of Electronic Notes in Theoretical Computer Science, pages 43–51. Elsevier, Jan. 2006. Edinburgh, Scotland.

M. Moskal. Rocket-fast proof checking for smt solvers. In K. Jesen and A. Podelski, editors, TACAS, LNCS 4963, pages 486–500. Springer, 2008.

T. Weber. Efficiently checking propositional resolution proofs in isabelle/hol. volume 212 of CEUR Workshop Proceedings, 2006.