DYNAMIC ENFORCEMENT OF KNOWLEDGE-BASED SECURITY POLICIES Michael Hicks University of Maryland,...
-
Upload
cecil-york -
Category
Documents
-
view
219 -
download
0
Transcript of DYNAMIC ENFORCEMENT OF KNOWLEDGE-BASED SECURITY POLICIES Michael Hicks University of Maryland,...
DYNAMIC ENFORCEMENT OF KNOWLEDGE-BASED SECURITY POLICIES
Michael HicksUniversity of Maryland, College Park
Joint work with Piotr Mardziel, Stephen Magill, and Mudhakar Srivatsa (IBM)
2
My research• Programming languages-based techniques for more
reliable, available, and secure software
3
My research• Programming languages-based techniques for more
reliable, available, and secure software
• Recent projects• Dynamic software updating (for C and Java)
• Blending PL and Crypto
• Adapton: Demand-driven Incremental Computation
• Druby: Static types for dynamic languages
4
Why am I here? http://www.pl-enthusiast.net/2014/09/08/probabilistic-programming/
21,858 pageviews,as of yesterday
5
Today’s talk
Using
probabilistic programming
to enforce
knowledge-based security policies
1.The setting, the problem, and our solution2.A technique for probabilistic inference that
is sound with respect to security• Based on abstract interpretation
6
“Free” sites
• Good: Useful service, no direct charge• Bad: Free use paid for by use of personal information• How to balance the two?
7
Status quo: they have your data
your data
you
query/filter
page
Photography
advertisers
+ ads
8
Take back control
Photography
advertisers
query/filterpage +query/filter
queryresponses
your data
you
9
Simpler model (for exposition)
response
querier
you
queryPhotography
10
Useful and non-revealing
Photography
out = 24 ≤ Age ≤ 30& Female?
& Engaged? *
true
* real query used by a facebook advertiser
querier
you
11
Reveals too much!out = (age, zip-code, birth-date) *
reject
* - age, zip-code, birth-date can be used to uniquely identify 63% of Americans
12
The problem: how to decide to answer based on the information revealed?
out := (age, zip-code, birth-date)
out := ageout := 24 ≤ age ≤ 30
& female? & engaged?
Q1 Q2 Q3
?
13
Approach• Maintain a representation of the querier’s belief about secret’s possible values• Each query result revises the belief; reject if actual secret becomes too likely
• Cannot let rejection defeat our protection.
time
Q1 Q3… …Q2
Reject
Belief ≜probabilitydistribution
Bayesian reasoningto revisebelief OK (answer) OK (answer)
14
Contributions
1. Knowledge-based security policies • Dynamic enforcement (CSF’11, JCS’13)
• Threshold to decide when an answer reveals too much• Query rejection does not reveal information
• Risk prediction (IEEE S&P’14, FCS’14)• Model series of interactions between adaptive adversary
and a defender able to change the secret• What is risk of certain change strategies?
All work collected in Piotr Mardizel’s dissertation
15
Contributions
2. Implementing checking and belief revision efficiently
• Treat queries as probabilistic programs, where inputs are distributions that represent beliefs
• Perform probabilistic inference via abstract interpretation, using a novel probabilistic polyhedra
• Domain associates polyhedron with overlapping descriptions of probability mass, for improved precision
• We track upper and lower bounds for sound conditioning and normalization
All work collected in Piotr Mardizel’s dissertation
16
Meet Bob
=0 ≤ bday ≤ 364
1956 ≤ byear ≤ 1992
Bob (born September 24, 1980)bday = 267byear = 1980 Secret
bday
byear
1956
1992
0 364
Assumption: this is accurateeach equally likely
17
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
|= (out = 0)
1956
1992
0 364259 267
1956
1992
0 364
18
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
|= (out = 0)
1956
1992
0 364259 267
=
Problem Policy: Is this acceptable?
19
Idea: policy as knowledge threshold
• Answer a query if, for querier’s revised belief, Pr[my secret] < t• Call t the knowledge threshold
• Choice of t depends on the risk of revelation
20
Bob’s policy
=0 ≤ bday ≤ 364
1956 ≤ byear ≤ 1992
Bob (born September 24, 1980)bday = 267byear = 1980 Secret
PolicyPr[bday] < 0.2Pr[bday,byear] < 0.05
bday
byear
1956
1992
0 364
CurrentlyPr[bday] = 1/365Pr[bday,byear] = 1/(365*37)
Pr[bday = 267] …
each equally likely
P
21
Bob’s policies
PolicyPr[bday] < 0.2
• bday specifically should never be certainly known
22
Bob’s policies
PolicyPr[bday,byear] < 0.05
• the pair of bday,byear should never be certainly known
• doesn’t commit to protecting bday or byear alone• allows queries that need high precision in one, or low
precision in both
23
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
|= (out = 0)
1956
1992
0 364259 267
1956
1992
0 364
24
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
|= (out = 0)
1956
1992
0 364259 267
PotentiallyPr[bday] = 1/358 < 0.2Pr[bday,byear] = 1/(358*37) < 0.05
=
25
bday-query2today := 261;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
Next day …
1956
1992
0 364259 267
|= (out = 1)
1956
1992
0 267 So reject?Pr[bday] = 1
P
26
1956
1992
0 267
if bday ≠ 267 if bday = 267
1956
1992
0 364259 268
Querier’s perspective
will get answer will get reject
Assume querier knows policy
27
Rejection problem
| =reject
• Policy: Pr[bday = 267 | out = o] < t• Rejection, intended to protect secret, reveals secret!
28
Rejection revised
• Policy: Pr[bday = 267 | out = o] < t
• Solution?• Decide policy independently of secret
• Revised procedure
• for every possible output o,• for every possible bday b,
• Pr[bday = b | out = o] < t
• So the real bday in particular
29
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
accept
initial belief
30
bday-query2today := 261;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
reject
(regardless of what bday actually is)
(after bday-query1)
31
bday-query3today := 266;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
(after bday-query1)
accept
32
Implementing belief revision
• Idea: Use probabilistic programming!• Query is a probabilistic program• Inputs are distributions representing adversary beliefs• Observations are actual/possible outputs
• Posterior is conditioned on these
33
• BUT: Enumeration/sampling-based probabilistic interpretation is problematic• In the limit, time taken is proportional to the running time of the
query and the size of the state space• Approximations used may not be sound (may underestimate risk)
• Can address the latter problem using abstract interpretation• Sound approximation of enumeration-based semantics
Implementing belief revision
34
Three slides on abstract interpretation• Given a programming language
• e ::= x | n | e * e
• The concrete interpretation is its actual semantics• Concrete domain is natural numbers• Concrete interpretation Eval(e) is multiplication
• 1 * (2 * 3) = 6, etc.
• An abstract interpretation approximates the semantics• Abstract domain may be intervals of natural numbers [n1,n2]
• Abstract interpretation Aeval(e): [n1,n2] * [n3,n4] = [n1*n3, n2*n4]• [1,4] * ([2,2] * [3,6]) • = [1,4] * [6,12] • = [6,48]
35
Relating abstract and concrete• Concretization function γ relates the abstract domain A
to the concrete one C• γ : A C⟶• γ ([n1,n2]) = { n | n1 ≤ n ≤ n2 } – here, C is the power-set of naturals
• Abstraction function α relates the concrete domain C to the abstract one A• α : C A⟶• α(S) = [n1,n2] where n S implies n∈ 1 ≤ n and n ≤ n2
• Galois insertion: S γ(α(S)) and α(γ(A)) = A⊆
36
Soundness• Aeval(e) (where x appears in e) is sound iff
• Aeval(e)[x=A] = A’• forall n γ(A), Eval(e)∈ [x=n] = n’ implies n’ γ(A’)∈• (With obvious generalization to multiple variables)
• Suppose e is x * 5• Abstract interpretation, assuming x=[1,3]
• Aeval(e)[x=[1,3]] = [5,15]
• Concrete interpretation, assuming x=2 γ([1,3])∈• Eval(e)[x=2] = 10
• Notice that 10 γ([5,15])∈
37
Concrete domain: probability distributions
• Distributions δ : States Real numbers⟶• A state σ is a map from variables to integers
• Semantics of a statement S under distribution δ is written ⟦S⟧δ. Informally, concrete interpretation is:• For each state σ in the support of δ, let p be its probability• Execute S in that state, producing σ’ with probability mass p
• Add to it the probability mass of all other input states that produce σ’
• Note that distributions’ probabilities may not sum to 1• Frequency distributions are more efficient, simplify development
Back to our setting of probabilistic inference:
38
1956
1992
0 364
bday-query1today := 260;if bday ≤ today && bday < (today + 7) then out := 1 else out := 0
Implementation by enumeration
1956
1992
364
259267
out = 1
out = 0
| (out = 0) *
* δ | (out = 0) = normalize(δ | (out = 0))
input state
output state
=
41
Abstract domain: probabilistic polyhedra
P1: 0 ≤ bday ≤ 259, 1956 ≤ byear ≤ 1992, out = 0p = 0.000074
P2: 267 ≤ bday ≤ 364, 1956 ≤ byear ≤ 1992, out = 1p = 0.000074
…
1956
1992
0 364
P1
P1: 0 ≤ bday ≤ 364, 1956 ≤ byear ≤ 1992p = 0.000074
1956
1992
0
364
259267
P 1
P 2out = 0
P 3
out = 1
Key elements of Prob. Poly. P: Constraints describing polygons Ci
Probability p per point within the polygon
42
Performance / precision tradeoff
• (Sets of) polyhedra are a standard abstract domain, so we can reuse (parts of) existing tools to operate on them directly
• Can end up with many small regions• May be expensive to manipulate
• Approach• Collapse regions: specify the “boundary”
of a region and bounds on the number and probability of the points within it
• Nonuniform regions hurt precision but fewer regions improve performance
43
nasty-query1… many disjuncts …
age := 2011 – byear;if age = 20 || … || age = 60 then out := true else out := false;
1991
1956
0 364259 267
1961
1971
1981
1992
Too many regions
Let = belief after bday-query1
| (out = false)
44
Approximation
P1: 0 ≤ bday ≤ 259, 1956 ≤ byear ≤ 1992p = 0.000067s = 8580 ( < true size = 9620)
P2: 267 ≤ bday ≤ 364, 1956 ≤ byear ≤ 1992p = 0.000067s = 3234 ( < true size = 3626)
1956
0 364259 267
1961
1971
1981
1992P1
P2
P1: 0 ≤ bday ≤ 259, 1992 ≤ byear ≤ 1992p = 0.000067
P2: 0 ≤ bday ≤ 259, 1982 ≤ byear ≤ 1990p = 0.000067
… (ten total)P10
p,s refer to possible (non-zero probability) points in region
1956
0 364259 267
1961
1971
1981
19921991
approximate
P1 P2
exact
45
Abstraction imprecision
1956
0 364259 267
1961
1971
1981
19921991
abstract
P1 P2
1956
0 364259 267
1961
1971
1981
1992
1956
0 364259 267
1961
1971
1981
1992
1956
0 364259 267
1961
1971
1981
1992
exact
46
nasty-query-2… many disjuncts …
age := 2011 – byear;if age = 20 || … || age = 60 then out := true else out := false;
1956
0 364259 267
1961
1971
1981
1992
Non-uniform regions
Takes the true branch one time out of ten Thus, out = true implies age is a decade or the coin flipped in our favor; former more likely
| out = true
pif 0.1 then out := true
… probabilistic choice:
47
Approximation
1956
0 364259 267
1961
1971
1981
19921991
approximate
P1 P2
P1: 0 ≤ bday ≤ 259, 1956 ≤ byear ≤ 1992p ≤ 0.000074 (less-equal)s = 9620
P2: 267 ≤ bday ≤ 364, 1956 ≤ byear ≤ 1992p ≤ 0.000074s = 3626
1956
0 364259 267
1961
1971
1981
1992P1
P2
P1: 0 ≤ bday ≤ 259, 1992 ≤ byear ≤ 1992p = 0.0000074
P2: 0 ≤ bday ≤ 259, 1991 ≤ byear ≤ 1991p = 0.000074
… (18 total)P18
for policy checkPr[bday = b | out = o] < t
48
Final abstraction
1956
0 364259 267
1961
1971
1981
19921991
P1 P2
For each Pi, storeregion (polyhedron)p: upper bound on probability of each possible points: upper bound on the number of (possible) pointsm: upper bound on the total probability mass (useful)
Also storelower bounds on the above
Pr[A | B] = Pr[A ∩ B] / Pr[B]
ProbabilisticPolyhedron
Relating concrete and abstract
Ci
δ ∈ γ(P) iffsupport(δ) ⊆ γ(C)pmin ≤ δ(σ) ≤ pmax for every σ support(∈ δ)smin ≤ |support(δ)| ≤ smax
mmin ≤ mass(δ) ≤ mmax
support(δ) = {σ | δ(σ) > 0}
pimin, pi
max si
min, simax
mimin, mi
max ∈ γ( )…
δ1 δ2
δ3
δ ∈ γ({P1,P2}) iff δ = δ1 + δ2 with δ1 ∈ γ(P1) and δ2 ∈γ(P2)
similarly for more than two
Abstract Semantics• Define
• ⟦S⟧P• Soundness: if δ γ(P) then ∈ ⟦S⟧δ γ (∈ ⟦S⟧P)
• Need Abstract operations• P1 + P2
• if δi γ(P∈ i) for i = 1,2 then δ1 + δ2 γ(P∈ 1 + P2)
• P | B• if δ γ(P) then ∈ δ | B γ(P ∈ | B)
• p*P• if δ γ(P) then p * ∈ δ γ(p*P)∈
• …• (and likewise for sets of P)
Abstract operation exampleδ1 + δ2 – combine mass from both
δ1 + δ2 = λσ. δ1(σ) + δ2(σ)
δ1 δ2 δ1+δ2
s1max = 100 s2
max = 30
P1
C1
P2
C2
P3 = P1 + P2
s3max ≤ s1
max + s2max = 130
| C1 ⋂ C2 | = 20
20
20
80
| C1 | = 100
| C2 | = 40
s3max = 120
What is the maximum number of possible points in the sum?• determine minimum
overlap (20)
Operations• P3 = P1 + P2
• C3 – convex hull of C1, C2
• s3max – what is the smallest overlap?
• s3min – what is the largest overlap?
• p3max – is overlap possible?
• p3min – is overlap impossible?
• m3max – simple sum m1
max + m2max
• m3min – simple sum m1
min + m2min
• Other operations, similar, complicated formulas abound• Need to
• count number of integer points in a convex polyhedra• Latte
• maximize a linear function over integer points in a polyhedron
• Latte
• convex hull, intersection, affine transform• Parma
± P
PPL
53
Implementation and experiments• Interpreter written in Objective Caml for a simple
imperative language• Calling out to LattE and Parma, as mentioned
• In addition to full-precision polyhedra, implemented probabilistic versions of two other domains• Intervals – constraints have form c1 ≤ x ≤ c2
• Can implement counting without LattE
• Octagons – constraints have form ax + by ≤ c with a,b {-1,0,1}∈
• Experiments on several queries• Results for a Mac Pro with two 2.26 GHz quad-core Xeon
processors using 16 GB of RAM
54
Queries• Birthday query 1• Birthday query 1+2• Birthday query 1+2+special• Pizza
• User in particular age range, in college, lives within 2 mile square
• Photo• User in particular age range, female, engaged
• Travel• Lives in particular country, speaks English, over 21, completed high
school
55
Scales better than enumeration
=0 ≤ bday ≤ 364
1956 ≤ byear ≤ 1992
=0 ≤ bday ≤ 364
1910 ≤ byear ≤ 2010
1 pp
> 1 pp
each equally likely
each equally likely
bday1 small
bday 1 large
56
Performance/precision tradeoffBirthday query 1+2+special
57
Intervals very fast generallyQuery Intervals Octagons Polyhedra
Bday1 (small) 0.01 1.87 2.81
Bday1+2 (small) 0.01 2.9 5.25
Bday1+2+spec 0.47 17.8 23.0
Bday1 (large) 0.01 2.1 2.48
Bday1+2 (large) 0.02 3.02 4.52
Bday1+2+spec 0.58 33.6 46.5
Pizza 0.26 92.7 125.5
Photo 0.02 5.47 7.98
Travel 0.48 126.9 154.5
Times in secondsAll achieve maximum precision when given unlimited polyhedra
58
LattE is the bottleneck
59
Limitations: Prototype impoverished• No real or rational numbers• No probabilistic choice where weight is unknown• No widening/narrowing operator for better handling loops• Little support for structured data
• These things should all be possible, starting from standard domains for abstract interpretation
60
Ongoing work, other applications• Knowledge-based security policies in Symmetric setting
• A and B are private inputs to Alice and Bob, and Charlie computes f(A,B) and returns the result to both
• What can Alice and/or Bob learn about the other’s input?• See our paper in PLAS’12
• Location privacy• Apply knowledge policies to locations as secrets• Must consider time-varying nature of current location• Ongoing work with Neil Toronto and David Van Horn
• More … !
61
Summary of contributions• Knowledge-based security policies• Implementation of belief tracking using probabilistic
abstract interpretation• Performance results using intervals are promising
62
BACKUP
63
Creating dependency• Dependencies can be created.
1956
1992
0 364
P1
1956
1992
0 364
P1
strange-queryif bday – 1956 ¸ byear - 5 Æ bday – 1956 · byear + 5 then out := 1 else out := 0
* not to scale and/or shape
64
Differential privacy• Encourage participation in a database by protecting inference of a record.• Add noise to query result.• Requires trusted curator.• Suitable for aggregate queries where exact results are not required.• Not suitable to a single user protecting him(her)self.
• Using differential privacy on a query with 2 output values results in the incorrect query output almost half the time (for reasonable ²)
65
Composability / Collusion• If collusion is suspected among queries, one can make them share belief
• query outputs are assumed to have been learned by all of them• assumes deterministic queries only
• Confusion problem: non-deterministic queries can result in a revised belief which is less sure of the true (or some) secret value than the initial belief
• If collusion is suspected but not present, this can result in incorrect belief• Abstract solution: abstract union of the revised belief (colluded) and the
prebelief (not colluded)P1 [ P2
if ±i 2 °(Pi) for i = 1,2 then ±1, ±2 2 °(P1 [ P2)
66
Setting, Known Secret• Measures of badness that weigh their values by the distribution of the secret
(or the distribution of the outputs of a query) can be problematic.• An unlikely user could theorize a query is safe, but know it is not safe for
them• Consider conditional min-entropy
• (see Information-theoretic Bounds for Differentially Private Mechanisms)
H1(X | Y) = -log2 y PY(y) maxx PX|Y(x,y) > - log2 t
probability of certain query output
• Our policy:• -log2 maxy maxx PX|Y(x,y) > - log2 t• so that -log2 maxx PX|Y(x,yreal) > - log2 t
67
Abstract Conditioning
1956
0 364259 267
1961
1971
1981
1992
1956
0 364259 267
1961
1971
1981
1992
68
Abstract Conditioning
1956
0 364259 267
1961
1971
1981
19921991
approximate
P1 P2
1956
0 364259 267
1961
1971
1981
1992
1956
0 364259 267
1961
1971
1981
1992
1956
0 364259 267
1961
1971
1981
1992
69
Initial distribution
bday
byear
1956
1992
0 364
P1
1/(37*365) - ² · probability · 1/(37*365) + ²37*365 · # of points · 37*365
• Need attacker’s actual belief to be represented by this P1
• Much easier task than knowing the exact querier belief.