Towards a Better Understanding of Accountability

Work in progress (joint with Joan Feigenbaum and Rebecca Wright)

10 December 2010Security & Privacy Day

Partially supported by NSF

Aaron D. JaggardRutgers (DIMACS)/Colgate (CS)

Alice and Bob (only) can read this email exchange

• Run key-establishment protocol• Exchange encrypted email• Take care to protect the plaintext

• Note that it is Alice and Bob themselves who are enforcing this policy.

3

Law-enforcement officials (and ONLY they)may access the Alert Database.

Requester AccessController

AlertDB

PolicyDB

L.E. Cred.DB1

L.E. Cred.DBi

……

Req.; L.E. Cred.

Data

“©2010, Disney. All rights reserved.”

• DRM systems allow only authorized users to access the content and restricts the manner in which they can use it.

• Under the Fair-Use provisions of copyright law, certain categories of uses do not require authorization by the rights holder.

• A user may need to access the work in order to determine how he wants to use it (and thus whether he needs authorization).

Eavesdropping without a warrant is permitted if (and ONLY if) the source is not a US person• The source of an Internet traffic stream (or even its

geographic location) is hard to determine• As in the copyright case, the requester may need to

access the data (now the traffic stream) in order to (try to!) determine whether he needs a warrant

• What should he do with a US person’s traffic while he waits for a warrant, and how can he prove that this is what he has done?

Cloud services for Yale undergraduates will be provided in accordance with some contract• The data are owned by the student or by the university

(as appropriate).• Deletion by the owner will cause all copies of the data

item to be destroyed (within time T).• Data will not be stored in any of the following

countries ...• ...

• How can compliance with such a contract be adjudicated, e.g., how can a cloud-service provider prove that it has not done something?

Issues Raised

• Most security and privacy policies that we know how to specify and implement are preventive. They are about authorization before the fact.

• We know less about accountability after the fact.• Accountability will be increasingly important;

architectures for data usage should incorporate this.– One goal: Guarantee sufficient “accountability” to

effectively deter violations.

Need for Accountability

Weitzner et al., CACM 2008: “For too long, our approach to information

protection policy has been to seek ways to prevent information from ‘escaping’ beyond appropriate boundaries, then wring our hands when it inevitably does. This hide-it-or-lose-it perspective … on privacy, copyright, and surveillance is increasingly inadequate. … As an alternative, accountability must become a primary means through which society addresses appropriate use.”

Need for Accountability

Lampson, CACM 2009: Misplaced emphasis on prevention

(“security based on locks”) rather than accountability (“security based on deterrence”) has resulted in unusable security technology that people do not understand and often work around.

Research Goal: Define “Accountability”

• Seems to be consensus that “accountability” is important in online activity, but people disagree about what it means.

• Users will resist the construction of a “cyber architecture for accountability” if they think its cost (in, e.g., privacy, speed, or convenience) will be too high.

• Progress on definitions and terminology may defuse this resistance and identify desiderata for architectures and protocols.

Accountability in Law and Political Science

• “Accountability” [in global-scale interactions] implies “that some actors have the right tohold other actors to a set of standards,to judge whether they have fulfilled their

responsibilities in light of these standards,and to impose sanctions if they determine that

these responsibilities have not been met.”[Grant & Keohane]

Accountability in Law and Political Science

• This presupposes some sort of larger framework within which the nation-states are interacting– A nation-state unilaterally defending its interests,

even if codified in a treaty, is not viewed as an accountability mechanism [Grant & Keohane]

Administrative Law and International Relations

• Interest in, and discussion of, various accountability mechanisms (without necessarily precisely stating what is meant by “accountability”)– Elections– Superior/subordinate relationships– Delegation of authority– Fiscal– Legal

Definitions of Accountability

• “Accountability is a protean concept, a placeholder for multiple contemporary anxieties.” [Mashaw]

• “[A]ccountability has not yet had time to accumulate a substantial tradition of academic analysis. ... [T]here has been little agreement, or even common ground of disagreement, over the general nature of accountability or its various mechanisms.” [Mulgan]

Definitions of Accountability

• “Accountability is the ability to hold an entity, such as a person or organization, responsible for its actions.” [Lampson]

• “An accountability protocol gives [an agent] lasting evidence, typically digitally signed, about the actions performed by his peer.” [Bella & Paulson]

• For example:– Non-repudiation: provide both sender and receiver with evidence of

the other’s participation– Certified email: provide sender with non-repudiation of receipt

(receiver reads the message iff sender gets return receipt) [Abadi et al.]; non-repudiation for both sender and receiver [Nenadic et al.]

Accountability via Policy Awareness and Adjudication

• Cyber-architectural components:– Policy languages– Policy-reasoning systems– Policy-aware monitoring and logging

• http://dig.csail.mit.edu

17

Examples of Accountability in DIG • Logging, analysis, and revision of policies and queries

– Policy assurance in Private Information Retrieval– Data exchange in Fusion Centers

• Flagging but not stopping non-compliant actions– Policy-aware mashups– License validation in Creative Commons– Social-web privacy

• DIG projects use Semantic-Web technology for policy expression and reasoning.

Another Approach

• Accountable Internet Protocol– Addresses derived from public keys

Identity plays a major role.

What Accountability Protocols Provide

• A judge agent in a protocol might deliver a verdict about a protocol run [Kuesters—Truderung—Vogt]

• The judging agent should be– Fair: Agents who follow the protocol shouldn’t be

blamed– Complete: If the protocol fails due to misbehavior, then

the judge should blame at least one of the misbehaving participants

• Similarly, auditors blaming agents when a bad message is seen [Jagadeesan et al.]

An Alternative Formulation [FJW]

• Working Definition: An entity is accountable with respect to policy P (or accountable for obeying P) if, whenever the entity violates P, then, with some positive probability it is punished (or it could be punished).– We separate accountability from identifiability

Accountability and Identifiability

• One example: E-cash– Provides anonymity if no coin is spent more than

once– Agent’s identity is revealed if the agent double-

spends• In some approaches, agent’s other spending is then

linked• Provides evidence (with identity)

Accountability and Identifiability

• Another Example: Blacklistable Anonymous Credentials without TTPs [Tsang—Au—Kapadia—Smith]– Anonymous credentials for authentication– Participants who misbehave can be blacklisted• No TTP is used to de-anonymize• Identity of misbehaving participants is not revealed

Accountability and Identifiability

• Many approaches to accountability rely on some sort of identity/identification– Implicit in approaches in administrative law and

political theory– If evidence is used or produced, this typically

identifies the participant(s)– A judgment typically specifies the guilty party to

be punished– Shift focus to punishment to try to minimize need

for identity

Questions on Identifiability

• “Closed” systems– Recall international-relations example of non-

accountability• Subset/delegated accountability– Don’t (immediately) have individual punishment– Reduce level of identifiability– How to induce participation?

An Alternative Formulation [FJW]

• Working Definition: An entity is accountable with respect to policy P (or accountable for obeying P) if, whenever the entity violates P, then, with some positive probability it is punished (or it could be punished).– We separate accountability from identifiability– Punishment = expected utility is decreased.– Relaxes Lampson’s definition (to allow automatic/passive;

examples later)– Decreased w.r.t. what? Idea of “normal” trace. (Cf. ideas

used by Halpern for causality.)– This is a separate question from whether the punishment is

effective as a deterrent.

Automatic vs. Mediated Punishment

• Intuitively, punishment after a policy violation:– Is mediated if it happens as a result of some other

action that depends on the violation– Is automatic otherwise

Automatic Punishment without Identifiability

• Second-price auctions (policy is “Bid your true value”)– With some non-vanishing distribution on the values

of the other bidders, the bidder cannot improve his utility by bidding falsely, but with positive probability his utility will be decreased.

– No punishing action is taken; this is automatic punishment

– The violator isn’t identified!– Nobody else even knows that there was a violation!!

Automatic Punishment with Identifiability

• Self-destructing stolen goods (policy is “Don’t steal the good”)– A car dealer attaches a device to each car on his lot.– The device will harm anyone [in a non-permanent but

utility-decreasing manner] who tampers with it and anyone in the car if it is driven off the lot with the device attached.

– Punishment is meted out without any action that depends on the violation.• Automatic punishment

– Note that, unlike second-price auctions, the violator is identified (and the existence of a violation is revealed).

Automatic Punishment Isn’t Necessarily Incentive Compatibility

• Automatic punishment need not uniformly disincentivize rule violation– Even if punishment is generally effective (in the

sense of deterring most potential violators), some agents might have extreme utility functions.

• Mediated punishment might disincentivize rule violation– E.g., shoplifting uniformly deterred by large

enough fines and watchful/fast enough guards.

Research Goal: Explicate Relationship Between Accountability and other S&P Properties

Accountability

Identification Authorization

Compensation Detection

PunishmentClosedSystems

Other Issues and Questions

• Should we reserve “accountability” for approaches that require identification? That might be consistent with common uses of “to hold someone accountable.”– This may not be the fundamental goal; we may really

be after deterrence.– One can be deterred even if one will not be identified.

• Possible approach: Allay fears (about privacy, etc.) by promoting “deterrence” instead of “accountability.”

Summary

• Lots of views of accountability• These usually involve some mix of evidence,

judgment, and punishment– Lots of related ideas: Compensation, detection, ...

• Role of identity– Commonly used, but may prompt concerns

• Shift focus to later in the evidence—judgment—punishment spectrum– If we focus on punishment, can we still get deterrence

with less of a need for identifiability