Combinatorial Agency Michal Feldman ( Hebrew University)

Combinatorial Agency

Michal Feldman(Hebrew University)

Joint with: Moshe Babaioff (UC Berkeley)

Noam Nisan (Hebrew University)

Hidden Actions

Algorithmic Mechanism Design: computational mechanisms to handle Private Information.

(Classical) Mechanism Design Private Information Hidden Actions

We study hidden actions in multi-agents computational settings

Example Quality of Service (QoS) Routing [FCSS’05]:

We have some value from message delivery. Each agent controls an edge:

succeeds with low probability by default. succeeds with high probability if exerts costly effort

Message delivered if there is a successful source-sink path.

Effort is not observable, only the final outcome.

source sink

Modeling: Principal-Agent Model

AgentPrincipal

exerts effortcost: c >0

Does not exert effortcost: 0

Project succeeds with high probability

Project succeeds with low probability

Motivating rational agents to exert costly effort toward the welfare of the principal, when she cannot contract on the effort level, only on the final outcome

“Success Contingent” contract. The agent

gets a high payment if project succeeds,

gets a low payment if project fails

Our focus is on multi-agents technologies

Our Model n agents Each agent has two actions (binary-action):

effort (ai=1), with cost c>0 (ci(1)=c) no effort (ai=0), with cost 0 (ci(0)=0)

There are two possible outcomes (binary outcome): project succeeds, principal gets value v project fails, principal gets value 0

Monotone technology function t: maps an action profile to a success probability: t: {0,1}n [0,1] t(a1,…,an)=success probability given (a1,…,an) i t(1, a-i) > t(0,a-i) (monotonic)

Principal designs a contract for each agent Project succeeds agent i receives pi (otherwise he gets 0)

Players’ utilities, under action profile a=(a1,…,an) and value v: Agent i: ui(a) = t(a)·pi – ci(ai) Principal: u(a,v) = t(a)·(v –Σipi)

Agents are in a game, reach Nash equilibrium.

The Principal’s design parameter: Used to induce the desired equilibrium

The Principal’s “input” parameter.

Example: Read-Once Networks A graph with a given source and sink

Each agent controls an edge, independently succeeds or fails in his individual task (delivering on his edge) Succeeds with probability ɣ<½ with no effort Succeeds with probability 1-ɣ (>½>ɣ) with effort

The project succeeds if the successful edges form a source-sink path.

example: t(1, 1, 0) = Pr { x1 (x2 x3) =1 | a=(1,1,0) } = (1- ɣ) (1- ɣ(1-ɣ))

source sinka1=1

a2=1

a3=0Pr {x1=1}=1- ɣ

Pr {x2=1}=1- ɣ

Pr {x3=1}=ɣ

Nash Equilibrium

Principal’s best contract to induce eq. a=(a1,…,an):

pi= c / i(a-i) for agent i with ai=1

pi= 0 for agent i with ai=0 e.g., (1,0) (1,1)

Agent i’s utility

ui( 1,a-i ) = pi· t( 1,a-i ) – c ui( 0,a-i ) = pi· t(0,a-i )

exerts effort Does not exert effort

),0(),1(),0(),1(

iiiiiii atat

cpauau

i(a-i)

)0,0()0,1(1 tt

cp

)0,1()1,1(2 tt

cp

)1,0()1,1(1 tt

cp

P2=0

Optimal Contract

the principal chooses a profile a*(v) that maximizes her optimal equilibrium utility

1: ),0(),1(

)(),(iai ii atat

cvatvau

Probability of success

Total payments

Research Questions

How does the technology affect the structure of the optimal contracts? Several examples (AND, OR, Majority …) General technologies

What is the damage to the society due to the inability to monitor individual actions? “price of unaccountability”

What is the complexity of computing the optimal contract? Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing edges

from the graph?

Optimal Contracts: simple AND technology2 agents, = ¼, c=1

t(0,0) = 2 = (¼)2=1/16 t(1,0) =t(0,1)= = 3/16; 0 =t(1,0)-t(0,0)=3/16 - 1/16 = 1/8 t(1,1) = = 9/16Principal’s Utility 0 agents exert effort: u((0,0),v) = t(0,0)·v = v/16 1 agent exerts effort: u((1,0),v) = t(1,0)·(v-c/0) = =3/16(v-1/(1/8))=(3/16)v-3/2 2 agents exert effort: u((1,1),v) = t(1,1)·(v-2c/1) = 9v/16-3

s tx1 x2

1: ),0(),1()(),(

iai ii atat

cvatvau

-4

-2

0

2

4

6

8

0 5 10 15

v

U(v)

At value of 6 there is a “jump” from 0 to 2 agents

v v

Optimal Contract Transitions in AND and OR

AND

s tx1 x2

s t

x1

OR

x2ɣ=1/4 optimal to contract with 0 agents up to 6, then with 2 agent

2

Optimal Contract Transitions in AND and OR

Theorem: For any AND technology, there is only one transition, from 0 to n agents.

Theorem: For any OR technology, there are always n transition (any number of agents is optimal for some value).

• We characterize all technologies with 1 transition and with n transitions.

Proofs Idea-AND’s single transition Observation (monotonicity): number of contracted

agents monotonically non-decreasing in v. Proof for AND’s single transition:

At the indifference value between 0 and n agents, contracting with 0<k<n agent has lower utility.

By the above observation, a single transition.

-4

-2

0

2

4

6

8

0 5 10 15

v

U(v

)

The 0 and n indifference value

Transitions in AND and OR

Proof (AND):k: number of contracted agents

this function has a single minimum point, thus maximized at one of the edges 0 or n

)21()1()1(

)1()()()(

1

kknkkn ckv

ktkt

ckvktku

Proofs Idea – OR’s n transitions Let vk be the indifference point between k

and k+1 agents ( u(k,vk) = u(k+1,vk) )

We show that for OR: vk+1> vk

This ensures that k is optimal from vk-1 to vk

-12

-2

8

18

28

38

48

58

0 20 40 60 80

v

U(v

) 0

1

2v0: The 0 ,1

indifference value.

v1: The 1 ,2

indifference value.

v1>v0

Transitions in AND and OR

k: number of contracted agents

solve for v: u(k) = u(k+1), and let v(k) be the solution

we have to show: v(k+1) > v(k) E.g., n=3

v(0)v(1)

v(2)

General Technologies In general we need to know which agents exert

effort in the optimal contract Examples:

In potential, any subset of agents (out of 2n subsets) that exert effort could be optimal for some v.

Which subsets can we get as an optimal contract?

s s

(a) OR-of-ANDs technology

t

A1

A2B2

(b) AND-of-ORs technology

A1

A2

B1

B1

B2

And-of-Ors (AOO) Technology Example: 2x2 AOO technology

Theorem: The optimal contract in any AOO network (with identical OR components) has the same number of agents in each OR-component

Proof: by induction based on following lemmas: Decomposition lemma: if S=TUR is optimal on

f=hg on some v, then T is optimal for h on v·tg(R) and R is optimal for g on v·th(T)

Component monotonicity lemma: the function vth(T) is monotone non-decreasing (same for vtg(R) )

s

A1

A2

B1

B2

t

v{A1,B1} {A1,B1,A2,B2}

s ts t

f = h g

T R

Decomposition Lemma

Proof:

Rigi

i

Tihi

i

Rigi

i

Tihi

i

Rifi

i

Tifi

i

Sifi

i

iR

cRg

iT

cvRgTh

iRTh

cRgTh

iTRg

cRgvRgTh

iS

c

iS

cvRgTh

iS

cvSf

)\()(

)\()()(

)\()()()(

)\()(

)()()(

)\()\()()(

)\()(),( vSU

)\()()\(,,

)\()()()\,0()()\,1()\(,

iRThiSRiSimilarly

iTRgRgiThRgiThiSTigi

fi

hi

fi

f = h g

T R

if S=TUR is optimal on f=hg on some v, then T is optimal for h on v·tg(R) and R is optimal for g on v·th(T)

Component Monotonicity Lemma

Proof: S1 = T1 U R1 optimal on v1

S2 = T2 U R2 optimal on v2<v1

By monotonicity lemma: f(S1) ≥ f(S2) Since f=g·h, f(S1)=h(T1)·g(R1) ≥ h(T2)·g(R2) = f(S2) Assume in contradiction that h(T1) < h(T2).

Since h(T1)·g(R1) ≥ h(T2)·g(R2) , we get g(R1) > g(R2). By decomposition lemma, T1 is optimal for h on v1·g(R1), and T2 is

optimal for h on v2·g(R2) As v1 > v2, and g(R1) > g(R2), T1 is optimal for h on a larger value than T2. Thus, by monotonicity lemma, h(T1) ≥ h(T2)

h gT1R1

f:T2

R2

The function vth(T) is monotone non-decreasing (same for vtg(R) )

And-of-Ors

Theorem: The optimal contract in any AOO network, composed of nc OR-components (of size nl) contracts with the same number of agents in each OR-component. Thus, |orbit(AOO)| ≤ nl+1

Proof: by induction on nc

Base: nc=2assume (k1,k2) is optimal on some v, assume by contradiction k1>k2 (wlog), thus h(k1)>h(k2).By decomposition lemma:

k1 optimal for h on v·h(k2)k2 optimal for h on v·h(k1)>v·h(k2)

but if k2 optimal for a larger value, k2≥k1. in contradiction.

s t

x11

xnlnc

x1nc

xnl1

And-of-Ors

assume (induction) that claim holds for any number of OR components < nc

Assume 1st component has k1 contracted agents

Let g be the conjunction of the other (nc-1) comp.

By decomposition lemma, contract on g is optimal at v·h(k1), thus by induction hypothesis has same number of agents, k2, on each OR component.

Let h2 be conjunction of first two comp.

By decomp. Lemma, contract on h2 is optimal for some value and by induction hypothesis has same number of agents, k3

We get k1=k3 (in first comp. k1 agents contracted), and k2=k3 (in second comp. k2 agents contracted), thus k1=k2

g

hhhh

k1 k2k2k2

h2

k3k3= ==

The Collection of Optimal Contracts Given t we wish to understand how the optimal

contract changes with v (the “orbit”).

Monotonicity Lemma: The optimal contract success probability t(a*(v)) is monotonic non-decreasing with v So is the utility of the principal, and the total

payment Thus, there are at most 2n-1 changes to the

optimal contracts (|Orbit(t)| ≤ 2n)

Is there a structure on the collection of optimal contracts of t?

The Collection of Optimal Contracts Observation 1: in the observable-actions case, only one set of

size k can be optimal (set with highest probability of success)

Observation 2: not all 2n subsets can be obtained Only a single set of size 1 can be optimal (set with highest

probability of success)

Thm: There exists a tech. with optimal contracts

Open question 1: is there a read-once network with exponential number of optimal contracts?

nn

n2

Can a technology have exponentially many different optimal contracts?

Exponential number of optimal contracts (1) Thm: There exists a tech. with optimal contracts Proof sketch:

Lemma 1: all k-size sets in any k-admissible collection can be obtained as optimal contracts of some t

Lemma 2: For any k, there exists a k-admissible collection of k-size sets of size Based on error correcting code

Lemma 3: for k=n/2 we get a k-admissible collection of k-size sets of size , as required.

k

n

n

1

nn

n2

S1

S2

S3

S4

Collection of sets of size k, in which every two sets in it differ by at least two elements

nn

n2

Proof of Lemma 1

S

S\i S\i

k

k-1

1

n

t(S)= ½ - S

t(S\i)= ½ - 2S

S’

S’\i S’\i

t(S’)= ½ - S’

t(S’\i)= ½ - 2S’

• marginal contribution of i S is: t(S) – t(S\i) = S

Define t to ensure that the marginal contribution of at least one agent is very small

Claim: at vs=(ck) / 2S2, the set S is

optimal:• S better than any other set in col. (by derivative of u(S,v))• S better then any other set not in col. (too high payments)

Si iStSt

cvStvSu

)\()()(),(

vkcvSu

SS

20),(

Let vs be v s.t. vkc

S 2

Exponential number of optimal contracts (2) Lemma: For any n ≥ k, there exists an admissible collection of

k-size sets of size

Proof: take error correcting code that corrects 1 error. Hamming distance ≥ 3 admissible Known: codes with (2n/n) code words. Construct a code with sufficient # of k-weight words

XOR every code word with a random word r. weight k w/ prob

Expected number of k-weight code words There exists r such that the expectation is achieved or

exceeded

k

n

n

1

n

k

n2/

k

n

n

1

Research Questions

How does the technology affect the structure of the optimal contracts?

What is the damage to the society / principal due to the inability to monitor individual actions? “price of unaccountability”

What is the complexity of computing the optimal contract?

Can the principal gain utility from mixed strategies? Can the principal gain utility from a-priory removing

edges from the graph?

Observable-Actions Benchmark (first best) Actions are observable Payment: an agent that exerts effort is paid

his cost (c)

Principal’s utility: u(a,v) = v·t(a) – i|ai=1 c

Principal’s utility = social welfare sw(a,v). The principal chooses a*OA, the profile with

maximum social welfare.

Social Price of Unaccountability Definition: The Social Price Of Unaccountability

(POUS) of a technology is the worst ratio (over v) between the social welfare in the observable-action case, and the social welfare in the hidden-action case:

a* - optimal contract for v in the hidden-action case a*OA - optimal contract for v in the observable-action case

Example: AND of 2 agents:

),(

),(sup

*

*

0 vasw

vaswPOU OA

vS

v

0 2Hidden actionsObservable actions 0 2

s t

Principal’s Price of Unaccountability Definition: The Principal’s Price Of Unaccountability

(POUP) of a technology is the worst ratio (over v) between the principal’s utility in the observable-action case, and the principal’s utility in the hidden-action case:

a* - optimal contract for v in the hidden-action case a*OA - optimal contract for v in the observable-action case

),(

),(sup

*

*

0 vau

vauPOU

p

OApvP

Price of Unaccountability - Results Theorem: The POU of AND technology is

unbounded for any fixed n≥2, when unbounded for any fixed ½ when n

Theorem: The POU of OR technology is bounded by 2.5 for any n

111

11n

POU

Research Questions






Complexity of Finding the Optimal Contract

Theorem: There exists a polynomial time algorithm to compute (a*,p), if t is given by a table (exponential input).

Theorem: If t is given by a black box, exponentially many queries may be required to find (a*,p).

Proof: for value v = c(k+ ½),

S’ is optimal Any algorithm must query

all sets of size k=n/2 to find S’ in the worst case

Input: value v, description of t Output: optimal contract: (a*,p)

t(S)=0

t(S)=1

100 0 00 sets of size n/2

sets of size 1

sets of size n

S’

Complexity of Finding the Optimal Contract

Theorem: For read-once networks, the optimal contract problem is #p-hard Proof: reduction from network reliability problem

Open problem 3: is it polynomial for series-parallel networks?

Open problem 4: does it have a good approximation?

Input: value v, description of t Output: optimal contract: (a*,p)

Best Contract Computationin Read-Once Networks

Proof (sketch): an algorithm for this problem can be used to compute t(E) (probability of success)

Player x will enter the contract only for very large value of v (only after all other agents are contracted), call this value vc

At vc, principal is indifferent between E and EU{x}

Gs t t

G’

x ½

v

cEt

Et

c

iE

cvEt

iE

cvEt

x

x

Ei xtix

xEi

tix

x

2)21(

)1()(

)21)(()\()1()1()(

)\()(

Research Questions






Mixed Strategies

In the non-strategic case: NO (convex combination) What about the agency case?

Extended game: qi : probability that agent i exerts effort

t( qi,q-i ) = qi·t(1,q-i )+ (1-qi )·t(0,q-i )

Marginal contribution: i(q-I ) = t(1,q-i ) - t(0,q-i ) ≥ 0

Can mixed-strategies help the principal ?

What is the price of purity ?

Nash Equilibrium in Mixed Strategies Claim: agent i’s best-response is to mix with probability

q (0,1) only if she is indifferent between 0 and 1

Agent i’s utility:

Principal’s utility:

),0(),1(),0(),1(

ii

iiiiii qtqt

cpququ

i

iiii q

q

qtcqu

)(

)()(

0| )()(),(

iqi ii

i

q

cvqtvqu

Agent i’s utility

ui( 1,q-i ) = pi· t( 1,q-i ) – ci ui( 0,q-i ) = pi· t(0,q-i )

High effort Low effort

Example:OR with two agents Optimal contract for v=110

Pure strategies: both agents contracted: u = 88.12... Mixed strategies: q1=q2=0.96..: u=88.24...

Two observations: q1=q2 in optimal contract Principal’s utility is improved, but only slightly

How general are these observations?

s t

=0.25

=0.25

Optimal Contract in OR Technology Lemma: For any anonymous OR (any ,n,c,v), k{0,1,…,n}

agents exert effort with equal probabilities q1=…=qk (0,1], and n-k agents shirk. i.e. optimal profile: (0n-k, qk)

Proof (skecth): suppose by contradiction that (qi,qj,q-ij) s.t. qi,qj (0,1) and qi > qj is optimal

qj

qi

(qi,qj,q-ij)

(qi-ε,qj+yε,q-ij)j

i

j

i

q

q

qt

qty

)12(1

)12(1

/

/

For a sufficiently small ε , success probability increases, and total payments decrease. In contradiction to optimality

Optimal Contract in OR TechnologyExample: OR with 2 agents:

Price of Purity (POP)

Definition: POP is the ratio between principal’s utility in mixed strategies and in pure strategies

)*(

0)(|*

0

)())(*(

))(())(*(

)(*

vSi ii

i

vqi ii

i

v

ac

vvSt

vqc

vvqt

SuptPOP i

Optimal pure contract

Optimal mixed contract

Price of Purity

Definition: technology t exhibits increasing returns to scale (IRS) if for any i and any b ≥ a

t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i) decreasing returns to scale (DRS) if for any i and any b ≥ a

t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i)

Observations: AND exhibits IRS, OR exhibits DRS

Theorem: for any technology that exhibits IRS, optimal contract is obtained in pure strategies e.g., AND

Price of Purity

For any anonymous DRS technology, POP ≤ n For anonymous OR with n agents, POP ≤ 1.154.. For any anonymous technology with 2 agents, POP ≤

1.5 For any technology (not necessarily anonymous, but with

identical costs) with 2 agents, POP ≤ 2

Observation: the payment to each agent in a mixed profile is greater than the min payment in a pure profile and smaller than the max payment in a pure profile

Research Questions






as before

Free-Labor So far, technology was exogenously given Now, suppose the principal has control over the technology in

that he can ex-ante remove some agents from the graph

Example: OR with 2 agents

Action set of agent i: ai {1,0,} 1: exert effort – succeed with probability d. cost=c 0: do not exert effort - succeed with probability d.

cost=0 : do not participate – succeed with probability 0. cost=0

Action “wastes free-labor” since action “0” increases the success probability with no additional cost

s t s t

Free-Labor

The answer is: YES Example: OR technology, n=2, =0.2

Theorem: for technologies with increasing marginal contribution (e.g., AND), utilizing all free-labor is always optimal

Are there scenarios in which the principal gains utility from “wasting free-labor”?

s t

=0.2

=0.2

v0 1 2

1 removed

Analysis of OR

Lemma: for any OR with n agents and which is small enough, there exists a value for which in the optimal contract one agent exerts effort and no other agent participates

=0.49=0.25=0.01

Version of the Braess’s Paradox A project is composed of 2 essential components: A and B And-of-Ors (AOO): allow interaction between teams

Or-of-Ands (OOA): don’t allow interaction between teams

Obviously, AOO is superior in terms of success probability

s t

B2

B1

A2

A1

s t

A1 B1

B2A2

project succeeds if at least one of the following pairs succeed: (A1,B1) ; (A1,B2) ; (A2,B1) ; (A2,B2)

project succeeds if at least one of the following pairs succeed: (A1,B1) ; (A2,B2)

Version of the Braess’s Paradox

s ti =1

B2

B1A1

A2

s t

A1 B1

B2A2

remove middle edge

s t

B2

B1

A2

A1

don’t remove middle edge

Or-of-Ands “wastes free-labor”. Could the principal gain utility from removing middle edge?

s t s t

u(2,2) = 75.59..

Example: =0.2, v=110

u(1,1) = 74.17..>

Conclusion: it may be beneficial for the principal to isolate the teams

And-of-Ors

Or-of-Ands

Summary “Combinatorial Agency”: hidden actions in combinatorial

settings

Computing the optimal contract in general is hard

Natural research directions: technologies whose contract can be computed in

polynomial time Approximation algorithms

Many open questions remain

Thank You

[email protected]

Related Literature

[Winter2004] Incentives and discrimination The effect of technology on optimal contract (full implementation)

[Winter2005] Optimal incentives with information about peers

[Ronen2005][Smorodinsky and Tennenholtz2004,2005] Multi-party computation with costly information

[Holmstrom82] Moral hazard in teams Budget-balanced sharing rules

Combinatorial Agency Michal Feldman ( Hebrew University)

Documents

Transcript of Combinatorial Agency Michal Feldman ( Hebrew University)