Software Agents

CS 486January 29, 2003

What is an agent?

• “Agent” is one of the more ubiquitous buzzwords in computer science today.– It’s used for almost any piece of software

• “I know an agent when I see one” (and the paperclip is not one.)

Examples

• News-filtering agents• Shopbots/price comparison agents• Bidding agents• Recommender agents• Personal Assistants• Middle agents/brokers• Etc.

Real-world agents

• Secret Agents• Travel Agents• Real Estate Agents• Sports/Showbiz Agents• Purchasing Agents

• What do these jobs have in common?

What is an agent?

An agent is a program with:• Sensors (inputs)• Effectors (outputs)• An environment• The ability to map inputs to outputs

– But what program isn’t an agent, then?

What is an agent?

• Can perform domain-oriented reasoning.– Domain-oriented: a program has some specific

knowledge about a particular area.– Not completely general.

• Reasoning – What does this mean?– Does the program have to plan ahead?– Can it be reactive?– Must it be declarative?

What is an agent?

• Agents must be able to:– Communicate– Negotiate

But what do these terms mean? Language?Are pictures, GUIs communication?How sophisticated is “negotiation”?Communication should “degrade gracefully”

What is an agent?• Lives in a complex, dynamic environment

– Getting at the notion of a complicated problem

• Has a set of goals– An agent must have something it intends to do.(we’ll return to this idea.)

• Persistent state• Distinguishes agents from subroutines, servlets, etc.

What is an agent?

• Autonomy/Autonomous execution• Webster’s:

– Autonomy: “The quality or state of being self-governing”

• More generally, being able to make decisions without direct guidance.– Authority, responsibility

Autonomy

• Autonomy is typically limited or restricted to a particular area.– “Locus of decision making”

• Within a prescribed range, an agent is able to decide for itself what to do.– “Find me a flight from SF to NYC on

Monday.”• Note: I didn’t say what to optimize – I’m allowing

the agent to make tradeoffs.

What is an agent?• Not black and white• Like “object”, it’s more a useful

characterization than a strict category

• It makes sense to refer to something as an agent if it helps the designer to understand it.– Some general characteristics:

• Autonomous, goal-oriented, flexible, adaptive, communicative, self-starting

Objects vs. Agents

• So how are agents different from objects.• Objects: passive, noun-oriented, receivers

of action.• Agents: active, task-oriented, able to take

action without receiving a message.

Examples of agent technology, revisited.

• Ebay bidding agents– Very simple – can watch an auction and increment the

price for you.• Shopping agents (Dealtime, evenBetter)

– Take a description of an item and search shopping sites.– Are these agents?

• Recommender systems (Firefly, Amazon, Launch, Netflix)– Users rate some movies/music/things, and the agent

suggests things they might like. – Are these agents?

More examples of agents• Brokering

– Constructing links between• Merchants• Certificate Authorities• Customers• Agents

• Auction agents– Negotiate payment and terms.

• Conversational/NPC agents (Julia)• Remote Agent (NASA)

The Intentional Stance• We often speak of programs as if they are

intelligent, sentient beings:– The compiler can’t find the linker.– The database wants the schema to be in a different

format.– My program doesn’t like that input. It expects the last

name first.• Treating a program as if it is intelligent is called

the intentional stance.• It doesn’t matter whether the program really is

intelligent; it’s helpful to us as programmers to think as if it is.

The Knowledge Level• The intentional stance leads us to program agents

at the knowledge level (Newell).– Reasoning about programs in terms of:

• Facts• Goals• Desires/needs/wants/preferences• Beliefs

• This is often referred to as declarative programming.

• We can think of this as an abstraction, just like object-oriented programming.– Agent-oriented programming

Example• Consider an agent that will find books for me that

I’m interested in.• States: a declarative representation of outcomes.

– hasBook(“Moby Dick”)• Facts: Categories of books, bookseller websites,

etc.• Preferences – a ranking over states

– hasBook(“Neuromancer”) > hasBook(“MobyDick)– hasBook(B) & category(B) == SciFi > hasBook(b2) & category(b2) == Mystery

Example

• Goals: find a book that satisfies my preferences.

• Take actions that improve the world state.

• Beliefs: used to deal with uncertainty– May(likes(Chris, “Harry Potter”))– Prob(likes(Chris, “Harry Potter”)) == 0.10

Rational Machines

• How do we determine the right thing for an agent to do?

• If the agent’s internal state can be described at the knowledge level, we can describe the relationship between its knowledge and its goals.

• Newell’s Principle of Rationality:– If an agent has the knowledge that an action

will lead to the accomplishment of one of its goals, then it will select that action.

Preferences and Utility• Agents will typically have preferences

– This is declarative knowledge about the relative value of different states of the world.

– “I prefer ice cream to spinach.”• Often, the value of an outcome can be

quantified (perhaps in monetary terms.)• This allows the agent to compare the utility

(or expected utility) of different actions.• A rational agent is one that maximizes

expected utility.

Example• Again, consider our book agent.• If I can tell it how much value I place on different

books, it can use this do decide what actions to take.

• Prefer(SciFi, Fantasy) • prefer(SciFi, Mystery)• Like(Fantasy), Like(Mystery) like(book) & not_buying(otherBook) -> buy(book).How do we choose whether to buy Fantasy or

Mystery?

Example

• If my agent knows the value I assign to each book, it can pick the one that will maximize my utility. (value – price).

• V(fantasy) = $10, p(fantasy) = $7• V(mystery) = $6, p(mystery) = $4

– Buy fantasy.• V(fantasy) = $10, p(fantasy) = $7• V(mystery) = $6, p(mystery) = $1

– Buy mystery.

Utility example• Game costs $1 to play.

– Choose “Red”, win $2.– Choose “Black”, win $3.

• A utility-maximizing agent will pick Black.

• Game costs $1 to play.– Choose Red, win 50 cents– Choose Black, win 25 cents

• A utility-maximizing agent will choose not to play. (If it must play, it picks Red)

Utility example• But actions are rarely certain.• Game costs $1.

– Red, win $1.– Black: 30% chance of winning $10, 70%

chance of winning 0. • A risk-neutral agent will pick Black.• What if the amounts are in millions?

Rationality as a Design Principle

• Provides an abstract description of behavior. • Declarative: avoids specifying how a decision is

reached.– Leaves flexibility

• Doesn’t enumerate inputs and outputs.– Scales, allows for diverse envoronments.

• Doesn’t specify “failure” or “unsafe” states.– Leads to accomplishment of goals, not avoidance of

failure.

Agents in open systems• Open systems are those in which no one

implements all the participants.– E.g. the Internet.

• System designers construct a protocol– Anyone who follows this protocol can

participate. (e.g. HTTP, TCP)• How to build a protocol that leads to

“desirable” behavior?– What is “desirable”?

Protocol design

• By treating participants as rational agents, we can exploit techniques from game theory and economics.

• “Assume everyone will act to maximize their own payoff – how do we change the rules of the game so that this behavior leads to a desired outcome?”

• We’ll return to this idea when we talk about auctions.

Agents and Mechanisms• System designers can treat external

programs as if they were rational agents.• That is, treat external programs as if they

have their own beliefs, goals and agenda to achieve.– For example: an auction can treat bidding

agents as if they are actually trying to maximize their own profit.

Agents and Mechanisms

• In many cases, a system designer cannot directly control agent behavior.– In an auction, the auctioneer can’t tell people what to

bid.• The auctioneer can control the mechanism

– The “rules of the game”• Design goal: construct mechanisms that lead to

self-interested agents doing “the right thing.”

Mechanism Example• Imagine a communication network G with

two special nodes x and y.– Edges between nodes are agents that can

forward messages.– Each agent has a private cost t to pass a

message along its edge.– If agents will reveal their t’s truthfully, we can

compute the shortest path between x and y.• How to get a self-interested agent to reveal

its t?

Solution• Each agent reveals a t and the shortest path is

computed.• Costs are accumulated

– If an agent is not on the path, it is paid 0.– If an agent is on the path, it is paid the cost of the path –

the cost of the shortest path that doesn’t include it - t.– P = gnext– (gbest – t)

• For example, if I bid 10 and am in a path with cost 40, and the best solution without me is 60, I get paid 60 – (40 – 10) = 30

• Agent compensated for its contribution to the solution.

Analysis

• If an agent lies:– Was on the shortest path, still on the shortest path.

• Payment lower – no benefit to lying.

– Was on the shortest path, now not on the shortest path. • This means the lie was greater than gnext– gbest • But I would rather get a positive amount than 0!

– Not on the shortest path, but now are.• Underbidding leads to being paid at the lower amount, but still

incurring higher cost. • Truth would be better!

Example• Cost = 2, SP = 5, NextSP = 8• My payment if I bid truthfully:

8 – (5 – 2) = 5. Net: 5 – 2 = 3.If I underbid, my payment will be lower and net

cost higher.If I overbid, I either get 0, or the same utility.

e.g – if I bid 3, I get 8 – (5 – 3) = 6, but my net is 6 –3 = 3.Therefore, truthtelling is a dominant strategy.

Adaptation and Learning

• Often, it’s not possible to program an agent to deal with every contingency– Uncertainty– Changing domain– Too complicated

• Agents must often need to adapt to changing environments or learn more about their environment.

Adaptation

• Adaptation involves changing an agent’s model/behavior in response to a perceived change in the world.

• Reactive• Agents don’t anticipate the future, just

update

Learning

• Learning involves constructing and updating a hypothesis

• An agent typically tries to build and improve some representation of the world.

• Proactive• Try to anticipate the future.• Most agents will use both learning and

adaptation.

Agents in e-commerce

• Agents play a fairly limited role in today’s e-commerce.– Mostly still in research labs.

• Large potential both in B2C and B2B– Assisting in personalization– Automating payment

Challenges for agents

• Uniform protocol/language– Web services? XML?

• Lightweight, simple to use, robust.– Always a challenge

• Critical mass– Enough people need to adopt

• “Killer app”– What will the agent e-mail/IM be?