Intelligent Systems Lecture 2 - Intelligent Agents - TIME.mk

Intelligent Systems

Lecture 2 - Intelligent Agents

In which we discuss the nature of agents, the diversity of

environments, and the resulting menagerie of agent types.

Dr. Igor Trajkovski

Dr. Igor Trajkovski 1

Outline

♦ Agents and environments

♦ Rationality

♦ PEAS (Performance measure, Environment, Actuators, Sensors)

♦ Environment types

♦ Agent types


Agents and environments

?

agent

percepts

sensors

actions

environment

actuators

Agents include humans, robots, softbots, thermostats, etc.

The agent function maps from percept histories to actions:

f : P∗ → A

The agent program runs on the physical architecture to produce f


Vacuum-cleaner world

A B

Percepts: location and contents, e.g., [A, Dirty]

Actions: Left, Right, Suck, NoOp


A vacuum-cleaner agent

Percept sequence Action[A, Clean] Right

[A, Dirty] Suck

[B, Clean] Left

[B, Dirty] Suck

[A, Clean], [A, Clean] Right

[A, Clean], [A, Dirty] Suck... ...

function Reflex-Vacuum-Agent( [location,status]) returns an action

if status = Dirty then return Suck

else if location = A then return Right

else if location = B then return Left

What is the right function?Can it be implemented in a small agent program?


Rationality

Fixed performance measure evaluates the environment sequence– one point per square cleaned up in time T ?– one point per clean square per time step, minus one per move?– penalize for > k dirty squares?

A rational agent chooses whichever action maximizes the expected value ofthe performance measure given the percept sequence to date

Rational 6= all-knowing– percepts may not supply all relevant information

Rational 6= foreseeing the future– action outcomes may not be as expected

Hence, rational 6= successful

Rational ⇒ exploration, learning, autonomy


Rationality (cont.)

What is rational at any given time depends on four things:

• The performance measure that defines the criterion of success.

• The agent’s prior knowledge of the environment.

• The actions that the agent can perform.

• The agent’s percept sequence to date.


PEAS

To design a rational agent, we must specify the task environment

Consider, e.g., the task of designing an automated taxi:

Performance measure??

Environment??

Actuators??

Sensors??


PEAS

To design a rational agent, we must specify the task environment

Consider, e.g., the task of designing an automated taxi:

Performance measure?? safety, destination, profits, legality, comfort, . . .

Environment?? US streets/freeways, traffic, pedestrians, weather, . . .

Actuators?? steering, accelerator, brake, horn, speaker/display, . . .

Sensors?? video, accelerometers, gauges, engine sensors, keyboard, GPS, . . .


Internet shopping agent

Performance measure??

Environment??

Actuators??

Sensors??


Internet shopping agent

Performance measure?? price, quality, appropriateness, efficiency

Environment?? current and future WWW sites, vendors, shippers

Actuators?? display to user, follow URL, fill in form

Sensors?? HTML pages (text, graphics, scripts)


Environment types

The range of task environments that might arise in AI is obviously vast.We can, however, identify a fairly small number of dimensions along whichtask environments can be categorized.

• Fully observable vs. partially observable

• Deterministic vs. stochastic

• Episodic vs. sequential

• Static vs. dynamic

• Discrete vs. continuous.

• Single agent vs. multiagent.


Environment types (cont.)

Solitaire Backgammon Internet shopping TaxiObservable??Deterministic??Episodic??Static??Discrete??Single-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic??Episodic??Static??Discrete??Single-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic?? Yes No Partly NoEpisodic??Static??Discrete??Single-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic?? Yes No Partly NoEpisodic?? No No No NoStatic??Discrete??Single-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic?? Yes No Partly NoEpisodic?? No No No NoStatic?? Yes Semi Semi NoDiscrete??Single-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic?? Yes No Partly NoEpisodic?? No No No NoStatic?? Yes Semi Semi NoDiscrete?? Yes Yes Yes NoSingle-agent??


Environment types

Solitaire Backgammon Internet shopping TaxiObservable?? Yes Yes No NoDeterministic?? Yes No Partly NoEpisodic?? No No No NoStatic?? Yes Semi Semi NoDiscrete?? Yes Yes Yes NoSingle-agent?? Yes No Yes (except auctions) No

The environment type largely determines the agent design

The real world is (of course) partially observable, stochastic, sequential,dynamic, continuous, multi-agent


Agent types

The job of AI is to design the agent program that implements the agentfunction mapping percepts to actions.

We assume this program will run on some sort of computing device withphysical sensors and actuators-we call this the architecture:

agent = architecture + program


Table driven agents

function TABLE-DRIVEN-AGENT( percept) returns actionstatic: percepts, a sequence, initially empty

table, a table, indexed by percept sequences, initially fully specified

append percept to the end of perceptsaction LOOKUP( percepts, table)return action

• Consider the automated taxi: the visual input from a single camera comesin at the rate of roughly 27 megabytes per second (30 frames per second,640 x 480 pixels with 24 bits of color information). This gives a lookuptable with over 10250,000,000,000 entries for an hour’s driving.

• Even the lookup table for chess - a tiny, well-behaved fragment of thereal world-would have at least 10150 entries.


Table driven agents - problems

The daunting size of these tables (the number of atoms in the observableuniverse is less than 1080) means that

• no physical agent in this universe will have the space to store the table

• the designer would not have time to create the table

• no agent could ever learn all the right table entries from its experience

• even if the environment is simple enough to yield a feasible table size, thedesigner still has no guidance about how to fill in the table entries


Agent types

Four basic types in order of increasing generality:– simple reflex agents– reflex agents with state– goal-based agents– utility-based agents

All these can be turned into learning agents


Simple reflex agents

Agent

En

viron

men

t

Sensors

What the worldis like now

What action Ishould do nowCondition−action rules

Actuators


Example


if status = Dirty then return Suck

else if location = A then return Right

else if location = B then return Left

or

if car-in-front-is-braking then initiate-braking


Reflex agents with state

Agent

En

viron

men

t

Sensors

What action Ishould do now

State

How the world evolves

What my actions do

Condition−action rules

Actuators



Example


static: last A, last B, numbers, initially ∞

if status = Dirty then . . .


Goal-based agents

Agent

En

viron

men

t

Sensors

What it will be like if I do action A


State


What my actions do

Goals

Actuators



Utility-based agents

Agent

En

viron

men

t

Sensors

What it will be like if I do action A

How happy I will be in such a state


State


What my actions do

Utility

Actuators



Learning agents

Performance standard

Agent

En

viron

men

tSensors

Performance element

changes

knowledgelearning goals

Problem generator

feedback

Learning element

Critic

Actuators


Summary

Agents interact with environments through actuators and sensors

The agent function describes what the agent does in all circumstances

The performance measure evaluates the environment sequence

A perfectly rational agent maximizes expected performance

Agent programs implement (some) agent functions

PEAS descriptions define task environments

Environments are categorized along several dimensions:observable? deterministic? episodic? static? discrete? single-agent?

Several basic agent architectures exist:reflex, reflex with state, goal-based, utility-based


Intelligent Systems Lecture 2 - Intelligent Agents - TIME.mk

Documents

Transcript of Intelligent Systems Lecture 2 - Intelligent Agents - TIME.mk