Pat LangleyPat Langley

Institute for the Study of Learning and ExpertiseInstitute for the Study of Learning and ExpertisePalo Alto, CaliforniaPalo Alto, California

andandCenter for the Study of Language and InformationCenter for the Study of Language and Information

Stanford University, Stanford, CaliforniaStanford University, Stanford, California

http://www.isle.org/~langleyhttp://www.isle.org/~langley

langley@csli.stanford.edulangley@csli.stanford.edu

Adaptive User InterfacesAdaptive User Interfacesfor Personalized Servicesfor Personalized Services

As information and choices become more available, users need As information and choices become more available, users need help in finding, and selecting among, the many alternatives.help in finding, and selecting among, the many alternatives.

The Need for Intelligent AssistanceThe Need for Intelligent Assistance

This has led to the development of This has led to the development of recommendation systemsrecommendation systems, , which attempt to locate and recommend relevant items.which attempt to locate and recommend relevant items.

At the same time, society is becoming ever At the same time, society is becoming ever more diversified.more diversified. Differences in private and professional Differences in private and professional preferences are growing.preferences are growing.

Internet users are becoming increasingly Internet users are becoming increasingly selective about what they want to see and selective about what they want to see and purchase.purchase.

We need We need personalizedpersonalized systems that can give systems that can give users the information or product they want. users the information or product they want.

But personalized response requires some But personalized response requires some modelmodel or or profileprofile of the user. of the user.

The Need for Personalized AssistanceThe Need for Personalized Assistance

• manual creation by individual users (e.g., MyYahoo); manual creation by individual users (e.g., MyYahoo);

• manual creation of stereotypes and assignment of users based manual creation of stereotypes and assignment of users based on demographic or behavioral data;on demographic or behavioral data;

• offline learning of stereotypes from demographic/behavioral offline learning of stereotypes from demographic/behavioral data and assigning users to them;data and assigning users to them;

• online learning of individual user models from traces of their online learning of individual user models from traces of their interactions.interactions.

Approaches to User ModelingApproaches to User Modeling

There are four distinct approaches to creating and utilizing user There are four distinct approaches to creating and utilizing user profiles for personalized services:profiles for personalized services:

We will refer to systems of the last sort as We will refer to systems of the last sort as adaptive user adaptive user interfacesinterfaces. .

• Given:Given: a set of tasks that require some user decision a set of tasks that require some user decision

• Given:Given: descriptions for each of these tasks descriptions for each of these tasks

• Given:Given: traces of the user’s decision on each task traces of the user’s decision on each task

• Find:Find: mappings from task features to user decisions mappings from task features to user decisions

The Problem of Learning Individual ModelsThe Problem of Learning Individual Models

There exist two broad approaches to describing the user’s task: There exist two broad approaches to describing the user’s task:

We can state the problem confronting adaptive user interfaces as: We can state the problem confronting adaptive user interfaces as:

Our work focuses on content-based approaches to user modeling.Our work focuses on content-based approaches to user modeling.

• collaborativecollaborative methods refer to other users’ responses to the task methods refer to other users’ responses to the task

• content-basedcontent-based methods refer to measurable features of the task methods refer to measurable features of the task

Examples of Adaptive User InterfacesExamples of Adaptive User Interfaces

Adaptive interfaces have been developed for many different tasks: Adaptive interfaces have been developed for many different tasks:

These efforts cover a wide spectrum but also raise common issues.These efforts cover a wide spectrum but also raise common issues.

• Command and form completionCommand and form completion• Email filtering and filingEmail filtering and filing• News selection and layoutNews selection and layout• Browsing the World Wide WebBrowsing the World Wide Web• Selecting movies and TV showsSelecting movies and TV shows• On-line shoppingOn-line shopping• In-car navigationIn-car navigation• Interactive schedulingInteractive scheduling• Dialogue systemsDialogue systems

Definition of an Adaptive User InterfaceDefinition of an Adaptive User Interface

that reducesuser effort

by acquiringa user model

based on pastuser interaction

a softwareartifact

Definition of a Machine Learning SystemDefinition of a Machine Learning System

that improvestask performance

by acquiringknowledge

based on partialtask experience

a softwareartifact

Steps in Developing an Adaptive InterfaceSteps in Developing an Adaptive Interface

Formulatingthe Problem

Engineering theRepresentation

CollectingUser Traces

Utilizing ModelEffectively

Gaining UserAcceptance

ModelingProcess

Computational route advisors already exist in both rental carsComputational route advisors already exist in both rental carsand on the World Wide Web.and on the World Wide Web.

A decision-making task that confronts drivers can be stated as:A decision-making task that confronts drivers can be stated as:

However, they do not give However, they do not give personalizedpersonalized navigation advice to navigation advice to individual drivers.individual drivers.

• Given: Given: The driver’s current location The driver’s current location C;C;

• Given: Given: The destination The destination DD that the driver desires; that the driver desires;

• Given: Given: Knowledge about available roads (e.g., a digital map); Knowledge about available roads (e.g., a digital map);

• Find: Find: One or more desirable routes from One or more desirable routes from CC to to D.D.

The Task of Route SelectionThe Task of Route Selection

Here is a one approach to learning route preferences, though not Here is a one approach to learning route preferences, though not the first we considered:the first we considered:

This method learns a user model with respect to the This method learns a user model with respect to the entireentire route. route.

• Formulation: Formulation: Learn a “subjective” function to evaluate entire routesLearn a “subjective” function to evaluate entire routes

• Representation: Representation: Global route features computable from digital mapsGlobal route features computable from digital maps

• Data collection: Data collection: Preference of one complete route over anotherPreference of one complete route over another

• Induction: Induction: A method for learning weights from preference dataA method for learning weights from preference data

• Utilizing model: Utilizing model: Apply subjective function to find “optimal” routeApply subjective function to find “optimal” route

An Approach to Route SelectionAn Approach to Route Selection

In this way, it avoids two important problems: In this way, it avoids two important problems: data fragmentationdata fragmentation and and credit assignmentcredit assignment..

The Adaptive Route AdvisorThe Adaptive Route Advisor

We have implemented this approach in the We have implemented this approach in the Adaptive Route Advisor Adaptive Route Advisor (Fiechter, Rogers, & Langley, 1999), which:(Fiechter, Rogers, & Langley, 1999), which:

• models driver preferences in terms of 14 global route featuresmodels driver preferences in terms of 14 global route features

• gives the driver two gives the driver two alternativealternative routes he might take routes he might take

• lets the driver lets the driver refinerefine these choices along route dimensions these choices along route dimensions

• uses driver choices to refine its model of his preferencesuses driver choices to refine its model of his preferences

• invokes the driver model to recommend future routesinvokes the driver model to recommend future routes

Note that providing drivers with choices lets the system collect Note that providing drivers with choices lets the system collect data on route preferences in an unobtrusive manner.data on route preferences in an unobtrusive manner.

The Adaptive Route AdvisorThe Adaptive Route Advisor

The Adaptive Route Advisor represents the driver model as a The Adaptive Route Advisor represents the driver model as a weighted linear combination of route features. weighted linear combination of route features.

Training data: [x0, x1, x2, x3] is better than [y0, y1, y2, y3].Training data: [x0, x1, x2, x3] is better than [y0, y1, y2, y3].

TimeTimeDistanceDistance

IntersectionsIntersectionsTurnsTurns

CostCost

w0w1w2w3

Driver Model and Training CasesDriver Model and Training Cases

The system uses each training pair as constraints on the weights The system uses each training pair as constraints on the weights found during the learning process.found during the learning process.

Experiments with 24 subjects show the Route Advisor improves Experiments with 24 subjects show the Route Advisor improves its predictive ability rapidly with experience.its predictive ability rapidly with experience.

Experimental Results on Route AdviceExperimental Results on Route Advice

Analyses also show that personalized user models produce better Analyses also show that personalized user models produce better results than generalized models, even when given more data. results than generalized models, even when given more data.

Experimental Results on Route AdviceExperimental Results on Route Advice

Many recommendation systems are available on the World Wide Many recommendation systems are available on the World Wide Web, including ones that suggest restaurants.Web, including ones that suggest restaurants.

Aother type of decision that confronts drivers can be stated as:Aother type of decision that confronts drivers can be stated as:

However, they are ill suited for use in the driving environment, However, they are ill suited for use in the driving environment, where visual attention is a limited resource. where visual attention is a limited resource.

• Given: Given: The driver’s current location The driver’s current location C;C;

• Given: Given: Some goal the driver wants to achieve (e.g., eat lunch);Some goal the driver wants to achieve (e.g., eat lunch);

• Given: Given: Knowledge about available nearby destinations; Knowledge about available nearby destinations;

• Find: Find: One or more destinations One or more destinations DD that achieve the goal that achieve the goal..

The Task of Destination SelectionThe Task of Destination Selection

Conversational Destination AdviceConversational Destination Advice

• FormulationFormulation: Learn to predict features user cares about in : Learn to predict features user cares about in itemsitems

• RepresentationRepresentation: Conditions/weights on attributes and values: Conditions/weights on attributes and values• Data collectionData collection: Converse with user to help him make : Converse with user to help him make

decisions, noting whether he accepts or rejects questions and decisions, noting whether he accepts or rejects questions and items items

• InductionInduction: Any supervised induction method: Any supervised induction method• Utilizing modelUtilizing model: Guide the dialogue by selecting informative : Guide the dialogue by selecting informative

questions and suggesting likely valuesquestions and suggesting likely values

One approach to destination recommendation can be cast as: One approach to destination recommendation can be cast as:

This formulation relies on the notion of a This formulation relies on the notion of a conversationalconversational user user interface.interface.

Spoken-language versions of this approach appear well suited Spoken-language versions of this approach appear well suited to the driving environment.to the driving environment.

We implemented this approach in the We implemented this approach in the Adaptive Place AdvisorAdaptive Place Advisor (Langley, Thompson, Elio, & Haddadi, 1999), which:(Langley, Thompson, Elio, & Haddadi, 1999), which:

• carries out spoken conversations to help user refine his goalscarries out spoken conversations to help user refine his goals

• incorporates a dialogue model to constrain this processincorporates a dialogue model to constrain this process

• collects and stores traces of interaction with the usercollects and stores traces of interaction with the user

• personalizes both its questions and recommended itemspersonalizes both its questions and recommended items

Our work to date has focused on recommending restaurants to Our work to date has focused on recommending restaurants to users who want advice about where to eat.users who want advice about where to eat.

The Adaptive Place AdvisorThe Adaptive Place Advisor

We believe this approach to recommendation will work well for We believe this approach to recommendation will work well for drivers, but it also has broader applications.drivers, but it also has broader applications.

Inquirer: Inquirer: Where do you think I should eat tonight? Where do you think I should eat tonight?

Advisor: Advisor: Well, what type of cuisine would you like? Well, what type of cuisine would you like?

Inquirer: Inquirer: What types are there? What types are there?

Advisor: Advisor: Some examples are Chinese, Indian, and Some examples are Chinese, Indian, and Mediterranean. Mediterranean.

Inquirer: Inquirer: Oh, maybe Indian. Oh, maybe Indian.

Advisor: Advisor: What quality rating would you like? What quality rating would you like?

Inquirer: Inquirer: I don't care, as long as it's cheap. I don't care, as long as it's cheap.

Advisor: Advisor: How do you want to pay? How do you want to pay?

Inquirer: Inquirer: No, I think I'd like Chinese instead. No, I think I'd like Chinese instead.

Advisor: Advisor: Okay, we'll switch to Chinese food. What city do you Okay, we'll switch to Chinese food. What city do you prefer? prefer?

Inquirer: Inquirer: How about Berkeley? How about Berkeley?

Advisor: Advisor: I know three cheap Chinese restaurants in Berkeley. I know three cheap Chinese restaurants in Berkeley. One is the Long Life Vegi House on 2129 University One is the Long Life Vegi House on 2129 University Avenue. Does that sound alright?Avenue. Does that sound alright?

Inquirer: Inquirer: Sure, that sounds fine.Sure, that sounds fine.

A Sample Destination DialogueA Sample Destination Dialogue

Speech Acts Per ConversationSpeech Acts Per Conversationwith Place Advisorwith Place Advisor

0

2

4

6

8

10

12

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

dialogue number

num

ber o

f spe

ech

acts

modeling

control

Time Per ConversationTime Per Conversationwith Place Advisorwith Place Advisor

0

50

100

150

200

250

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

dialogue number

tim

e i

n s

with modelling

control

Many automated scheduling systems are used in industry, and Many automated scheduling systems are used in industry, and some interactive schedulers exist.some interactive schedulers exist.

Another complex problem that people encounter can be stated:Another complex problem that people encounter can be stated:

But these systems do not generate But these systems do not generate personalizedpersonalized schedules that schedules that reflect the preferences of individual users.reflect the preferences of individual users.

• Given: Given: A set of tasks or jobs that must be done soon;A set of tasks or jobs that must be done soon;

• Given: Given: A limited set of resources available for these jobs;A limited set of resources available for these jobs;

• Given: Given: Knowledge about constraints on jobs and resources; Knowledge about constraints on jobs and resources;

• Find: Find: One or more schedules that complete the jobs.One or more schedules that complete the jobs.

The Task of Resource SchedulingThe Task of Resource Scheduling

Here is one approach to preferences about learning schedules:Here is one approach to preferences about learning schedules:

This method is similar to that used in the Adaptive Route Advisor.This method is similar to that used in the Adaptive Route Advisor.

• Formulation: Formulation: Learn a utility function to evaluate entire schedulesLearn a utility function to evaluate entire schedules

• Representation: Representation: Global features computable from the scheduleGlobal features computable from the schedule

• Data collection: Data collection: Preference of one complete schedule over anotherPreference of one complete schedule over another

• Induction: Induction: A method for learning weights from preference dataA method for learning weights from preference data

• Utilizing model: Utilizing model: Apply subjective function to find a good scheduleApply subjective function to find a good schedule

An Approach to Personalized SchedulingAn Approach to Personalized Scheduling

However, it assumes search through a space of complete schedules However, it assumes search through a space of complete schedules (a (a repair spacerepair space), which), which needs a way to generate an initial schedule.needs a way to generate an initial schedule.

The INCA SystemThe INCA System

We have developed INCA (Gervasio, Iba, & Langley,1999), an We have developed INCA (Gervasio, Iba, & Langley,1999), an interactive scheduler that:interactive scheduler that:

• retrieves an initial schedule from a personalized case libraryretrieves an initial schedule from a personalized case library

• suggests to the user improved schedules from which to selectsuggests to the user improved schedules from which to select

• lets the user lets the user directdirect search to improve on certain dimensions search to improve on certain dimensions

• collects user choices to refine its personalized utility functioncollects user choices to refine its personalized utility function

• stores solutions in the case base to initialize future schedulesstores solutions in the case base to initialize future schedules

• invokes the user model to recommend future schedule repairsinvokes the user model to recommend future schedule repairs

As before, providing users with choices lets the system collect As before, providing users with choices lets the system collect data on schedule preferences in an unobtrusive manner.data on schedule preferences in an unobtrusive manner.

Experiments with the INCA scheduling system suggest that:Experiments with the INCA scheduling system suggest that:

• it improves its ability to predict user choices over timeit improves its ability to predict user choices over time

• personalized case libraries are more effective than genericpersonalized case libraries are more effective than generic

• its advice provides greater benefit on harder problemsits advice provides greater benefit on harder problems

• linear models give useful predictions even when falselinear models give useful predictions even when false

• more detailed guidance speeds the user-modeling processmore detailed guidance speeds the user-modeling process

These studies (Gervasio et al., 1999) used a mixture of human These studies (Gervasio et al., 1999) used a mixture of human and synthetic subjects.and synthetic subjects.

Experimental Results with INCAExperimental Results with INCA

Experiments with INCA suggest that retrieving personalized Experiments with INCA suggest that retrieving personalized schedules helps users more as task difficulty increases.schedules helps users more as task difficulty increases.

INCA and Task CharacteristicsINCA and Task Characteristics

Personalized BookmarksPersonalized Bookmarks

Personalized Music DeliveryPersonalized Music Delivery

Challenges of Adaptive InterfacesChallenges of Adaptive Interfaces

Adaptive user interfaces have clear attractions but also pose some Adaptive user interfaces have clear attractions but also pose some challenges to developers: challenges to developers:

These challenges overlap with other applications of machine These challenges overlap with other applications of machine learning, but also raise some new issues. learning, but also raise some new issues.

• formulation of user modeling as an induction taskformulation of user modeling as an induction task

• engineering of representation to support learning processengineering of representation to support learning process

• unobtrusive collection of training data from usersunobtrusive collection of training data from users

• effective application of learned user modeleffective application of learned user model

• requirement for some form of requirement for some form of onlineonline learning learning

• necessity for induction from necessity for induction from fewfew training cases training cases

The Promise of New SensorsThe Promise of New Sensors

Adaptive interfaces rely on user traces to drive their modeling Adaptive interfaces rely on user traces to drive their modeling process, so they stand to benefit from developments like: process, so they stand to benefit from developments like:

As such devices become cheap and widespread, they will give us As such devices become cheap and widespread, they will give us new sources of data and support new types of adaptive services.new sources of data and support new types of adaptive services.

• GPS and cell phone locatorsGPS and cell phone locators

• robust software for speech recognitionrobust software for speech recognition

• accurate eye and head trackersaccurate eye and head trackers

• real-time video systemsreal-time video systems

• wearable body sensors (GSR, heart rate)wearable body sensors (GSR, heart rate)

• portable brain-wave sensorsportable brain-wave sensors

Concluding RemarksConcluding Remarks

In summary, adaptive interfaces integrate ideas from machine In summary, adaptive interfaces integrate ideas from machine learning, intelligent agents, and human-computer interaction.learning, intelligent agents, and human-computer interaction.

These adaptive systems promise to change the way we interact These adaptive systems promise to change the way we interact with, and think about, computer software. with, and think about, computer software.

• has many examples already in regular and successful use,has many examples already in regular and successful use,

• but many unexplored niches for research and application,but many unexplored niches for research and application,

• and challenges involving integration rather than new methods.and challenges involving integration rather than new methods.

This approach to automated personalization of online This approach to automated personalization of online services: services:

• records of keystrokes and mouse clicks records of keystrokes and mouse clicks

• click streams from Web browsersclick streams from Web browsers

• outcomes of electronic purchasesoutcomes of electronic purchases

• user responses to presented optionsuser responses to presented options

Behavior TracesBehavior Traces

An important source of data comes from An important source of data comes from tracestraces of user activity. of user activity.

Collection of user traces can occur during the normal course of Collection of user traces can occur during the normal course of interacting with a system or service. interacting with a system or service.

Examples of such trace data include: Examples of such trace data include:

Thus, this approach is less obtrusive than either questionairres or Thus, this approach is less obtrusive than either questionairres or rating forms.rating forms.

Elements of an Adaptive InterfaceElements of an Adaptive Interface

human userhuman user user modeluser model

adaptiveadaptivealgorithmalgorithm

useruserinterfaceinterface

Elements of Machine LearningElements of Machine Learning

environment knowledge

learningalgorithm

performanceelement

Five Paradigms for Machine LearningFive Paradigms for Machine Learning

RuleRuleInductionInduction

Decision-TreeDecision-TreeInductionInduction

Case-BasedCase-BasedLearningLearning

NeuralNeuralNetworksNetworks

ProbabilisticProbabilisticLearningLearning

Problem FormulationProblem Formulation

The first hurdle of an adaptive interface developer can be stated:The first hurdle of an adaptive interface developer can be stated:

• Given:Given: Some task that an intelligent assistant could aid; Some task that an intelligent assistant could aid;

• Find:Find: Some formulation that lets the assistant improve its Some formulation that lets the assistant improve its performance by learning a user model from experience.performance by learning a user model from experience.

Since most robust learning methods focus on supervised learning, Since most robust learning methods focus on supervised learning, most adaptive interfaces formulate the task in these terms.most adaptive interfaces formulate the task in these terms.

This decision includes making clear design choices about: This decision includes making clear design choices about:

• the aspect of user behavior to be predicted;the aspect of user behavior to be predicted;

• the level of description (what constitutes a training case).the level of description (what constitutes a training case).

Representation EngineeringRepresentation Engineering

Another stage in developing an adaptive interface can be stated:Another stage in developing an adaptive interface can be stated:

• Given:Given: A formulation of some task in machine learning terms; A formulation of some task in machine learning terms;

• Find:Find: Some representation for behavior and user models that Some representation for behavior and user models that makes the learning task tractable.makes the learning task tractable.

Since most robust learning methods assume an attribute-value Since most robust learning methods assume an attribute-value formalism, most adaptive interfaces take this approach.formalism, most adaptive interfaces take this approach.

This decision includes making clear design choices about: This decision includes making clear design choices about:

• the information to be used when predicting behavior;the information to be used when predicting behavior;

• the internal encoding of that information in the system.the internal encoding of that information in the system.

Collecting User TracesCollecting User Traces

A third step in designing an adaptive interface can be posed as:A third step in designing an adaptive interface can be posed as:

• Given:Given: A problem formulation in terms of machine learning A problem formulation in terms of machine learning and a representation of user behavior; and a representation of user behavior;

• Find:Find: An effective way to collect traces of this behavior. An effective way to collect traces of this behavior.

Since people seldom like extra burdens, an ideal adaptive interface Since people seldom like extra burdens, an ideal adaptive interface requires no extra user effort to collect such traces. requires no extra user effort to collect such traces.

This decision includes making clear design choices about: This decision includes making clear design choices about:

• how to transform these traces into training instances;how to transform these traces into training instances;

• what action the user must take to generate the traces.what action the user must take to generate the traces.

Using the Learned ModelUsing the Learned Model

Another essential step in the development process can be stated:Another essential step in the development process can be stated:

• Given:Given: An approach to learning a user model for some task; An approach to learning a user model for some task;

• Find:Find: Some way to invoke the model that helps the user Some way to invoke the model that helps the user perform the task more effectively.perform the task more effectively.

The ideal adaptive interface lets the user take advantage of good The ideal adaptive interface lets the user take advantage of good predictions and ignore bad ones. predictions and ignore bad ones.

This decision includes making clear design choices about: This decision includes making clear design choices about:

• when and how to present the model’s predictions to user;when and how to present the model’s predictions to user;

• how to handle cases in which these predictions are wrong.how to handle cases in which these predictions are wrong.

Gaining User AcceptanceGaining User Acceptance

A final important facet of the development process can be stated:A final important facet of the development process can be stated:

• Given:Given: A complete adaptive user interface for some task; A complete adaptive user interface for some task;

• Find:Find: Ways to get people to try the system and to become Ways to get people to try the system and to become long-term users.long-term users.

However, a system that is well-designed and easy to use is more However, a system that is well-designed and easy to use is more likely to retain users over long periods.likely to retain users over long periods.

Attracting first-time users involves marketing much more than Attracting first-time users involves marketing much more than technology, but, without it, a good system may be ignored. technology, but, without it, a good system may be ignored.

Training case: Sched[x1, x2, x3] is better than Sched[y1, y2, y3].Training case: Sched[x1, x2, x3] is better than Sched[y1, y2, y3].

Updating the User ModelUpdating the User Model

The system determines weights from such data in the same way The system determines weights from such data in the same way as does the Adaptive Route Advisor.as does the Adaptive Route Advisor.

SpilledSpilled

BurnedBurned

Health RiskHealth Risk

UtilityUtilityw1w1w2w2w3w3

INCA represents its user model as a weighted linear combination INCA represents its user model as a weighted linear combination of schedule features (three for the HazMat domain). of schedule features (three for the HazMat domain).

Setting for an Adaptive InterfaceSetting for an Adaptive Interface

Task / Task / Content / ProfilesContent / Profiles

Adaptive InterfaceAdaptive Interface User / CustomerUser / Customer

Adaptive Interfaces as Psychological ModelsAdaptive Interfaces as Psychological Models

One can view adaptive interfaces as automatically creating One can view adaptive interfaces as automatically creating cognitive simulations, in that they:cognitive simulations, in that they:

• develop knowledge structures to describe user preferencesdevelop knowledge structures to describe user preferences

• make explicit predictions about the user’s future behaviormake explicit predictions about the user’s future behavior

• explain individual differences through personalizationexplain individual differences through personalization

But we can distinguish two approaches to cognitive simulation:But we can distinguish two approaches to cognitive simulation:

Both should play a role in HCI, but content models seem more Both should play a role in HCI, but content models seem more relevant for the adaptive aspects of user interfaces. relevant for the adaptive aspects of user interfaces.

• processprocess models that embody architectural principles models that embody architectural principles

• contentcontent models that describe behavior at the knowledge level models that describe behavior at the knowledge level