Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7....

11
Articles SUMMER 2015 63 I n today’s world of ubiquitous computing, an abundance of information is constantly at hand wherever you go, whenever you desire. This can make it challenging for users to find the actual information they need in between these vast amounts of resources. Additionally, unrelated information can easily distract users from the original work they set out to do, leading separate activities to intertwine. Purposefully multitasking or not, there is an overhead asso- ciated with managing different parallel activities. The pres- ence of too much information negatively affecting the user’s work is commonly referred to as information overload. Providing contextualized and relevant information to the user can alleviate this problem, hiding irrelevant data while pushing important information to the foreground. However, the question remains what this “context” is composed of. Historically, context is a broadly interpretable term usually encompassing the where, when, who, how, or why of any given situation. Different areas of research are trying to provide contextual- ized information to the user, each approaching the problem from a different angle. Although there are clear overlaps among fields, a discrepancy on focus and terminology is apparent. Personal information-management (PIM) research tries to empower users by providing extensive tool support by Copyright © 2015, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602 Activity-Based Computing: Computational Management of Activities Reflecting Human Intention Jakob E. Bardram, Steven Jeuris, Steven Houben An important research topic in artifi- cial intelligence is automatic sensing and inferencing of contextual informa- tion, which is used to build computer models of the user’s activity. One approach to building such activity- aware systems is the notion of activity- based computing (ABC). ABC is a com- puting paradigm that has been applied in personal information-management applications as well as in ubiquitous, multidevice, and interactive surface computing. ABC has emerged as a response to the traditional application- and file-centered computing paradigm, which is oblivious to a notion of a user’s activity spanning heterogeneous devices, multiple applications, services, and information sources. In this article, we present ABC as an approach to contex- tualize information, and present our research into designing activity-based computing technologies.

Transcript of Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7....

Page 1: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Articles

SUMMER 2015 63

In today’s world of ubiquitous computing, an abundanceof information is constantly at hand wherever you go,whenever you desire. This can make it challenging for

users to find the actual information they need in betweenthese vast amounts of resources. Additionally, unrelatedinformation can easily distract users from the original workthey set out to do, leading separate activities to intertwine.Purposefully multitasking or not, there is an overhead asso-ciated with managing different parallel activities. The pres-ence of too much information negatively affecting the user’swork is commonly referred to as information overload.

Providing contextualized and relevant information to theuser can alleviate this problem, hiding irrelevant data whilepushing important information to the foreground. However,the question remains what this “context” is composed of.Historically, context is a broadly interpretable term usuallyencompassing the where, when, who, how, or why of anygiven situation.

Different areas of research are trying to provide contextual-ized information to the user, each approaching the problemfrom a different angle. Although there are clear overlapsamong fields, a discrepancy on focus and terminology isapparent. Personal information-management (PIM) researchtries to empower users by providing extensive tool support by

Copyright © 2015, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Activity-Based Computing: Computational Management

of Activities Reflecting Human Intention

Jakob E. Bardram, Steven Jeuris, Steven Houben

� An important research topic in artifi-cial intelligence is automatic sensingand inferencing of contextual informa-tion, which is used to build computermodels of the user’s activity. Oneapproach to building such activity-aware systems is the notion of activity-based computing (ABC). ABC is a com-puting paradigm that has been appliedin personal information-managementapplications as well as in ubiquitous,multidevice, and interactive surfacecomputing. ABC has emerged as aresponse to the traditional application-and file-centered computing paradigm,which is oblivious to a notion of a user’sactivity spanning heterogeneous devices,multiple applications, services, andinformation sources. In this article, wepresent ABC as an approach to contex-tualize information, and present ourresearch into designing activity-basedcomputing technologies.

Page 2: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Articles

64 AI MAGAZINE

which to manage and access their information.Human-computer interaction (HCI) follows a moreuser-oriented approach, where the user is usuallyplaced central during system design. Artificial intelli-gence (AI) research focuses on making systemssmarter, making them context-aware and even respon-sive to ongoing activities using activity recognition. Inother words, PIM, HCI, and AI emphasize informa-tion, users, and automation respectively. These differ-ent perspectives all lead to useful insights and solu-tions to what is in essence a common problem.

In this article, we present our approach to contex-tualizing information called activity-based comput-ing (ABC), applying methods and insights gainedfrom all three fields. We reflect back on 10 years ofresearch of this particular approach, providing anoverview of the state of the art of ABC and its rela-tionship to research in AI. ABC defines activity as thecontext associated to human intention, thus alsoincluding the cognitive context. By definition, whatdefines a concrete activity is user specific, since itrelies on which intention is expressed by or is mean-ingful to the user. Although a standardized and com-mon ontology has still not been established, thebasic model of ABC has not changed over the yearsand rather has been extended to explore differentaspects of what constitutes an activity in more detail.We have applied the notion of ABC in system sup-port for many different domains, including personalinformation management in an office setting, mobileand collaborative work in hospitals, wet-lab researchin biology labs, and in software engineering. Thisabundance of empirical evidence has provided uswith a deep insight into the benefits and challengesof applying the concepts and technologies of ABC.We use this as the basis for a discussion of recurringissues in ABC, and how this relates to research in AI.

Context and Human IntentionA common approach to providing contextualizedinformation to the user is context-aware computing.Its traditional definition is intentionally broad:

A system is context-aware if it uses context to providerelevant information or services to the user, where rel-evancy depends on the user’s task. (Dey, 2001, p. 5)

An early criticism against context-aware comput-ing is “there are human aspects of context that can-not be sensed or even inferred by technologicalmeans” (Bellotti and Edwards 2001). A key challengeis detecting human intention — what drives users toperform certain actions or tasks. Context-aware sys-tems cannot be designed always to act correctly onour behalf. To prevent possible conflicts due to incor-rect presumptions of human intention, intelligibilityhas been brought forward as a design principle.

Intelligibility — Context aware systems that seek to actupon what they infer about the context must be ableto represent to their users what they know, how they

know it, and what they are doing about it. (Bellottiand Edwards, 2001, p. 201)

We argue that taking into account human inten-tion is not only important for context-aware systems,but for any system that provides contextualizedinformation, automated or not. Having digital sup-port of users’ activity contexts, aligned with the user’smental model of them, allows for new interactiontechniques otherwise not possible. Although activityrecognition traditionally focused on detecting low-level human actions, more recent work also focuseson capturing goal-oriented stateful activities (Brdicz-ka and Bellotti 2011). The resulting activity aware-ness could allow systems to respond to users’ activi-ties in meaningful ways. For example, by monitoringand detecting so-called activities of daily living (ADL)of patients in a nursing home, early warning signalscan be triggered automatically when irregular behav-ior is detected (Tentori and Favela 2008). WithinABC, human intention and its surrounding contextare part of the main perspective considered duringsystem design.

Activity-Based ComputingA large number of observational studies show thatusers often structure their work within the context ofhigher-level activities in desktop environments(González and Mark 2004; Boardman and Sasse 2004;Bergman, Beyth-Marom, and Nachmias 2006). Usersthus not only reason within the context of activitiesbut sometimes also feel a need to externalize themwhen using current computing systems. Althoughthis can be seen as a necessity due to limitations ofcurrent computing systems, it also has some advan-tages. Manually subdividing work within the contextof activities makes them more explicit, which sup-ports episodic memory during later revisitation(Whittaker 2011).

In the early 1980s Bannon et al. (1983) described avision in which the computer provided “an alterna-tive organization of user commands which preservestheir task specificity.” One of the earliest implemen-tations of this vision was the Rooms system (Hender-son and Card 1986). Activity-based computing (ABC)as an interaction paradigm was originally coined byApple Research (Norman 1999), which makes activi-ties first-class computational objects. Apple neverpublished this or implemented it in any of its solu-tions, but subsequently a number of activity-centeredsystems have been researched, each focusing on dif-ferent application domains and technological aspects.Figure 1 provides an overview of many of these sys-tems, which we will introduce and discuss next.

Our research group has been researching ABC formore than 10 years with a special focus on providingABC support for ubiquitous computing (Christensenand Bardram 2002). The central goal is to provide acomputing platform that allows the user to focus on

Page 3: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Articles

SUMMER 2015 65

Figure 1. A Historical Overview of Activity-Based Computing Systems Since 2003.i 1 i i l O i f i i d C i S Si 2003

UMEA

2003 20092006 2012

ActivityExplorer

TaskTracerActivity Bar

Activity-Based Computing Project

CAAD

Giornata

co-Activity Manager

ReticularSpaces

ActivityDesk

UMEA

ActivityExplorer

TaskTT TracerActivity Bar

Activity-Based Computing Project

CAAD

Giornata

co-Activity Manager

ReticularSpaces

ActivityDesk

higher-level collaborative activities rather than low-level application and data management. We situatethis goal in Mark Weiser’s original ideas of trans-parency and computation as a ubiquitous back-ground resource. In a ubiquitous computing world,where users are using a multitude of heterogeneouscomputing devices, the need for supporting the usersat the activity level becomes essential.

ABC PrinciplesOur research on ABC has been crystallized into sixABC principles (Bardram 2009). These principles aregrounded both in theoretical models of human cog-nition and activity, as well as in empirical researchinvolving the design and evaluation of ABC tech-nologies and applications. Although the principlesthemselves evolved over time from their original def-inition, the core concepts remain unchanged.

Activity-CenteredWork is organized into activities, which are higher-level computational constructs that encapsulate allresources, tools, and communication mechanismsinto one goal-oriented interaction model. By movingaway from classic application-oriented interfaces tomultidevice activity-oriented work spaces, users arepresented with logical units of work combined withthe tools required to perform that work.

Activity MultiplexingBy supporting activity suspension and resumption,users can easily switch between different activitycontexts. Suspending an activity means its state isstored and removed from the active work space,while resuming an activity restores it. This feature

supports parallel activities (multitasking) and inter-ruptions in work.

Activity RoamingActivities are stored in an infrastructure and hencecan be accessed from multiple devices. This allows auser to suspend an activity on one device and resumeit on another, thereby allowing the user to roambetween devices. The context of an ongoing activitycan also be spread across devices, allowing for multi-device interaction on one common activity context.Users are presented with awareness cues andoverviews on the distributed state and accessibility ofthe activities.

Activity Adaptation Activities adapt to the capabilities of the device(s) onwhich they are resumed. Hence, an activity mightlook quite different whether it is resumed on a wall-sized display or on a smartphone. A subset of anactivity’s context can be displayed when it is spreadacross several devices.

Activity SharingBecause activities are distributed, they can also beshared among users. Shared activities can be accessedand modified by all related participants. Accessingactivities simultaneously allows for synchronous col-laborative setups. Alternatively, asynchronousexchange of information is possible when separateusers suspend and resume an activity. By attachingmessages or other objects to the activity, all relatedparticipants are notified of changes, thus providingusers with awareness about what changed and onwho is working on what activity.

Page 4: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Context-AwarenessSince activities are computational constructs thattranscend a single user or device, they need to beaware of their usage context such as location, type ofdevice, amount of users and other factors. Theprocess of detecting, selecting and managing theactivity and its resources is a semi-automatic processinvolving both the users as well as automated sensingand inferencing of context information.

Activity-Based Technologies and Systems

Over the years, a wide range of activity-centered com-puting technologies and applications have beenbuild. In this section we provide an overview of sys-tems that were designed with the ABC principles inmind, or adhere to them to a large extent. We willdiscuss the systems within the particular contextthey were designed for, demonstrating the generaliz-ability of the model.

Desktop SystemsSince the seminal work on Rooms (Henderson andCard 1986) many desktop systems supporting activi-ties have been described. UMEA (Kaptelinin 2003) isa first example of a system that monitors users’behavior within self-defined projects. Activity con-text is built up automatically within the currentlyselected activity, providing the user with an overviewof interaction history.

TaskTracer (Dragunov et al. 2005) and CAAD (Rat-tenbury and Canny 2007) use advanced data collec-tion frameworks to monitor the user’s interactionwith the system, automatically creating activity rep-resentations. Although the work of defining activitiesis automated, TaskTracer still allows for manual con-struction as well.

Activity Explorer (Muller et al. 2004), the ActivityBar (Bardram, Bunde-Pedersen, and Soegaard 2006),and Giornata (Voida, Mynatt, and Edwards 2008) areexamples of systems that fully rely on users to definemeaningful activities. Giornata and the Activity Barreframe the desktop interface for personal computersto be activity-centered, for OS X and Windows XPrespectively. They provide activity-centered manage-ment and sharing of context like windows, files andcontacts by integrating with the traditional desktopoperating system. Activity Explorer (Muller et al.2004) is an example of a multiuser communicationand collaboration tool resembling an email client.Although being an external application which is lessintegrated with the operating system, it was the firstsystem to show that you can meaningfully structurecommunication and collaboration processes withinshared activity abstractions.

Extending our research on the Activity Bar, wehave recently introduced the co-ActivityManager(Houben et al. 2013). The co-ActivityManager is

shown in figure 2 and is an activity-centered desktopsystem supporting three main features; (1) activity-centered computing in dedicated activity workspaces; (2) activity sharing and collaborationthrough shared status updates, sharing of resources,and online communication; and (3) activity man-agement through a dedicated activity task bar. Thegoal of the system is explicitly to integrate commu-nication and collaboration channels into activity-centered computing support.

The co-ActivityManager was deployed for a periodof 14 days in a multidisciplinary software develop-ment team. The study showed that the activity-cen-tered work space supports different individual andcollaborative work configuration practices and thatactivity-centered collaboration is a two-phaseprocess consisting of an activity sharing and peractivity coordination phase. An analysis of the activ-ities created by users showed different granularitiesof goals and time spans used. Participants organizedactivities in three categories: (1) ad hoc activities(such as a to-do), (2) short-term activities (for exam-ple day to day work) and (3) long-term activities (forexample, ongoing collaborative projects). Despitethis new insight into how users appropriate differentgranularities of activities within activity-centeredsystems, it is still unclear how these different activi-ties relate to each other and how they fit into theentire shared life cycle of daily work.

Ubiquitous Computing SystemsABC has also been applied to ubiquitous computingand distributed user interface systems, demonstrat-ing its merit as a context model in a multideviceenvironment. When moving away from a single useror device scenario to a more pervasive environmentmany of the multitasking and interruption problemsare greatly amplified.

Our initial research into activity-based computingtook its outset in the design of a ubiquitous comput-ing infrastructure to support the nomadic, collabo-rative, and time-critical work in hospitals (Chris-tensen and Bardram 2002; Bardram 2009). Thisresearch introduced an activity-based infrastructureconsisting of distributed middleware and specializeduser interfaces optimized for medical work. All serv-ices, applications, and resources related to patientcare are bundled in distributed activities. For exam-ple, medical records, information on the medicineadministered, and medical images for a patient arelinked together in an activity, which is shared acrossclinicians involved in the patient’s treatment andcare. This infrastructure was extended to incorporatelarge interactive displays as depicted in figure 3(Bardram et al. 2009) and automatic activity detec-tion (Doryab, Togelius, and Bardram 2012), resultingin increased activity awareness.

More recently, we have extended this research tocreate support for distributed activities across multi-

Articles

66 AI MAGAZINE

Page 5: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

ple devices in the ReticularSpaces infrastructure(Bardram et al. 2012). ReticularSpaces is an activity-based smart-space system designed to support unifiedinteraction with applications and documents throughReticUI, a novel distributed user interfaces design;management of the complexity of tasks between usersand displays; mobile users in local, remote, ornomadic settings; and collaboration among local andremote users. ReticularSpaces was deployed in a smart-space environment and was exposed to end usersthrough scenario-based evaluation.

The study showed that users found the use of activ-ities intuitive in a multidevice context as it allowedthem to move tasks between devices and collaborat-ing users. However, it also highlighted a number ofopen issues with device-specific visualization of activ-ities and cross-device interaction. While ReticUIduplicated the same user interface on all devices withonly small adaptations, users argued that this adap-tation should go much further and that interfaces aswell as information density should be much moretailored to the type of device.

Interactive Surface and Multidevice InteractionMobile devices, such as smartphones and tablets,have become an intrinsic part of people’s everydaylife. Together with laptops and desktop computers,these devices have become part of a device ecologyin which each device acts as a specialized portal intousers’ personal or shared information space. Theuser-device mapping is quickly changing from beinga one-to-one to a one-to-many or even to a many-to-many relation. However, using multiple devicesintroduces a configuration overhead as users have tomanually reconfigure all devices according to ongo-ing activities. Especially in an environment such asan office (figure 4), where the use of multiple devicesis more common, the process of configuring them incontext of ongoing activities is cumbersome.

The ActivityDesk system shown in figure 5(Houben and Bardram 2013) is our latest research insupport for multidevice management. ActivityDeskis an activity-centered interactive desk that supports

Articles

SUMMER 2015 67

Figure 2. The Interface of Co-ActivityManager.

This interface (Houben et al. 2013) consists of (A) a per activity work space, (B) an activity task bar to visualize activities to the user, (C) anactivity start menu to manage activities and applications, and (D) a collaboration manager to interact and share with contacts. Each con-tact is visualized with an avatar, name, and status field. The interaction menu (E) can be used to share a folder, chat, or share an activity(F).

Startbutton Startbutton Activity Dock

Contacts

Activity Tray

User Name

KurtQTY project-

Jennifer-Reading papers-

Kurt-QTY Project-

John-away-

Anne Marie-Correcting exams-

Peter-Programming-

Presence

Display Filters

Interaction Menu

Current Activity Name

Context Menu

Quick Launch Area

Home Activity button

Avatar

Avatar

share folderchatshare activity

Start Menu

B

C

A

F

D

E

Page 6: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

multidevice configuration work and work-spaceaggregation into a personal ad hoc smart space forknowledge workers. The main goal of ActivityDesk isto reduce the configuration work required to usemultiple devices simultaneously by using an interac-tive desk as an activity-centered configuration space.Through ActivityDesk a shared work space acrossdevices can be set up in a tangible and visible way,after which information is automatically distributedacross all participating devices.

Lessons LearnedBased on our research on ABC and the design of dif-ferent activity-centered systems we are able to derivesome common findings. While ABC has been shownto be particularly advantageous in certain settings,there are still several recurring questions that remainunanswered. In this section we provide an overview

of our main findings so far and the open issues thatwill need to be addressed in future work.

BenefitsOver the many research projects, the core idea ofactivity-centered computing has proven to beextremely robust; significant improvements inhuman-computer interaction are achieved by allow-ing users to organize computational resources intological bundles of activities, which can be subject todistribution, sharing, adaptation, and suspensionand resumption.

Activity AppropriationUsers found computational activities a useful con-struct to organize information and communicationprocesses. They reported being more focused andappreciated the ability to switch quickly between dif-ferent information contexts. In desktop environ-ments, we observed that users generally closed fewer

Articles

68 AI MAGAZINE

Figure 3. The Original ABC Project.

This original project (Bardram 2009) focused on supporting mobile and collaborative workflows in hospitals using activity as an organiza-tional unit to structure patient data. This image shows a nurse engaged in an online activity sharing session (including video) with a radi-ologist while sharing resources like X-ray images on a large interactive display.

Fi 3 ThT O i i l ABC P j t

Page 7: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

resources, since they could simply be hidden byswitching to another activity. Activities within ABCallow for additional ways by which to organize work,on top of traditional organizational strategies. Sys-tems such as co-ActivityManager do not impose a for-mal activity model on users but rather provide themwith an additional activity abstraction that can beused to organize and structure work on their desktop.Activities were created both up front and retrospec-tively as activities evolve over time. Post hoc activitycreation usually occurs when an ongoing activitycontext becomes too large and is split into multipleactivities. Users appropriate activities differently,including differences in intention and duration.Activities were created for short undefined ad hocwork, to long-running collaborative projects.

Long-Term UseThere is a learning curve associated with using activ-ities. Initially most users are reluctant to create toomany activities as they see the effort of doing sogreater than the estimated benefits. However, afterhaving used an ABC system over longer periods oftime and having experienced the advantages firsthand, users start to create activities even for smaller

one-hour tasks. Structuring work within the contextof activities becomes especially interesting oncemore elaborate work needs to be done. All users whowere confronted with multitasking on a daily basispreferred ABC over the traditional approach.

CollaborationUsers reported that the process of constructing andsharing activities with each other helped themreflect on their work and that of their collaborators.Activity management and sharing thus increases theawareness of users about ongoing processes withinthe working context.

Within a collaborative setting, users found thatsharing information and setting up collaborativeprocesses in the context of an activity significantlyhelped them manage ongoing work. Users would, forexample, asynchronously share an activity (which ina desktop configuration contained files and contacts)with other users to create a shared starting point fora collaborative project. After this initial sharingprocess, the scope and definition of the activity canbe changed and appropriated by all users dependingon their role inside the project. Communication andcollaboration channels such as logs, chat windows,

Articles

SUMMER 2015 69

Figure 4. RecticularSpaces.

This figure (Bardram et al. 2012) depicts an activity-based smart-space setup, comprising large wall-based displays, horizontal tabletop dis-plays, laptops, and tablet computers that all run ReticUI, the unified user interface.

Figi ugg rer 4 Rectitt cularSr pS aces

Page 8: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

or shared folders can be included in the activity rep-resentation.

MultideviceWithin a distributed setting where information isspread across several devices, activity-centered com-puting proved to provide additional advantages tousers. A computational unit representing an activityacross devices and locations offers a more consistentmental model to the user. Information is automati-cally distributed across devices within the work con-text it relates to, removing the overhead of having totransfer resources manually. Devices can becomeactivity visualizers that are part of a larger activityecology rather than independent computing entities.Activities can be replicated across all devices to sup-port a synchronized view, but additionally activityresources can also be divided among several devices,allowing for cross-device activity representations.Different roles can be allocated to devices. For exam-ple, in ActivityDesk the desk itself becomes a masteractivity manager that manages the rendering ofactivities and resources on “slave” devices.

Open IssuesOur work with ABC has, however, also left us with arange of open questions, which we would like toaddress — potentially in collaboration withresearchers working in the field of AI. These issues arein particular concerned with the modeling of humanactivity and how activities are managed and handledin daily use.

Human IntentIdeally, computational activities reflect user intent.For example, in a hospital setting, an activity-centeredelectronic medical record would provide support forpatient-related activities, like a “prescribe medicinefor patient Hansen” activity. Such an activity wouldhelp a physician to locate and bring up relevant infor-mation on a public display or on a mobile tablet com-puter. This, however, requires the system to knowwhat is relevant in this activity, and — as argued inthe introduction — in order to prevent possibleannoyances due to incorrect presumptions of humanintention, intelligibility needs to be supported.

During our close collaboration with users (includ-ing hospital clinicians), it is evident that activity-

Articles

70 AI MAGAZINE

Figure 5. The ActivityDesk System.

ActivityDesk (Houben and Bardram 2013) is an activity-based information space for knowledge workers that aggregates different devicesand their containing resources into one interactive desktop space.

Page 9: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

based computing support should avoid the need forusers to provide lengthy descriptions whenever theydefine a new activity. Even the relatively simpleaction of naming an activity when it is created isoften too distracting. Coupling activity with intentneeds to be light weight or should be automated. Forexample, light-weight support could involve suggest-ing activity descriptions based on names of resourcesused during post hoc activity construction. Depend-ing on the scenario (such as a critical hospital envi-ronment) even fully automated activity constructionmight be preferred.

Early work on task-based computing (Sousa andGarlan 2002) actually argued that given more sophis-ticated context monitoring, the less the task-basedcomputing system has to rely on explicit indicationsfrom a user concerning their intentions. Thus intelli-gent context-aware and activity-recognition systemscan help identify and capture user activity and intenton the fly. Early in our research, we experimentedwith contextual triggering of activities (Christensen2002), but this can be extended to have automaticsensing of contextual information, which then isused for semiautomatic generation and maintenanceof the activity model in the computer system.

Activity Life CycleOne particularly challenging part of the ABC princi-ples is the light weightness of activities. Because anactivity carries no semantic, it is often hard to distin-guish one activity from another. One of the recurrentobservations was that one activity merged intoanother — there were no clear demarcations betweenthe ending of one activity and the beginning ofanother. For example, while prescribing medicine toone patient, a physician would look up medical datafor another patient. This was a source for much con-fusion, as, for example, an activity labeled “Prescrip-tion for Mr. Hansen” would end up also displayingmedical data for Mrs. Pedersen. Furthermore, it wasnot always easy to judge when to create a new activ-ity. Should a new activity be created per patient, orper type of activity (such as prescribing medicine)?

Adding semantic information to activities is anobvious challenge for data mining and data process-ing approaches. For example, the linkage between anactivity, its main object (such as a specific patient),and associated data and tools could be maintainedthrough continuous data mining techniques. More-over, semantic technologies would help maintainconsistent semantics across the ABC system.

Organizing and Managing ActivitiesAnother conceptual as well as practical challenge withthe ABC principles concerns scalability; it is hard totell how well the ABC principles would scale based onour current research systems and implementations. Areal-world deployment of an activity-centered elec-tronic medical record system in a modern hospitalwould be required to handle a significant number ofpatients, users, physical artifacts, and real-world activ-

ities. It is by no means straightforward to scale the cur-rent research systems to these numbers. If not care-fully designed, a user would in no time accumulate asignificant number of activities that are being outdat-ed at a fast pace. Hence, automatic tools and methodsfor handling, linking together, and navigating in acomplex web of activities are needed. Furthermore,ways of cleaning up more or less automatically areessential. All of this calls for automated data mining,learning, and pattern-matching techniques.

ConclusionIn this article we presented activity-based computingas an approach to contextualize human-computerinteraction. We have surveyed findings from morethan 10 years of research in ABC systems: how usersperceive computational activities, how they definethem, which ones they find useful, and how theyaffect their work. Our current research includes fur-ther investigation of the activity life cycle, and pro-viding support to organize and manage large collec-tions of shared activities as part of a collaborativeworkflow. In particular, we are exploring how glob-ally distributed software development can benefitfrom ABC’s principles.

Creating contextualized information systems is amultidisciplinary challenge. To this end we are look-ing more into what we can learn from related fields.We hope by providing an overview of ABC and itsopen issues, new cross-disciplinary insights can begained. Empirical results within personal informa-tion management can tell us which PIM tools areused and how they relate to the activity life cycle. Inparticular we see opportunities for the AI communi-ty to automate parts of activity management, possi-bly addressing some of the open issues. However,likewise we argue for areas where explicit userprocesses should be maintained. We suggest a hybridapproach of explicitly defined activities, enhancedby AI to predict and suggest activity operations. Thiscan reduce activity construction costs while simulta-neously supporting episodic memory by makingoperations more explicit.

AcknowledgmentsThe research on activity-based computing has beensupported by several partners, including the DanishCouncil for Strategic Research and the EU MarieCurie Program under grant number 264738 (theiCareNet network).

ReferencesBannon, L.; Cypher, A.; Greenspan, S.; and Monty, M. L.1983. Evaluation and Analysis of Users’ Activity Organiza-tion. In Proceedings of the 1983 ACM SIGCHI Conference onHuman Factors in Computing Systems, 54–57. New York:Association for Computing Machinery.

Bardram, J. E. 2009. Activity-Based Computing for Medical

Articles

SUMMER 2015 71

Page 10: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Norman, D. A. 1999. The Invisible Computer:Why Good Products Can Fail, the PersonalComputer Is So Complex, and InformationAppliances Are the Solution. Cambridge, MA:The MIT Press.

Rattenbury, T., and Canny, J. F. 2007. CAAD:An Automatic Task Support System. In Pro-ceedings of the 2007 ACM SIGCHI Conferenceon Human Factors in Computing Systems,687–696. New York: Association for Com-puting Machinery. dx.doi.org/10.1145/1240624.1240731

Sousa, J. P., and Garlan, D. 2002. Aura: AnArchitectural Framework for User Mobilityin Ubiquitous Computing Environments.In Proceedings of the Third Working IEEE/IFIPInternational Conference on Software Architec-ture, volume 224 of IFIP Conference Pro-ceedings, 29–43. Dordrecht, Netherlands:Kluwer.

Tentori, M., and Favela, J. 2008. Activity-Aware Computing for Healthcare. IEEE Per-vasive Computing 7(2): 51–57.dx.doi.org/10.1109/MPRV.2008.24

Voida, S.; Mynatt, E. D.; and Edwards, W. K.2008. Re-framing the Desktop InterfaceAround the Activities of Knowledge Work.In Proceedings of the ACM SIGCHI Conferenceon User Interface Software Technology, 211–220. New York: Association for ComputingMachinery.

Whittaker, S. 2011. Personal InformationManagement: From Information Consump-tion to Curation. ARIST 45(1): 1–62.

Jakob E. Bardram (itu.dk/people/bardram)is a professor at the IT University of Copen-hagen (ITU) and runs the Pervasive Interac-tion Technology (PIT) Lab. His researchinterests are ubiquitous computer systems,object-oriented software architecture, com-puter-supported cooperative work (CSCW);and human-computer interaction (HCI).The main application area of this research iswithin health care, especially pervasivehealth care.

Steven Jeuris is a Ph.D. student in the PITLab at the IT University of Copenhagen,working on next-generation technologiesfor global software development (NexGSD)where he applies ABC principles through-out the life cycle of software developmentto improve tool integration and knowledgetransfer between coworkers.

Steven Houben is a research associate at theIntel Collaborative Research Institute onSustainable and Connected Cities (ICRI-Cities) and UCL Interaction Centre (UCLIC)working on multidevice interaction tech-niques, physical computing and sensor-based systems..

Work in Hospitals. ACM Transactions on Com-puter-Human Interactaction 16(2) Article 10.

Bardram, J. E.; Bunde-Pedersen, J.; Doryab,A.; and Srensen, S. 2009. Clinical Surfaces:Activity-Based Computing for DistributedMulti-Display Environments in Hospitals.In Proceedings of the Conference on HumanComputer Interaction, volume 5727 of Lec-ture Notes in Computer Science, 704–717.Berlin: Springer.

Bardram, J. E.; Bunde-Pedersen, J.; and Soe-gaard, M. 2006. Support for Activity-BasedComputing in a Personal Computing Oper-ating System. In Proceedings of the 2006 ACMSIGCHI Conference on Human Factors in Com-puting Systems, 211–220. New York: Associa-tion for Computing Machinery.

Bardram, J. E.; Gueddana, S.; Houben, S.;and Nielsen, S. 2012. ReticularSpaces: Activ-ity-Based Computing Support for Physical-ly Distributed and Collaborative SmartSpaces. In Proceedings of the 2012 ACMSIGCHI Conference on Human Factors in Com-puting Systems, 2845–2854. New York: Asso-ciation for Computing Machinery. dx.doi.org/10.1145/2207676.2208689

Bellotti, V., and Edwards, K. 2001. Intelligi-bility and Accountability: Human Consider-ations in Context-Aware Systems. Human-Computer Interaction 16(2–4): 193–212.dx.doi.org/10.1207/S15327051HCI16234_05

Bergman, O.; Beyth-Marom, R.; and Nach-mias, R. 2006. The Project FragmentationProblem in Personal Information Manage-ment. In Proceedings of the 2006 ACMSIGCHI Conference on Human Factors in Com-puting Systems, 271–274. New York: Associa-tion for Computing Machinery.

Boardman, R., and Sasse, M. A. 2004. StuffGoes into the Computer and Doesn’t ComeOut: A Cross-Tool Study of Personal Infor-mation Management. In Proceedings of the2004 ACM SIGCHI Conference on Human Fac-tors in Computing Systems, 583–590. NewYork: Association for Computing Machin-ery. dx.doi.org/10.1145/985692.985766

Brdiczka, O., and Bellotti, V. 2011. Identify-ing Routine and Telltale Activity Patterns inKnowledge Work. In Proceedings of the IEEE2011 Conference on Semantic Computing, 95–101. Piscataway, NJ: Institute for Electricaland Electronics Engineers.

Christensen, H. B. 2002. Using Logic Pro-gramming to Detect Activities in PervasiveHealthcare. In Proceedings of the 2002 Inter-national Conference on Logic Programming,volume 2401 of Lecture Notes in ComputerScience, 421–436. Berlin: Springer. dx.doi.org/10.1007/3-540-45809-3_8

Christensen, H. B., and Bardram, J. E. 2002.Supporting Human Activities — ExploringActivity-Centered Computing. In Proceed-ings of the 2002 International Conference onUbiquitous Computing, volume 2498 of Lec-

ture Notes in Computer Science, 107–116.Berlin: Springer.

Dey, A. K. 2001. Understanding and UsingContext. Personal Ubiquitous Computing5(1):4–7. dx.doi.org/10.1007/s007790170019

Doryab, A.; Togelius, J.; and Bardram, J.2012. Activity-Aware Recommendation forCollaborative Work in Operating Rooms. InProceedings of the 2012 ACM InternationalConference on Intelligent User Interfaces, 301–304. New York: Association for ComputingMachinery. dx.doi.org/10.1145/2166966.2167023

Dragunov, A. N.; Dietterich, T. G.; John-srude, K.; McLaughlin, M.; Li, L.; and Her-locker, J. L. 2005. TaskTracer: A DesktopEnvironment to Support Multi-TaskingKnowledge Workers. In Proceedings of the10th International Conference on IntelligentUser Interfaces, 75–82. New York: Associa-tion for Computing Machinery.

González, V. M., and Mark, G. 2004. Con-stant, Constant, Multi-Tasking Craziness:Managing Multiple Working Spheres. InProceedings of the 2004 ACM SIGCHI Confer-ence on Human Factors in Computing Systems,113–120. New York: Association for Com-puting Machinery. dx.doi.org/10.1145/985692.985707

Henderson, Jr., D. A., and Card, S. 1986.Rooms: The Use of Multiple Virtual Work-spaces to Reduce Space Contention in aWindow-Based Graphical User Interface.ACM Transactions on Graphics 5(3): 211–243.dx.doi.org/10.1145/24054.24056

Houben, S., and Bardram, J. E. 2013. Activi-tyDesk: Multi-Device Configuration WorkUsing an Interactive Desk. In CHI 2013Extended Abstracts on Human Factors in Com-puting Systems, 721–726. New York: Associa-tion for Computing Machinery.

Houben, S.; Bardram, J. E.; Vermeulen, J.;Luyten, K.; and Coninx, K. 2013. Activity-Centric Support for Ad-Hoc KnowledgeWork: A Case Study of Co-Activity Manager.In Proceedings of the 2013 ACM SIGCHI Con-ference on Human Factors in Computing Sys-tems, 2263–2272. New York: Association forComputing Machinery.

Kaptelinin, V. 2003. UMEA: TranslatingInteraction Histories into Project Contexts.In Proceedings of the 2003 ACM SIGCHI Con-ference on Human Factors in Computing Sys-tems, 353–360. New York: Association forComputing Machinery.

Muller, M. J.; Geyer, W.; Brownholtz, B.;Wilcox, E.; and Millen, D. R. 2004. One-hundred Days in an Activity-Centric Col-laboration Environment Based on SharedObjects. In Proceedings of the 2004 ACMSIGCHI Conference on Human Factors in Com-puting Systems, 375–382. New York: Associa-tion for Computing Machinery. dx.doi.org/10.1145/985692.985740

Articles

72 AI MAGAZINE

Page 11: Activity-Based Computing: Computational Management of …bardram/pdf/aaaimagazine15.pdf · 2015. 7. 26. · ciated with managing different parallel activities. The pres-ence of too

Reproduced with permission of the copyright owner. Further reproduction prohibited withoutpermission.