Proceedings of the 2007 11th International Conference on Computer Supported Cooperative Work in Design

An Observation Model for the Collaborative Analysis of Real Workplaces

Renata Guanaes Machado', Marcos R. S. Borges', Stephanie Guerlain2 Jose Orlando Gomes'Graduate Program in Informatics, Federal University ofRio de Janeiro, Brazil

rguanaes@ufrj. br, tmborges, joseorlandol@nce. ufrj. br2 Systems and Information Engineering Dept., University of Virginia, Charlottesville, VA, USA

guerlain@avirginia. edu

Abstract

Many approaches to work analysis have beenproposed so as to enhance the requirements elicitationfor systems design. However, systems delivered atdynamic, complex and socio-technical workplaces, havestillfailed at satisfying the needs of users successfully,mainly because they are unable to support activitiesentirely, especially those related with cognition andcollaboration. This paper describes the ObservationConceptual Model, as a complementary approach, toelicit high-quality requirements through systematic fieldstudies about organizations. We also include some ideasfor the elaboration of a collaborative system thatsupports the conceptual model.

Keywords: Ethnography, Cognition, CSCW,Collaboration.

1. Introduction

Traditional requirements elicitation techniques aregrounded on users interviews, where they discuss actualproblems and expectations with the system to bedeveloped. In case of greater uncertainty, prototypingmethods may be included. Some other techniques canbe used, such as organization modeling, use cases,scenarios and task analysis [ 13].

Despite this high variety of techniques fordiscovering the requirements, there is a gap between itselicitation process and the reality of workplaces. Usersoften do not describe their activities entirely, either dueto lack of time, low recall capability, omission, orbecause it is burdensome to articulate their abilities andtacit knowledge. System developers often neglect thecollaborative and cognitive aspects of activities, as wellas have focused to expose requirements formally ratherthan to analyze if they are really appropriated.Furthermore, considerable communication obstacles arecommon due to distinct mental models: users havedomain specific knowledge, whereas developers arefamiliar with requirement methodologies [ 17].

Moreover, such techniques are focused on technicalaspects of systems, its data and operations, rather thantrying to understand the organization domain and

activities, so that to uncover how these can be bettersupported by the systems. Consequently, a great deal ofeffort and time is spent with gathering and validation ofrequirements throughout the system design process.

Instead of asking users to describe what they do, itmay be more effective to apply a work-centeredperspective that aims to capture the actual activitiesbeing carried out in the context of work settings,together with their aspects of social interactions,collaboration and cognition, so as to derive moreauthentic and complete system requirements.We propose an Observation Conceptual Model that

provides organized field studies for complexorganizations. Taking into account that these fieldstudies should be performed by groups, a preliminarysketch of a collaborative system supporting this modelis included, so as to facilitate the coordination ofobservation activities in the group, as well as to enhancethe collaboration, interaction and communication amongthe participants. Finally, it may empower the collectiveelaboration of representations about the workplaceinvestigated, translating them into system requirements.

Cognitive and ethnographic approaches are brieflydiscussed in Section 2, so as to introduce theObservation Conceptual Model in Section 3, andSection 4 suggests a case study to evaluate it. The lastsection concludes the paper.

2. Emerging approaches

There have been some undertakings toward thedevelopment of more adequate methods to address therequirements elicitation, such as HCI, Ergonomics,Human Factors and Cognitive approaches. Two of theseare briefly introduced, as they are adapted for use in ourproposed approach.

2.1. Cognitive methods

Any analysis of work, no matter if at management oroperational level, usually involves both physical andcognitive elements [1], the latter being more and morepredominant. Thus, cognitive approaches are concernedwith understanding topics related to human thinking andknowledge, such as reasoning and judgment processes,situation awareness, decision making and the like, fairly

1-4244-0963-2/07/$25.00 ©2007 IEEE.

292

exposed in activities of planning, execution, problemdetection, diagnosis and resolution.CWA (Cognitive Work Analysis) is an example of a

work-centered conceptual framework that deliverscomplex information systems requirements [18]. Itinvestigates the workplace and its adaptations directedat coping with complexity, along with the variables thatshape human-information interaction [5].

According to Figure 1, CWA presents distinctdimension levels, each one containing a body ofvariables for in-depth analysis. These levels arecontrasted to the "divide and conquer" paradigm, sincethe overall problem is not decomposed into smallercomponents, but the organization is analyzedsimultaneously through different perspectives.

The actual Work domainwork env i- analysisruonmento irltermnsof\ ~~m eanis-en-dsnl i s tr u ct re Activity ana ys

nt s of nTask situton

of work and m iaWl rk

> / *~~~~terms Ineackisingn

{ / __~~~~~~thatcn be used

Organizational

in termns of divfisionl t />

o rg anrizati o n (

CharacteristicEr_ nr_, - --Ergonornic anatiyrisRemoving actions alte ves- of perception-actionDefining behavior shaping con. capabilitiesstraints at progressively narro11envelopes

Figure 1: Dimensions of analysis [5]

The focus is not to describe variables that affecthuman-information interaction, but the discovery anddeliberation about factors that delineate and influencethat interaction. Thus, the socio-technical workplace inits contextual environment is evaluated first, andafterwards, requirements for systems investigated in thiscontext are developed. It differs from systemsengineering processes, in which the general activitiescan be abbreviated to the specification, design,implementation first, and the validation with users later.

Despite its visible contributions in opening up newways of thinking about workplace analysis, CWArequires experts, intensive field studies and does notprovide guidance, nor computer-based tools to supportits implementation in practice. More important, it doesnot afford a systematic process for translating theanalysis results into system requirements.

The last problem is treated in ACWA (AppliedCognitive Work Analysis) approach, an adaptedmethodology for decision support systems design [4]. Itproposes a structured process that starts with domainanalysis through knowledge elicitation and ends up withdesign requirements.

According to Figure 2, displayed only to give anoverview of its steps, ACWA begins at decomposingthe work domain in a Functional Abstraction Network.Being similar with a mental model of an expert, itdescribes the goals and the functional means available

would provide the information already processed, asopposed to raw data that would have been interpreted bythe human decision maker. Therefore, the systemintuitively adapts itself to human cognition,complementing its decision making.

The second step refers to cognitive demands that areneeded for each component of the domain model,corresponding to recognition, reasoning and decisionmaking for problem solving. The third step identifiesthe information required for the resolution of eachcognitive demand. Finally, the last two steps are tospecify design requirements associated with informationrepresentation and implementation.

Figure 2: Process of analysis [4]

2.2. Ethnography

Ethnography methodology, derived from SocialAnthropology, provides detailed descriptions of humanactivity in primitive societies, along with its behavior,social interactions, techniques, and cultural practices. Itresults from an observer, participant or not, situated inthe environment for very prolonged periods. Similarly,bringing real aspects of workplace through naturalisticfield studies is a means of discovering actual workingknowledge and practices: how people interact withsystems and technology; how they cooperate withothers; how they use cognition to cope with complex orunanticipated situations, detecting and solving problemsas they happen. Moreover, communication problemsbetween developers and users may be reduced, sinceobserving what they do allow better acknowledging oftheir point of view.

The use of ethnographic methods in systems designstarted with Suchman [15], so as to consider the socialaspects and situated actions belonged to workplaces.Since then, there have been many studies reportingactual working practices in different domains. Thus,ethnography has gained considerable importance as atechnique for informing requirements for complexorganizations, and also for CSCW systems because ofits social dimension.

for achieving them. Once implemented in a system, it

293

However, ethnography has raised challengingquestions. It asks for various observers and for a verylong time to observe and even to analyze, so it isdifficult to practice with tight schedules and budgets ofmost system design. Further, it is oriented towarddetailed portrayal of the situation in an extensive,unstructured textual format that does not fit thecharacteristics of abstraction and graphical notations ofsystem developers. Perhaps the most important issue isto move explicitly from ethnography to requirements.

Faster methods have already been proposed as quickand dirty, concurrent and evaluative ethnographystudies [9], as well as rapid ethnography [12]. The lattercan be done through limited scope of observation, betterselection of key informants, multidisciplinary teams andbetter data analysis tools such as cognitive maps andstorytelling techniques.

As a strive to structure the results of field studies,Designer's NotePad is a hypertext tool created tosupport the representation of abstract models andsoftware designs, once grounded in ethnographicfindings, through multiple viewpoints [16]. Allinformation stored can be reorganized flexibly to avariety of interested parties, each one most focused insome attributes of work setting. Later work suggested a

framework for organizing the results around thedimensions: distributed coordination; plans andprocedures; and awareness of work [8]. Finally, it ledrecently to Coherence method, which integratesethnographic analysis into UML notations [19].

2.3 Approaches summary

Table 1 summarizes the strengths and weaknesses ofthe emerging approaches discussed here. Cognitive andethnographic methods can be combined to provide amultidisciplinary and integrated approach for workanalysis, so as to deliver more accurate requirements.

However, cognitive approaches require guidance forconducting field studies and domain knowledgeelicitation, while ethnography is still an untried methodin the wider community of system engineering. Moresignificantly, the intrinsic analysis activities amongobservers during either a cognitive or ethnographicstudy are not mentioned, so as to explain how theycome themselves to the findings. Further, we believethat adding collaboration research into our proposedmethodology holds the promise to improve productivityof field studies.

Table 1. Outline features of the some approaches to requirement elicitation in system design

Approach General CharacteristicsPrompt deliberation about users

problems and needs.

Work investigation through multi-level perspectives analysis.

Domain analysis for specification ofhuman decision making process.

Naturalistic observation ofworkplaces.

StrengthsMore simple and straightforwardmethod, easier to be performed.

Holistic approach to unfold aspectsdelineating human-information

interaction.

Organized process since analysis torequirements including guidance.Unfold contextualized social,

cognitive and collaboration aspects.

WeaknessesSocial and cognitive aspects missed,

resulting in failed requirements.

Intensive field studies; lack of aprocess to its implementation.

Intensive field studies; lack ofguidance to elicit domain knowledge.Unsystematic, lengthy and expensive

method; textual format.

3. Observation Conceptual Model

3.1 Overview of the model

Approaches focused on technical features of workmiss social and cognitive aspects, like situated action[15] and distributed cognition [10].Thus, our proposal isan OCM (Observation Conceptual Model) frameworkthat combines both methods discussed and some ideasof CSCW research [11], so as to support effectively theethnographic analysis in multidisciplinary groups. Thismodel is primarily concerned with supporting thepractical efforts of observing and analyzing people'sreal activities and performance in the context ofworkplaces, so as to capture those missing aspects.

As shown in Figure 3, OCM is composed of a multi-level Dimension of Analysis, and implements a

systematic Process of Analysis, in which collected andanalyzed field data according to the dimensions maydeliver work representations and system requirements.

From Dimensions of Analysis, OCM specifies "whatto observe" and "how to observe". The former refers tothe most relevant work domain variables that must becollected from the field, so as to focus observation as

suggested in [12]. The latter describes possible ways tocapture those variables, such as through audio andvideo equipment or by multiple observers situated in theworkplace, supported by manual or technologicalappliances as field notes, portable computers andobservation templates (prompted forms for checkingdata instead ofmaking transcription manually).

Once activities data are stored on a central repositoryof heterogeneous and multimedia data, the model alsodescribes "how to analyze", driving the teamworkendeavors for collective elaboration of sketches, modelsand other representations about the work setting. Moresignificantly, it can be done through controlled andcollaborative virtual sessions of ideas brainstorming,debating, deliberation, and negotiation of conflictingissues. People from workplaces investigated, being.

294

CWAapproach

ACWAapproach

Ethnography

im ensions of Analysis Analysis Process

( adap >ted1>fro CWA (aape fro AWA)

Figure 3: Components of Observation Conceptual Model

observed or not, are strongly recommended toparticipate in the session, confirming the analysis orsharing domain knowledge not yet captured

The Process of Analysis involves creatingsucceeding representations until system requirements,and for each of its steps the collaborative processeshappens repeatedly. Moreover, observation is notlimited to initial analysis, but may occur in all phases ofsystems development whenever an ongoingconfirmation or a deeper analysis is needed.

Finally, after ending the collaborative sessions, "howto represent" refers to viewing the analysis results atdifferent viewpoints through computational interfaces.Based on [16], each viewpoint addresses a particularaspect of the work setting, so as to allow navigationthrough the most relevant information, according to theuser preferences, and also traceability to requirements.Both activities of analyzing and representing resultsshould be supported by groupware, so as to stimulatecollaboration and cooperative work analysis.OCM is applied exclusively for organizations with

information systems already implemented, so as not tochange dramatically its actual activities.

3.2 CSCW elements in the model

Due to intense division of labor, it is evident thatpeople do not work alone, but in multi-skilled, oftengeographically dispersed teams. Thus, besidesobserving people working together, the observationitself should also be performed with groupware,considering it supports teams working cooperativelyand distributed over many locations. Moreover, the

emphasis is using the computer not only to solveproblems, but to facilitate human interaction [3].

Therefore, as a part of the OCM, we briefly discusshere some premises related to CSCW: collaboration,cooperation, coordination, awareness andcommunication. They are assumed within the model soas to facilitate the execution of field studies bymultidisciplinary groups.

Any work within a group provides better resultswhen there is a harmonic interaction among itsmembers [14]. Consequently, a virtual collaborativespace is required so as to afford a trusting, fellowshipkind of environment. Members give information abouttheir profile, such as personal data, formal education,previous experience and skills in software engineering,ergonomics, cognitive approaches and the like.

Coordination of observation activities is essential forplanning individual tasks of each member, so as toimprove team productivity and also to allow for laterchecking, review and further analysis. Withoutcoordination, there is the risk that participants wouldget involved in conflicting or repetitive task [2].

Thus, coordination is achieved through definition ofroles and responsibilities, displayed in Table 2, thatcould be possibly allocated to the members during thecourse of an ethnographic study. Multiple roles may beallocated to a single person; new roles may be createdwhenever required.

Cooperation is achieved through enabling peoplecollecting, analyzing and delivering representations ofworkplace together through coordination of the roleswithin collaborative sessions of brainstorming,debating, deliberation, and negotiation.

295

Table 2. Coordination of Roles and responsibilities in the Observation Conceptual Model

Role Responsibilities

Implements tools and analytical techniques in a methods database, such as observation templates, considering that they maybe adapted for analysis of different workplace domains.Similar to a team manager responsible for: settling upon Dimensions of Analysis and defining roles, tasks and deadlines for

Moderator each member. All of these decisions depend on the scope of investigation of the workplace environment. Further, itestablishes discussions, controls interactions and manages conflicting issues during collaborative sessions.Similar to someone that has greater experience in ethnographic and cognitive studies, so as to choose which tools and

Expert techniques to apply in a specific workplace to be analyzed.Observer Performs an observation, collecting data or accessing field data captured by audio and video.

Analyst Performs an analysis of data collected, delivering representations of the workplace.

Developer Delivers suggestions for possible requirements for system design.

Editor Creates a final version of a representation of the workplace after ending a collaborative session of analysis.

Reader Reads representations under development or finalized, so as to validate or to complement with further details. It isrecommended for those belonging to workplaces, being observed or not.

Thus, these representations express informationgathered from a set of knowledge, ideas and insights ofa group. Multidisciplinary teams, each one with specificknowledge and abilities implements the collaboration.

Workspace awareness is knowledge about others'interaction with a shared workspace [7], including itsactions. Information about who are connected to asession, what are their roles, which artifacts they areworking on and what are their new comments should beavailable. It allows collaborators to perceive the actualcourse of a groups' work, without interrupting anddistracting members for this information.

It is important that each member knows how othermembers' work relate to his/her own contribution [6].Thus, context information should be available for eachrepresentation, such as member name, date, time andcomments made by that person.

Finally, communication among members is requiredif they are geographically dispersed, or even to usemember's time more productively. Generally, it can beachieved by e-mails, chats, video conference and so on.

3.3 Technical aspects

The initial prototype of a groupware tool for theOCM implementation is being built using JSPtechnology, to provide a distributed and interactiveenvironment. It uses PostgreSQL database for datarepository and it is integrated with a workflowmanagement system, in order to create and managemember's tasks, coordinate collaborative work overtime, and to inform others' work status.

After capturing work activities through audio andvideo equipment, the use of Java Media FrameworkAPI whithin the groupware tool enables creation, accessand manipulation of these time-based media, that couldbe updated with textual transcriptions, mark-up andannotation alongside the material made by differentmembers. Consequently, members work alone ortogether with a variety of media: texts, datavisualizations and digital material displayed on screens.

The member responsible for the initial capturecreates a preliminary model of the current worksituation and makes it available to others who canaccess it and interact with other members incollaborative sessions. A final version is elaborated bythe member in the editor role, considering all commentsmade. This leads to a more complete model. A log ofthe interactions is associated with the model, making iteasier to follow the discussion thread.

4. Planning of a case study

Studies of work in several organizations areconducted for six months by groups of graduatestudents, based on manual transcription of unstructuredinterviews. It is a laborious task, often at a ratio of eighthours per hour of tape. Our initial goal is to facilitateinterview-based ethnographic work, addingcollaboration aspects so as to make better use of theirefforts and also to assess the OCM proposed. Learnedlessons will be useful for groupware tool improvements.

For a group of four students, the professor acts as themoderator and determines tasks for them, as depicted inTable 3. Based on existing documentation, Analyst 1 isup to explicit the formal process, indicating its goalsand means according to CWA approach, while Analyst2 links specific words to a glossary of terms, explainingthem in more detail. Observers 1 and 2 can access themso as to prepare themselves for the interview.

During the interview, Observer 1 conducts dialogwhile Observer 2 controls its progress, checking whichquestions are still missing, asking for more clarificationand taking notes. Both can check how the formalprocess differs from real activities to be told by theinterviewers. As already familiarized with workdomain, both Analyst 1 and 2 access the interview,either synchronously or asynchronously. While Analyst1 insert markers at specific moments that areworthwhile to be transcribed, Analyst 2 goes to markedsegments and begins to transcribe them.

296

Table 3. Task allocation for student group and its collaborative aspects

Task Role

Planning the field study

Dim. of AssignedAnalysis to

Moderator

Features of Collaboration

Coordination: Specifies observation tools and templates;Professor Creates individual tasks; and assigns them for members.

Definition of formal process Analyst 1

Definition of a domain Analyst 2vocabulary

ObserverInterview 1, 2

Index interview Analyst 1

Transcription Analyst 2

WorkDomainAnalysis

Student 1

Student 2

Student3, 4

Activity tl A.on,t 1Analysis

5. Conclusion

It is evident that work analysis through field studiesis coming into prominence. Therefore, the ObservationConceptual Model presented here is a complementaryapproach extending and integrating previousethnographic, cognitive and CSCW research. Webelieve it is able to improve shared understanding aboutthe working activities and context, so as to deliver morecomplete system requirements that better support users,expanding its human potential and knowledge.

Although the case study and the construction of agroupware tool is further work under development, wealso think OCM brings a contribution for recognizingthat such cognitive and ethnographic approaches do notaddress the potential of collaboration topics between itswork analysts themselves.

References[1] Clark, A. Being there: Putting brain, body and worldtogether again. MIT Press, London, 1997.[2] Cruz, A. J. A. da, Raposo, A. B., e Magalhaes, L.P.Coordination in Collaborative Environments -A GlobalApproach. Proc. of the 7th International Conference onCSCWD, Rio de Janeiro, 2002, 25-30[3] Ellis, C.A., Gibbs, S.J., and Rein, G.L. Groupware: someissues and experiences. Communications of the ACM, 34, 1,January 1991, 38-58[4] Elm, W.C., Potter, S.S., Gualtieri, J.W., Roth, E.M., andEaster, J.R. Applied cognitive work analysis: a pragmaticmethodology for designing revolutionary cognitiveaffordances. In: Hollnagel, E. (Ed.), Handbook for CognitiveTask Design. Lawrence Erlbaum Associates, London, 2003,357-382[5] Fidel, R., and Pejtersen, A.M. From information behaviorresearch to the design of information systems: the CWAframework. Information Research, 10, 1, Oct. 2004.[6] Fuks, H., and Assis, R. Facilitating perception on virtuallearning based environments. Journal of Systems andInformation Technologies, Vol. 5, no. 1, Edith CowanUniversity, 2001, 93-113.

Collaboration: each member has a task to perform.Cooperation: both members explicit formal domain knowledge

Collaboration: one conducts the dialog while another controls it

3LUUCllL I Collaboration: one indexes while another make the transcription

Student 2 Cooperation: both members creates the same document

[7] Gutwin, C., Roseman, M., and Greenberg, S. A usabilitystudy of awareness widgets in a shared workspace groupwaresystem. Proc. ofCSCW'96, Boston, Massachusetts, Nov. 1996[8] Hughes, J.A., O'Brien, J., Rodden, T. and Rouncefield, M.Designing with Ethnography: A Presentation Framework forDesign. Proc. DIS'97, Amsterdam, ACM Press, 147-159[9] Hughes, J., King, V., Rodden, T. and Andersen, H.,Moving out from the control room: ethnography in systemdesign. Proc. CSCW'94, Chapel Hill, ACM Press, 429-439.[10] Hutchins, E. The technology of team navigation. In:Gallagher J., Kraut R., Egido C. (eds.). Intellectual teamwork:social and technical bases of collaborative work. LawrenceEarlbaum, Hillside, 1990.[11 ] Khoshafian, S., and Buckiewicz, M. Introduction toGroupware, Workflow, and Workgroup Computing. JohnWiley and Sons, 1995[12] Millen, D.R. Rapid ethnography: time deepeningstrategies for HCI field research. Proc. of the conference onDesigning interactive systems: processes, practices, methods,and techniques, NY, 2000, 280-286.[13] Nuseibeh, B. and Easterbrook, S. Requirementsengineering: a roadmap. Proc. ofInternational Conference onSoftware Engineering, 4-11 2000, Limerick, Ireland.[14] Pinheiro, M.K., Lima, J.V., e Borges, M.R.S. Aframework for awareness support in groupware systems.Computers in Industry, 52, 1, 2003, 47-57.[15] Suchman, L. Plans and situated actions: the problem ofhuman-machine communication. Cambridge University Press,Cambridge, 1987.[16] Twidale, M., Rodden, T. and Sommerville, I. TheDesigners Notepad: supporting and understanding cooperativedesign. Proc. ECSCW'93 (Milan, 1993) Kluwer, 93-108.[17] Valenti, S., Panti, M. and Cucchiarelli, A. Overcomingcommunication obstacles in user-analyst interaction forfunctional requirements elicitation. ACM SIGSOFT SoftwareEngineering Notes, vol. 23, not., January, 1998.[18] Vicente, K. (1999). Cognitive work analysis: Towardsafe, productive, and healthy computer-based work. Mahwah,NJ: Erlbaum.[19] Viller, S. and Sommerville, I. Ethnographically informedanalysis for software engineers. Int. J Human ComputerStudies 53, 169-196, 2000.

297