The New Equipment is Here, Now Comes the Hard Part: …1) taming the complexity associated with...

ARL-TR-8322 ● MAR 2018

US Army Research Laboratory

The New Equipment is Here, Now Comes the Hard Part: Cognitive and Sociotechnical Challenges in Network-Enabled Mission Command by John K Hawley and Michael W Swehla Approved for public release; distribution is unlimited.

NOTICES

Disclaimers

The findings in this report are not to be construed as an official Department of the Army position unless so designated by other authorized documents. Citation of manufacturer’s or trade names does not constitute an official endorsement or approval of the use thereof. Destroy this report when it is no longer needed. Do not return it to the originator.

ARL-TR-8322 ● MAR 2018

US Army Research Laboratory

The New Equipment is Here, Now Comes the Hard Part: Cognitive and Sociotechnical Challenges in Network-Enabled Mission Command by John K Hawley Human Research and Engineering Directorate, ARL Michael W Swehla US Army Mission Command Center of Excellence Approved for public release; distribution is unlimited.

ii

REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE (DD-MM-YYYY)

March 2018 2. REPORT TYPE

Technical Report 3. DATES COVERED (From - To)

October 2013–September 2017 4. TITLE AND SUBTITLE

The New Equipment is Here, Now Comes the Hard Part: Cognitive and Sociotechnical Challenges in Network-Enabled Mission Command

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

John K Hawley and Michael W Swehla 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

US Army Research Laboratory ATTN: RDRL-HRF-D Aberdeen Proving Ground, MD 21005-5425

8. PERFORMING ORGANIZATION REPORT NUMBER

ARL-TR-8322

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

10. SPONSOR/MONITOR'S ACRONYM(S)

11. SPONSOR/MONITOR'S REPORT NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution is unlimited.

13. SUPPLEMENTARY NOTES 14. ABSTRACT

The distinguished cognitive scientist and design consultant Donald Norman writes that managing complexity is a partnership: Designers have to produce things that tame complexity, and users have to take the time to learn the skills associated with their use. This report describes the consequences of failure to manage complexity during the introduction of new mission-command equipment suites to tactical US Army units. The challenge facing developers and users of such systems is to address Norman’s twin objectives of 1) taming the complexity associated with equipment design and 2) properly preparing individuals, crews, teams, and units to employ those complex systems. Anticipated levels of system performance can be achieved, but only if complexity is managed and modernized systems are “understandable, sensible, and meaningful” to their user populations. That is where the “hard part” in this report’s title comes into play. This research looks at current Army modernization efforts through the lens of sociotechnical systems theory. A sociotechnical system is a work setting consisting of people in interaction with a technology suite intended to accomplish a specific organizational function. In the present case, that work setting is a command post and the organizational function is network-enabled mission command. 15. SUBJECT TERMS

human–systems integration, HSI, Network Integration Evaluations, NIEs, sociotechnical systems, cognitive load, mission command, network-enabled operations, macrocognitive work system

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT

UU

18. NUMBER OF PAGES

40

19a. NAME OF RESPONSIBLE PERSON

John K Hawley a. REPORT

Unclassified b. ABSTRACT

Unclassified

c. THIS PAGE

Unclassified

19b. TELEPHONE NUMBER (Include area code)

915-568-2896 Standard Form 298 (Rev. 8/98)

Prescribed by ANSI Std. Z39.18

Approved for public release; distribution is unlimited. iii

Contents

List of Figures iv

1. Introduction 1

2. Background 1

3. Methodology 3

3.1 A Framework for Characterizing Cognitive Load in Network-Enabled CPs 4

3.2 Observations on Cognitive Load in NIE CPs 7

3.2.1 Mission-Command System Functionality and Ergonomics 10

3.2.2 Mission-Command Component Integration 11

3.2.3 Training, Practice, and Experience 12

3.2.4 Design to Tame Complexity 17

3.2.5 Training on, Experience with, Network-Enabled Mission-Command Systems 19

3.3 Perspectives on HSI at System-of-Systems and Unit Levels 22

4. Conclusion 23

5. References 29

List of Symbols, Abbreviations, and Acronyms 32

Distribution List 34

Approved for public release; distribution is unlimited. iv

List of Figures

Fig. 1 Conceptual model of macrocognitive work .......................................... 6

Approved for public release; distribution is unlimited. 1

1. Introduction

Inside the front cover of his book Living with Complexity, the distinguished cognitive scientist and design consultant Donald Norman writes that managing complexity is a partnership:

Designers have to produce things that tame complexity. But we too have to do our part: we have to take the time to learn the structure and practice the skills. This is how we mastered reading and writing, driving a car, and playing sports, and this is how we can master our complex tools [Norman 2011].

The 3 issues raised in Norman’s theme statement—complexity, system design, and training/learning—serve to frame the issues discussed in the current report. As Norman asserts, when considering new tools—such as those used in network-enabled mission command—it is necessary to manage their complexity. The discussion to follow highlights the consequences of failure to manage complexity during the introduction of new and more complex mission-command equipment suites and supporting network gear to tactical US Army units. The challenge facing developers and users of such systems is to address Norman’s twin objectives of 1) taming the complexity associated with equipment design and 2) properly preparing individuals, crews, teams, and units to use those complex systems. Contrary to much popular opinion, Norman asserts that complexity sometimes is necessary; apparent simplicity can be misleading. Complex work often requires complex tools. Anticipated levels of system performance can be achieved, but only if complexity is managed and those modernized systems are “understandable, sensible, and meaningful” to their user populations. That is where the “hard part” in the title of this report comes into play.*

2. Background

For the past 5 years (2013–2017), a team led by personnel from the US Army Research Laboratory (ARL) Human Research and Engineering Directorate’s (HRED) Field Element at Fort Bliss, Texas, has provided human–systems integration (HSI) support to the Army’s Brigade Modernization Command (BMC) for the Network Integration Evaluations (NIEs). The NIEs are a series of semiannual, brigade-level exercises intended to integrate and mature the Army’s

* This ARL technical report is an expanded version of a paper originally presented in November 2017 at

the International Command and Control Research and Technology Symposium in Los Angeles, California.


tactical networks in an operational context. The broad scope of the NIEs permits human-performance effects and HSI issues to be assessed at the individual Soldier–system level as well as at the system-of-systems and organizational levels.

The initial focus of ARL’s HSI support to the BMC was the cognitive load associated with network-enabled mission command. Excessive cognitive load in modernizing command posts (CPs) was an expressed concern of the then-Chief of Staff of the Army. Simply stated, cognitive load is defined as the aggregate mental load placed on multi-echelon commanders and battle-staff personnel by an increasingly complex mission-command work setting. In this respect, network-enabled CPs were treated as complex sociotechnical systems of systems embedded within a multi-echelon unit context. A sociotechnical system is a work system consisting of people in interaction with a technology suite intended to accomplish a specific organizational function. In the case of CPs, that organizational function is mission command.

An additional opportunity provided by the NIEs was the ability to address nonmateriel (i.e., Doctrine, Organization, Training, Leadership and education, and Personnel [DOTLP]) adaptations necessary to make effective use of the systems and technologies underpinning network-enabled mission command. Previous force-modernization research, along with results from the ARL team’s NIE support work, suggested robust nonmateriel adaptations are on a par with materiel as contributors to enhanced mission-command performance and unit effectiveness (Gonzales et al. 2005). Unfortunately, nonmateriel developments and modifications often take a back seat to equipment-related concerns during the development, testing, and fielding of modernizing systems. Moreover, tactical doctrine—the manner in which the new systems might be most effectively used—often lags well behind the edge of technical advances (Holley 2004). Units receiving new systems are often left mostly “on their own” to determine how to best use that equipment to meet mission objectives. This lack of emphasis on nonmateriel issues early on can cause misjudgments of the potential military utility of new systems and technology as well as the nature of the DOTLP package required to adequately support those systems in tactical organizations.

Based on results across NIEs, ARL’s HSI team recommended mitigation in 3 broad areas affecting CP and mission-command complexity and associated cognitive load: 1) mission-command system functionality and ergonomics, 2) component integration, and 3) individual and team-oriented battle-staff training. Mission-command component integration was further divided into interoperability effects and operational integration challenges. Operational integration refers to incorporation of new materiel solutions into CP and mission-command processes and procedures as well as adapting existing processes and procedures to reflect the


capabilities provided by new technologies. New technologies often change the nature of the work processes they are intended to support (Wickens et al. 2013). Individual Soldiers, functional teams, commanders, and supporting staff elements must adapt their operations to reflect the new capabilities they now possess. Much of the discussion to follow is taken from more detailed descriptions of the ARL HSI support team’s activities across NIEs provided in Hawley (2014, 2015, and Hawley and Swehla (2016).

3. Methodology

Data relevant to the topics noted above were obtained from a variety of sources. These included

• field observations in CPs during NIE operations

• field interviews and focus-group sessions with commanders and brigade- and battalion-level battle-staff members

• discussions with network-support personnel such as unit S-6 (Signal/Communications) Soldiers and contractor field service representatives (FSRs)

• discussions with supporting exercise observer/analysts

• reviews of NIE database entries

During field observations in CPs, ARL’s HSI analysts were accompanied by a mission-command subject matter expert (SME) provided by the Army Training and Doctrine Command’s Mission Command Center of Excellence (MCCoE). This SME was an experienced military analyst familiar with CP operations, mission-command procedures, the Military Decision Making Process (MDMP), and NIE equipment and objectives. He assisted ARL personnel in 1) gaining access to unit CPs, 2) making essential introductions to unit battle staff and network-support personnel, 3) understanding what was transpiring as mission-command operations were observed, and 4) focusing follow-on interviews on critical aspects of cognitive load in mission command and network (i.e., S-6) operations. ARL HSI analysts also used this SME after the fact to assist in making sense of and clarifying observations, conclusions, and recommendations. During and after NIE 16.1, ARL’s HSI team was augmented by Warfighter Information Network–Tactical (WIN–T) SMEs from that system’s prime contractor, General Dynamics Mission Systems. WIN–T is the communications “backbone” for network-enabled operations. These SMEs were a source of background information on the network and its capabilities, and also served as interface between the ARL team and network


FSRs supporting the exercise. The WIN–T SMEs were retired field-grade military officers with extensive command and staff experience. This operational experience coupled with their in-depth knowledge of mission-command systems such as Command Post of the Future (CPOF) and the enabling network (WIN–T) made them invaluable team members.

The investigative approach used by ARL’s HSI team is an example of what has come to be termed ethnographic research (Anderson 2009). Ethnographic research stresses interacting with system users in their natural environment while observing and listening in a nondirected way. Such field-centered work also focuses extensively on post hoc analyses of critical incidents (Flanagan 1954). The goal is to “see” and understand users’ behavior from their point of view and not through the lens of any particular technical domain and with a minimum of preconceived notions. Ethnographic research applied to the investigation of cognitive issues in technology-dominated work systems such as a CP is consistent with the views expressed in Hutchins (1995, p. xiii). Hutchins argues that “human cognition is always situated in a complex [socio/technical/cultural] world and cannot be unaffected by it”. Functional systems of systems such as a CP composed of Soldier teams in interaction with a tool suite display cognitive properties that are radically different from the properties of those individuals acting alone or in isolation. What is necessary in such cases is to develop an understanding of naturally situated cognition in which the unit (of measure) of cognitive analysis is work as it is performed by a functional team operating in its natural operational setting. Hutchins refers to the study of naturally situated cognition as cognition in the wild.

3.1 A Framework for Characterizing Cognitive Load in Network-Enabled CPs

As noted, cognitive load refers to the aggregate mental load placed on commanders and battle-staff members by an increasingly complex mission-command work setting. Kantowitz (1987, p. 97) further characterizes mental load as “a subjective experience caused by … motivation, ability, expectations, training, timing, stress, fatigue, and circumstances in addition to the number, type, and difficulty of tasks performed, effort expended, and success in meeting requirements”. As the term is used here, cognitive load is related but not identical to the more familiar concept of cognitive workload. Commanders and battle-staff members likely do not experience excessive cognitive workload in the same sense that a console operator might. That is, commanders and their staffs typically are not overwhelmed by the sheer number of discrete physical or cognitive actions required to perform their duties—as might be the case with a mission-command information system (MCIS) operator. That is not to say that exercising mission command using information


from multiple and often complex equipment sources is not a cognitively challenging requirement for commanders and supporting battle-staff members. Engaging in the sensemaking and mental model-development activities required to establish and maintain appropriate situation awareness is a cognitively challenging activity, although not demanding in the same sense as with a MCIS operator. The distinction sometimes used to differentiate these classes of performances is macrocognitive versus microcognitive work (Hoffman and Best 2012). In keeping with this distinction, the primary focus of the discussion to follow is macrocognitive work as performed in modernizing tactical CPs. To a considerable extent, command and staff work performed in tactical operations centers is macrocognitive work.

In order to better understand the individual Soldier- and team-performance dynamics underlying and contributing to cognitive load and ensuing mission-command performance, consider the model of macrocognitive work presented in Fig. 1. Elements of this model are adapted from Hoffman and Best (2012). Their model of macrocognitive work comprises 4 quadrants or related sets of processes. The upper-left quadrant, Quadrant 1, is labeled Sensemaking in the World, which is the key cognitive-performance requirement underlying effective mission command. This activity involves observing the world and making actionable inferences about it. The intent of Sensemaking in the World is to determine how best to act upon that world to achieve one’s objectives. Hoffman and Best refer to the “act upon” process as Flexecution with the World (Quadrant 3). Flexecution is short for “flexible execution.” Satisfactorily acting on the world cannot always be achieved following a rigid, unthinking set of procedures—rote drills. Critical thinking and problem-solving skills, applied within the context of a dynamic mental model of the tactical situation, are necessary to act flexibly as opposed to acting rigidly to achieve one’s objectives. Flexible execution depends on these skills and often requires suitable training and extensive on-the-job experience with relevant feedback.

Sensemaking in the World (Quadrant 1) is mediated, in whole or in part, by the available technology suite: sensors, computational systems, information displays, and so forth. Contemporary mission command is technology-mediated cognitive work. Commanders and their staffs are required to understand and make sense of the technology suite available to them to aid job performance. Hoffman and Best refer to this activity as Sensemaking in the Technology (Quadrant 2). Trust in supporting technology is a product of the interplay of Sensemaking in the World and Sensemaking with (or in) the Technology. If the technology used to support Sensemaking in the World is “clunky” and unreliable or if users simply are unfamiliar with it, trust in technology will suffer. In a worst-case situation, commanders and battle-staff members will simply refuse to use technology they do


not understand or trust and fall back on more proven methods. Falling back on older, more proven tools and methods in stressful situations was frequently observed and documented across the NIEs.

Fig. 1 Conceptual model of macrocognitive work (after Hoffman and Best 2012)

Flexible execution of one’s actions in the observed/controlled world (Quadrant 3) depends on the user’s ability to flexecute with the technology (Quadrant 4). Reliance on (or willingness to use) that technology suite results from successful interplay between flexecution of the work in the world and the user’s ability to flexibly execute using that technology. Command and staff personnel must be able to realize their intentions using the technology suite available to them. The ability to execute flexibly with the available technology suite is a function of that suite’s design (both individual systems and those systems in the aggregate) along with users’ familiarity with that technology used in a representative mission setting. Also note that the ability to flexibly execute with the technology represents a performance “step beyond” simply being trained on how to operate discrete elements of that technology suite. This performance capability also requires time and role-relevant experience with feedback to develop. Routine Army New Equipment Training (NET) emphasizing equipment operation versus proficient use of that equipment in the target job environment is not sufficient to enable flexible execution with new technology (Hawley 2015).

When performing macrocognitive work as in mission command, users have to devote time and effort to making sense of their technology as well as making sense of the observed or controlled world. They have to learn how to use that technology;


they have to understand what the technology does and does not do. Users then have to learn to flexibly execute using that technology suite. They often have to cope with its shortcomings and awkwardness and create workarounds. If a user has to abruptly shift attention from the quadrant of Sensemaking in the World and focus instead on trying to make sense of the supporting technology suite, user attention and effort are shifted away from primary mission goals and performance can be impacted. Macrocognitive work will suffer due to such distractions and associated cognitive demands (Koopman and Hoffman 2003).

NIE observations, interview results, and database entries frequently referred to distractions resulting from unreliable equipment or being unfamiliar with its use. Difficulties or disruptions within or across any of the 4 quadrants comprising the Hoffman–Best model of macrocognitive work increase perceived system complexity and cognitive load and, potentially, adversely impact mission-command performance. As discussed in the next section, that is exactly what ARL’s HSI support team repeatedly observed across multiple NIEs.

3.2 Observations on Cognitive Load in NIE CPs

The ARL HSI support team’s objective across NIEs was to characterize the nature of cognitive load within the contemporary mission-command performance environment by addressing the following questions:

1) Do commanders, battle-staff members, or mission-command system operators perceive cognitive load to be a problem?

2) Which aspects of the contemporary mission-command performance setting appear to be driving cognitive load?

3) Going forward, what are some potential solutions to the problem of growing cognitive load on commanders and battle-staff members?

Results across NIEs as they apply to these questions are now discussed in turn (in this section and Section 3.2.3).

Question 1: Do commanders, battle-staff members, or other mission-command equipment operators perceive cognitive load to be a problem?

The short answer to this initial question is “Yes”. High levels of cognitive load were reported to be an issue for some levels of command, some battle-staff members, and some MCIS users. Interview results and observations across NIEs indicated that cognitive load was most often a problem during high-operational-tempo events or when a unit was surprised and quick action was necessary. The term most often used by NIE participants to describe cognitive load was “information overload”.


As one company commander put it: “We have too much information to be processed in the time allowed. It’s too much for one person to handle. Sometimes, I don’t know what I’m missing.” The remark that “It’s too much for one person to handle” was supported by multiple database entries indicating company-level commanders preferred command vehicles having more than one mission-command workstation. Having more than one workstation permitted them to share the information processing load among several personnel. A frequent, associated comment was that commanders did not want to be “tied” to their computer screens and keyboards. They frequently opined that being tied to mission-command equipment interfered with effective command. Several commanders commented favorably on the idea of using their executive officer (XO) as an intermediary between them and mission-command-related information sources—referred to as the “Digital XO” concept. In their view, this permitted them to command their unit effectively while still staying abreast of relevant information provided by mission-command support systems. In essence, the Digital XO served as a “cognitive filter” or “router” for information flowing to and from the commander. This informal adaptation in response to perceived information overload has been tried out but not formally or rigorously evaluated. For example, there is likely to be an increased coordination burden associated with the Digital XO concept.

The level of command that appeared to be most impacted by excessive cognitive load was the company/battery/troop level and below. Mission-command equipment suites at these levels provide considerable information that must be processed, assimilated, and acted upon by command and staff personnel. There also are considerable demands for information to be provided to upper command echelons to maintain their situation awareness. These information processing requirements place a significant load on company-level (and below) command personnel. Lower-level command echelons typically do not have formal staff elements like those found at battalion or brigade. Company-level commanders often adapted to excessive information-processing demands by forming ad hoc company-level staff elements—when their command vehicles permitted this adaptation—or by relegating the bulk of their information processing requirements to their XO or a senior noncommissioned officer.

At the battalion and brigade levels the staff member who appeared to be most impacted by excessive information-processing requirements and the resulting cognitive load was the battle captain. The battle captain is the integrating and coordinating agent for information flowing into and out of the CP. The battle captain in a brigade or battalion CP is a nondoctrinal position for which there currently is no formal training. This can be a very demanding role, particularly for an inexperienced officer. During an after-action interview, an admittedly


inexperienced battle captain from a field artillery (FA) battalion remarked that an FA battalion-level battle captain has to integrate fires-related information from 13 (his count) different informational sources. The captain said he was not familiar with the individual strengths and weaknesses of each of these informational sources and often had to rely on the individual system operators in the CP to provide necessary clarifications—if time permitted. This interviewee also noted he was not fully aware of role expectations for a battle captain in an FA battalion. He indicated he had received no specific orientation, training, or practice getting ready for this critical role. The battalion XO and more experienced battle-staff members provided mentoring and guidance when opportunity and the tactical situation allowed, but that level of intermittent performance support was not judged to be sufficient. There was also some question whether adequate facilities existed at home station to train command and staff personnel and teams to handle the rising demands of contemporary mission command outside a large-scale field exercise like an NIE. The FA battle captain stated he would like to have attended an orientation course or session to better prepare himself for the role of battle captain. In essence, this interviewee was saying he had no effective mental model to guide his activities in this key role. He had to develop his mental model “on the fly” over the course of the event. An appropriately detailed and role-related mental model is crucial to the sensemaking activities (in Quadrant 1 of Fig. 1).

ARL’s HSI support team noted a similar pattern in other CPs. Many of the battle captains observed and interviewed were rather junior (one was a second lieutenant with less than a year in service), inexperienced in and unfamiliar with CP staff operations or the equipment used to support those operations, and upon questioning did not appear to fully understand their role requirements or how to carry them out. As discussed later, training and on-the-job experience are key factors in managing perceived complexity and cognitive load. The NIEs are a good setting for providing essential training and experience for command and staff personnel. However, one must be cautious making inferences about equipment and concept suitability based on results obtained using incompletely trained and inexperienced personnel. Training, equipment testing, and concept evaluation often present conflicting objectives (Hawley 2007).

Question 2. Which aspects of contemporary mission command appear to be driving cognitive complexity and load?

The previous paragraphs make a reasonable case for the conclusion that excessive cognitive load is an emerging HSI and human-performance issue in contemporary mission command. Question 2 now addresses the sources of the problem. To answer that question, it is necessary to accept the proposition that contemporary mission command is an intrinsically complex activity. Network-enabled mission-


command equipment suites provide a wealth of information to be processed, comprehended (made sense of), and acted upon by commanders and battle-staff personnel. Moreover, a large and growing number of often-complex individual systems are supplying this information. Effective information management can be a serious problem.

As noted previously, cognitive load is defined as the aggregate mental load placed on the battle staff or other CP personnel by an increasingly complex mission-command work setting. From an HSI perspective, Cognitive Load Theory (CLT) can be used as a framework to guide the development and evaluation of work settings such as a CP that efficiently uses people’s limited cognitive processing capacity to support effective job performance. A central tenet of CLT is the notion humans’ working memory architecture and its limitations should be a major consideration when designing or evaluating a macrocognitive work system (Plass et al. 2010), which is what an Army CP is.

CLT distinguishes between 2 types of cognitive load: intrinsic and extraneous. Intrinsic cognitive load is primarily determined by the nature of the job being performed. Intrinsic load is high when job performance requires a large number of interactions involving a number of workplace components. Extraneous cognitive load is the additional performance load beyond the intrinsic level, primarily resulting from poorly designed or integrated components along with inadequate levels of job-performer expertise. A high level of extraneous load is argued to interfere with effective job performance. Because intrinsic load and extraneous load are considered additive from an overall cognitive-load perspective, it is important the total performance load associated with a work setting should not exceed individual or team capabilities. The cognitive-load “drivers” most amendable to short-term reduction or elimination fall into the extraneous load category. As Norman (2011) asserts, “good” component design, appropriate component integration, and adequate preparation of job incumbents are the keys to managing complexity and mitigating extraneous cognitive load in complex macrocognitive work systems. Although he was not, Norman could have been describing the inherent challenges in modernizing Army CPs.

ARL’s HSI team identified 3 primary contributors to extraneous cognitive load in CPs as observed across NIEs.

3.2.1 Mission-Command System Functionality and Ergonomics

Many of the individual systems used to support mission command in NIE CPs were neither user friendly nor sufficiently reliable. Moreover, the components composing the CP were developed and evaluated mostly in isolation and often by


different proponents and vendors. Their relationship with other CP components was not always considered and neither was their overall assembly to form the CP based on an understanding of complex cognitive work in context. Consequently, the pieces of the CP “puzzle” did not always fit together smoothly or comprehensively to support mission command as an integrated warfighting function. Battle-staff members were often required to compensate for both component design inadequacies vis-à-vis role requirements and functionality shortfalls or gaps. These deficiencies contributed to perceived system-of-systems complexity and drove extraneous cognitive load.

Conventional HSI assessments focus mostly on answering the question, “From an ergonomic perspective, did we build the system right?” A question that is not typically asked is, “Did we build the right system?” That is, does this mission-command system, when considered as part of the CP, readily support mission command as integrated cognitive work? In the case of CPs observed across NIEs, the answer to both of these questions was “Not always.”

3.2.2 Mission-Command Component Integration

Many of the individual systems within CPs were not suitably integrated to support mission command as cognitive work. When used within the context of a discussion of complexity and cognitive load, the following aspects of integration must be addressed:

Physical integration primarily refers to mission-command-component connectivity and interoperability. Do data flow as they should? Does this data flow facilitate effective information exchange across mission-command systems and command echelons? From a cognitive-load perspective, the most important aspect of physical integration is component interoperability. Suitable component interoperability is one of the foundations of effective mission command. Interoperability shortfalls, such as the inability to share maps or map overlays across mission-command systems or command echelons, were common across NIE CPs.

Operational integration involves the incorporation of new mission-command materiel solutions into battle-staff processes and procedures. It has been observed that new technology often changes the nature of the work that technology is intended to support. Operational integration is the organization’s necessary response to such work changes. ARL’s HSI support team observed that operational integration was not adequately addressed in NIE CPs. This shortfall often meant that new mission-command systems were used in the same manner as older systems. Units did not adapt work practices to reflect their new capabilities. Consequently, units often failed to “get the most out of” those new capabilities. A


common criticism of new mission-command systems was that users thought they had to “work harder” using the new systems to achieve the same results obtained using older and often simpler systems. The potential benefits of modernized systems was not always apparent.

Effective physical and operational integration help users make sense of information transmitted via technical connections, understand the implications of that information, and respond appropriately. Effective component integration, interoperability, and information management are the foundations of effective mission-command performance. Observations across NIEs indicated that inadequate physical integration coupled with a failure to address the critical issue of operational integration contributed to perceptions of CP complexity and staff cognitive load. Referring to Fig. 1, these integration shortfalls routinely interfered with Sensemaking in the Technology (Quadrant 2) and with users’ ability to successfully flexicute using that technology (Quadrant 4).

3.2.3 Training, Practice, and Experience

Many of the personnel using mission-command systems across NIEs had not been adequately trained on them individually or as a systems of systems forming the CP. For example, across NIEs the most common response of battle-staff members when asked if they had received formal training on their mission-command workstation (most often CPOF) was “No.” Most battle-staff members reported they had learned to use the system on their own during the start-up portion of the exercise. Moreover, battle-staff personnel considered as a team had not been provided sufficient time to become familiar with the equipment suites used to support mission command as an integrated warfighting function. As illustrated in Fig. 1, trust in and reliance on technology emerge from familiarity and positive experiences with that technology. The level of expertise required to effectively use mission-command support technology cannot be developed as part of the Army’s traditional “drive-by” approach to NET or a short, follow-on collective training program within a receiving unit. Observations and interview results indicated that hands-on experience gained across NIEs with equipment items and equipment suites mattered significantly. Also, there were numerous remarks in NIE databases concerning “lack of trust” in the equipment provided to NIE participants. There were suggestions that some of this lack of trust was derived from lack of equipment familiarity. NIE participants simply had not had time to become comfortable with mission-command equipment or equipment suites and conversant with their potential uses.

Results across NIEs indicated the 3 factors cited in Sections 3.2.1–3.2.3 (functionality and ergonomics, component integration, and training and experience)


combined and acted to increase the aggregate level of perceived complexity and cognitive load on CP personnel. The mission-command role in modernizing CPs itself is intrinsically complex and demanding. However, a work setting with a large number of design-related “rough edges” will give the impression of being more complex and intimidating than one that has been better designed and integrated. While some of the cognitive load associated with mission command in NIE CPs is intrinsic to battle-staff roles, high reported levels of extraneous cognitive load were needless consequences of insufficient attention to HSI in individual component design and integration coupled with inadequate training for individual system users and for battle staffs operating as a team. As used here, the term “inadequate training” refers to training that is 1) too short to produce necessary levels of Soldier competence, 2) ill-focused in the sense that training content does not address critical individual or team skills, or 3) inappropriate in that the instructional methods used are not suitable for the job’s skill content or required level of proficiency.

Battle-staff integration is a term used to characterize the process by which the individuals comprising a unit’s battle staff learn to work together as an effective mission-command team (Olmstead 1992). Inadequate individual and team-oriented training, staff-member inexperience, and rapid personnel turnover within battle staffs can prevent effective battle-staff integration. This failure impacts both mission-command performance and unit effectiveness (Thompson et al. 1991; Sauer 1996). It is arguable that few of the battle staffs observed across NIEs had received sufficient training or were stable enough across time to have achieved the level of battle-staff integration required to effectively employ the mission-command equipment suites in modernized CPs.

An additional aspect of training for network-enabled mission operations surfaced during later NIEs. This aspect of training pertains to the preparation of commanders and their senior supporting staffs to command a network-enabled unit. As noted previously, technology often changes the nature of the work that technology is intended to support. Such changes also potentially impact command-level concepts and practices. Wallace (2014) insightfully remarks that the contemporary network can be “maneuvered,” not in the traditional sense of unit maneuver but in the sense the “pipes” through which the information flows can be technically adjusted to the needs of the mission. He further comments that this maneuverability demands awareness, training, and a degree of network-related technical understanding on the part of commanders and senior staff officers. Authority and responsibility for network operations cannot be left in the hands of Signal specialists absent clear direction and understanding of the commander’s intent. Commanders and their supporting staffs must learn to “command the network” much as they would any other critical and limited resource. Similarly, Signal personnel must learn to


coordinate with the battle staff to “maneuver the network” in accordance with the unit’s scheme of maneuver. Suitable training and on-the-job practice on the part of both parties are required.

Wallace (2014) further observes that one of the most frequently cited concerns about “the network” is lack of training in its use. The previous discussion of NIE findings addressed aspects of training and on-the-job experience related to the technical and operational use of the network as well the MCIS, such as CPOF, attached to it. What was not addressed in this previous discussion is training focused on commanding the network or maneuvering the network in accord with the commander’s intent, as Wallace uses those terms. The former is primarily the command team’s responsibility, while the latter is the responsibility of leadership in the unit’s Signal (i.e., S-6) section. These 2 groups must learn to work together to achieve a common end in ways they have not had to do in the past.

ARL’s HSI support team focused on aspects of this staff-section interaction during observations of mission-command operations during later NIEs. However, it is not clear that what is required in this respect (i.e., doctrinal concepts and associated knowledge, skills, and competencies) has been fleshed out to the extent that suitable orientation and training could be developed. Wallace further remarks (2014, p. 2) that the speed at which the network has evolved “has eluded contemporary organizational and institutional training solutions”. The Army’s formal training institutions typically are slow to respond to rapid evolutionary change. Moreover, most tactical organizations are not fully aware that such change is happening or are not in a position to develop or deliver essential training.

As noted, commanding the network refers to adapting traditional concepts and practices for mission command to take advantage of the resources now provided by the network and associated digital mission-command systems. If commanders of networked units were to continue to practice the art of mission command in the old way, a lot of capability might be left unused or underutilized. The Army might find itself in the situation often referred to as the productivity paradox: We see information technology everywhere in the workplace but in the productivity statistics. It took leading, private-sector organizations several decades to break out of the productivity paradox by adjusting job structures, work methods, training, and personnel practices to take advantage of their new information and communications technologies and raise organizational productivity levels. One aspect of that adjustment was adapting leadership and management practices to match the new tools of work. This latter adaptation is analogous to the issues underlying commanding the network. That process is now well underway in leading firms and has involved considerable experimentation over time. Moreover, there is an emerging literature on leadership and management in information-intensive


organizations, but implementable specifics generally are lacking. The Army has to face those same challenges and travel that same path. It is not enough merely to change equipment and hope that new work practices, command styles, and productivity enhancements will automatically follow.

HRED’s HSI support team observed considerable variety in network-enabled command practices across units observed during the NIEs. Some unit commanders obviously used the new tool sets to command their units much as things were done in the past—that is, prior to the introduction of modernizing equipment. Other commanders demonstrated some degree of innovation in how they commanded their units using these new capabilities. However, none of these approaches to unit command has been subject to any serious scrutiny directed at identifying best practices: what works and under which conditions. Some variability in command practices across units and missions is to be expected, but a common core of doctrinal concepts and tactical procedures underpinning network-enabled mission command might prove beneficial.

When questioning commanders and senior staff about commanding the network, the team encountered a number of views. The most common view was that the modernizing mission-command equipment definitely changes the “science” of mission command. The tool set used to support the process of mission command is different and must be integrated into staff practices. That load obviously falls on the battle staff and is one reason why operational integration and effective battle-staff integration are now more important than in the past. Commanders and senior staff are less certain whether or how the new tool sets impact the “art” of mission command. Several commanders and senior staff interviewed stated the new tools do not impact the art of mission command. Their view was the art of mission command is still the art of mission command and has not changed much as technology support has changed over time. Other commanders were not so certain about that position. They opined that the new tools likely impact the art of mission command, but were uncertain how the tools impact that art and should be reflected in command practices. Respondents in both camps agreed that, at a minimum, it is necessary for commanders and senior staff to engage with the new technologies in order to understand their capabilities and limitations. That cannot be left to the battle staff alone or to the S-6 staff section. This view does not imply, for example, that commanders have to become CPOF operators—but, they must be familiar with the command-support capabilities that CPOF (or similar systems) provides. Whether that level of engagement with the new tools of mission command is sufficient to effectively command the network remains an empirical question open to future experimentation.


In a case study of the evolution of network-enabled operations in early Stryker brigades, Gonzales et al. (2005) provide results that support the observations reported in the previous paragraphs. These authors reported that tactical procedures in initial Stryker brigades emphasized the importance of explicitly training Soldiers and leaders to operate on the network. They went on to remark that achieving an appropriate level of understanding required leaders who were “well-versed in all aspects of Stryker brigade doctrine, the network-enabled operational concepts contained therein, and in the capabilities and limitations of the networking and battle command systems of the Stryker brigade” (p. 37). The brigade had to deliberately focus on reengineering processes, procedures, training, and leadership thinking to make the enhanced unit capabilities provided by the network a reality. Enhanced unit capabilities are the justifying promise of network-enabled operations.

Recent results from the Army’s Combat Training Centers (CTCs) support Gonzales et al.’s and Wallace’s views regarding the importance of reorienting and training leaders to operate in a network-enabled setting. CTC results indicate that unit commanders often do not understand the newer generation of mission-command systems and do not appreciate what those systems can do for them (CTCs 2017). CTC observer/controllers also assert there is a need to train unit leaders to take advantage of the mission-command capabilities now available to them. If commanders are not familiar with the benefits associated with new systems, the proper emphases will not be placed on them during training or subsequent field operations. Similar conclusions on educating (versus simply training) future Army leaders to use future advanced systems were also provided in a recent report from the Army’s Mission Command Battle Laboratory (MCBL 2017). Educating leaders how to employ future advanced systems versus simply training them how to operate those systems is consistent with Wallace’s concept of commanding the network—learning how to effectively command a network-enabled unit to take advantage of new capabilities. This view also has significant support in the extensive historical literature on the interplay of technology and military innovation (e.g., Corum [1992]).

Question 3. Going forward, what are some solutions to the problem of growing cognitive load on commanders and battle-staff members?

The ARL team’s final question involved potential solutions: What can be done to remedy the problem of growing cognitive load on commanders and battle-staff members? Previous paragraphs identify and discuss a number of issues contributing to extraneous cognitive load in network-enabled mission command. At one level, the remedy to these contributors (which mostly deal with design, system and operational integration, and training) appears straightforward: Focus follow-on


work toward solving those issues. However, that might be easier said than done in current system-development and institutional and collective training settings. In keeping with Norman’s (2011) view that managing complexity involves 1) creating designs that “tame” complexity and 2) taking the time to learn how to use new suites of systems effectively, the following discussion focuses on critical underlying issues and institutional impediments associated with meeting these challenges in the contemporary force-modernization environment. The ensuing discussion also illustrates that sometimes these 2 seemingly separate goals cannot be addressed in isolation. Design and training considerations are often confounded.

3.2.4 Design to Tame Complexity

The first challenge to be addressed is a design to tame complexity. Prior to NIE 14.1, the then-BMC commanding general (CG) directed ARL’s HSI support team to conduct what he referred to as a “CP Ease of Use” study. Based on personal observations, the CG had become concerned that modernized CPs as observed during the NIEs were not easy to use in the aggregate, and this shortfall might interfere with effective mission command. The ARL team’s observations across NIEs support the CG’s view. Moreover, recent results from the Army’s CTCs indicate many commanders have difficulty maintaining a shared and accurate understanding of their operational environment. Mission-command component systems are not being integrated to provide a useful common operating picture (COP) for commanders. In current usage, the COP is a single display of relevant information within a commander’s area of interest tailored to the user’s requirements and based on common data and information shared by more than one command echelon (HDA 2014). A shared and accurate COP is essential to effective mission command.

During a follow-up discussion of mission-command complexity and cognitive load, the BMC CG digressed at length on the issue of the analog “wing board” versus current digital displays in NIE CPs. The CG quipped he could stand in front of a properly laid-out wing board and get the gist of the tactical situation in less than 30 s—but not as readily with current digital displays in CPs. HSI team observations of CP operations across NIEs, along with the creeping addition of analog displays into modernizing CPs, suggest several questions related to the CG’s remarks:

• In assisting commanders’ performance of mission command, is there an attribute to a highly evolved analog tool (e.g., wing board) that is difficult to achieve with current digital displays?

• Does an analog wing board facilitate “cognitive fusion” of essential information in ways that are difficult to achieve using contemporary digital


displays? (When asked about his preference for analog situation displays—paper map versus digital displays—one battalion-level XO pointed to his mapboard and quipped, “That’s where we do our thinking.”)

• Do the HSI team’s observations in NIE CPs reflect something intrinsically limiting about digital display technology, or simply reflect the limits or poor design of current digital displays?

Well-designed operations centers must support decision makers at all levels with the insight and foresight required to make effective choices, manage associated risks, and consider second- and subsequent-order effects. This involves the cognitive ability “at a glance” to see and understand a tactical situation and thereby enable independent decisions and correct actions. These capabilities, sometimes labeled cognitive fusion (or perceptual integration), are the product of appropriate display design, component technical integration, and operational integration of materiel components into the MDMP and operations-center practices. Cognitive fusion is not a process, per se. Rather, adequate system design coupled with suitable technical and operational integration create the conditions for cognitive fusion on the part of experienced decision makers. Technology can enhance, but not substitute for, command experience.

Klein (1997) argues that an excessive focus on decision-support technologies coupled with too little consideration of the actual cognitive mechanisms underlying expert decision making can reduce rather than improve decision makers’ performance. He asserts that improperly structured information technologies can interfere with the expression of expertise on the part of skilled commanders and battle staff. Based on observations across NIEs, there is little doubt the current focus of much of contemporary Army network modernization is information technology, almost to the exclusion of whether that technology actually supports the mechanisms underlying expert command decision making.

Moving beyond a strict consideration of design, is what the ARL team has observed a training- and experience-related phenomenon? Does the observed preference for analog displays in NIE CPs reflect a commander’s and staff’s lack of experience with digital mission-command systems? Are these personnel simply accustomed to doing things “the old way” and have not yet adapted their command style and staff practices to a digital mission-command setting? In this respect, Gonzales et al. (2005) reported that in the absence of adequate training and follow-on practice, commanders and supporting staffs in early Stryker brigades abandoned network-enabled systems and resorted to more familiar analog systems and methods. Also, should analog media and digital displays be considered complementary? If so, in what sense should they be considered complementary? It is interesting that a recent


broadcast email (dated 10 April 2017) from the Director of the Center for Army Lessons Learned (commonly known as CALL) solicited field-tested best practices for “maintaining an analog and digital COP”.

The fact that older display media, such as paper maps, are now ubiquitous in NIE CPs after once being “banned” might be telling us something. Perhaps that “something” is simply a lack of familiarity with new digital technologies on the part of commanders and their supporting staffs. But, it may also reflect something deeper that might be essential to effective mission command, as Klein’s extensive body of work on naturalistic decision making might suggest (Klein 2009).

3.2.5 Training on, Experience with, Network-Enabled Mission-Command Systems

The ARL HSI support team’s reporting on NIE field operations repeatedly emphasized the importance of training and follow-on experience to user perceptions of mission-command complexity and cognitive load as well as success in using new mission-command systems. Based on the team’s observations across NIEs, it is arguable the single largest contributor to perceived complexity and extraneous cognitive load during the NIEs is that participants simply did not know how to use digital mission-command systems individually or collectively. Recent CTC results and reports from other venues such as the MCBL support this conclusion. Moreover, observed battle staffs often did not know how to function as an integrated CP team. Complex cognitive work is teamwork. There were obvious issues with mission-command system interoperability and data flow within and across CPs, but it is not clear how much of that actually was attributable to lack of individual and team proficiency in using new mission-command systems. Complex technology and limited user experience can be a dangerous combination. Addressing training deficiencies for individual users and battle-staff teams is a critical aspect of cognitive-load mitigation and mission-command effectiveness going forward.

In addressing the training problem, 3 issues of individual, team, and unit performance must be stressed:

1) Preparation of individuals on the battle staff, along with suitable training and orientation for commanders and senior staff.

2) Unit collective training to reinforce individual task-work skills and develop essential teamwork skills.

3) Maintenance of core battle-staff expertise over time as personnel arrive at and depart the unit.


Detailed recommendations pertaining to the first 2 issues are provided in Hawley and Swehla (2016) and are not repeated here. Based on NIE observations and anecdotal reports, the third issue is an important aspect of unit performance at the intersection of unit training and local personnel management. It has been noted the primary function of a peacetime military is maintaining readiness. The glue that holds materiel systems together and makes them more than a collection of hardware is Soldier expertise; however, there are high rates of personnel turnover in all military organizations. The human parts keep passing through the system, so to speak. Thus, even though a unit is combat-ready one day, it might not be the next unless the expertise of the personnel departing is continually replaced by the newly acquired skills of those who have recently arrived. This high turnover rate of personnel and the need for continual replenishment of expertise is an important consideration in unit-training planning and local personnel management. Maintaining essential levels of unit expertise cannot be left solely to the whims of the Army’s formal personnel-assignment system. Units must be proactive in managing available Soldier expertise, as with any other critical resource. During discussions of this topic with brigade and battalion staff members, several remarked that maintaining a consistent level of mission-command expertise in CPs is a challenge for the unit’s leadership. Loss of a few key specialists often has a significant impact on the unit’s performance capabilities—single points of personnel-related performance risk. A complicating factor in local personnel management is that expertise in network-enabled mission command systems and operations is not deep across the Army.

To emphasize these points, the draft “Command Post 2025 Concept of Operations” developed by the MCCoE included the following:

Current Army command posts are the result of unsynchronized requirements and efforts of multiple programs and are not integrated as a system. … This lack of synchronization results in a level of complexity that cannot be overcome by the unit’s organization, people, or available training resources. Current CPs do not easily support mission command as cognitive work [MCCoE 2014, p. 6].

The key issue in training for more-effective, network-enabled operations is the development of higher levels of expertise on the part of individual Soldiers, crews, teams, senior staff, and commanders. Much of the technology on display during the NIEs is “skill-biased” in the sense that it requires high levels of developed skill for effective use. Developing expertise should not be confused with the results of conventional Army training. Expertise is a function of suitable formal training followed by extensive job-relevant experience with feedback over time. Part of the solution to developing higher levels of expertise in tactical units certainly involves


additional training time—time that tactical units will be hard-pressed to provide in the contemporary garrison environment. Results across NIEs speak volumes to this observation. But new objectives for and approaches to individual, team, and collective training also will be required, particularly for critical high-skill areas such as mission command or network operations (e.g., Hoffman et al. 2014).

A central question to be addressed going forward is whether the training “load” (time, resources, and need for specialized training expertise, etc.) associated with the new generation of network-enabled systems exceeds the Army’s ability to provide essential training. It has been noted, for example, that developing good training for a complex task is itself a complex task. The complexity curve associated with new systems individually and in the aggregate has been rising steadily for some years. At the same time, the Army’s ability to train has stayed constant, or perhaps even declined, particularly when considering qualitative aspects of the training resources the Army now requires. In the formative days of Army HSI (circa 1985), General Max Thurman—sometimes referred to as the godfather of Army Manpower and Personnel Integration (MANPRINT), now HSI—cautioned that whenever a new system or operating concept required the Army to go outside the personnel and training “footprint” of the predecessor, trouble would follow. For many systems, the Army has been getting further and further outside the personnel and training footprint of predecessor systems, and the result has been increasing difficulty supporting those systems.

In a recent journal article addressing necessary levels of training and expertise for complex sociotechnical systems, Strauch (2017) argues that developers of new systems must consider the expected level of operator expertise when designing and building such systems. Strauch further observes that because of limitations in time and resources, available training programs often do not provide users the expertise necessary to effectively employ complex systems. Training shortfalls require users to obtain necessary expertise ad hoc during system operations. As a result, many users do not acquire essential levels of system expertise. That has certainly been seen in the ARL team’s mission-command-related observations across NIEs. An obvious flaw in Strauch’s argument is that sometimes complex tasks require complex equipment suites. It is not clear that if equipment design were constrained to reflect expected user-performance capabilities (assuming such a thing could be practically achieved) that the resulting systems would be adequate for an increasingly complex and demanding operational setting.

Concerns such as those expressed previously are not new and have been central to HSI since program inception. Indeed, similar concerns provided much of the impetus for the initial development of the Army’s formal HSI program (then termed MANPRINT) in the mid-1980s. Given that HSI has been mandated and applied


during Army system development for more than 30 years, why are so many systems fielded that are inadequate to their users’ cognitive needs or require more training or personnel resources than can reasonably be provided? This question was recently posed by the US House Armed Services Committee, and it presents a challenge for both HSI and Army force modernization going forward. Several perspectives on that challenge are addressed in the next section.

3.3 Perspectives on HSI at System-of-Systems and Unit Levels

Army HSI efforts have traditionally been applied at the individual system level for programs of record, and that has been the case with the mission-command network (i.e., WIN–T) and most of the individual MCISs, such as CPOF, comprising NIE CPs. What has not been adequately addressed in these efforts is the evaluation of HSI issues in the relationships between Soldiers and technology, not just at the individual system level but also at the system-of-systems and organizational levels. Some of the most demanding and problematic aspects of mission-command operations, as observed across NIEs, are emergent issues that only show up when the individual systems composing the CP are brought together, configured in a particular way, and placed in a unit context. These emergent performance issues might not show up in an isolated assessment of individual mission-command systems. Several of the contributors to mission-command complexity and extraneous cognitive load discussed previously (e.g., interoperability shortfalls and inadequate operational integration) are emergent issues associated with cobbling the individual systems together to support mission command as an integrated warfighting function. It is arguable the aggregate performance effects of these emergent issues exceed those associated with design features and training for the individual mission-command systems considered in isolation. Yet the focus of most HSI assessments is on individual mission-command systems considered mostly in isolation.

HSI for a system of systems such as a CP involves more than simply rolling up the assessments for the individual components while asserting that the resulting composite picture accurately reflects the whole (Vicente, 2006; Walker et al. 2009). In complex systems of systems such as a CP, the performance effects of the whole are more and often different than the sum of the effects of its parts. One lesson to be taken away from the ARL HSI team’s NIE support work is that HSI assessments for a system of systems such as a CP must reflect the macrocognitive, team-based nature of the tasks performed in that integrated work setting.

Beyond system-of-systems-level concerns, additional HSI issues are encountered when equipment suites such as a CP are embedded within their broader operational


context. Functional systems of systems such as a CP composed of teams in interaction with a tool suite show cognitive properties that are radically different from the properties of those individuals acting alone and apart from their natural work setting (Hutchins 1995). Hutchins asserts that what is necessary in such cases is an assessment of naturally situated cognition in which the unit (of measure) of cognitive analysis is work as it is performed by that functional team operating in its natural operational setting. For example, consider the performance impacts associated with Wallace’s (2014) notions of commanding and maneuvering the network. Results from venues such as the CTCs and the NIEs suggest these issues are important with respect to a unit’s mission-command effectiveness. Yet these issues might not have surfaced in an assessment of individual mission-command systems, the network (i.e., WIN–T), or even during a more holistic assessment of the CP considered out of its natural operational context. To observe the impact of these issues, it was necessary to place CPs in their multi-echelon-unit context and observe the interplay of the functional teams consisting of battle staffs and S-6 (Signal) personnel while pursuing the unit’s mission objectives.

Given the HSI issues cited here, why is there so little emphasis on holistic system-of-systems-level assessments along with a deeper look at “cognition in the wild”? The simple answer to this question goes back to the funding mechanisms for HSI assessments. Army HSI assessments are paid for by program managers (PMs) for programs of record. At present, a CP consists of a collection of systems developed by individual PMs who pay for HSI assessments of “their” systems at milestone decision points. The same is true when those systems are taken to formal operational tests. There is no comparable PM for the CP considered as a system of systems. Consequently, a holistic HSI assessment of the CP considered as a system in and of itself is not performed. Taken together, system-of-systems-level analysis and a consideration of what might be termed cognition in the wild represent new and important challenges and opportunities for HSI.

4. Conclusion

In many respects, the title of the current paper, “The New Equipment is Here, Now Comes the Hard Part …”, captures the primary lesson to be carried away from the ARL HSI support team’s work across NIEs. The previous discussion paints a picture of unnecessary CP complexity, excessive extraneous cognitive load, and Soldier- and team-performance deficiencies that arguably are the result of failure to adequately address the “hard part” that comes after the introduction of modernized systems into tactical organizations. The new equipment is introduced, and routine New Equipment Fielding and NET practices are followed. But that is not enough to get receiving units across the “valley of death” between the


introduction of those systems and reaching an effective operational capability based on them (Hawley 2015). The support team’s observations across NEs indicated that units “struggle to fight” using their new mission-command suites. Unfortunately, similar results are observed at the CTCs when standard tactical units receive versions of the same equipment used during the NIEs.

A common belief encountered across the NIEs and elsewhere is the notion that if a complex system, such as any of the MCISs or the CP considered as a whole, were more “intuitive,” problems associated with effective use would be reduced or eliminated. Also, the need for extensive training would be reduced or virtually eliminated. The underlying belief here might be summarized as, “Just make that system intuitive, and the usability problems routinely observed will be eliminated.” Used in this context, intuitive means knowing how to use a piece of equipment with little or no formal training. Most people, for example, are able to pick up a smart phone or tablet computer and use it with little or no formal training or orientation on the system’s use. These devices are said to be intuitive. However, much of that ability results from previous experience with similar items of equipment. A smart phone is intuitive because that new user has a mental model of how such a device should work based on previous experience with similar equipment.

From a Soldier-performance perspective, the high hopes pinned on mitigating complexity by developing intuitive equipment items or software are partly a myth. It is accurate that providing design commonality across equipment items or software used together can reduce or even eliminate training for basic system operations: the so-called “buttonology” of that system or software package. Following this approach, equipment and software use can also be made less error-prone through reductions in the potential for various forms of proactive and retroactive interference across systems or applications. However, a common look and feel across items of equipment will not eliminate the need for users to learn to use those equipment items or applications to perform the various job roles in their workplace. Basic system operability should not be confused with effective use, particularly with respect to complex tasks. An intuitive user interface for a word-processing program will not, for example, make anyone a better writer. And, neither will a more intuitive interface for CPOF, or any of the MCISs, make the user a proficient practitioner of the “art” of mission command. The art aspect of mission command is not intuitive, in the sense the term is used here. That next step from basic system operation to smooth, flexible, and effective use in the workplace requires formal training and extended practice directed at different and more complex instructional objectives. As often used, the phrase “make it intuitive” is a simplistic and potentially misleading “folk solution” to a more complex problem—


a silver bullet approach to dodging the hard work associated with the introduction of new systems.

Beyond design considerations, we should not be seduced into being too optimistic about our ability to reduce the amount of training and experience required for the effective use of complex systems used to execute complex tasks. Mission command in the contemporary operating environment is a complex and demanding activity. Suitably designed mission-command support systems can aid commanders and battle staffs in the performance of these activities, but such systems will not (and perhaps should not be expected to) substitute for in-depth human expertise in the art of mission command. That in-depth human expertise has to be developed over time through exposure to the right kinds of training and formative on-the-job experiences. Methods exist to accelerate the development of mission-command expertise (e.g., Hoffman et al. [2014]), but much of the hard work associated with expertise development still must be done.

Complexity and resulting cognitive load can be managed but cannot and perhaps should not be eliminated without careful consideration of essential and potentially beneficial information needs. Scientific work on complex systems (e.g., Hollnagel and Woods [2005]; Hoffman and Woods [2011]) supports a broad consensus that intrinsic complexity often cannot be reduced. Complex work often requires complex tools. To adequately deal with the issue of complexity, it is necessary to accept complexity as a persistent and pervasive fact and manage it. It is hazardous to attempt to avoid complexity by making reductive assumptions about it and attempting to implement simple, quick-fix solutions. Quick-fix solutions rarely if ever cope with or eliminate complexity. Simplistic solutions merely transform the root problem of complexity, most often by hiding it from users. Such solutions can lead to unpleasant “surprises” during later operational use.

Stepping back and taking a broader view of force modernization and looking beyond the specifics of the findings discussed in this report, it is arguable that many of the factors contributing to mission-command complexity and extraneous cognitive load in NIE CPs stem from a failure to manage complexity during the design, development, and fielding of new mission-command equipment suites. It is also arguable that failure to manage complexity is an artifact of 3 “sins” associated with the way the Army approaches the development of new systems and technologies. These sins are characterized as follows:

1) An excessive preoccupation with equipment (hardware and software) during concept development, system design, materiel development, and operational testing. Sociotechnical and nonmateriel aspects of system development, testing, and effective use typically are given less emphasis


and are generally not considered in much depth until the system is near fielding. Much of this preoccupation with materiel is driven by funding considerations. That said, it is difficult to manage complexity from the point of view of users after a system is nearly fully developed and users must adjust to that system after the fact.

2) Lack of an overarching system-of-systems focus for team-based work systems such as a CP. ARL’s HSI team observed and reported that many of the issues driving complexity and extraneous cognitive load in NIE CPs result from the fact that CPs are cobbled together from components developed by individual proponents and vendors. In many cases, these components were developed and evaluated in isolation with little consideration given to how they “fit together” to support mission command as integrated cognitive work. It is left to battle staffs to knit these components together to support mission command as an integrated warfighting function. From the perspective of the battle staffs conducting network-enabled mission command, the usability of NIE CPs considered as systems of systems was seriously deficient.

3) Failure to consider organizational learning processes during the period leading up to system evaluation or introduction to tactical units. The issue of organizational learning is important with respect to how well a unit will use new equipment suites during an exercise such as the NIEs or after those equipment suites are introduced to tactical units. It is unrealistic to expect that complex new equipment suites can be “dumped into” an organization without having an effect, often detrimental initially, on how well that organization performs its intended mission. As noted several times herein, new technology often changes the nature of the work that technology is intended to support. A receiving unit must adapt to this new technology and learn how to use it effectively. Adaptation takes time and often requires more than traditional NET. Traditional NET assumes the receiving organization will use new systems and technology in much the same fashion as it used older equipment. The tacit assumption is that not much adaptation or organizational learning is necessary. To the extent that assumption is not true—and new organizational forms, doctrinal concepts, processes, and procedures are required to take advantage of new materiel—traditional NET will be inadequate. Moreover, multiple and concurrent equipment changes can have a cumulative and possibly nonadditive impact on unit performance (i.e., 1 + 1 > 2). That is, multiple equipment changes requiring corresponding and possibly interactive DOTLP changes will increase the complexity of the unit’s learning and adaptation processes, increase the


length of the adaptation period, and possibly deepen the initial performance drop-off. Change is disruptive, and multiple changes are cumulatively disruptive.

It is clear, based on observations across NIEs, that the introduction of complex new equipment suites—such as those supporting network-enabled mission command—into tactical units requires considerably more than what is typically done under contemporary approaches to new equipment fielding and follow-on NET. An observation from the Army’s successful experience in developing and deploying Stryker Brigade Combat Teams (BCTs) is relevant to this point:

Stryker BCTs are complex organizations. Transformation of the BCT is much more than conducting NET and essentially is a holistic effort required to convert to a new organization, receive new equipment, and ultimately train to a higher level of unit proficiency [BCC 2003].

There are many parallels between the development of Stryker BCTs and the deployment of modernizing mission-command equipment suites to tactical units. In essence, the modernizing unit is being “transformed” (as that term is used in the above quote). The development of Stryker BCTs is generally considered to be an example of best practices with respect to fielding a new type of unit with new kinds of equipment. There are lessons in the Stryker experience both for preparing a unit for meaningful participation in exercises such as the NIEs and for fielding complex new equipment suites to tactical units.

These issues affecting successful force modernization are not new. Binkin (1986) and Demchak (1991) discuss at length the Army’s experiences during the “great wave” of force modernization during the late 1970s into the 1980s. For the Army, this was the period during which the “Big 5” (Abrams main battle tank, Bradley fighting vehicle, Apache attack helicopter, Black Hawk helicopter, and Patriot air-defense system) were introduced into the force. Many of the receiving units’ initial experiences with these systems were not positive. Modernization often did not proceed smoothly. A number of the observed problems resulted from the fact system developers failed to consider the “information load” these systems would impose on receiving units and the organizational, procedural, and training impacts that would result (Demchak 1991).

It is arguable the lessons from this earlier force modernization are still relevant today. In fact, those lessons may be even more relevant today given the shift in types of modernizing technologies between that period and the present. Modernization during the 1970s and 1980s primarily involved electro-mechanical technologies. The use and effect of information and communication technologies (ICTs) was considerably less than today. Contemporary modernization initiatives


such as those observed in NIE CPs primarily involve ICTs. Levy and Murnane (2012) argue that the increasing use of ICTs in the workplace fundamentally changes the nature of work and the skill, knowledge, and experience requirements of the people who perform that work. ICT-dominated work is more cognitive and conceptual in nature. It might be said that ICTs are doubly skill-biased in the sense they often require higher levels of aptitude (mental ability) as well as higher levels of education, training, and experience for effective use.

Successful force modernization involves far more than simply giving a unit new equipment and assuming that Soldiers somehow will make it work, or that the organizational forms, doctrinal concepts, usage concepts, and work practices of the past will prove sufficient going forward. Adapting organizational forms, doctrinal concepts, usage concepts, and work practices to reflect new system capabilities followed by suitable training for all concerned represents the “hard part” in the title of this report. Enhanced unit performance using new materiel will not be achieved by focusing on widgets alone. As long as the Army equates force modernization solely with equipment change and not organizational transformation, success in modernization initiatives will be elusive. This point is also emphasized in the unclassified summary of the 2018 National Defense Strategy: “Success no longer goes to the country that develops a new fighting technology first, but rather to the one that better integrates it and adapts its way of fighting” (DOD 2018, p. 10).


5. References

Anderson K. Ethnographic research: a key to strategy. Harvard Bus Rev. 2009 Mar.

[BCC] Stryker key transformation and unit set fielding (USF) lessons learned, lessons learned summary. Fort Lewis (WA): Stryker Brigade Coordination Cell, Army Battle Command Training Capability (US); 2003 May.

Binkin M. Military technology and defense manpower. Washington (DC): The Brookings Institution; 1986.

Corum JS. The roots of blitzkrieg: Hans von Seeckt and German military reform. Lawrence (KS): University of Kansas Press; 1992.

[CTCs] Combat training centers: NTC/JRTC/7th ATC. Briefing presented at: Mission Command Network Summit No. 4; 2017 Apr 11; Fort Leavenworth, KS.

Demchak CC. Military organizations, complex machines: modernization in the US armed forces. Ithaca (NY): Cornell University Press; 1991.

[DOD] Summary of the 2018 national defense strategy of the United States of America: sharpening the American military’s competitive edge. Washington (DC): Department of Defense (US); 2018.

Flanagan JC. The critical incident technique. Psych Bul. 1954;51(4):327–359.

Gonzales D, Johnson M, McEver J, Leedom D, Kingston G, Tseng M. Network-centric operations case study: the Stryker brigade combat team. Santa Monica (CA): The RAND Corporation; 2005.

Hawley JK. Training and testing: a complex and uneasy relationship. ITEA J Test Eval. 2007;27(4):34–40.

Hawley JK. Manpower and personnel integration (MANPRINT) and network integration evaluation (NIE) 13.2: observations on cognitive load in mission command. Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2014 Mar. Report No.: ARL-TR-6848.

Hawley JK. Human-systems integration (HSI) and the network integration evaluations (NIEs), part 2: a deeper dive into mission command complexity and cognitive load. Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2015 Mar. Report No.: ARL-TR-7238.


Hawley JK, Swehla MW. Human-systems integration (HSI) and the network integration evaluations (NIEs), part 3: mitigating cognitive load in network-enabled mission command. Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2016 Jun. Report No.: ARL-TR-7698.

[HDA] Commander and staff organization and operations. Washington (DC): Headquarters, Department of the Army (US); 2014 May. Field Manual No.: FM 6-0.

Hoffman RR, Best BJ. An integrated model of macrocognitive work and trust in automation. Pensacola (FL): Institute for Human and Machine Cognition; 2012. Technical report.

Hoffman RR, Ward P, Feltovich P, DiBello L, Fiore SM, Andrews DH. Accelerated expertise: training for high proficiency in a complex world. New York (NY): Taylor and Francis; 2014.

Hoffman RR, Woods DD. Beyond Simon’s slice: five fundamental trade-offs that bound the performance of macrocognitive work systems. IEEE Intel Sys. 2011;26(6):67–71.

Holley IB Jr. Technology and military doctrine: essays on a challenging relationship. Maxwell Air Force Base (AL): Air University Press; 2004.

Hollnagel E, Woods DD. Joint cognitive systems: foundations of cognitive systems engineering. New York (NY): Taylor and Francis; 2005.

Hutchins E. Cognition in the wild. Cambridge (MA): MIT Press; 1995.

Kantowitz BH. Mental workload. In: PA Hancock, editor. Human factors psychology. Amsterdam (The Netherlands): North-Holland; 1987. p. 81–121.

Klein G. Implications of the naturalistic decision making framework for information dominance. Wright-Patterson AFB (OH): Air Force Armstrong Laboratory (US); 1997 July. Report No.: AL/CF-TR-1997-0155.

Klein G. Streetlights and shadows: searching for the keys to adaptive decision making. Cambridge (MA): MIT Press; 2009.

Koopman P, Hoffman RR. Work-arounds, make-work, and kludges. IEEE Intel Sys. 2003;18(6):70–75.

Levy F, Murnane RJ. The new division of labor: how computers are creating the next job market. Princeton (NJ): Princeton University Press; 2012.

[MCBL] Mission command, commander s toolkit study, version 3.0: examining the commander’s information, technology, tools needs to support future mission


command. Fort Leavenworth (KS): Army Mission Command Battle Laboratory (US); 2017 Sep.

[MCCoE] Command Post 2025: concept of operations (CONOPS). Fort Leavenworth (KS): Army Mission Command Center of Excellence (US); 2014 Nov. Draft report.

Norman DA. Living with complexity. Cambridge (MA): MIT Press; 2011.

Olmstead JA. Battle staff integration. Alexandria (VA): Institute for Defense Analyses; 1992 Feb. IDA Paper No.: P-2560.

Plass JL, Moreno R, Brunken R. Cognitive load theory. Cambridge (UK): Cambridge University Press; 2010.

Sauer GG. Battle staff integration: the key to battle tracking in battalion command posts. Fort Leavenworth (KS): Army Command and General Staff College (US); 1996.

Strauch B. The automation-by-expertise-by-training interaction: why automation-related accidents continue to occur in sociotechnical systems. Hum Fac. 2017;59(2):204–228.

Thompson TJ, Thompson GD, Pleban RJ, Valentine PJ. Battle staff training and synchronization in light infantry battalions and task forces. Alexandria (VA): Army Research Institute (US); 1991 Dec. Research Report No.: 1607.

Vicente K. The human factor: revolutionizing the way people live with technology. London (UK): Routledge; 2006.

Walker G, Stanton N, Salmon P, Jenkins D. Command and control: the sociotechnical perspective. Surrey (UK): Ashgate; 2009.

Wallace WS. Commanding the network: thoughts on mission command in the 21st century. Professional development seminar. Fort Bliss (TX): 2nd Brigade Combat Team, 1st Armored Division, Army (US); 2014 Sep.

Wickens CD, Hollands JG, Banbury S, Parasuraman R. Engineering psychology and human performance. 4th ed. New York (NY): Taylor and Francis; 2013.


List of Symbols, Abbreviations, and Acronyms

ARL US Army Research Laboratory

BCT Brigade Combat Team

BMC Brigade Modernization Command

CG commanding general

CLT Cognitive Load Theory

COP common operating picture

CP command post

CPOF Command Post of the Future

CTC Combat Training Center

DOTLP Doctrine, Organization, Training, Leadership and education, and Personnel

FA field artillery

FSR field service representative

HRED Human Research and Engineering Directorate

HSI human–systems integration

ICT information and communication technology

MANPRINT Manpower and Personnel Integration

MCCoE Mission Command Center of Excellence

MCIS mission-command information system

MDMP Military Decision Making Process

NET New Equipment Training

NIE Network Integration Evaluation

PM program manager

S-6 Signal/Communications

SME subject matter expert

TRADOC US Army Training and Doctrine Command


WIN-T Warfighter Information Network–Tactical

XO executive officer


1 DEFENSE TECHNICAL (PDF) INFORMATION CTR DTIC OCA 2 DIR ARL (PDF) IMAL HRA RECORDS MGMT RDRL DCL TECH LIB 1 ARL (PDF) RDRL HRB B T DAVIS BLDG 5400 RM C242 REDSTONE ARSENAL AL 35898-7290 8 ARL (PDF) SFC PAUL RAY SMITH CENTER RDRL HRO COL H BUHL RDRL HRF J CHEN RDRL HRA I MARTINEZ RDRL HRR R SOTTILARE RDRL HRA C A RODRIGUEZ RDRL HRA B G GOODWIN RDRL HRA A C METEVIER RDRL HRA D B PETTIT 12423 RESEARCH PARKWAY ORLANDO FL 32826 1 USA ARMY G1 (PDF) DAPE HSI B KNAPP 300 ARMY PENTAGON RM 2C489 WASHINGTON DC 20310-0300 1 USAF 711 HPW (PDF) 711 HPW/RH K GEISS 2698 G ST BLDG 190

WRIGHT PATTERSON AFB OH 45433-7604

1 USN ONR (PDF) ONR CODE 341 J TANGNEY 875 N RANDOLPH STREET BLDG 87

ARLINGTON VA 22203-1986 1 USA NSRDEC (PDF) RDNS D D TAMILIO 10 GENERAL GREENE AVE

NATICK MA 01760-2642

1 OSD OUSD ATL (PDF) HPT&B B PETRO 4800 MARK CENTER DRIVE SUITE 17E08 ALEXANDRIA VA 22350 ABERDEEN PROVING GROUND 12 ARL (PDF) RDRL HR J LOCKETT P FRANASZCZUK K MCDOWELL K OIE RDRL HRB D HEADLEY RDRL HRB C J GRYNOVICKI RDRL HRB D C PAULILLO RDRL HRF A A DECOSTANZA RDRL HRF B A EVANS RDRL HRF C J GASTON RDRL HRF D A MARATHE J HAWLEY

The New Equipment is Here, Now Comes the Hard Part: …1) taming the complexity associated with...

Documents

Transcript of The New Equipment is Here, Now Comes the Hard Part: …1) taming the complexity associated with...