21H E A LT H C A R E T R A N S F O R M AT I O N

The Art and Science of Data AggregationJoseph C. Nichols, MD

As the era of big data commences, many analysts are

wondering if the additional volumes of data will lead

to more meaningful information. Answering this

question requires diving deep into the data’s source

and definition to determine if the information built from this

data will be reliable. In other words, a thorough understanding

of healthcare information critically depends on a thorough

understanding of the data that created it.

Data aggregation—the grouping of raw data elements to

facilitate the assessment of higher level concepts—makes this

task easier. The most sophisticated statistical manipulation

cannot assure that information is accurate and reliable unless

the underlying data elements and aggregation of those elements

are clearly defined and accurately implemented.

Healthcare data is no exception. Data aggregation groups

codes and other values to assess higher level concepts, which

Quality data does not automatically translate to useful information.

Data aggregation is an essential interim step that must be as accurate

and reliable as the data itself.

• Clear definition is an art. A definition must strive to remove ambiguity and the likelihood of unintended interpretations. Tests ensure that the definition meets these goals.

• Regardless of the specific aggregation intent, it is important to have a single point of truth: a particular code grouping must consistently map to one concept.

• Defining codes and higher level concepts cannot be left to programmers or analysts with limited knowledge of clinical subject areas or coding principles.

• Subject matter experts should be asked to review both the aggregation method and test case outputs to ascertain that aggregation results meet expectations.

Inside Track

serve a wide variety of business and clinical functions, including

but not limited to

• analytics,

• payment policies,

• operation processes,

• key metrics, and

• triggers for system interventions.

For example, data aggregation must first identify the patient

population with Down syndrome before an organization can

assess service use, apply special coverage rules, or create metrics

to evaluate the quality and efficiency of care.

In its current practice, data aggregation tends to vary

according to the high-level concept. Instead of having a

consistent standard definition of “Down syndrome” and an

aggregation that maps to the intent of this definition—why the

user is interested in Down syndrome—the definition tends to

vary according to use. Consequently, data aggregation is often

plagued by inconsistencies in both process and terminology.

Most organizations fail to recognize that the clarity and

accuracy of high-level concepts depend heavily on the clarity and

accuracy of an aggregation that meets the concept definition’s

intent. They tend to ignore aggregation quality and concentrate

on raising data and analytic quality. Less effort is devoted to

examining the quality of the aggregation process itself, largely

because organizations lack the insight to establish and ensure

that quality.

Raising aggregation quality is within reach of healthcare

organizations but it requires establishing clear definitions of intent

and a consistent mapping of aggregated codes to a particular

intent—essentially making aggregation agnostic to purpose.

A G G R E G AT I O N Q U A L I T y

A N O B L I S P U B L I C AT I O N22

From facts to informationAs Figure 1 shows, data aggregation falls between data and

analysis. Logically, then, even if the process of converting facts

to information starts with high-quality data, it will not end with

high-quality information if data aggregation lacks quality.

To illustrate data aggregation’s central role, consider a user

who needs information about the cost of caring for someone with

secondary diabetes. The data needed to get to this information

relies on conditions defined by codes based on the International

Classifications of Diseases (ICD). The aggregation of those codes

should represent data related to the higher level concept of

secondary diabetes.

With the code descriptions in hand, the user’s goal is to

determine an aggregation of the appropriate codes, which

theoretically should capture all data related to patients with

secondary diabetes.

Unfortunately, the process can break down at this point

because of uncertainty about what codes that aggregation

should include. The user might inadvertently exclude codes that

should be included or select some that should be left out. This

confusion is understandable given the number of codes that

can relate to a higher level concept. As Table 1 shows, ICD-9

contains 20 codes related to secondary diabetes, most of which

refer to the term explicitly. In ICD-10, 183 codes are related to

secondary diabetes from a conceptual standpoint, but the term

itself does not appear in any of these code descriptions.

Lacking a clear definition of “secondary diabetes,” the user can

only guess at which codes capture the most relevant data. This

guesswork feeds into the analysis step, generating information

that at best only partially satisfies the user’s need and at worst is

inconsistent with the user’s intent.

Raising aggregation qualityFrom this simple example, it is clear that raising aggregation

quality is integral to raising information quality. Intelligent healthcare decisions depend critically on reliable information,

which means that healthcare organizations need to take a second look at their data aggregation process. Aggregation quality has three main elements:

• a clear definition of the user’s intent, as in as in what a user seeks in asking...,

• the selection of codes that match that intent, and

• the choice of an appropriate aggregation model.

Quality aggregation also depends on the establishment of standards that ensure a particular code grouping consistently maps to one concept, as well as a commitment to aggregation

quality monitoring and maintenance.

Clear definition of intentThe primary cause of poor quality information is a lack

of clearly defined, clearly accepted, and fully communicated definitions that remove ambiguity and the likelihood of unintended interpretations. The sidebar “Linking Brain Injury to Behavior” describes the importance of clear, standardized

definitions in the reading of brain MRIs, for example.

Figure 1. Why data aggregation affects information quality. Attention has been on raising data quality, but even if the translation from data to information starts with high-quality data, information quality can be suspect because aggregation quality will affect analytic quality.

The Facts

DataQuality

AggregationQuality

Information Quality

AnalyticQuality

Secondary Diabetes

ICD-9 Codes Description

24901 Secondary diabetes mellitus without mention of complication, uncontrolled

24910 Secondary diabetes mellitus with ketoacidosis, not stated as uncontrolled, or unspecified

… Total of 20 ICD-9 codes in this category

ICD-10 Codes Description

E08610 Diabetes mellitus due to underlying condition with diabetic neuropathic arthropathy

E08621 Diabetes mellitus due to underlying condition with foot ulcer

E0900 Drug or chemical induced diabetes mellitus with hyperosmolarity without nonketotic hyperglycemic-hyperosmolar coma (NKHHC)

… Total of 183 ICD-10 codes in this category

Table 1. An example of aggregation of ICD-9 and ICD-10 codes to represent the aggregated concept of secondary diabetes.

23H E A LT H C A R E T R A N S F O R M AT I O N

Michael Jarrett and Dennis V. Kennedy

In October 2011, the National Intrepid Center of Excellence (NICoE) launched a study to better understand how abnormalities identified in brain MRIs relate to behavioral issues. The Neuroimaging Common Data Element Validation Study is part of NICoE’s mission to advance the understanding of novel and innovative techniques to treat traumatic brain injuries and post-traumatic stress disorders sustained in combat.

At present, analysts only vaguely understand the relationship of MRI-revealed abnormalities to behavior, in large part because brain injuries are characterized in broad terms that have little specificity, such as mild, moderate, or severe traumatic brain injury.

The NICoE study will enable far more descriptive terminology and an increased understanding of the specifics of brain injuries and their resulting behavioral influences, enabling doctors to more effectively link injury with possible cognitive impairment. Typically, a radiology review is a long text note describing any abnormalities that the MRI revealed. There are no standard terms to use in completing the review, and terminology and specificity tend to vary across image readers.

Although this kind of review is adequate for clinical use, it creates problems when asking questions across a larger patient population, such as, what behavioral issues do people with right frontal cortex lesions share, or do the MRIs for people with sleep issues reveal consistent brain abnormalities?

Developing consistent understandingUsing standard forms ensures that image readers create the same set of structured data for interpreting an MRI and reviewing a patient’s results

and provides some way to impose control over the terminology used. With that idea in mind, a prototype system was developed that directs each reader to select applicable brain injuries from a list of 35 specific injury types and to provide standardized descriptive information about the injuries, such as location in the brain or size of a particular finding. Figure A shows a partial screenshot of a review in progress. The descriptions closely follow those developed by the National Institutes of Health’s (NIH’s) Common Data Elements Project to standardize the collection of investigational data. The overall aim is to facilitate comparison of results across studies and more effectively aggregate information into significant metadata results.

Validating the prototypeNeuroradiologists at the NICoE reviewed 50 image studies, with each image assigned to two readers. Both readers used the prototype system to describe and assess the abnormalities observed. Comparison of the resulting readings highlighted several injury types that required more clearly defined specifications to help ensure that different readers will classify a specific type of brain abnormality in the same way.

Noblis is refining the system to provide the neuroradiologists at the NICoE with clearer definitions of some injury types and reporting tools that inform readers about common discrepancies in their findings.

Michael Jarrett is founder and CEO of QuesGen Systems. Contact him at mike.jarrett@quesgen.com.

Dennis V. Kennedy is a manager at Noblis. Contact him at dennis.kennedy@noblis.org.

Linking Brain Injury to Behavior

Figure A. Partial screen shot from a system to help neuroradiologists be more consistent in reporting MRI findings. The injury descriptions are narrowly defined, and can appear as dropdowns or checkboxes, allowing for easy comparison of findings across multiple image readers. This reader has identified a diffuse axonal injury from the selections at left (not shown) and three lesions.

A N O B L I S P U B L I C AT I O N24

Clear definition is an art and requires testing to ensure that

the definition is indeed clear, as well as being acceptable to

likely users and easy to communicate. Evolving such definitions

requires clinical and coding knowledge that can lead to the

identification of all the codes appropriate to that definition.

Data use starts with the intent to define some concept that

is usually an abstraction of the underlying data elements. This

abstraction should hold regardless of its use, but it must be

unambiguous. Aggregation of codes or other values cannot

represent this intent if the definition is easily subject

to interpretation.

For example, the definition of the higher level concept,

“secondary diabetes” might be:

“A condition of abnormal metabolism of glucose and/

or the production of insulin by the body that is caused

indirectly by some other health condition, external drug,

or chemical.”

This definition is clear, likely to be understood and accepted

regardless of its purpose, and is specific enough to disallow

too much interpretation about what should be aggregated to

support the definition. The last characteristic is central to quality

aggregation, since part of defining intent is to explicitly identify

concepts that the aggregation should include or exclude. For

example, secondary diabetes means different things to different

people, and the code aggregation will reflect those differences.

Defining higher level concepts in a way that is clear to everyone

involved in the aggregation process allows an analyst to

define those codes to meet this shared understanding of what

“secondary diabetes” should be.

For example, if the intent was to exclude gestational diabetes

as part of the concept of “secondary diabetes,” the previous

definition should state as much. Thus, in addition to defining the

basic concept, the author might add, “This definition excludes

gestational diabetes.”

There are no absolutes in judging clarity, but if most people

reading the definition can articulate the same concepts and

act consistently in identifying related codes, the definition is

probably clear enough.

Selecting codes that match intentOnce analysts have properly defined the aggregation’s intent,

they must choose codes that will represent that intent. Each

category or aggregation consists of a code set, but selecting that

set is not always straightforward.

For example, in looking for codes that represent “fractures

of the proximal femur” analysts must consider “hip fractures,”

“subcapital fractures,” “intracapsular fractures,” and

“intertrochanteric fractures,” (to name a few)—all of which are

proximal femur fractures. Unless the analyst has sufficient

expertise in understanding what a proximal femur fracture could

entail, the inclusion of these codes will not be obvious.

Healthcare data is about patient conditions and the services

delivered to improve or maintain those conditions. Clinical

insight into the definition of categories and into the appropriate

codes that relate to those categories is critical to assure the

accurate abstraction of data to develop correct and relevant

knowledge. This process cannot be left to programmers or

analysts with limited knowledge of clinical subject areas or

coding principles. Rather, developing reliable and accurate

Figure 2. Two aggregation schemes associated with the classification of “Streptococcal Pneumonia.” Taxonomies can help in bucketing data into specific categories, which aids comparisons.

25H E A LT H C A R E T R A N S F O R M AT I O N

aggregation requires a team comprising clinicians, business

analysts, executive sponsors, database experts, programmers,

and coding specialists. All members must closely collaborate

to ensure that the intent of aggregation modeling yields

a consistent, accurate data interpretation. The sidebar

“Aggregation Team Task List” describes some tasks that an

aggregation team should complete.

Creating an aggregation modelSeveral models are available to assist data aggregation,

including taxonomies, ontologies, and groupers—aggregation

based on multiple factors. No model is superior; rather,

suitability depends on the aggregation’s intent.

TaxonomiesTaxonomies generally involve categorizing concepts related to

a dataset so as to stratify data exclusively and hierarchically.

The taxonomy establishes a scheme for bucketing data into

specific categories and rolling up those categories exclusively

to categories at a higher level. Taxonomies enable a category

comparison and balanced reporting with no double counting of

data. In accounting terms, a taxonomy is a balance sheet: the

sum of all dollars paid in all categories balances to the sum of

the entire dataset independent of aggregation.

Combination codes, in which a single code reflects multiple

concepts, present a particular challenge, since the modeler

must decide how to structure the taxonomy. Figure 2 shows two

possible choices. For example, if the modeler decides to structure

the taxonomy on a model of body systems, then aggregation

for streptococcal pneumonia would include codes categorized

to “Respiratory System,” “Lower Respiratory Infections,” and

“Pneumonia.” If the modeler decides to structure the taxonomy

on cause or etiology, aggregation would include codes categorized

to “Infectious Agents,” “Bacteria,” and “Streptococci,” and

“Streptococcal Infections.”

Developing models involving combination codes requires

considerable work to ensure that the aggregation scheme is

communicated sufficiently that individuals using these models

understand the rationale for developing the taxonomy.

Models involving combination codes are challenging when the

goal is to aggregate historical codes from different standards in

the same database, such as the aggregation of codes from both

ICD-9 and ICD-10. Definitions for categories in these types of

models will need to be at a high enough level to be comparable

regardless of code standard.

OntologiesAggregation can also be independent of the underlying

scheme. An ontology aggregation is agnostic as to its place in a

hierarchical scheme, but simply represents a logical standalone

code grouping. Unlike taxonomies, ontologies provide the latitude

to include all codes that meet the category’s criteria without

mandating that the categorization be exclusive or have some

restrictive relationship with other categories or groups. Rather,

any given code can belong to any number of aggregations or

categories and have a limitless number of relationships of any

number of types between categories and other elements.

Ontologies are ill suited for category comparisons or in

aggregations that require data balancing or the bucketing of

data to a single category. The strong advantage of ontologies is

their ability to gather all relevant data for a specific or focused

intent. Table 2, for example, shows the possible relationships of

“Streptococcal Pneumonia” code to other conceptual aggregations.

Ontologies are also helpful in resolving issues of mixed

granularity by creating canonical concepts at a level where codes

in different standards and at different levels of granularity can

map to one or more canonical concept. Figure 3 illustrates the

concepts defined in a clinical scenario and how those concepts at

a canonical level map to codes in different standards.

GroupersIn many cases, aggregation is based on multiple data elements

that are grouped according to various factors. Third-party

groupers such as diagnosis-related groups (DRGs), risk adjusters,

and episode groupers often use a variety of data elements related

q Create a data governance structure to oversee and approve decisions and to set criteria for information quality.•Who approves?

•What is the process?

•What are the documentation standards?

q Identify the strategy for aggregating codes and other data.•Will this be a fully defined hierarchical model?

•Will we use nonhierarchical aggregation?

•Is all aggregation to a unique bucket, or can we aggregate data into multiple buckets?

•Will we use a third-party grouper?

q Clearly define the aggregation intent for each category.

q Use the appropriate tools and resources to ensure the inclusion of all codes and only those codes appropriate for each category defined.

q Establish and implement a clear process for oversight and approval of all logic used to create the aggregation.

q Establish analytic and reporting models to monitor and test all aggregation schemes to ensure that they met the aggregation's intent.

Aggregation Team Task List

A N O B L I S P U B L I C AT I O N26

to demographics, services, timeframes, diagnostic codes, or other

key healthcare data parameters to arrive at some categorization

or score for a specific purpose. For example DRGs aggregate

claims data not by diagnostic or procedural codes, but by

combinations of these, as well as by age, sex, and other factors

that drive a particular data aggregation for payment analysis.

This aggregation model is considerably different from one based

on a single code dimension.

Many groupers become a functional black box operating under

the assumption that all inputs are correct. However, if the inputs

are not correct or the internal aggregation logic is flawed, the

output could be totally unreliable.

Confirming aggregation quality can be challenging if the logic

is hidden and no one can validate assumptions about the input.

The ability of these groupers to perform their intended functions

is particularly constrained when code standards are mixed, such

as in the transition from ICD-9 to ICD-10.

Establishing standardsRegardless of the specific aggregation intent, it is important

to have a single point of truth: a particular code grouping must

consistently map to one concept. If a variety of rules or reports

reference “secondary diabetes,” but each group codes differently,

confusion and unanticipated results are likely to compromise

data aggregation quality. Standards ensure a central point for

the definition and the code set that represents the definition’s

intent so that all analysts who might reference that definition

will be empowered to use the same code grouping.

Monitoring information qualityCode standards and aggregation intent are likely to change

over time as analytic reporting or other uses of aggregated

data expose aggregation errors. To maintain information quality,

organizations must put in place a defined process for upholding

data aggregation quality assurance. This process should be

a standard part of warehouse and datamart development,

testing, and maintenance. The organization should also

bring in subject matter experts to review both the aggregation

method and test case outputs to ascertain that aggregation

results meet expectations.

The Agency for Healthcare Research and Quality and the

National Committee for Quality Assurance have extensive

processes to monitor a variety of code standard changes so

that updates of their aggregation categories are consistent with

the changes in the underlying standards. Although most other

organizations have processes to maintain and update codes sets,

many lack the same rigor in updating the aggregation of codes in

warehouses, datamarts, or reports. The result is a disconnect in

aggregation use for analytics or other business purposes.

Streptococcal Pneumonia

Relationship Ontological Concept

Is a type of Pneumonia

Is a type of Infection

Is a condition of Pulmonary system

Is a condition of Lung

Is caused by Streptococcus

Is a Communicable disease

Table 2. Possible relationships between “streptococcal pneumonia” and other concepts in an ontology.

Figure 3. Mapping clinical concepts to code standards. Ontologies are useful in resolving issues arising from multiple standards mapping to single concept. (Image courtesy of Health Data Consulting.)

Medical Scenario:

A [27 year old] [male] patient is seen in [follow-up] for a [Smith’s fracture] on the [right] that was exposed through an [open wound] with [minimal opening tissue damage]. The fracture has [not healed after 6 months].Though not explicitly stated in this scenario certain expressions imply other concepts:“Smith’s fracture”>>[fracture], [radius], [distal], [volar displacement], [extra-articular], [displaced]“minimal opening and minimal tissue damage”>>[Gustilo classification 1]“not healed after 6 months”>> [nonunion]

ICD-10-CM552541M

Subsequent Encounter

Nonunion

Fracture

Right

Radius

Open

Distral

Displaced

Dorsal Angulation

Extra-articular

Gustilo Type I

Smith’s Fracture

Subsequent Encounter

Nonunion

Fracture

Fracture

Fracture

Right

Radius

Open

Distral

Radius

Open

Distral

Displaced

Dorsal Angulation

Extra-articular

Radius

Open

Distral

Displaced

Dorsal Angulation

Extra-articular

Gustilo Type I

Smith’s Fracture

Smith’s Fracture

Male

Age 27 Years

ICD-981352

SNOWMED-CTDD-12392

ConceptsConcepts in Scenario

27H E A LT H C A R E T R A N S F O R M AT I O N

Supporting aggregation qualityWithout a solid infrastructure to support requirements

such as standardization and monitoring, any attempt to raise

data aggregation will soon lose momentum. Organizations can

establish this infrastructure gradually, taking the steps in the

“Action Items” sidebar.

Awareness of aggregation qualityOne of the first steps toward establishing that infrastructure

is to raise awareness across the organization about the

importance of data aggregation and the role of clearly defined,

clearly documented, consistent, and validated aggregation

models. Everyone must be educated about the indispensible

role of aggregation in generating reliable and accurate

healthcare information.

Governance structureA governance structure is another essential part of support.

For example, a large healthcare quality initiative in New York

State aggregates data across multiple payers to produce

information about the quality of care delivered by providers

in certain clinical areas. A major part of this process was to

establish a governing body, requirements for data quality and

data aggregation, and a system for accepting or rejecting data.

The initiative created key data quality metrics that measured

the data’s ability to meet specifications that all participants had

reviewed and agreed on. The governance model includes ongoing

monitoring and reporting of data issues and challenges so that

all participants and information users are aware of the resulting

information’s potential limitations.

Regardless of the particular structure, the governing

body must have the authority to demand data quality as an

organizational commitment. It must also have the authority to

hold other parts of the organization acc ountable to institute

and maintain that quality through the monitoring of test data,

aggregation, and higher level analysis.

Investment in process and peopleInvestment in process and people is also part of an

infrastructure to support quality aggregation. Whether the effort

is as large as a multipayer, multistate aggregation or as small

as an in-house information initiative quality data, aggregation

and information don’t just happen. Research is needed to evolve

clear definitions and code mappings and to create aggregation

models that are accurate, reliable, and consistently applied. This

investment will require a commitment of time, people, and tools

necessary to do the job well. The people commitment entails not

only staff training but also the enlistment of subject experts to

validate data and aggregation quality at the business and clinical

levels. Experts can then clearly define requirements on the basis

of this validation to support aggregation that is consistent with

user intent.

Data aggregation is both an art and a science and

must be supported and maintained as a standard

with the same care given to the standards that

define the underlying data elements. Getting to the

proper aggregation of data and ultimately reliable and accurate

information requires a shared understanding, sense of values,

and vision. A shared understanding recognizes the challenges

associated with the underlying data elements. A shared sense

of values recognizes the importance of good quality data and

the need to confirm that quality. A shared vision recognizes

the priority of obtaining quality data, quality aggregation, and

quality information.

Ultimately, all three of these dimensions must be part of any

effort to raise data aggregation, particularly the institutional

capacity to support a commitment to that effort. As healthcare

becomes increasingly more complex, there will be less room

for suspect data aggregation. Organizations must work

toward evolving a system that turns data into reliable,

quality information. n

Dr. Joseph C. Nichols is a principal at Health Data Consulting (http://healthdataconsulting.com/) and a board-certified orthopedic surgeon with considerable experience in health information technology. Contact him at joenichols@healthdataconsulting.com.

High-quality data aggregation is within reach of many organizations, but these steps require a concerted effort across the entire enterprise.

• Raise awareness organization wide about the importance of data aggregation and the role of sound aggregation models in supporting reliable healthcare information.

• Define a governance structure that has the accountability and the authority to demand data quality.

• Invest in the research needed to create reliable aggregation models that are consistently applied.

• Establish standards that specify data values and aggregation schemes in sufficient detail to create measures of information quality.

• Enlist the subject matter experts necessary to validate data quality and data aggregation quality.

• Monitor information quality through well-defined and consistently used metrics.

Action Items