White Paper

by Gary Angel, President and CTO, Semphonic

Effective Measurement and Analytics in a Multichannel WorldTaming the Many-Headed Digital Marketing Beast

ABSTRACT

The explosion of social media, digital media and digital channels has created a whole set of challenges to all marketing

organizations. Not the least of these challenges is how to measure, analyze, and optimize this multi-headed beast. Traditional

media tracking focuses on a small number of quasi-independent channels measured in the simplest terms (reach and

audience). Digital is different in every respect. Digital channels are tightly bound, and the performance of each is heavily

dependent on the performance of the entire system. Both cannibalization and support are common. Digital channels lack a

single common metric because there‘re no equivalent to GRPs (gross rating points). Digital channels lack even a common

type of metric – with Awareness, Engagement, Conversion and Retention (and a host of sub-metrics under these categories)

vying for attention and credibility. Worse, focusing on top-line or siloed metrics will nearly always miss the point.

All of this makes the effective measurement and optimization of digital marketing efforts extremely challenging.

Challenging or not, however, it’s a problem that can’t be avoided. As marketing budgets shift into ever-increasing digital and

social share, it’s no longer acceptable to ignore the problems around digital channel measurement and optimization. So the

creation of enterprise-wide, digital marketing dashboards has become a priority at many leading businesses today.

However, digital marketing dashboards, as commonly developed, are uninformative at best and misleading at worst.

In this white paper, we’ll show why traditional best-practices in digital reporting fail to capture both the nature of the

customer journey and, perhaps more surprising, even the right information about behavior WITHIN the silos. We’ll present

a methodology for doing the job right. We’ll also highlight a technology solution that makes the integration of the necessary

data, as well as the construction of the necessary analysis and reporting possible.

For the purposes of the examples in this whitepaper, we’ll be using Anametrix for multichannel analytics and reporting, Google

Analytics for web analytics, DART for ad serving, ViralHeat for social media analytics and Salesforce.com for CRM.

THE PROBLEM OF INTEGRATED MULTICHANNEL MARKETING MEASUREMENT

Digital marketing in today’s world requires integrated multichannel measurement and analysis. However, enterprise

measurement and dashboarding systems attempting to provide that measurement have suffered from three primary

problems: (1) they lack methods for joining and understanding customer engagement across silos; (2) they focus on top-

line performance without providing the necessary context to create real understanding of the interrelationships leading up

to them; and (3) they capture only the state of the system, but completely fail to describe the system itself and its potential

levers. In the next three sections, we’ll explore each of these problems and show how they cripple most digital marketing

measurement and analysis efforts.

1 The Problem With SilosTraditional mass-media buying featured relatively little interaction between channels. While Adstock might increase

or decrease depending on exposure in multiple channels, the basic measurements of reach and audience were

independent. You didn’t reduce or improve your radio GRPs by buying TV. In digital, however, that just isn’t true.

Digital marketing works quite differently. Your search traffic (including your PAID search traffic) is heavily dependent on your

offline media spend, your display ad budget and even your organic search listing efficiency. This deep interdependence is

true across the board for digital: from social to mobile to fixed web to search and display. Unfortunately, as media buying and

measurement have migrated over to digital, many of the traditional habits and expectations borne from traditional marketing

have accompanied this transition.

Most marketing measurement and analysis efforts are based on the aggregation of siloed channel data. When this data is

aggregated, it’s not integrated at the prospect or customer level. There’s usually little or no effort made to understand the full

customer journey or the multiple marketing and digital touchpoints along the way.

This creates severe problems with campaign optimization and attribution. You can’t properly credit campaign success

without understanding the full set of customer touchpoints and the sequence in which they’ve occurred. Reports showing

revenue is up in every marketing channel but down in reality is all too common. If you are relying on the integration of

siloed campaign measurement for each channel for optimization and dashboarding, you’re almost certainly seriously

misrepresenting the reality of your marketing effectiveness.

The problems don’t end with attribution. Siloed data makes almost every type of analysis problematic.

• Want to understand the role of mobile in the customer journey? You can’t do that unless you can integrate

mobile and fixed web data.

• Want to identify segments to target by channel? It’s impossible without integrating customer data and

demographics with channel behavior.

• Want to measure the true ROI of your social marketing efforts? How do you do it without reviewing how social

media correlates to changes in campaign returns across each marketing channel?

• Want to understand campaign performance relative to your industry? You need to integrate competitive set

and NPD -type online research data with your reporting.

• Want to understand the true value of a content consumer? You need to integrate your DART-type ad serving

data with your web behavioral data.

• Want to understand the total impact of your spend across all channels? You simply have to combine spend,

impact and outcome data in a single view to evaluate marketing performance.

At every level of marketing, the failure to integrate data across silos creates huge – and dangerous – gaps in understanding.

It’s dangerous because without the holistic understanding that only multichannel analytics can deliver, marketers will all too

often come to incorrect conclusions regarding the effectiveness of their marketing spend WITHIN each silo.

2 The Problem With Focus on Top-Line MetricsIt’s probably no surprise that siloed metrics are a plague on your measurement house. At least, inside each

channel you have a clear and powerful strategy for reporting on success. Your measurement department has

almost certainly discovered the current religion in digital measurement: don’t overload on numbers. The key

to successful dashboarding and reporting is finding a small set of marketing key performance indicators (KPIs) that are

understandable and immediately actionable.

Chances are, that’s exactly what your enterprise has adopted – a small set of key metrics like Site Conversion Rate,

Campaign Conversion Rate, Cost per Conversion (CPC), Customer Retention Rate, Customer Referral Rate, Average

Number of Service Calls per Day, and other similar KPIs throughout the customer engagement lifecycle, all laid out in big

numbers with great fonts, pretty colors, big trend arrows and lots of Tufte-inspired whitespace.

Unfortunately, these reports deliver neither understanding nor actionability on their own.

Suppose I walk into your office and tell you that Site Conversion rate is up 5 percent. You’ll probably be delighted. Now

suppose I walk into your office and tell you that your Site Search Engine traffic is down 20 percent. That’s bad, right? But

would you realize that, in all probability, the two metrics are related and are, in fact, telling you exactly the same story? As you

drive less early-stage traffic to your site via natural search, your Conversion Rate will go up. Understanding the relationship

between parts of the system is fundamentally different and more important than understanding the state of any single

performance indicator.

What happens if you find out that the CPC of a specific campaign is quite low compared to others? You’re happy about the

high ROI, right? What if you found out your CRM call volume is up during that same time? That should signal bad news.

Unless you look at the entire system, what you may not realize is that the high ROI campaign that you were delighted with

happens to be driving customers that also lead to more service calls or higher return rates, significantly eroding your profit

margins.

What about revenue? Surely, it’s impossible for a revenue increase to be bad! Not only isn’t it impossible, it’s common.

For example, company A creates a re-marketing program that sends a 10 percent discount offer to all cart abandons.

Unfortunately, consumers quickly game the system and abandon the cart to get the coupon. To make matter worse, they

share the news of the cart abandonment discount in social media to let their friends take advantage of this great offer! Gross

Revenue increases with the remarketing program and the ensuing social buzz that you may or may not be tracking. However,

Net Revenue declines due to reduced margins. Short-term revenue gains that sacrifice margin are frequent. Poor reporting

systems make this type of system interdependency and correlation difficult or impossible to spot or understand. Worse yet,

they can create the wrong indicators of and incentives for actual performance.

There just is NO SINGLE METRIC THAT CAN BE MEANINGFULLY INTERPRETED WHEN VIEWED IN ISOLATION.

What’s needed is a way to create meaningful KPI context so that movement and level can be easily consumed and

understood.

3 The Problem With Showing the Current StateYou are the manager of the long-play videos site section. And, hurrah, your company has really gotten its act

together. Your performance is tied to specific site goals, and your company has established a clear target for

long-play video on the site: a 20 percent year-over-year increase.

That’s great. There’s nothing like clear goals tied to real incentives to sharpen the mind and drive performance.

You ask your analytics team to create a report on long-play videos. Here’s what they come back with:

This is the classic measurement view for an organization with a well-defined success metric, a clear goal and robust ability to

measure. You can, with this report, instantly know whether you are on-plan or not.

That’s great.

However, isn’t something missing from this report? While you can instantly see whether you’re on-plan or not, you have no

idea why. As long as you’re on-plan, you’ll never look at this report. It isn’t really a report at all. It’s an alert.

If the only function of a report is to alert you when you’re not on-plan, why not just deliver that:

Really! Why not just send this if the only meaningful intelligence in your report is that you have something to worry about. It’s

less expensive and more impactful.

When report builders consider this kind of question, they often decide that they do, in fact, need to deliver a little more. So

they start adding to the report. They add traffic sources since changes to Search Traffic might easily drive changes to the

long-play video site section. They add a report on top Exit Pages, since maybe some videos lost traffic too easily. They add

a report on top Entry Pages to help spot videos that might be getting more direct traffic and could be further supported. And

so on. Pretty soon, there’s a report for every different kind of data that might help a decision-maker understand why they are

not on-plan.

Now there’s too much data. With no connections between all these reports it’s nearly impossible for a decision-maker to

navigate the reports and decide what’s interesting, what’s meaningful and what’s just noise. What’s needed is a method of

going beyond the “current state” to show the actual levers of change.

SOLVING THE PROBLEMS OF MULTICHANNEL MEASUREMENT: ONE HEAD OF THE BEAST AT A TIME

1 Multichannel Data Integration: Solving the Silo ProblemIn today’s digital world, there isn’t one single integration challenge – there are many. With the explosion in digital

and social channels and the increasing importance of understanding the relationship between offline and online

touchpoints, the idea of doing siloed measurement just isn’t reasonable. On the other hand, that same explosion

of sources makes integration daunting. It’s critical to have a real strategy about what you need to integrate and how that

integration can occur.

There are two common levels/types of integration in digital analytics:

1. Integration of Two Streams of Data at the Visitor Level

2. Integration of Stream Data Into Row-based Customer Data

The challenges in joining multichannel data streams are solved by the same segmentation techniques that address the

challenge of top-line metrics. Segmentation, while a classic, marketing-analysis technique, is also a very effective data

reduction and aggregation technique. By creating the two-tiered segmentation we’ve described (visitor/visit-type), you can

reduce a complex chunk of stream data into a small number of discrete variables attached to a single row. These variables

describe the type of the visit, its measured success and its recency. By summing visit types, successes, recency and

frequency at the visitor level, you have a powerful but very terse description of a large amount of customer behavior.

This type of aggregation by segmentation is an obvious solution to our problem #1 since it creates a record that can easily

be joined to a customer record. It also provides a ready-made solution to the first challenge. The beauty of the Two-Tiered

model is that it can be applied to a very broad number of data streams, including offline data streams. It’s perfectly possible

to model mobile engagement, social media buzz, web visits, calls to the call-center, ATM visits, branch or store visits and

much more with the same simple model of type, recency, frequency and success. By supplying a unifying model to different

types of data streams, each can be meaningfully reduced from stream data into discrete records that can then be intelligibly

joined at the customer level. This delivers a holistic understanding of how prospects and customers engage with the brand

across multiple channels throughout the lifetime of their engagement.

2 Segmentation Solves the Problem of Focus on Top-Line MetricsThe simple fact is that aggregate reach, engagement, conversion and retention metrics – from number of

Twitter followers and Facebook likes to Total Site Traffic to Conversion Rate to Campaign ROI to Churn Rate –

are nearly all worthless. To be meaningful, a metric needs to be placed in the context of “who” it’s about and “what” those

customers were trying to accomplish. Audience and Customer segmentations are the foundation of interesting measurement

and analysis.

If you walk into your bosses office and say something like, “Good news, traffic is up 5 percent on the Website,” or “bad news,

we have 5 percent more customer churn,” I expect the boss to ask “With whom?” followed by “why?”

Knowing the “who” behind a number is nearly always critical. With traffic, it’s particularly important. Bad traffic is ubiquitous

on the Web. It’s cheap and too easy to find or buy. If you are going to measure campaign performance, you must measure

something beyond traffic, otherwise you’ll almost certainly drown in unqualified and likely useless traffic.

Basically, if you don’t know the audience behind that traffic increase, you really don’t know anything. Which campaign and

channel is driving the visit? Do we have any demographic data to gain more insight into who the visitors are? What’s the

K-factor (virality) associated with the visitors? Are they driving social-media driven visits in addition to the paid campaign-

driven traffic?

In traditional marketing, the “who” question was pretty much all there was to segmentation. In digital, that’s not true. Because

once you’ve answered the “who” question, the next question I expect your boss to ask is “And why did they engage?”

In pretty much every digital channel, your customers set the agenda for an engagement or touch. It’s your job to figure out

what they had in mind and match your business goals to their intent. In essence, understanding what your customers are

trying to accomplish during every engagement is part and parcel of understanding whether you are successful or not. What

are your chances of making a product sale during a Customer Support visit? Zero. So if customer support visits are included

in your site-wide Conversion Rate, what are you really measuring? In site-wide metrics, typically all you’re seeing is noise.

This second type of segmentation – visit- or intent-based segmentation – is fairly unique to digital, and it’s why we say

digital requires a two-tiered segmentation. The idea is simple: every metric should be in the context of a “who” and a “what.”

Building this type of segmentation can be hard; indeed, creating visit-intent segments is one of the most challenging but

fruitful tasks in digital analytics. It’s fruitful because your dashboards and reporting are only useful if they’re built around this

concept.

In essence, building good dashboards, scorecards and report sets starts with segmentation, however segmentation is only a

piece of what’s required.

3 System Reports With Multichannel Integration and Embedded Segmentation Solve the “Current State” ProblemNearly all enterprise reporting has concentrated on finding variables that can show the state of a system.

There’s no denying that understanding “where you’re at” is useful. But if knowing “where you’re at” is important, it’s largely

in the context of knowing where you could go. We’ve developed great reports for showing latitude and longitude; we haven’t

done much for filling the rest of the map.

A really good report set will capture more than the “state” of the system by showing the connections between each variable

in the system and report on the changes in those connections. To achieve this kind of report requires building a model of the

system itself.

Suppose, that you want to understand how social media – whether social marketing efforts driven by your company or other

social buzz mostly out of your control – is impacting paid campaign effectiveness. The below Social Insights report from

Anametrix overlays data from social analytics solutions such as ViralHeat or Radian6 with your campaign data across each

channel to reveal interdependencies and relationships. It provides marketers with immediate insights into how social buzz

correlates to campaign performance, including traffic and conversion metrics, highlighting whether social activity is impacting

campaign results, or whether certain campaigns are going viral, and driving additional social media-driven visits.

Further segmentation and filtering also enable you to ascertain if certain channels and / or geographies are more likely to

being affected by any specific social buzz elements.

As you can see, when you incorporate these multiple data sources in a report, you’ve done so much more than capture the

state of the system. You’ve helped a decision maker understand the factors that drive the system, the degree to which each

factor contributes and the places where potentially significant improvement opportunities exist.

PUTTING IT ALL TOGETHER

In this white paper, we’ve described three fundamental challenges in digital measurement, analysis and reporting. Nearly

everyone can see and understand (and, unfortunately, has experienced!) these challenges. For each of these challenges,

we’ve also described a solution. Segmentation, multichannel integration based on intelligent handling of streams, and

system-based reporting using graphical flows, are the core methods that can drive your enterprise measurement to an

entirely new level.

It will probably be no surprise, however, that the most common existing toolkits for doing marketing analytics and reporting

aren’t really up to the job.

Web analytics tools have improved their segmentation capabilities considerably in recent years. However, many still lack

significant segmentation capabilities. In addition to frequent limits on Visitor-based segments, they often make it difficult or

impossible to overlay customer (visitor) segmentations from the offline world, lack capabilities for data-driven segmentation,

limit cross-tabulation of segments (such as Visitor x Visit Type), lack cohort capabilities, and provide no way to integrate

segments into back-end processes. Segmentation, as a result, is often unavailable in reports and analysis.

At the integration level, the lack of segmentation is fatal to the strategies suggested here. In addition, there are numerous

limits on the type of integrations that can be accomplished, the ability to effectively use integrated data and constraints

caused by the cost and timeliness of creating and supporting the integrations themselves. Any attempt to integrate CRM

data, Social Media Data, even Campaign Impression data into these systems is nearly always expensive AND fruitless.

Most organizations have decided to tackle these integrations in internal warehouse-based systems. Given the poorly

understood nature of joining digital streams, this strategy has not been widely effective. Furthermore, internal warehouses

suffer from a fatal lack of agility. Digital isn’t just one source. It’s many. From web to mobile to display data to PPC to Pandora

to Facebook to email, there’s a seemingly never-ending list of important channels and the associated and often quite

complex data. Traditional IT just can’t keep up.

Finally, at the user-interface level, it’s simply not possible to build really good “system” models in most measurement and

analytics tools. The visualizations are challenging to create, as are the embedded models. It’s critical to have a reporting

system focused on the customization of sophisticated dashboards instead of drag-and-drop of charts and tables.

The Anametrix Digital Analytics platform has been designed from the ground-up to solve exactly this set of problems.

Created by the team of analytics experts from WebSideStory, it’s engineered to tackle the next generation of digital analytics

challenges, specifically the multichannel integration, analysis and visualization of data. Unlike other solutions, it provides a

natural platform for the integration of digital and non-digital data sources. It even includes a number of built-in “external”

integrations (such as U.S Census data) that are often extremely useful in building system models. It provides an open data

model on which the User Interface can work, which is a critically important feature for handling multiple channels and types

of data.

As a segmentation engine, it provides all the front-end capabilities you’d expect including unlimited filtering at the Visit and

Visitor level. It also supports segmentation on the back-end at the data-level, making it possible to fully support the type of

integration with segmentation suggested here as the most appropriate method for joining disparate streams of data.

From the User Interface perspective, Anametrix is specifically designed to support rich customization of any of number of

data-extraction models, along with predefined reports that are customized to visualize the multichannel marketing process

throughout the customer engagement lifecycle. This interface is built on top of the data model, so it works very well for all

sorts of data types and data structures. It accommodates the most critical features in system-based reporting: extensive

filtering, custom elements and complex calculations.

Here, for example, is a sample Performance Overview Report designed to evaluate campaign ROI across channels…

…and an Executive Overview Report designed to understand channel performance compared to previous periods and

goals:

Both reports illustrate many of the fundamental challenges and solutions we’ve discussed in this whitepaper. Anametrix uses

multi-channel integration to measure channel performance for each campaign based on integrated cost data. It also uses

segmentation and modeling to help create a powerful view of campaign success, across each channel, based on customer

lifetime value.

In this next Anametrix dashboard, Ad, Web, Social, and Business Goal data are integrated along with Lifetime Value

modeling to produce a simple, comprehensive A/B comparison of end-to-end campaign effectiveness.

Typically, a decisionmaker would focus on the first three-to-five metrics extracted from email marketing tools and web

analytics, to determine performance. In this case, the top five metrics clearly favor the campaign on the right. It’s only after

reviewing the entire system, including campaign virality, customer lifetime value and true ROI derived from CRM, social media

and other data sources, that we realize the campaign on the left is the real winner.

As you can see, combining digital segmentation, multichannel integration, and system-based reporting gives decision

makers a holistic perspective into how customers are engaging with their brand through out the lifetime of their engagement.

It lets them identify the correct levers to continually optimize marketing performance to further drive revenue and profitability.

Ultimately, driving your digital marketing effectiveness to the next level takes both methodology AND technology.

SUMMARY

Traditional approaches to digital marketing measurement, analytics, reporting and dashboarding have assumed that it was

possible to treat customer engagement with the brand in silos, to focus on high-level campaign or site metrics and to capture

the “state” of a key variable or system.

Each of these assumptions is wrong, and they have fed the creation of an ever-growing set of enterprise digital reports and

dashboards that are neither interesting nore useful.

The remedies to these problems aren’t necessarily simple or easy, but remedies do exist.

By framing digital data in terms of audience and intent, it’s possible to create a meaningful context around the customer

journey and eliminate the errant focus on top-line metrics. By navigating the tricky task of integrating digital stream data from

multiple channels, it’s possible to drive down to a unified view of customer behavior – across all engagement touch points

– that can solve the silo problem. By embedding models into reporting, it’s possible to capture not just the state of a digital

system, but the levers of change, as well as the connections and interdependencies between each of them.

As with most things (not just digital), good solutions take real work and require good tools. To solve these problems, you need

a different set of capabilities than are inherent in most of today’s digital reporting tools. You need an open data model capable

of flexible integration of stream data. You need to be able to build segmentations on the back end to facilitate that integration.

You need to be able to create complex dashboards that can handle different data types, provide embedded segmentation,

allow for the integration of models and calculations, and support the visual representation of flows and relationships and

interdependencies.

Digital Marketing is, truly, a many-headed beast. It’s multiplicity of channels present deep challenges to any technology and

any approach. Traditional IT approaches lack the agility to cost-effectively integrate so many complex data sources. Web

analytics solutions lack the openness to bring multichannel data together and lack the power to analyze and report on it

effectively.

It’s, nevertheless, a beast that must be tamed. Measurement is the key to effective digital marketing. And multichannel

integration is absolutely essential to effective digital measurement and analysis. Putting the right methods and the right

technology together can make all the difference for ensuring marketing is, indeed, a predictable and effective revenue driver

for your business.

About Semphonic

Semphonic is a cutting-edge digital measurement and data

analytics consulting firm providing our clients with deep

insight and strategy into their complex customer interaction

challenges across the digital channel. Semphonic has deep

roots in the analysis of large-scale databases, combining

the power of segmentation, predictive modeling, and deep

statistical analysis to help clients identify and execute high

ROI, data-driven marketing campaigns. Founded in 1997,

Semphonic’s US headquarters are in the San Francisco

Bay Area with offices in Boston, New York, Washington DC,

Portland and European headquarters in Berlin.

For more information visit:

Website: semphonic.com

Twitter: @semphonic

Blog: semphonic.blogs.com

ABOUT ANAMETRIX

Anametrix transforms marketing with digital analytics. We

collect, analyze, and make sense out of data across all

channels in real time to enable marketers to discover new

truths about customers, prospects, and the market at large.

Anametrix delivers 360-degree visibility into business data

to uncover new trends and hidden correlations, explore new

relationships, and deliver a bigger and more predictable

impact on revenue. Founded in 2010 by the trailblazing

web analytics team behind WebSideStory, Anametrix has

headquarters in San Diego, Calif.

For more information visit:

Website: anametrix.com

Twitter: @anametrix

Blog: http://anametrix.com/blog