Unlocking the Value in Telecom Data (BI)

8/3/2019 Unlocking the Value in Telecom Data (BI)

http://slidepdf.com/reader/full/unlocking-the-value-in-telecom-data-bi 1/8

Unlocking the value intelecommunications data

A Detica white paper

© Detica 2010



Unlocking the value in

telecommunications data

2 Unlocking the value in telecommunications data

Executive summary

In today’s increasingly competitive

and cash-constrained operating

environment, telecommunications

companies are seeking to extract

maximum business value from the

data they collect. To address thesechallenges they are turning to data

warehouses which offer tried and

tested technology and familiar

processes. However, more and

more of the business insights are

locked up in human data, which has

traditionally sat outside the data

warehouse. By identifying these

new data sources and implementing

new approaches for processing them,

telecommunications companies can

make the efciency savings theyneed, increase cross-selling and

improve customer experience – and

ultimately make an impact on their

bottom line.

A missed opportunity

In an increasingly competitive, regulated

and nancially-constrained market,

the future for telecommunications

companies – telcos – looks challenging.

The price they are able to charge for

‘business-as-usual’ voice trafc has

declined by 10-15% per annum over the

last decade, and European legislation

is continuing to drive down roaming

charges and termination rates. The ‘data

explosion’ has driven up network costs

yet revenues from consumer data services

have failed to ll the gap because ‘over-

the-top’ service providers are invading

markets traditionally dominated by

large telcos. To add to their woes, the

current economic crisis is also reducing

discretionary spend on communications.

These challenges are increasing the

pressure on telcos to deliver new revenue

streams while simultaneously improving

operational efciency, and identifying

revenue leakage from fraud and bad

debts. In their attempts to achieve this

difcult balance, operators are turning

to data warehouses – monolithic stores

of structured data collected from across

the enterprise – to gain operational and

customer insight.

However, telcos are failing to unlock the

full value in their data. More and more

of the data they are able to collect or

access is unstructured – such as customer

service complaints or emails – or does

not t the ordered model needed by

most data warehouses. This data is

often ignored because, although it might

provide valuable business intelligence,

it is too difcult to analyse or simply

not perceived as being important. For

example, on the operational side,this data typically records hundreds

of millions of detailed transactions

each day – for instance, call records or

network events – and therefore generates

vast data volumes which can only be

stored for short periods before being

overwritten. Low-level transactional data

often requires extensive manipulation

and transformation before it is in a

t state to be analysed and exploited

effectively. This pre-processing requires

extensive data-processing infrastructure

and specialised skills, both of which

many operators lack. To make matters

worse, much of this data is stored in

core operational systems where large-

scale extraction of data is neither

easy nor common practice, and some

information systems departments have

been traditionally resistant to scheduling

large data extraction tasks which may

potentially interfere with mission-critical

production.

Moreover, the timescales for datawarehouse development cannot keep

pace with the constantly evolving needs

of the business. Many end users do not

have predictable data needs – they may

need one type of report for a number of

months and then unexpectedly require

new information because the business

environment or internal priorities have

changed. For example, understanding

data usage and its impact on the

customer experience has recently become

a key issue for both xed-line and mobile

Internet Service Providers (ISPs). In

spite of this, most operators have not

contemplated the inclusion of detailed

network usage data into their customer

data warehouses, which will signicantly

limit their ability to understand the

link between usage, experience and

protability.

In the end, the data warehouse is

undermined by a lack of trust by

users and the inevitable sproutingof spreadsheet-based ‘skunkworks’ –

unofcial approaches to handling data

– around the business. Gartner estimates

that despite data warehouses achieving



3

a 50% adoption rate among end users, data exploitation efforts

have struggled to move beyond basic reporting and customer-

targeting activities. Thus, while data warehouses have proven

useful for some core operational reporting and relatively stable

insight requirements, in the long term telcos will need to adopt

new approaches to data exploitation if they are to generate

the insights needed to achieve operational efciency and

competitive advantage in a progressively challenging commercial

environment.

Where is the data?

Telcos are generating a huge data footprint outside the

traditional data warehouse. Areas such as customer behaviour

and internal business performance generate vast volumes of data

on a daily basis, capturing in-depth information on customerneeds and preferences, as well as detail on internal process

failures, capacity bottlenecks and enterprise risk.

Unstructured data

Emails and text-based documentation, as well as plain-text

comment elds in structured databases, have historically not

been included in the data warehouse and also contain vast

amounts of valuable data.

For example, many business-to-business telcos store their

contract documents in MS-SharePoint™, which cannot be

searched using traditional data warehouse techniques, thuspotentially exposing them to huge losses or unbilled revenue

– because contractual terms may not be being enforced.

Furthermore, some telcos may also nd themselves delivering

Service Level Agreements (SLAs) which are far more onerous

than those enshrined in their contractual documentation, or not

charging for certain services – because they are unable to easily

retrieve the data.

Email – with appropriate treatment to comply with the Data

Protection Act (see Detica’s white paper ‘Data protection

compliance: x your roof while the sun is shining’) – also

contains valuable information about process performance acrossan organisation. Understanding how work actually gets done,

by referring to communications between employees, can unlock

efciency savings.

Network perormance data

Network development is the largest capital expenditure item

for telecoms operators. Network operations centres are also a

signicant operational expenditure item. Operators generate

and store many gigabytes of event and fault data every day from

both their external telecoms and internal corporate networks.

The data from these various network layers (service, network andtransmission) are rarely brought together but would, if properly

correlated, provide a consolidated view of network performance,

delivering a rich source of insights into the root causes of alarms,

network performance issues and other key network events.

Network trafc data

Online connection speeds and reliability are critical drivers of

overall customer experience and satisfaction. Both xed-line

and mobile ISPs are able to capture data on the type of datapassing over their networks (such as, HTML, P2P, VoIP, streaming

etc.), and therefore identify both the type of trafc, the level of

bandwidth usage and type of user whose online activity causes a

disproportionate impact on the experience of other subscribers.

This type of information is important for pursuing an intelligence-

led approach to trafc management and network investment.

Call (and other) data records (CDRs/xDRs)

With the exception of Call Data Records (CDRs) captured and

retained for law-enforcement purposes, only a minority of

operators ingest CDRs into their data warehouse. The majorityeither outsource CDR retention or store CDRs in a digital archive,

with no easy access possible by the business. Telecoms operators

need to increase their analysis of CDRs and other data records

(xDRs) to understand detailed calling patterns, which can provide

rich customer behavioural insights valuable for enhancing

customer experience.

Web logs and call centre records

Call centre records and web logs contain a signicant proportion

of free text, which cannot be easily warehoused or analysed by

mainstream statistical techniques. While most operators usetrafc analysis tools to analyse trafc on their websites, this is

usually conducted at a site or page level rather than from the

perspective of a customer journey. To make efciency savings,

telecoms operators need to move customers away from labour-

intensive manned channels to automated processing. Driving a

successful channel migration strategy means building a better

understanding of the customer’s experience by automating

analysis of the free text obtained from contact forms on the

operator’s website and combining it with transcripts from calls

made to call centres.

Techniques for unlocking value

With the immense computing power available today, new

techniques have emerged which can handle the huge quantities of,

and generate value from, ‘human’ data – data that is disorganised,

unstructured, disparate and compromised by errors and

inconsistencies. Data which has previously been ignored by the

data-warehouse builders becomes capable of delivering valuable

insights.

Advanced network analytics

By analysing the relationships between disparate and seeminglyunconnected data from across the organisation, new insights are

revealed which could not be derived by looking at the individual

data in isolation.




Looking at data in the context of its network, that is to say in

the context of the interconnections between other data points

or entities, is a technique which has been deployed to great

effect in other elds and industries, and has also delivered

unanticipated benets.

For example, banks and insurance companies have been

able to dramatically reduce fraud by building up a picture

of their customers’ relationships with other customers and

by understanding the networks connecting phone numbers,

addresses, product applications and a host of other data points.

By identifying a network with multiple fraudulent ‘nodes’ (people

or data points such as phone numbers), any new customers with

a relationship to that network are treated with greater caution

– an approach which has seen many tens of millions of pounds

saved.

The corollary to this fraud reduction has been the creation of

a single view of the customer – often one of the key goals of

an enterprise data warehouse. Different products procured at

different addresses would have once been assumed discrete.

Through the observation of interconnected networks and ‘fuzzy

matching’ – a technique which enables the analyst to identify

when two apparently different subscribers are in fact the

same person, based on similarities in name, address and other

identifying details – these products can be revealed to relate to

the same customer.

In other situations, missing data may in itself be signicant – forexample if a network component is not recording data accurately

– which can form an integral part of, rather than be an obstacle

to, the approach.

In addition, as these techniques are inherently designed to bring

together data which appear unrelated (e.g. ‘Acacia Avenue’ and

‘Accacia Street’), this single view of the customer has been found

to help organisations cut through the kind of poor data quality

which would historically break a data warehouse-based analysis.

Unstructured text analytics

The analysis of unstructured ‘human’ data sources, which arenot typically found inside an enterprise data warehouse – such

as emails, text documents, blog entries, customer contact

transcriptions – can provide new insights for an organisation

looking to better understand its customers or discover how

internal processes are working in reality.

Seeing how information travels through an organisation via email

can shed new light on the informal processes that employees

create in order to do their day jobs, providing invaluable

information for any process re-mapping activity carried out to

reduce operational expenditure. Analysing call-centre records

and feedback forms can provide new insight into what customersare saying or feeling about the service being delivered, giving

organisations the chance to take action to prevent churn.

As unstructured data make up an estimated 80% of a typical

organisation’s information, text analytics will become an

essential tool in the quest to exploit internal data to support

more intelligent decision-making.

Data-driven thinking

If the analysis of information networks or unstructured

data allows an organisation to make sense of human data,

management decisions can be made on the basis of empirical

evidence rather than on instinct. For example, looking at the

Formula 1 industry we can see how a data-driven approach can

deliver results: when Lewis Hamilton passed Timo Glock on the

nal corner of the Brazilian Grand Prix to win the 2008 Formula

1 World Championship, a computer programme was cited as the

brains behind the victory. Faced with a decision about whether

to change tyres, race tacticians turned to empirical evidence.

The software told them that Hamilton’s lap times on new tyres

would enable him to catch Glock. This was an example of how,

in the most critical moments, trusting data can deliver hugely

benecial results.

Exploiting the data

To get the most from the new data exploitation techniques and

tap into the power of unstructured data, telcos need to take a

different approach to collecting and analysing data. Given its

volume and nature, the way the data is sampled and extracted,processed, secured and assured all need to evolve. The following

section outlines some of the changes required.

Construct a rapid analytics environment

As data outside the data warehouse becomes more important

for supporting decision-making, telcos will need to develop

efcient processes for extracting large volumes of data from

operational systems and making them available for analysis. If

the data is ‘free-standing’ – in other words, it can be extracted

with no impact on other important data sources – the operator

may save a considerable amount of time, and cost, manipulatingand applying their analytical tools directly onto the data without

the constraints of a data warehouse. This is a particularly useful

option where there is no requirement to integrate the data with

the structured set held within the data warehouse. This approach

also enables the operator to develop a suitable environment for

the analysis to take place. Typically, this involves the creation

of ‘sandpit’-type capabilities – computing environments

constructed with sufcient disk-space and processing power to

accommodate large data sets. Here, the operator can rapidly

set up and tear down various advanced analytics capabilities to

determine the benet to be gained from different approaches.



5

Introduce sampling

Taking a sample, rather than full data set, signicantly reduces

data volumes while still providing sufcient data to generate

robust insights which the organisation can use with condence.Sampling overcomes one of the key barriers to exploiting many

of the data sources: the scale of the computing infrastructure

(especially disk space and processing power) required to perform

the analysis. The sampling methodology does need to be

thought through carefully, however. For example, a random

sample of call records across the country will be less useful for

most analyses than all the call records for a group of customers

selected from a specic part of the country. Similarly, the web

logs for a pre-selected sample of customers will yield more

useful results than a random sample of all web logs, as it will

preserve the integrity of user sessions to support better insights

into customer behaviour. Network analysis often benets fromgeographic-based sampling initially, so that the approach can

be demonstrated on a region before the analysis is gradually

extended nationally.

Furthermore, if the analysis only needs to be run periodically,

then one-off snapshots of the data may be sufcient. For

example, if an ISP wishes to analyse internet trafc data to

support ‘intelligent’ trafc management policies, usually it needs

the analysis refreshed on a quarterly basis, given the timescales

for updating the policy and measuring the effect on customer

behaviour. Therefore, in this example, the ISP could extract the

required data on an ad-hoc basis rather than attempting to builda complex data feed that is only used on an intermittent basis.

Increase adoption o analytics

Advanced data exploitation often requires a shift from ‘self-

service’ type analytics, where end users select and generate

their own data, to a ‘black-box’ model where advanced tools

or specialist teams develop highly-targeted analytics to suit a

specic business area. The outputs are highly tailored towards

supporting operational decision-making; for example, lists of ‘at

risk’ customers, details of faulty network components or trafc

management policy recommendations. However, the underlyingcalculations are less transparent to the business, whose trust in

the outputs is therefore primarily based on their efcacy rather

than the approach. This in turn implies a culture shift is needed

within the organisation, in which people become more accepting

of the use of analytics to drive decision-making but have less

involvement or ownership of the analytics process.

Ensure security protocols are robust

Telcos will also need to ensure they have robust and efcient

security protocols in place to ensure the data is kept safe and

used appropriately once extracted. These protocols will need tocover the security of the analysis environment and supporting

infrastructure, the anonymisation or encryption of particularly

sensitive data, restrictions on which personnel can access or view

the data, and clear rules on how the outputs of any analysis can

be distributed or used.

Achieving ‘better for less’

As the pressure increases on telcos to derive more value from

their data, they must reach out to sources of data that have not

been traditionally captured by a data warehouse. In so doing,

they must also embrace unstructured ‘human’ data from across

the enterprise, which exists in non-standard formats, and may

have patchy integrity, yet nevertheless holds the potential to

unlock huge business value.

Using new analytical methods, such as advanced network

analytics or unstructured text analytics (proven in other

industries such as nancial services), and adopting data-

driven decision making – whereby data is trusted to inform

management decisions – telcos can signicantly increase

the level of insight they generate about their operations and

customers.

Network performance data, trafc data, CDRs and weblogs have

the potential to deliver insights that can:

• vastly reduce network monitoring effort in the network

operations centre;

• optimise capital expenditure on the network;

• deliver smarter trafc-management policy decisions;

• prevent fraud and protect revenue;

• improve understanding of customer behaviours.

Changes are required to deliver on these promises, with the

establishment of rapid analytics environments, sampling of data

and the incorporation of more powerful analytical tools. The

costs of doing so are a fraction of those which have been soaked

up by data warehouses, and their related constructions, and do

not require the adoption of draconian data integrity measures

What is more, the benets of adopting these new approaches are

already being proven by telcos in a number of areas.




Understanding customer experience to prevent churn

At least two telco operators have recently started to apply text

analysis techniques to call centre, email and online web logs, to

identify key word combinations which indicate poor customer

sentiment which may translate into downstream churn or

product downgrades. In addition, some operators have begun to

stitch together disparate data (IVR logs, provisioning, call centrerecords, helpdesk etc.) to develop a ‘single service view of the

customer’ as a basis for identifying, via statistical analysis, key

failure points in the customer journey. Initial results indicate

that proactively following up with customers identied as

having received relatively poor levels of service has a signicant

impact on service-related customer churn and complaint levels.

Improving up-selling and marketing efciency

Some mobile operators are developing sophisticated analysis

of the social networks present within their subscriber bases,

by applying network level analysis techniques to billions of

call data records. These operators are seeking to identify

the ‘key inuencers’ within their subscriber base that have a

disproportionate impact on the churn and up-sell decisions

of others. One operator recently found a threefold increase

in mobile broadband take-up from viral adoption rather than

direct targeting. Another operator discovered it could double

the accuracy of its churn prediction models by including social

network type effects, such as whether a key inuencer in a

subscriber’s social network recently churned.

Improving detection o raud

Network level analysis is also proving increasingly valuable

in identifying and stopping organised fraud through the build

up of a ‘risk picture’ of individuals based on their network of

relationships with other individuals, and the overall behaviour of

that network. CIFAS reported a 57% increase in fraud related to

communications products between 2008 and 2009. Whilst the

approach is only starting to be adopted by telecoms operators,

other industries have experienced a tripling of fraud investigator

productivity through this approach, based on nding both more

suspicious behaviour, and reducing the number of false leads.

Procurement and security departments in the telecoms and

other industries are also looking to use network level analytics

to extend data-driven fraud detection into insider and supply

chain fraud.

Minimising network capital expenditure

One ISP analysed protocol data generated from subscribers’

online activities, to identify the type of trafc and bandwidthusage prole of its subscriber base. It then segmented its

subscribers into distinct behavioural clusters, based on their

network trafc proles. Based on an objective view of which

groups consumed the most bandwidth and contributed the

most to usage peaks, the ISP was able to accurately identify the

customer experience impact of different trafc management

policies. As a result, it has instituted an ‘intelligent’ trafcmanagement policy which protects the customer experience of

over 99% of its subscribers, and enabled it to avoid over £25m

network development costs in 2009.

Reducing operating expenditure in network operations centre

A xed-line operator recently consolidated performance and

alarm data from its service, network and transmission layers,

along with trouble ticket data, to identify and strip out false

alarms to create a view of the true state of network health, and

to reduce the volume of alarms reaching agents. As a result of

this analysis, the operator established that 60% alarms were

part of a repeating pattern, generating signicant numbers of

unnecessary work tickets. In addition, it was able to codify

the characteristics of major incidents, enabling immediate

automated notication of a major incident in 65% of cases.

Case studies



7



About Detica

We live in a complex and constantly changing world.

Organisations everywhere are looking or ways to expand the

services they provide to their customers while simultaneously

tightening their belts and conronting new hazards.

Detica helps government and commercial organisations exploit

inormation to deliver critical business services more eectively

and economically. We also develop solutions to strengthen

national security and resilience, enabling citizens to go about

their lives reely and with confdence.

By combining technical innovation and domain knowledge,

we integrate and deliver world-class solutions – oten applying

our own unique intellectual property – to our clients’ most

complex operational problems. Our services range rom strategyormulation and business change through to sotware and

hardware technologies, systems integration and managed service

delivery.

Detica is part o BAE Systems, the premier global deence,

security and aerospace company. BAE Systems delivers a ull

range o products and services or air, land and naval orces, as

well as advanced electronics, security, inormation technology

solutions and customer support services.

Find out more:

If you require further information please contact:

Detica Limited

SRP3

Guildford

Surrey

GU2 7YP

T +44 1483 816000

F +44 1483 816592

[email protected]

www.detica.com

© 2010 Detica Limited. ALL RIGHTS RESERVED. This document is copyright o DeticaLimited and/or its ailiated companies. Detica, the Detica logo and/or names o Deticaproducts reerenced herein are trademarks o Detica Limited and/or its ailiatedcompanies and may be registered in certain jurisdictions. Other company names, marks,products, logos and symbols reerenced herein may be the trademarks or registeredtrademarks o their owners. Detica Limited is registered in England under number 1337451and has its registered oice at Surrey Research Park, Guildord, England, GU2 7YP.

A BAE Systems Company

Unlocking the Value in Telecom Data (BI)

Documents

Transcript of Unlocking the Value in Telecom Data (BI)