Copyright 2014 © by STORM Insights, Inc. All rights reserved. 1

An Executive Guide to Analytics InfrastructureImperatives for Advanced Analytics

Adrian J. Bowles, PhDinfo@storminsights.com

January 2014

Sponsored by

Imperatives for Advanced Analytics

Modern business is increasingly dependent on the timely analysis of large volumes of data for strategic and operational decisions. As C-suite executives come to rely on advanced analytics, the importance of a robust IT infrastructure cannot be overstated.

When a man-made system fails for no apparent reason - a building collapsing on a clear day, a bridge suddenly twisting until it plunges into a river, or a software application that fails to deliver expected results after long periods of reliable performance - we know that there is always a reason. When the dust settles and the source of the failure is diagnosed, it can usually be traced to faulty assumptions - often undocumented during the system design phase - that found their way into infrastructure decisions. A robust IT infrastructure that resists failure and enables change and growth doesn’t happen by accident. It is the result of careful consideration of possible sources of failure through hardware or software errors, or changing demands imposed by new workloads. The sooner fault sources and contingencies are identified, the more confidence one can have in the operational system. The more we rely on data to make decisions, the more important it becomes to build a fault- and growth-tolerant infrastructure from the very beginning.

The widespread availability of public data sources, from news, weather, legal, and municipal government sources etc, combined with new internal sources of data about operations, sales, and even customer sentiment, create unprecedented opportunities to improve decision-making with analytics. From the simplest descriptive analytics, which report on what is or what has been, to predictive analytics, which give a window into what may be, to prescriptive analytics, which automate decision making, differentiating with data has never been more important. As advanced analytics become part of the fabric of an enterprise, IT infrastructure investments become a C-Suite issue.Executives and managers need an IT infrastructure for analytics that delivers results:

In time - when analytics are used to drive activity, it is imperative that the results are delivered in time for the action to have impact. From an infrastructure perspective, that requires hardware and software optimization when data streams are sufficiently large, complex, or fast-moving. Business processes tend to have both sequential and parallel activities, so any event that results in a task failing to meet a latency performance requirement may result in additional delays downstream.

Everywhere - today, data are being created and captured within and external to the enterprise, and the results of analysis must be delivered to staff, customers and partners wherever they may be. That requires distributed sensors and mobile access, which imposes new requirements for security, management, and networks. As more sources for data emerge, and more users of the analytics seek to leverage this data, applications must be designed to handle more concurrent data streams being uploaded, downloaded, and managed during analysis. While some of these requirements favor a cloud-based implementation, the responsibility for ensuring that infrastructure requirements are met cannot be ignored or delegated to the provider.

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 2

Every time - At the heart of the matter for analytics infrastructure is reliability. Modern processors and components are generally more reliable than those from previous generations, but when a business depends on analytics to make decisions that drive operations, even small failures can lead to cascading errors that quickly become catastrophic to the reputation of the enterprise.

In Figure 1, we see that regardless of the sophistication of the analytics, and the speed, complexity and volume of data sources, the path from data to action goes through the IT infrastructure. No matter how much one pre-processes, cleans, compresses or otherwise acts on the data with applications, hardware performance will affect the outcome. It is popular, but shortsighted, to think that moving an application to the cloud to benefit from on-demand resource allocation and op-ex instead of cap-ex accounting, diminishes the need to be concerned with the underlying infrastructure. One can delegate these decisions to a third-party provider, but one must know enough about the architecture and infrastructure to ensure that the performance terms of a service level agreement can be met. When performance is a consideration - and it is always a consideration in high volume or low latency applications - it is important to remember that all processing takes place on-premises somewhere. That is where the results of poor infrastructure decisions become acutely visible.

Enabling Analytics-based Action

Differentiating with data and analytics requires access to better data than your competition ( this may be more data, or more accurate data, or more refined data) and/or better analytics for your data. The ability to make better decisions - those that are made faster, made by people closer to the problem, or more execution-oriented - depends on reducing the “degree of difficulty” of interpreting the appropriate data from all input streams. While these three dimensions are sometimes lumped together as “big data”, the steps required to manage them are different. Think of the combination of attributes as similar to the “degree of difficulty” metric used for competitive diving. The first step in preparing to make decisions based on data is managing the volume and complexity while capturing or sampling high velocity data in a meaningful way.

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 3

In general, raw data - as it is captured by systems and sensors - has potential value, much like crude oil after discovery but before processing. In Figure 2, we see that as the volume, complexity and speed are managed during analysis, the value to the business goes up. Once decisions have enabled action, the value is really seen. This is, of course, a simplification and abstraction of the role of analytics, drawn to make three points:

Preparing data for analysis - upstream refining - incurs costs and takes time, but adds value.

Perfect information is rarely attainable, so decisions must be made under uncertainty. The box showing a range of time for specific decisions models a range of acceptability - this is similar to choosing the optimal time for changing gears in an automobile. With data one trades off accuracy for immediacy, like trading performance for gas mileage.

The real value to an organization come from acting on data.

Managing any of these dimensions - speed, complexity, volume - can be addressed in hardware or software. At the extremes, though, it usually requires an optimization strategy that includes both. That requires a basic understanding of the demands analytics workloads put on IT.

Emerging Workloads Demand Infrastructure Flexibility

New and changing workloads often require changes to data/storage, processing and network services capabilities. Analyzing and accommodating the impact of system workload changes has traditionally been the province of IT. However, it is increasingly important for other executives to understand the demands their analytics requirements put on IT infrastructure. By participating early in the process and communicating their expectations, requirements and constraints, these executives can drive better infrastructure decisions - ones that support their own objectives and priorities. Key infrastructure choices involve tradeoffs. Not everyone needs to become an IT expert, but when a choice impacts a business outcome (eg trading speed of delivery for operational performance) the effected executive should be involved as soon as possible, when changes are

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 4

A system workload is a set of requirements - for data/storage, processing, and network services – that characterize a particular set of defined and repeatable tasks. As C-suite executives adopt more sophisticated analytics to improve their performance, we see new workloads emerging. Without careful planning and investment in the appropriate IT infrastructure to support these workloads, dramatic failures are inevitable.

cheaper. Executives who are active participants in the infrastructure requirements phase will also better understand the investment required to assure ongoing access to these systems in the context of changes in the enterprise and its environment.

Examples:

CFO - New data sources and analytics tools enable improvements in planning and performance management, human capital management, financial disclosures and risk management. In Figure 3, the “Z” represents one example that might require additional infrastructure capabilities: adding newly available customer-retention data derived from sentiment analysis to the system of record used for financial projections may require additional processing resources to ensure that results are available every time, in time for closing the books.COO - Well-established systems like ERP (enterprise resource planning) and BPO (business process operations) systems may already strain system resources. As new data data sources from within the organization and its environment (more granular supply chain data and customer sentiment data, for example) become available, they place more demands on IT infrastructure to support predictive and prescriptive analytics. The COO concerned with managing operations and inventory in a widely distributed environment may find that further investments in networking and mobile apps(“Y”) are warranted.CMO - Traditional marketing campaign planning and assessment systems are being augmented or replaced by new systems to address customer acquisition and retention. These workloads may include sentiment analysis from social media sources, and personalized marketing campaigns that rely on fast analysis of transaction data in the context of personal profiles and current events, to provide custom offers just in time for action that builds loyalty. Based on the volume and timing requirements, this could indicate a tipping point for flash storage or additional point of sale processing investments (“X”).

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 5

We are not suggesting in Figure 3 that every cell will be the focus of a new infrastructure investment, simply that workloads to support C-suite decision-making tend to interact, so for example a change ordered by the CMO may impact performance of a workload supporting the COO with unintended consequences. The important thing to recognize here is that over time, new workloads will inevitably emerge and the availability of new data will inevitably put new requirements on existing workloads. The need for a flexible architecture and infrastructure should be addressed early in the process of planning for advanced analytics, and revisited throughout the assessment lifecycle to avoid degraded performance due to these foreseeable new demands.

Findings and Recommendations

Analytics-oriented workload growth and diversity is inevitable in a growing business and should be welcomed as long as it can be managed. We believe that executives should foster opportunities for mass experimentation enabled by new data sources and tools. The emergence of these new sources and tools drives innovation, which creates new data that may be refined and used to drive new or better decisions. It is a virtuous cycle that should not be impeded by a rigid IT infrastructure. We close with a few thoughts on preparing for the future:

As the world becomes more instrumented (generating more data) the degree of difficulty for managing and monetizing data will accelerate the demand for modern analytics tools and techniques. Innovation is inevitable.

Options for analytics tools are getting more sophisticated - some projects will be constrained by the difficulty in attracting and retaining talent.

In parallel with the rapid adoption of descriptive, predictive, and prescriptive analytics, we see an increasing interest in cognitive systems, which learn from their interactions and experience. Similar to more conventional analysis workloads, cognitive systems operating on difficult data will perform best in a workload optimized environment.

The best infrastructure investments today are those that preserve options for tomorrow by providing a path to processor, storage, and networking capability changes to manage increasingly difficult data workloads. Plan early, and review often, to ensure that your infrastructure assumptions remain valid.

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 6

...mass experimentation enabled by new data sources and tools...is a virtuous cycle that should not be impeded by a rigid IT infrastructure.

Author Bio: Adrian Bowles

Adrian Bowles is the Founder of STORM Insights, Inc, a research and market intelligence firm focused on the role of emerging technologies in business transformation. Adrian has over 25 years of experience as an analyst, practitioner and academic in IT strategy and management.

Previously, Adrian founded the Governance, Risk Management & Compliance Roundtable for the Object Management Group, the IT Compliance Institute with 101 Communications, and Atelier Research. He has held executive positions at Ovum (Datamonitor), Giga Information Group, New Science Associates, and Yourdon, Inc. Adrian also held academic appointments in computer science at the State University of New York at Binghamton and Drexel University, and adjunct faculty positions in the business schools at NYU and Boston College. He began his career with research and application development roles at GTE Laboratories and IBM.

Adrian earned his BA in Psychology and MS in Computer Science from the State University of New York at Binghamton and his Ph.D. in Computer Science from Northwestern University.

Disclosure

This report was developed by Adrian Bowles of STORM Insights, Inc with IBM assistance and funding. This report may utilize information, including publicly available data, provided by various companies and sources, including IBM. The opinions are those of the report’s author and do not necessarily represent IBM’s position.

Learn about IBM's Big Data & Analytics Infrastructure solutions at:

www.ibm.com/big-data-infrastructure

Copyright 2014 © by STORM Insights, Inc. All rights reserved. 7