© Bolder Technology, Inc. 2011 Page 1

A BTI Case Study

Data Warehouse Migration Benefits: Oracle-to-Teradata Experiences

August 2011

This white paper examines the benefits, both expected and realized, for migrating data warehouses from Oracle platforms to the Teradata platform. It builds upon a previous white paper on migration strategies, for which several companies in different industries were interviewed. The study reveals that smart IT managers must balance a multi-faceted perspective on benefits: immediate payoffs versus long-term planning, tangible cost reduction versus intangible stability/agility, and business improvements versus infrastructure maturing. The success factors for reaping the desired benefits from migrating data warehouses are a well-conceived migration plan along with patience and perseverance.

Examples of Data Warehouse Migrations ...................... 3 It’s All about Reducing Costs ........................................... 5 More Bang for the Buck .................................................. 8 Knowing About Your Company ....................................... 9 Balancing the Benefits .................................................. 11 Synthesis ....................................................................... 11 About Bolder Technology, Inc. ...................................... 13 About the Sponsor......................................................... 13

© Bolder Technology, Inc. 2011 Page 2

After a discussion of the strategies for migrating data warehouses, the VP of Finance was wondering whether it was worth the effort. She asked, “Okay…I understand what needs to be done and how long the project will take. But, what are the benefits that we expect from doing this migration?”

She seemed convinced about the merits of the project but was looking for more concrete thoughts. In particular, she was worried about how she would convince colleagues that this DW migration was a good idea for the company. How much was simply technology for the sake of technology? And how much is necessary for the future of the company?

How would YOU address her concerns?

Data warehouse (DW) migration projects are a critical part of most IT agendas for two reasons. First, data warehouses have matured into a critical element of most IT architectures. It has often become the hub that uniquely integrates data from diverse business units and conveys a unified view of the enterprise. As the business environment becomes more complex, that unified view becomes a necessary ingredient for being a successful business. Second, the technology and practice of data warehousing has evolved and matured greatly over the past five years. Most production data warehouses were created ten or more years ago, limited by assumptions about the technology that are invalid today. In many subtle ways, these older data warehouses are hindering the growth and modernization of their companies. Older data warehouses are in need of a major overhaul and, hence, well-conceived projects for DW migrations.

Despite being critical, these DW migration projects are poorly understood by many IT professionals and especially by most business executives. The project seems deceptively simple, since one would imagine… Just move data from one SQL database to another. It would seem to be a simple ‘fork-lift’ project where we pick the data up, move it, and set it down in the new data warehouse. The real situation can be more complex, both from a business and technical perspective. Realistic expectations of the benefits from a DW migration are especially tricky to manage and to realize. This is the focus of this white paper.

In a previous white paper, we examined the strategies and issues for migrating data warehouses from Oracle platforms to the Teradata platform.1 The paper discussed the motivations and approaches to Oracle-to-Teradata migrations of data warehouses, gives examples of experiences, and concludes with benefits realized, lessons learned, and a synthesis of key points.

This white paper complements the previous one by examining the benefits of DW migration as the balancing three issues.

Immediate payoffs versus long-term planning

Tangible cost reduction versus intangible stability/agility

Business improvements versus infrastructure maturing

Let’s start with some examples of DW migration projects to provide a context for considering these benefits.

© Bolder Technology, Inc. 2011 Page 3

Examples of Data Warehouse Migrations

Several companies in different industries were interviewed and reported anonymously about their experiences with migrating their data warehouses over the past two to three years. In a previous white paper1 the strategies for DW migrations are described, while this paper focuses on the benefits realized from those DW migrations. Although this sample is small and biased toward Oracle-to-Teradata migration, the experiences were insightful into issues generic to IT management and to best practices for maturing data warehouses.

Manufacturing Company

The manufacturing company stressed sophisticated analyses for quality control of their manufacturing processes. Their daily data loads were taking 18 hours, indicating their Oracle data warehouse was “running up against the wall”.

There were several benefits from the DW migration to a Teradata system that were noted during the interview.

First, the company’s market share had consistently grown over recent years, surpassing the market share leader in their industry segment. This growth was described as ‘organically’ by selling more and more, rather than growth through acquisitions. Executives are convinced that the DW migration project contributed the enabling tools across the entire company to ensure this market success. This would not have been possible on their previous Oracle platform.

Second, the company’s obsession with quality improvements discovered via their Teradata data warehouse produced the best warranty accrual rate in the industry, implying the highest quality product in the industry. Both shareholders and customers appreciate this accomplishment.

Third, the company now has the lowest warranty returned units in the industry. Accurate forecasts have reduced their warranty reserves, along with more accurate reporting to stockholders. The company cited the increase in customer satisfaction as proof that investments in quality improvement had tangible payoff. The DW team attributes these three successes to their move to the Teradata system.

Fourth, the company has improved its precision at identifying defective units prior to shipment, thus reducing the number of units that need to be returned from customers and reducing the number of good shipments that would have been delayed. In addition, this improved precision increased the trust and loyalty of their larger customers.

Finally, the company was able to reduce reaction times to various operational problems from days/weeks to minutes/hours, resulting in higher efficiency in overall manufacturing processes. In other words, they were able to correct problems quickly with the Teradata system before those problems impacted the quality of thousands of units.

Healthcare Company

The healthcare company strived toward a scalable DW platform using the MPP capabilities inherent in the Teradata system. The transition from their SMP Oracle platform to the shared-nothing parallel DW architecture of Teradata was considered by the company as an “enterprise imperative” to support future business initiatives.

By standardizing on Teradata, Informatica, and SAP BusinessObjects (for reporting), the company adopted common corporate architectural standards, which have reduced the development effort on five projects so far. In particular, a tangible benefit has been the reduction in software licensing fees, which previously were paid to dozens of vendors. The effort of troubleshooting a problem has been reduced

© Bolder Technology, Inc. 2011 Page 4

with a small set of vendors. And, consolidating platforms has resulted in a smaller hardware footprint in the data centers and simplified maintenance. Finally, the three vendor partners created a pool of resources from which the healthcare company could draw upon for future projects.

With the linear scalability in processing data volumes by Teradata and Informatica, the company has avoided future technology investments that could be on the order of $10M every few years. Results from performance testing confirmed the scalability of the MPP system.

With the higher performance of the MPP platform, the company now has the capacity to load and deliver timely data globally, so that fresh reports are available to business users before they arrive at their office, regardless of their local time-zone. Previously, service levels were set regionally by maintaining three separate instances of the data warehouse for the Management Information Center.

Ongoing support costs for the DW operation were reduced by 25 to 30 percent, especially in the staffing of database administration and application-level support/services (monitoring user problem and performing minor enhancements). The support costs were reduced from $960K to $860K per year (or 10 percent) for the Financial Data Warehouse and from $2.3M to $1.6M per year (or 30 percent) for the Management Information Center

Beverage Company

A beverage company was coping with a major merger of two mature companies. The imperative was to consolidate more than fifteen data marts that used Oracle and DB2 platforms and quickly provide company-wide integrated reporting to manage the new company.

For this study, the DW team for the beverage company was again interviewed to probe their expected and realized benefits from their current DW migration project. The key business requirement was to have company-wide integrated reporting available as soon as possible to facilitate the merger. Thus, the center of attention was a new consolidated data warehouse.

One of the merged companies had been using IBM DB2 as its data warehouse, while the other company had approached its data warehouse as a set of fifteen Oracle data marts, each with a different subject area. The decision was made to migrate both platforms to the Teradata platform. The project team consisted of three lead database administrators working with ten off-shore developers for the ETL development.

The DB2 data warehouse was incorporated within three months, along with the biggest Oracle data mart. This data mart consisted of one billion rows requiring one terabyte of storage. The remaining data marts are estimated to contribute another terabyte to the new data warehouse.

When quizzed about performance improvements, the DW team confided that did not have ‘hard numbers’ about the performance of the older systems, so that performance comparisons were not possible. Informally, they felt that many processing steps went from ‘hours to minutes’, usually with ‘double or triple speed improvements’.

Instead of focusing only on performance improvements, their focus was ‘getting the information out of the data warehouse’, which was not sufficiently possible with their Oracle data warehouse. Relying on

© Bolder Technology, Inc. 2011 Page 5

the BusinessObject suite and MarginMinder tools, the company boosted their support to over 1,100 power users.

Retiring the Oracle data marts has proven to be a long-term endeavor. The Oracle data marts were designed to tap into specific legacy transaction processing (TP) operational systems. Because the TP systems were being consolidated and upgraded as part of another project, the ETL processes continued to rely on the Oracle data mart contents, rather than developing new ETL directly to the TP systems. Thus, the new data warehouse remained temporarily dependent on the data marts and was limited by their refresh rates. The DW team described the situation as ‘a truck coupled to a horse and buggy, all going at 23 mph’. The situation is slowly changing as new TP systems are being phased into production.

Automobile Distributor

The automobile distributor company was motivated for its DW migration by long and unpredictable report generation times, especially for their month-end processing. Month-end report processing would last for 2-3 days and often terminate unexpectedly.

The company’s old data warehouse used Oracle on SUN Solaris with about two terabytes of user data with roughly 400 tables. Cognos was used for report generation, and Informatica was used for ETL processing. They upgraded to a three-node Teradata 5555 system with seven terabytes of storage.

The strategy was a pure ‘forklift’ approach that planed for minimal data structure changes to minimize development risk and migration time. The project team consisted of five full-time employees, supplemented with five part-time employees and several Teradata specialists who contributed specific skills at various phases of the project. The DW migration took five months plus two months of parallel operation to verify data quality. There were data type conversions, data compression and index redesign, plus a small amount of PL/SQL recoding. About 230 Cognos report were converted, which was a quick and easy effort. The company had expected that the report conversion would take the major of development effort, while the ETL conversion would be quick. The opposite proved to be true, with report conversion was quick and easy and ETL conversion was time consuming.

They planned for a 10X improvement in performance, which they felt was achieved. Instead of days of processing, the month-end reports are now generated in 7-8 hours, reliably without surprise terminations. The SLA of delivering month-end reports by start-of-business the next day after month-end is now being met. The daily reports are now taking 45 minutes on average, as compared to the 7-8 hours with their previous Oracle DW platform.

The company cited several lessons learned: First, the reporting conversion was easier than the ETL conversion. A mature BI suite like Cognos enabled quick migration of their reports. Second, they attribute success to their simple Forklift strategy with limited ‘smart’ conversions of data types. They felt that they are now properly posed to re-architect their data as new business units and TP systems are incorporated into the data warehouse. For instance, they are incrementally migrating processing from using physical table definitions to logical view definitions.

It’s All about Reducing Costs

In our interviews, cost reduction was the most frequently cited motivation for driving DW migrations. Given today’s competitive market pressures, most IT projects are driven by cost reductions of one form or another. However, there are many forms of cost reductions when dealing with DW migrations. When justifying a DW migration project, one must be clear about the definition of cost reduction.

© Bolder Technology, Inc. 2011 Page 6

The Obvious

The obvious reductions are current tangible costs that appear on monthly financial statements. For data warehouses, these obvious costs are some variation of the following:

The obvious costs are visible and easily managed. If the business requirements for the data warehouse are static, the management of these costs should be straight-forward. One would concentrate on incremental cost reductions in specific areas, gradually evolving toward a leaner and meaner IT architecture.

The Expected

The typical company unfortunately does not have static IT requirements. Companies and their markets are constantly changing, driving IT to support new business objectives. The assumption is that expected costs in the future can be predicted based on historical cost trends.

Figure 1 show two cost trends. The horizontal axis is cost, while the vertical is capability of the data warehouse, such as number of concurrent users or volume of user data.

1. Personnel costs Database administrators, technical managers, user support, IT architects… Subcontractors, consultants, outsourcing…

2. System software costs Licensing, maintenance, upgrades Training

3. Development and maintenance costs ETL of existing and new data sources Application Data conversion and transfer

4. Data center costs Data warehouse equipment purchase, upgrades, and maintenance Backup, archive, restore equipment Disaster recovery equipment Test, Development and Quality Assurance equipment Network interconnection for external and internal nets Electricity and cooling of equipment General floor-space facilities, like security, lighting, fire prevention, A/C… Hosting or out-sourcing contracts

© Bolder Technology, Inc. 2011 Page 7

cost

capabilityA

B

a

b

Figure 1 – Simple Cost Trends

Trend A is a linear relationship of cost to capability. For instance, doubling the cost for the DW infrastructure will produce double the DW capability. This is often the case for small variations in cost and capability. Adding one extra disk drive, one more processing unit, or 20 percent more network bandwidth will likely generate incremental capability.

Trend B is the more likely case. Over time, additional cost generates less and less capability, until more cost does not generate any additional capability. We often say that “we hit the wall” with this infrastructure.

The contrast is highlighted with the dashed line. Your future plans for your data warehouse depend strongly on whether your infrastructure has a cost trend of A or B. Simply put, the system with trend B has no future!

This illustrates the concept of scalability. A system is scalable if additional cost (effort, materials…) generates a reasonable amount of additional capability (power, resources…). It would be excellent if the relationship was linear, like in Trend A. The implication is that a doubling in demand could be handled by a doubling of resources. The challenge is to understand the conditions over which a particular system infrastructure is scalable.

Scalability is important for data warehouses where business requirements can increase 10X and even 100X over the current capabilities, in terms of data volumes, query complexity, response times, and workload variations. DW technology is currently in the transition of SMP to MPP architectures, as was discussed in the previous white paper. Hence, many DW migrations involved this SMP-MPP transition because the company started small with inexpensive SMP systems and grew their business requirements to push them toward a MPP system like Teradata.

The Surprising

The nasty situation is when the cost trends surprise us! Instead of behaving nicely and predictably, the cost trends can go “non-linear” and exhibit “tipping points”. In other words, a given system architecture will suddenly and unpredictably break at some point. With data warehouses, we see many situations like this, say when workloads exceed 80 to 90 percent of capacity causing query times suddenly to explode from minutes to hours.

© Bolder Technology, Inc. 2011 Page 8

Figure 2 illustrates this point. As we put more resources (cost) into the data warehouse, its capabilities increase in a predictable manner, until a tipping point is reached. Obviously the challenge is to anticipate these tipping points and predict when critical assumptions about system behavior are no longer valid.

cost

capability

tipping point

Figure 2 – Non-Linear Cost Trends

More Bang for the Buck

There is a deeper sense behind the motivation for cost reduction. Technology innovations have dramatically altered cost trends, permitting significant cost reductions for the same level of capability or significant capability improvement for the same level of cost.

Figure 3 illustrates this point with two cost trends, one for older technology contrasted with one for newer technology. The assumption is that the newer technology will be more efficient and more scalable.

© Bolder Technology, Inc. 2011 Page 9

cost

capabilityb

a

newer

older

c

x

Figure 3 – Older-Newer Cost Trends

Let’s assume that your company is now at the cost/capability point X. What are your options?

You could stay with the older technology and push your costs toward the right. You faced diminishing returns on your efforts and possible tipping points. Or, you could migrate to newer technology, which (if chosen wisely) would offer several alternatives…

Point A: cost reduction with the same level of capability. For DW migration, this is often the situation when consolidating multiple data marts into a single data warehouse, thus reducing the overhead cost associated with multiple operations.

Point B: capability enhancement with same level of cost. For DW migration, this is the situation when the current data warehouse has significant performance problems and an immediate solution is needed.

Between Points A and B: A range of alternatives that mixes some cost reduction and some capability enhancement.

Toward Point C: ability to scale DW capabilities into the future.

For several companies interviewed, the business value of scalability in the DW infrastructure was the primary motivation for their DW migration. This scalability was not about 20 percent improvements but rather factors of 10 times greater capabilities as the company anticipated new business processes involved would involve customer self-service or social networking relationships.

The above forms of cost reduction are the basis for justify DW migrations. The clear definition of cost trends and the strategy for maximizing value to the company are the key success factors for DW migrations.

Knowing About Your Company

Cost factors may impact the entire company, not just a few items on the IT budget, as is discussed in this section.

© Bolder Technology, Inc. 2011 Page 10

For the beverage company, the fundamental justification for their DW is the integration of information that describes the business of the company. Initially this integration only involved a single functional area (e.g., retail store sales for the pharmacy department) with a couple of subject areas (e.g., customer, medications). This embryonic start belies the long-term effort involved with achieving real business value from an integrated view of the business. The complexity of any global company is mind-boggling, resulting in possible decision paralysis for many a company. This is a messy challenge that most business executives wish to avoid, instead stressing the priority to focus only on immediate business results for a specific business application.

However, the data warehouse is the only place where an integrated business view is possible. In particular, critical business opportunities (and business problems) are often hidden in the ‘cracks’ between functional areas. Understanding cross-functional dynamics becomes critical to growing the business in a changing global marketplace.

Facing a recent corporate merger, the beverage company recommended that their focus of DW migration should be on data integration so that the corporation can understand “how well they are doing.” Further, this migration must be implemented with a timeframe that delivers business value quickly.

Although data integration during the DW migration has huge business value, it is tough to do! The healthcare company chose a stepwise approach that balanced payoffs in business value with constraints in technical resources. The approach is to consolidate each domain into a common environment, then rationalize each domain, and finally integrate the common elements into a single enterprise-wide view of the business. This is a long-term incremental strategy that has a high probability of success and has flexibility to respond to unexpected industry changes.

Leverage the Opportunity

A DW migration specialist phrased the benefits of the migration project differently. Before a DW migration, the typical company has “hit a wall” and cannot see other ways of using the data. After the DW migration, the company is at “the next plateau” and can see many more uses for the data. However, he cautioned companies not to migrate for the sake of migrating. There must be a clear business justification.

Avoiding the Big Bang

Industry wisdom suggests the avoidance of a ‘big bang’ approach to enterprise-wide data warehousing. In other words, avoid attempts to integrate data across multiple functional areas of the enterprise as a single project. With big bang approaches companies usually attempt more than they can accomplish. Data integration is primarily a maturing of the organizational culture. A more iterative approach over the long term is more effective in achieving an integrated view of business processes.

The flip-side is the loss of business opportunities of NOT understanding key business processes, especially those that span functional boundaries. Many well-known companies have ceased to exist because their organizational cultures did not permit (condone, reward, encourage) thinking about their enterprise as a highly interconnected whole. The value of migrating to a mature DW platform like Teradata is the ability to understand key business processes across the enterprise.

Building Trust

DW migration projects are often in the center of these data integration opportunities and challenges. A mature project manager should understand and embrace the role as an educator or mentor to facilitate the organization change required to manage an enterprise as an integrated system. In particular,

© Bolder Technology, Inc. 2011 Page 11

creating trust among the diverse parties involved is a critical success factor. Patience and persistence of all parties are needed to reap the benefits from the DW migration.

Balancing the Benefits

In the realities of justifying and executing a DW migration project, there is a series of tough tradeoffs. In particular, the balancing of the following tradeoffs is critical for the companies interviewed.

Immediate payoffs versus long-term planning

This tradeoff is quite common among IT projects – fast and sloppy versus slow and precise. The truth is… it should not be a tradeoff. It is not one versus the other. Both can and should be pursued concurrently. In other words, a DW migration project should be managed to produce immediate (or fairly soon) payoffs to business users, while also following a long-term strategy.

Tangible cost reduction versus intangible stability/agility

This is another common tradeoff, which also should be managed so that both are pursued concurrently. In other words, a DW migration project should generate tangible cost reductions (usually via more efficient technology), while also being more stable (fault tolerate) and flexible (agility to adapt to changing business and technical requirements).

Business improvements versus infrastructure maturing

When resources are expended for a data warehouse, executives want to see specific improvements that support business units. Usually they do not see much value in resource devoted to infrastructure. If it’s not broken, then don’t fix it! This was a tough tradeoff to balance. However, technology advances are such that new capabilities (e.g., customer self-service) require infrastructure upgrades (e.g., MPP database services with workload optimization).

Synthesis

The success factors for DW migration are healthy amounts of patience and persistence along with trust among the diverse parties involved with the DW migration project.

The DW migration for the beverage company was biased toward data integration objectives, rather than cost reduction. However, their situation is typical of many corporations caught in dynamic business markets that force continual changes to their fundamental business processes.

It is not that cost reduction is ignored in certain cases. It is that cost reduction is interpreted in a larger context. Often, this context is cost avoidance – tangible costs that will be incurred by the company if the current strategies/trends are followed. One dollar of cost avoidance next quarter is just as tangible as one dollar of cost reduction, assuming a competent analysis of future costs. With DW operations, this cost analysis is often clear. Data volumes can be monitored, plotted and forecasted reliably, along with an increasing user base, daily processing loads and the like.

© Bolder Technology, Inc. 2011 Page 12

Cost avoidance is often extended to the tangible benefits of data integration that enables the company to respond to unexpected challenges of its global business. Like the beverage company, these companies have a sense of urgency that DW migrations must be accomplished quickly.

Finally, all the companies interviewed recognized that their new Teradata data warehouse has substantially more capability than their old data warehouses on Oracle. However, only a few companies recognized that this was an opportunity to challenge key business assumptions and to revolutionize their business processes. In other words, a DW migration project opens the door for business innovation. This opportunity can have surprising benefits. Do not be surprised! Be prepared to leverage the opportunity.

- - - - -

This paper presents the diverse issues that will assist in a successful migration of your data warehouse. DW migration is a major IT investment for most corporations, both in cost reductions and in business enhancements. Understand the business and technical issues. Build trust among the parties involved. Ensure sufficient patience and persistence to reap the desired benefits.

© Bolder Technology, Inc. 2011 Page 13

Appreciation is given to Teradata Corporation for its sponsorship of this research into data warehouse migration strategies and to the professionals who took time to discuss their experiences.

About Bolder Technology, Inc.

Bolder Technology is a twenty-year-old consultancy focused on Business Intelligence and Data Warehousing. The founder and president is Dr. Richard Hackathorn, who has more than thirty years of experience in the Information Technology industry as a well-known industry analyst, technology innovator, and international educator. He has pioneered many innovations in database management, decision support, client-server computing, database connectivity, associative link analysis, data warehousing, and web farming.

Richard was a member of Codd & Date Associates and Database Associates, early pioneers in relational database management systems. In 1982, he founded MicroDecisionware, Inc. (MDI), an early vendor of database connectivity products, growing the company to 180 employees. The company was acquired by Sybase, now part of SAP, in 1994. He is a member of the IBM Gold Consultants and the Boulder BI Brain Trust. He has written three books and has been a professor at the Wharton School and the University of Colorado.

About the Sponsor

Teradata is the world's largest company solely focused on data warehousing and integrated marketing management through database software, enterprise data warehousing, data warehouse appliances, and analytics. Teradata provides the best database for analytics with the architectural flexibility to address any technology and business need for companies of all sizes. Supported by active technology for unmatched performance and scalability, Teradata’s experienced professionals and analytic solutions empower leaders and innovators to create visibility, cutting through the complexities of business to make smarter, faster decisions. Simply put, Teradata solutions give companies the agility to outperform and outmaneuver for the competitive edge.

Teradata and the Teradata logo are registered trademarks of Teradata Corporation and/or its affiliates in the U.S. and worldwide.

EB-6435 > 0911

References 1 Richard Hackathorn, Bolder Technology. Data Warehouse Migration Strategies: Oracle-to-Teradata

Experiences, April 2011, http://www.teradata.com/t/white-papers/Data-Warehouse-Migration-Strategies-Oracle-to-

Teradata-Experiences/