Understanding the Value of In-Memory in the IT...

Understanding the Value of In-Memory in the IT Landscape

CITO ResearchResearch to solve your problems

February 2012

Sponsored by QlikView

Contents

The Many Faces of In-Memory 1

The Meaning of In-Memory 2

The Data Analysis Value Chain and Your Goals 3

Mapping Vendors to the Value Chain 4

Selecting the Right Technology Combination 10

1Understanding the Value of In-Memory in the IT Landscape


The Many Faces of In-MemoryThis paper aims to clear up the meaning of “in-memory,” a term that describes a vital collection of BI capabilities. If you are a CIO or CTO, you may have noticed the term “in-memory” popping up in articles and vendor announcements. You may think it is something new, but in fact in-memory has been around for a long time.

The term “in-memory” is used to describe many forms of technology related to busi-ness intelligence. QlikView introduced the term in-memory to the world of BI in 1993 as a way to empower people to explore data in a new way using an associative ar-chitecture. Today, SAP, Oracle, and others use the term to describe something quite different. Other companies that make solutions for complex event processing, mar-keting automation, high volume data processing, and visualization all use the term in-memory to describe different ways of helping their customers.

Do all these companies mean the same thing when they say in-memory? In one sense, they do. All these companies use in-memory to speed up processing. In the most im-portant sense, though, the answer is no. Speed creates a different kind of value for each company.

Does it matter how different companies use the term? In our view, it does. By gaining a sophisticated understanding of in-memory, technology leaders gain clarity about the emerging landscape and how to select and combine technologies from that land-scape to meet their needs.

In that spirit, CITO Research is writing this paper to answer three questions:• What does in-memory mean?

• How is the term used by different vendors?

• How can various in-memory capabilities work together?



The Meaning of In-MemoryFirst the basics. Remember that all processing in computers takes place in the CPU. Information is moved between the CPU and memory as a program runs. Most of the time, the information in RAM is moved from disk or from some other location on a net-work. An in-memory application works with data that is stored in RAM so that trans-ferring data does not introduce delays. Many in-memory processors also compress data, often by 10 orders of magnitude or more, which greatly speeds processing.

Applications that pull data from disk into memory are slow compared with applica-tions with all the data in memory. Delays present a huge challenge for computation-intensive applications, such as financial trading algorithms or scientific calculations, but delays are just as challenging for end-users who have to wait for an answer after they start a query.

When memory was expensive, in-memory was impractical. As the cost of memory dropped, it became economical to do calculations in-memory, and more vendors are exploiting the technology. That is why there is so much talk about in-memory these days.

In-Memory, Speed, and Business Value:The Heart of the Matter

The second thing to remember about in-memory technology is that it leads to speed. All vendors who offer in-memory solutions provide benefits based on speed. The key to making the best use of in-memory technology is understanding how speed creates business value.

Think of it in the same way that auto manufacturers highlight vehicles with a V-8 en-gine, which provides power. The power of a V-8 engine could be used for acceleration in a sports car, but in a completely different application, the power could be used by a construction truck to move heavy loads. The same engine is used, but applied and optimized to accomplish different goals.

With in-memory technology, speed is used to create value, as shown in Figure 1.



Figure 1. In-memory creates value through speed

The Data Analysis Value Chain and Your GoalsSo far, so good. In-memory provides speed, and speed provides business value. Now we can address the most important question: • How does in-memory technology help me achieve my goals?

This question really cannot be answered without a way to express different types of goals. Our solution is to categorize the ways people benefit from BI technology, which we call the Data Analysis Value Chain, shown in Figure 2.

Better waysto exploredata Faster

processingof queries

Interactivevisualizations

High volumeprocessing

Real-timeanalytics

SpeedIn-Memory



Figure 2. The Data Analysis Value ChainIn the next section, we map the way that different in-memory technologies provide business value along this value chain and how different vendors use in-memory tech-nology.

Mapping Vendors to the Value ChainNo one vendor satisfies every part of the value chain. In-memory technology can be applied at any of these stages. By combining products, in-memory technology can be applied in all stages using different technologies or combinations of technologies.

Each of the following sections examines the way that different vendors use in-memo-ry technology to support various elements of the value chain.

Oracle Exalytics and In-Memory TechnologyOracle recently introduced the Oracle Exalytics In-Memory Machine, a packaged sys-tem of hardware and software that supports analysis, modeling, and planning. The Exalytics In-Memory Machine boasts 40 available computation cores and 1 terabyte of memory and is capable of replicating an entire data warehouse—the workhorse of the traditional BI world—into memory. It is optimized to work with the Oracle Exadata database and uses columnar compression to expand capacity and speed up analyt-ics queries, which are launched through a SQL interface. Developers will tailor Oracle Exalytics to meet specific needs.

Data analysis value chain elements supported:

Store andSearchLargeDatasets

Distill andAggregateLargeDatasets

AdvancedAnalyticalTechniques

Real-TimeMonitoringand Analysis

AdvancedApplicationDevelopment

User-DrivenApplicationDevelopment

InteractiveExplorationandDiscovery

InteractiveVisualization






Typical user: Developer

Sample goals supported:• Optimize the response time of a financial trading or telecom billing application

• Build new analytical applications in an enterprise heavily invested in Oracle

Massively Scalable Spreadsheets and In-Memory Technology

1010data provides a large in-memory database that can process, present, and ana-lyze data in a way similar to a common spreadsheet. 1010data can handle spread-sheets with hundreds of billions of rows. End users interact directly with a snapshot of an entire dataset rather than interacting with a sample.

In-memory technology accelerates analysis of large amounts of data. This, added to the familiar spreadsheet paradigm, dramatically expands the number of end-users who can directly analyze large data sets.

Data analysis value chain elements supported:Typical user: Analyst

Sample goals supported:• Reduce the time interval for analysis (days rather than months)

• Make searching less constrained than traditional database searches

• Increase business user access to data (no SQL skills needed)







SAP HANA and In-Memory TechnologySAP HANA uses in-memory technology to combine into one the characteristics of relational databases used for transactional applications, data warehouses based on star schemas,1 and data cubes. SAP HANA enables one model to support both trans-actional and analytical use along with many types of common analytical and data pro-cessing functions in the database. SAP HANA transforms SAP’s portfolio of enterprise applications and BI technology into a more cohesive offering.

SAP HANA is not something that end users will use directly; it will primarily be used by developers. SAP HANA works in conjunction with analytical tools. It can be the foundation for designing custom applications and it can feed into business systems such as ERP.

In-memory technology is vital because analytics without star schemas requires very fast aggregation. The aggregation of data that happens in star schemas still must oc-cur, but because of the speed of the database, it happens in real time when the query is executed. The ability to serve both transactional and analytic applications from one database obviates ETL processes needed to migrate data from transactional databas-es to data warehouses.



Sample goals supported:• Reduce analytical tasks from hours or days to seconds

• Increase the amount of analysis performed

• Improve the speed of BI tools

• Process extremely large data sets with billions of rows

• Include both structured (transactional information) and unstructured informa-tion (such as email) in one database





1 Star schemas are designed to optimize user ease-of-use and retrieval performance by minimizing the num-ber of tables to join to materialize a transaction.



Complex Event Processing Systems and In-Memory TechnologyStreamBase Systems and similar vendors use in-memory technology to build com-plex event processing (CEP) applications for financial firms. The object behind Stream-Base is to process streaming data from financial sources or connected devices, putting structured and unstructured data in-memory. As the streams of data arrive in-memo-ry, complex rules and other analytics are applied to recognize important events that may be used to drive automated reactions. This sort of system is used in program trading and other real-time operational systems.



Sample goals supported:• Accelerate and improve the performance of real-time financial or manufacturing

applications

Marketing Automation Systems andIn-Memory TechnologyMetamarkets provides real-time analytics for digital marketing applications. Meta-markets in-memory database can sift through billions of marketing records up to 1,000 times faster than conventional databases. Agencies get a clear view into digi-tal marketing activities. In this case, in-memory technology supports a database and storage mechanism, along with a specially designed pipeline that allows parallel pro-cessing on Hadoop, which is optimized to keep up with huge volumes of data.






Typical user: Analysts

Sample goals supported:• Evaluate the effectiveness of digital marketing campaigns

• Understand the real-time impact of such campaigns

Business Discovery and In-Memory TechnologyQlikView uses in-memory to deliver Business Discovery to its customers, a method of analysis that allows everyone to gain insights using intuitive tools and work together collaboratively and concurrently.

Business Discovery helps people make decisions based on multiple sources of insight: data, people, and geography. Users can create and share knowledge and analysis in groups, across organizations. Business Discovery platforms help people ask and an-swer their own stream of questions and follow their own path to insight—on their own and in formal or informal groups.

QlikView uses in-memory technology because it is the only way to power an associa-tive architecture. When business users explore data in QlikView, they use associations and patterns that are more like those in the human mind than in a traditional data-base, releasing users from the limitations of databases.

In associative architecture, every data point is associated with every other data point. Picture a fiber-optic spider web, where everything is connected. Pulling on one thread lights up the related associations, showing you new paths through the data and re-vealing new kinds of connections.









Typical users: End users, analysts

Sample goals supported:• Collaborative analysis and decision-making across a shared data set

• Building user-driven analytical applications on the fly

• Determine where sales are underperforming and take immediate, corrective action

• Identify analytical gaps in traditional database queries

Visualization Software and In-Memory TechnologySoftware vendors that focus on interactive visualization use in-memory technology to provide instantly updated graphics.

Tableau Software and other vendors use in-memory technology to present interac-tive visualizations. Data can be presented on a graph or in some other complex visu-alization. Parameters can be changed to include or exclude data, and the visualization changes immediately. Users can manipulate information displays in real time and in multiple dimensions using data from predetermined queries.










Typical users: End users, analysts

Sample goals supported:• Communicating information that is difficult to understand in non-graphical

formats

Figure 3 provides a visual summary of the vendors and products just discussed.

Figure 3. Mapping Vendors to the Value Chain

Selecting the Right Technology CombinationThe best in-memory approach for you may involve a combination of technologies. For example, QlikView’s Business Discovery offering and the high-speed databases pro-vided by Oracle Exalytics and SAP HANA are not mutually exclusive. SAP HANA is ex-tremely powerful but does not inherently render data in a way that business users can exploit. Yet datasets processed in an in-memory appliance and delivered to QlikView for presentation in Business Discovery applications, using an associative architecture, could be very effective in completing the “last mile” to business value because busi-ness users can work with data right away. The user-friendly, collaborative nature of









Oracle Exalytics

1010data

StreamBase Systems

Tableau Tableau

SAP HANA

QlikView

Metamarkets

Oracle Exalytics

1010data



QlikView supports gaining new insights. It complements and expands the usefulness of heavy-duty appliances such as Exalytics or HANA, which are optimized to work with other Oracle and SAP software and reach deep into the machinery of a data-rich orga-nization, cutting a lot of the “red tape” in the process. This way, the investment you’ve made in enterprise technology is not only preserved; it’s actually enhanced.

Another common pattern is to use products for visualization along with heavy duty data crunching systems like Hadoop. In some cases, systems like Hadoop can feed data into SAP HANA or Oracle Exalytics and then feed that data into a system like QlikView for interactive processing.

The best way to determine which combination of in-memory technology will deliver the best value for you is to first examine the goals of your business. The powerful “V-8 engine” of in-memory technology is now available in many incarnations to serve those goals.

This paper was sponsored by QlikView and created by CITO Research

CITO ResearchCITO Research is a source of news, analysis, research, and knowledge for CIOs, CTOs, and other IT and business professioals. CITO Research engages in a dialogue with its audience to capture technology trends that are harvested, analyzed, and communicated in a sophisticated way to help practitioners solve difficult business problems.

Understanding the Value of In-Memory in the IT...

Documents

Transcript of Understanding the Value of In-Memory in the IT...