EBI: Ext EndEd BusI nEss Int EllIgEncE REvolut IonIzEs BI · formation environment of today’s...

3DS.

COM

/EXA

LEA

D

EBI: ExtEndEd BusInEss IntEllIgEncEREvolutIonIzEs BI

with

© 2012 Dassault SystèmesLogica & EXALEAD WP: Extended Business Intelligence

EBI (Extended Business Intelligence) refers to revolutionary new decision intelligence capabilities arising from the marriage of the technologies derived from the search engine market (advanced data collection, indexing and retrieval tools) with traditional BI (Business Intelligence) and CI (Competitive Intelligence) solutions.

“EBI profoundly revolutionizes the nature and use of decision platforms and portals.”

François Bourdoncle, Cofounder & Chief Strategy Officer, EXALEAD

“EBI transforms the extended information assets of the enterprise into a factual competitive advantage.”

Frédéric Brajon, Business Intelligence Division Manager, Logica Management Consulting

“EXALEAD’s CloudView TM technology, combined with traditional BI tools, enables faster, better decision-making.”

Ian Steiner, Expert EBI Consultant, IST Consulting

Logica & EXALEAD WP: Extended Business Intelligence© 2012 Dassault Systèmes

ExEcutIvE summaRy

What is Extended Business Intelligence (EBI)?

EBI represents a new approach to exploiting expanding enter-prise information assets to better support decision-making. Intended originally to address certain Business Intelligence(BI) platform limitations, it is now in fact redefining the fieldof decision intelligence by merging BI, Competitive Intelligence (CI), and innovative new technologies derived from theWeb search engine market.

current BI challenges

BI’s evolution from simple operational reporting tool to multi-dimensional analytical capabilities has fueled its widespread adoption as a mission-critical application. However, BI is now facing significant financial and technical barriers to its future efficacy:• limited data scope & RelevanceThough now fed by large data warehouses, BI systems still process only a fraction of relevant corporate information assets, and rarely if ever encompass the exploding volume of unstruc-tured data (email messages, presentations, Office documents, Web pages, instant messages, call transcripts, etc.) containing critical decision-making information.• complexity of useBI platforms typically employ multiple interfaces for access-ing various functions and data sources, and, though much improved, these interfaces are still overly complex, limiting adoption and use.• cost and complexity of scaling and IntegrationTo increase platform relevance, system architects continue to seek to expand the base of data sources feeding BI platforms, to integrate these platforms with other enterprise applications, and to incorporate the workflow and collaborative features users are increasingly demanding. But because these systems are comprised of often dozens of distinct components, these efforts have been highly complex, very costly, and of limited success. Current BI systems are simply difficult to evolve, scale and manage.

current cI challenges

Like BI, CI has matured significantly over the past three decades and has come to be regarded as an essential business activity. However, unlike BI, inherent limitations related to CI source data have hindered the development of robust IT platforms to support CI activities. In fact, CI solutions today consist mostly of Web-based tools of limited scope offering little to no integra-tion with other enterprise applications. The main challenge arises from CI’s reliance on unstructured data as an essential system feed. As noted above, a systematic, scalable solution for treating high volume unstructured content has so far not been available within the decision intelligence marketplace. This shortcoming has not only limited the development of compre-hensive CI platforms, but has been a core barrier to the long-desired merger of BI and CI systems.

the missing Piece: Web search Engine technology

Technologies derived from the Web search engine market are proving to be ideal complements to BI and CI technologies, capable of eliminating the significant financial and technicalbarriers confronting these essential systems, and opening the door to their integration within a unified decision intelligence platform. Web search engines are particularly adept at precisely those functions which present the greatest difficulties for BI and CI:• Handling staggering volumes of dataWeb search engines are designed to handle hundreds of mil-lions of gigabytes of data. This scalability is due both to their system architectures, and to the fact that indexing technol-ogy scales in a way relational database management systems (RDBMS) cannot.• collecting data from widely dispersed, highly heteroge-

neous sourcesThe Cloud environment of the Web increasingly mirrors the in-formation environment of today’s extended, mobile enterprise.• Efficiently processing unstructured dataThe semantic technologies at the core of search engines were specifically designed to analyze and process textual (and in-creasingly multimedia) data.• Ease of useIt is largely the Web’s user-friendly interfaces that have shaped today’s information consumer, leading BI users to demand features such as natural language querying, collaboration, workflow integration and rich, graphical tools.• Ease of scaling, low tcoWeb search engines are built with distributed architectures, making them ideally suited for the kind of cost-effective scaling needed to manage the exponentially growing datastores of the corporate information Cloud.

© 2012 Dassault SystèmesLogica & EXALEAD WP: Extended Business Intelligence

It is important to note, however, that not all search engines, nor search-engine derived technologies, are equal. Those capable of enabling an EBI system need to be able not just to index, but to effectively structure unstructured data, making it exploitable by meaningfully integrating it with existing structured data.In addition, the engines best suited to EBI feature SOA archi-tectures for rapid deployment and easy integration within the complete corporate information ecosystem. Finally, only en-gines that are enterprise-ready can provide the faceted naviga-tion and data security required in a corporate environment.

What EBI means for your Enterprise

Deploying EBI offers significant competitive advantages to organizations. It can help an enterprise better exploit existing structured data, open up access to new unstructured information channels, and unify access to these expanded assets. It can also enable new strategic insights through the dynamic, meaningful association of unstructured and structured BI and CI data, and through the deployment of more intuitive, responsive interfaces. With the right base technologies, the EBI platform can also provide solid data security even in a heterogeneous, fluid environment, and it can break through performance and scalability barriers while maintaining an advantageous TCO.

WE WElcomE youR FEEdBack

Whatever your role—BI platform user, IT analyst, business decision maker, system architect, security expert, or simply a curious reader—your feedback is important to us. We invite you to contact us at the addresses below with your comments, suggestions or questions.

Logica Management ConsultingFrédéric Brajon, Manager of the Business Intelligence [email protected]+33 1 58 22 46 01www.logica.com

EXALEADCarole Offredo, Marketing [email protected]+33 1 55 35 26 26www.3ds.com/exalead

Logica & EXALEAD WP: Extended Business Intelligence© 2012 Dassault Systèmes

taBlE oF contEnts

1. Architecture and Stakes in Business Intelligence (BI)…………..........………11.1 What is BI?.....………………......………………………………………………………………….11.2 Major Milestones in the Development of BI…………………………..……………………11.3 How BI Works……………………………………………………………………………………….2

2. Architecture & Stakes in Competitive Intelligence (CI)……………………...…42.1 What is CI?…......……………………………………………………………………………………42.2 Evolution of CI………………………………………………...…………………………………….42.3 CI Solutions…………………………………………………………………………………………..5

3. Limits of Traditional Decision Intelligence Systems…………………………….5 3.1 Limited Data Scope & Relevance....……………………………………………………………53.2 Limited Responsiveness/Timeliness…………………………………………………………..63.3 Limited Trend Detection…………………………………………………………………………..63.4 Complexity of the Tools…………………………………………………………………………..63.5 Complexity of Platform Implementation……………………………………………………..6

4. Extended Business Intelligence (EBI)……………………………….………………..74.1 Improved Data Scope & Relevance…………………………………………………………….74.2 Improved Timeliness of Data.…..…………………………………………………………….114.3 Better Trend Detection………………………………………………………………………….114.4 Easier Information Access.…..…………………………………………………………………114.5 Faster Implementation, Better Performance……………………………………………...13

FIguREs

Figure 1: Evolution of DIS Functionality……………………………………………………………1Figure 2: Feeding a Data Warehouse………………………………………………………………..2Figure 3: Data Warehouse, Data Marts and the BI Portal……………………………………..3Figure 4: Today’s Extended Enterprise Information Assets…………………………………..5Figure 5: Typical BI Platform Architecture…………………………………………………………7Figure 6: Companies are Generating Staggering Volumes of Data…………………………8Figure 7: Extended Information Assets Now Largely Exceed Corporate Boundaries....8Figure 8: Growing Importance of Extended Informational Assets (IDC)…………….......9Figure 9: Enriching the Dashboard with External Data………………………………...........9Figure 10: Association of Structured and Nonstructured Data in EBI Platforms……..10Figure 11: EBI Portal Benefiting from the Referential Framework of Metadata……..11Figure 12: EBI Portal Offering Natural Language Search……………………………………12Figure 13: Personalized Configuration and Assisted Navigation…………………..……...13

© 2012 Dassault SystèmesLogica & EXALEAD WP: Extended Business Intelligence 1

during this time. The first generation DIS, operational reporting, addressed this basic need for consolidated information, giving users access to highly valuable tables that aggregated elemen-tary data from the OIS.

1.2.2 1991-2005: From Reporting to AnalysisThe rise of client-server architectures contributed to the evolu-tion of DIS toward progressively more advanced data analysis, and extended the user base for such systems.

During this period, DIS evolved from simple reporting to moresophisticated analysis enabled by special tools (called OLAP tools for Online Analytical Processing) capable of combining multiple dimensions (time, geography, etc.) and indicators (sales, active clients, inventory levels, etc.). Additionally, in the mid ‘90s, the emergence of Web portals facilitated the massive diffusion of such applications.

1.2.3 2006-2008: Renewal of BICurrent third generation BI portals now integrate powerful multidimensional analysis functions, ad hoc reporting and even offer some workflow integration.

As a result of this pragmatic evolution (and a growing organi-zational awareness of the need to better manage all types of information), BI technology has gone mainstream, bringing BI into the fold of mission-critical applications like CRM, ERP and SCM.

This rise of “Operational BI” testifies to this mainstreaming. Operational BI seeks to improve day-to-day decision-making by pushing BI information closer to operation-level workers, usually by integrating BI data with existing front-office applications.

Just as the BI consumer base is expanding, so BI data source horizons have begun to expand, with current system designers struggling to capture valuable information residing outside

BI provides enterprise decision support

Ancestor of modern BI, operational reporting did not extend much beyond consolidating data fromheterogeneous and often isolated applications

The appearance of OLAP tools considerably enriched BI applications by enabling more advanced multidimensional analysis

1. aRcHItEctuRE and stakEs In BusInEss IntEllIgEncE (BI)In order to understand the purpose and revolutionary potential of Extended Business Intelligence (EBI), it is important to first understand the functionality, goals and primary limitations of traditional BI systems.

1.1 What is BI?

BI encompasses a range of IT tools, usually accessible through a common BI portal, that aid decision-making for enterprise analysts and managers. These tools include reporting on past performance, monitoring of current operations, performance benchmarking, free analysis of activities, file exportation, and more.In addition to aiding in analyzing past results and assessing a company’s current situation, BI can also serve a predictive role, detecting significant trends, forecasting potential risks, and identifying possible business opportunities.

1.2 major milestones in the development of BI

Since their inception in the mid ‘70s, Decision Information Systems (DIS), which were developed as a natural extension of Operational Information Systems (OIS), have undergone three major evolutions: Operational Reporting, Analytical Reporting, and Extended BI.

1.2.1 1975-1990: Operational ReportingInformation systems during this period were largely composed of numerous isolated applications, each one responding to a business unit’s specific needs, automating that unit’s functions, and adhering to unique management rules. Consequently, access to consolidated information was limited and difficult

Figure 1: Evolution of DIS Functionality

Logica & EXALEAD WP: Extended Business Intelligence© 2012 Dassault Systèmes 2

corporate databases, data that is often unstructured (for example, email messages, Web pages and Office documents) and located both behind and outside the company firewall.

1.3 How BI Works

BI solutions function by exploiting the data stored in a com-pany’s Operational Information Systems (OIS), sometimes (but rarely) including non-structured internal and external data, such as e-mail messages and text documents.

This data is collected from heterogeneous sources (commonly called silos) such as database management systems (DBMS, mostly relational), file servers, databases using proprietary formats (PDBs), and employee work posts.

The data is then filtered according to predefined rules, andfinally consolidated and grouped in data warehouses. Thesewarehouses serve as the foundation for the Decision Informa-tion System (DIS) operating at the core of each BI solution.

Numerous software solutions exist for feeding the warehouseand manipulating the data that it contains to perform functions including:• Producing predefined reports, adapted to questions asked

by analysts and decision makers• Constructing dynamic representations of business entities• Conducting statistical exploration of the data (multidimen-

sional analyses, ad hoc reporting, etc.)• Constructing information dashboards adapted to user

profiles• Automating data exploration to predict the development of

certain phenomena or to detect correlation among certain facts (data mining)

1.3.1 Collecting Data and Feeding a Data WarehouseCollection of data from various silos is entrusted to dedicatedtools referred to as ETLs (Extract, Transform and Load tools),the use of which was generalized in the mid ‘90s. Considering the large volumes of data to be collected, these tools generally operate in batch mode, even if some are capable of managing real-time updates of the warehouse for lower volumes of data.

ETLs use a temporary storage area, called an ODS (Operational Data Store) as a staging area for filtering, cleansing and homog-enizing the data prior to transferring it to the warehouse.

The most advanced BI portals extend multidi-mensional analysis possibilities even further by progressively integrating sophisticated work flow functions

BI uses a Decisional Information System (DIS)which exploits information stored in data ware-houses. Data warehouses are fed by a company’s Operational Information System (OIS)

BI portals offer more or less unified interfacesthat group various decision-making aids like:• Predefined reports • Ad hoc reporting• Detailed analyses • Trend detection

Collection of data used in the DIS is entrustedto specialized tools: ETLs (Extract, Transformand Load tools)

Figure 2: Feeding a Data Warehouse


tEcHnIcal annotatIonData Extraction Models

Data extraction is conducted according to two complimentary approaches:• Dynamic Model (Event-Driven)

The dynamic, or event-driven, approach is linked to the basic operations conducted over time by the company’s employees. The occurrence of these events triggers an update of the data from the OIS, such as for an accounting entry, reduction in stock of a product, creation of a new client, etc. These updates are propagated (by unit or by lot) with a minimum delay to the data warehouse.

• Static Model (Time-Driven)The static approach consists in taking instantaneous snapshots at judiciously chosen intervals (for example, upon completion of an accounting period) of factors characteristic of the activity of the company, for example, the number of clients having purchased product P, the inventory of product K, or the satisfaction index of a certain client group.

Data collection is otherwise a dynamic process. It can:• Unify heterogeneous data from several sources. It is

supported by a data reference framework, equipped or not, under the form of Master Data Management (MDM)

• Consolidate departmental data (recomposition of management objects, which are meaningful units of information that aid the decision-making process)

• Detect incoherent data and exclude it from the extraction/consolidation process

• Certify the data’s compliance with regulatory constraints and rules, and, if necessary, detect undesirable anomalies

To better address the needs of specific business units (account-ing, marketing, etc.), DISs often break data warehouses into specialized storage segments called data marts. Among other benefits, data marts significantly reduce of the volume of infor-mation to be processed.

Figure 3: Data Warehouse, Data Marts and the BI Portal

Data extracted from heterogeneous sourcesis filtered and aggregated using an Operational Data Store (ODS) prior to input to the global data warehouse

Users typically access the BI platform via a portal that combines various decisional tools, such as reporting, alerts, and tracking of key performance indicators (KPIs) like:• Prospect conversions by activity sector• Client satisfaction indices after receiving technical support• Ratios of defective products from a production chain• Inventory turnover within specified periods, etc.

1.3.2 Analysis Axes and IndicatorsOne may, thus, schematically consider BI as placing indicators in perspective according to multiple axes of analysis, called dimensions:• Indicators (including KPIs)

The indicators are all the facts the BI attempts to quantify, e.g. the number of active clients, returns of product P, inventory of product S, etc.

• Dimensions (Axes of Analysis)The dimensions are the various perspectives from which the indicators can be analyzed—by calendar period, geo-graphical area, market segment, business unit, etc.

In addition to providing multiple dimensions for reporting, BI provides analytical tools (often graphical) that enable the user to perform a variety of advanced operations on data, for example:• Visualization of indicators and exploration of relationships

among them• Exploration of indicators according to select dimensions, various levels of consolidation, and desired detail level• More or less complex data calculations (averages, statisti-

cal analyses, comparisons of aggregate data along several dimensions, etc.)


BI enables analysis of indicators according to several axes, or dimensions, like time, geography, or business unit

tEcHnIcal annotatIonTables, Cubes and Hypercubes

Within decision platforms, when analysis is conducted using two dimensions, one speaks of a table, with the indicators positioned at the intersection of the rows and columns (thedimensions, or axes) of the table, for example:• Dimension 1: temporal axis (time)• Dimension 2: market segment axis

At the intersection of these two axes, there are indicators representing the number of active clients, the number of dissatisfied clients, the number of new clients, etc. The table becomes cubic (three dimensional) as soon as one associates it with a third axis of analysis (a third dimension), such as adding geographical area to the preceding example. Finally, one speaks of a hypercube when dealing with more than three dimensions.

The following terms are used for BI reporting tools to indicate options for navigating among the dimensions of the tables, cubes, and hypercubes:• Drill down indicates the action of refining a search

according to a given dimension (zoom in - move ahead)• Drill up indicates the action of retreating within a cube

(zoom out -backing up) which leads to consolidation or summation of the data in question

• Slice and dice primarily involves altering the various dimensions in order to change the point of view within

the data analysis

2. aRcHItEctuRE & stakEs In comPEtItIvE IntEllIgEncE (cI)2.1 What is cI?

An outgrowth of the rise of the market economy and intelligence strategies developed in the context of the Cold War, CI is, broadly stated, the practice of legally, ethically collecting information about the players within a company’s market (competitors, customers, suppliers, distributors, regulatory agencies, etc.), analyzing this information within the context of a business’s operations, and using the knowledge gained for decision support.

2.2 Evolution of cI

The study of the strategies and tactics used by a company’scompetitors has been an indelible part of the market economy since its inception. However, an industrial and rational approach in this domain did not arise until the mid ‘70s, largely under the auspices of seminal work by Michael Porter (see Competitive Strategy, published in 1980).Numerous theorists have since enriched, extended, and com-pleted the analytic techniques proposed by Porter, such as Craig Fleisher, Rich Redmond, Babette Bensoussan and Frederick Rustmann, author the 1992 CIA Incorporated: Espionage and the Craft of Business Intelligence.

Today, companies use CI for many purposes, including:• Reinforcing their competitive position in a specific market

segment• Refining their overall strategy for approaching target

markets• Understanding and planning appropriate responses to

strategies implemented by their competitors and partners

CI is thus likely to have repercussions for both a company’s mid-term and long term strategies,including:• Competitive global positioning• Diversification strategies (internal, external, concentric,

by conglomeration, etc.)• R&D plans• Plans for mergers and acquisitions of competitors, part-

ners, clients, and suppliersCI can also affect a company’s short term strategies, such as product positioning, pricing and packaging.

Competitive Intelligence emerged as a disciplineduring the Cold War, and defined industrial principles for competitive analysis and strategies for the enterprise

CI enables the enterprise to understand their com-petitors’ strategies, and those of other important market players, and is particularly intended to identify nascent trends and new opportunities for growth and diversification


2.3 cI solutions

2.3.1 CI Information ChannelsWhereas BI relies principally on structured data stored in datawarehouses, CI relies on mainly non-structured, outside-the-firewall information. CI sources include:• Public websites and subscription-based Web services• Press and media, both generalized and specialized• Publications and reports by partners, suppliers, competi-

tors, etc.• External databases and subscription-based data aggrega-

tion services• Ad hoc studies conducted by specialized research bureaus• Market analyses conducted by independent institutions• Output from industry events like conferences, trad shows

and round tables

2.3.2 CI SystemsBecause of CI’s reliance on high volume, weak-signal resources like the Web, CI has remained a largely human endeavor, with internal or external specialists providing the bulk of CI informa-tion through custom-produced research products like company profiles, M&A due diligence reports and market and industry audits.

To date, the automated CI systems which do exist remain rela-tively limited in scope. These systems include:• Collaboration software that simply provides companies

with a central place for employees to share and address competitive information.

• Search engines that collect, aggregate, index select data sources (often limited to public Web resources and subscription news feeds). These systems are good at providing unified access to select resources, but often weak in analytical capability.

• RDBMSs that essentially serve as content management servers for CI-related data, often relying heavily on formal and official data from market studies and databases, and relying on built-in RDBMS indexing and search capabilities.

CI mainly exploits information channels external to the enterprise, with the Web being a preferred information channel

3. lImIts oF tRadItIonal dEcIsIon IntEllIgEncE systEms3.1 limited data scope & Relevance

3.1.1 CI Scope & Relevance IssuesMost CI systems are highly siloed, offering little to no integra-tion with other relevant systems such as BI, ERP or CRM.They also all hampered by:• The considerable volume of information to be handled

(they are ill-equipped to efficiently process very large volumes of data, and they require a highly customized, perfectly adapted and configured tool in order to produce relevant data)

• The very weak signal/noise ratio (SN ratio) of the preferred CI channel, the Web, with the attendant risks of missing important information and of misinformation (deliberate or involuntary)

3.1.2 BI Scope & Relevance IssuesBI systems face limitations in both accessing all relevant internal information, and in incorporating relevant external information.

Internally, the extreme specialization of data silos and the everincreasing complexity and diversity of information systems make it difficult to collect and manipulate all relevant in-house decision information. This applies especially to unstructured internal information like email messages, Office documents, and multimedia files.

The Web in particular is a challenging CI data source because of the strong dilution of informa-tion relevance and very weak signal-tonoise ratio

Users often complain about the low relevance of information available through their BI platforms. Users are increasingly aware that there is consid-erable pertinent information outside the reach of their BI platform

Figure 4: Today’s Extended Enterprise Information Assets


On the other hand, even when these traditional data sources are bridged, current BI platforms are still rarely able to effec-tively leverage relevant external data, especially the impressive mass of non-structured data available on the Web. In effect, neither BI nor CI can effectively leverage today’s Extended Enterprise Information Assets.

3.2 limited Responsiveness/timeliness

For BI, a lack of data freshness is a perennial complaint.Since input to data warehouses and data marts largely depends on ETL tools operating in batch mode, the freshness of the information and responsiveness of the decision platform can become critical. This is especially the case when decisions must be made based on the analysis of a series of eventsout of sync with the frequency of warehouse updates.

Some CI systems, specifically those leveraging some type of search engine technology, do provide more timely information than BI systems, but most are not robust enough to provide real-time updates for large data sets.

3.3 limited trend detection

The frequent restriction of a BI platform to structured dataderived from the company’s OIS poses a problem in the detec-tion of signals with a weak amplitude (common for unstruc-tured data) that could reveal potentially significant operational malfunctions or market trends. This is also a problem for CI tools, which are fed almost entirely by such weak signal data.

BI and CI systems not only need to do a better job of captur-ing and filtering such weak-signal information,the two types of systems to be bridged to make the correlations essential for effective trend detection.

3.4 complexity of the tools

DIS portals and dashboards have become more user-friendly. However, even with the widespread implementation of visual aids, they are still complicated to master, limiting the potential user base and restricting the creativity of analysts.

In addition to simpler, more intuitive interfaces, users are also demanding simpler query methods. DIS systems can success-fully respond to historical queries like “Which products at-tracted the highest number of clients in New York between last May and June?” and even sometimes answer prognostic ques-tions such as “Which products are likely to attract the greatest number of clients in the San Francisco six months from now?” but the methods of formulating and inputting these types of questions still operates within rather rigid boundaries.

This rigidity has led users to unanimously express the desire for natural language query and search functions, that is to say, for the ability to pose questions like those above verbatim, as they would in conversation.

According to Deirdre Serra, Senior BI Analyst at Forrester Research, “BI must evolve towards the integration in decision platforms of functions that enable users to express their queriesas simply as possible, and to better exploit the information available in their company’s extended information assets.”

3.5 complexity of Platform Implementation

Decision platforms must perform five fundamental operations:• Data collection• Data integration• Data processing (transformation)• Data access• Platform administration

Despite their marketing claims, most software providers do not offer an integrated solution for all these functions. In fact,most decision platforms are composed of up to dozens of dispa-rate components, integrated in ad hoc manner over time, with an attendant complexity in operations and management, and limitations in scalability and performance.

According to numerous users, BI platforms are notresponsive enough to meet operational constraints

The desire for better trend detection illustrates the need for convergence between BI and CI tools, a major challenge for future decisional platforms to meet

Users complain of the complexity of access inter-faces for decision tools, and unanimously express a desire for natural language query and searchfunctions


4. ExtEndEd BusInEssIntEllIgEncE (EBI)EBI is intended, primarily, to address certain shortcomings of traditional BI platforms, but it can also grant companies a con-siderably extended decision environment. It constitutesanother decisive step in the long awaited marriage of BI and CI.

Frédéric Brajon, Manager of the BI Division at Logica Manage-ment Consulting, summarizes the objectives of this approach: “The mission of EBI is to transform decision makers from ‘guided consumers’ of information into veritable ‘creative analysts’.” EBI enables users to pursue their analysis in a more intuitive fashion, democratizes access, and provides the type of timely, deeply contextualized information required for better, more rapid decision making. In doing so, EBI enables:• Better exploitation of existing data channels• Exploitation of new information channels (often unstruc-

tured data from sources such as the Internet, email servers, desktops and file servers)

• The intelligent association (or combination) of structured and non-structured data in order to yield new insights

Extended Business Intelligence (EBI) intends to transform decision makers into creative analysts

BI platforms sometimes encompass several dozentools. The complexity of this integration isa technical challenge for companies and limits the scalability of their DIS

Figure 5: Typical BI Platform Architecture

• Improved timeliness of data through the deployment of search engine-derived technologies

• Simplified, more intuitive data access and interfaces• The implementation of architectures offering better perfor-

mance, reliability, stability, and scalability• Overall improvement of the cost/relevance ratio of the

decision platform

4.1 Improved data scope & Relevance

4.1.1 Better Exploitation of Existing Structured DataAccording to Susan Feldman, Senior Consultant at IDC,“For enterprises, the main stakes in decision systems no longer reside in the collection of information, but in the intelligent assimilation thereof. Thus, in order to increase the relevanceof traditional decision portals, it is preferable, in the first place,to attempt to reinforce exploitation of existing data channels,rather than continue to accumulate superfluous data.”

Better Use of Existing Structured Data EBI platforms are supported by sophisticated data indexing and search infrastructures derived from the latest search engine industry technologies. These infrastructures provide advanced features such as the use of a natural query language to search the structured content of a DIS, content that was often previ-ously only accessible through the formulation of complex SQL queries or navigating rigid drilling functions. Such ease in querying can significantly improve the exploitation of existing structured data.

Better Use of Existing BI Data.

The EBI platform also indexes and processes information contained in existing BI reports, analyses, and dashboards. This processed content, both structured and rich in metadata, can then be integrated with data from new and emerging information resources (like digital audio transcripts, satisfaction surveys, blogs and forums) to create dynamic associations between structured and non-structured data.

EBI enables better use of existing data channels, exploitation of new information channels, anddynamic associations between structured andnon-structured data

EBI improves access to and assimilation of data from existing informational channels


4.1.2 Exploiting New Data ChannelsDan Rottenberg, Professor of Decision Technology at the University of California, Berkeley, writes, “The relevance of data stored in warehouses and data marts depends directly onan enterprise’s ability to exploit their extended information as-sets, composed of data internal and external to the enterprise.Handling unstructured data in synergy with structured datais undoubtedly a challenge that enterprises must confront.”

The volume of internal structured data has grown immensely, which is particularly explained by the generalization of ERP type programs. At the same time, semi-structured and un-structured data has exploded. Email, Office documents, PDF files, multimedia archives, etc., have become an integral part of an enterprise’s Extended Information Assets (EIAs). In fact, analysts predict unstructured content now comprises as much as 80% of EIAs.

Furthermore, the total volume of both unstructured andstructured data will continue its exponential volume increasefor the foreseeable future. Globally, businesses generated(created, captured, modified, etc.) nearly 281 million GBs ofdata in 2007, and that volume is estimated to increase six-foldby 2011 (IDC Research, 2008).

The use of new information channels becomesa determining factor in reinforcing the operational excellence and competitiveness of the enterprise

Part of this phenomenal growth is due to the emergence of new technologies which produce high volumes of unstructured data. Examples include:• Speech-to-text applications, capable of transforming

digital audio recordings of conversations between several speakers (a client, a marketing representative, a prospect, a telemarketer, or a support consultant, etc.) into clear text transcripts

• Image analysis applications, which use advanced statistical and machine-learning based techniques to independently generate metadata describing visual content

• Value-added syndication and collection applications (data mashups) that dynamically integrate local and external content (for instance, serving content that mixes Web data and data drawn form an internal database)

Other trends related to the reality of the “extended enterprise” are fueling this growth in non-structured data:• The reinforcement and industrialization of cooperationbe-

tween companies. This evolution has been supported by the automation of flow exchange and control processes, leading to the production of new unstructured data (docu-mentation, activity reports, etc.)

• The increasing reliance on external market studies and benchmarking information for increasing the relevance of certain decisional applications and CI

Of course this growing reliance on external data also encom-passes structured data, as represented, for instance, by the need to integrate external geolocalization data with certaindecisional indicators (such as the graphical representation of product stocks on a map of local warehouses).

Figure 6: Companies are Generating Staggering Volumes of Data

Analysts note an exponentially growing volumeof data created, acquired, and manipulated by enterprises around the world

Figure 7: Extended Information Assets Now Largely Exceed Corporate Boundaries

Numerous factors contribute to the increasing importance of the EIA, especially the expanding boundaries of the extended enterprise


Nonetheless the majority of these new external sources are of an unstructured nature, including vast resources available on the Internet (blogs, forums, RSS feeds, user produced content, etc.). To remain relevant, BI must assimilate this data.

Internet data is already a critical channel for CI, even if, for the reasons discussed, it is an underutilized channel. By better exploiting and integrating all these new channels, EBI signifi-cantly advances the possibility of integrating BI and CI within a single DIS.

4.1.3 Confronting the Challenges of the EIA

EBI intends to exploit in particular, and on a grand scale, the vast EIA resources represented by information from the Web. This will enable the enrichment of BI platforms with emotive analyses and product or brand notoriety using the information available in an unstructured format in blogs, forums, press sites, dedicate sites, testimonials, buyer comments, etc Exploitation of EIAs, nonetheless, poses these difficulties:

• Unstructured data sources are heterogeneous and widely dispersed inside and outside the enterprise. The challenge is thus to locate pertinent resources, and to extract, classify, and exploit the useful information they contain from the point of view of decision systems, e.g., effectively detecting named entities, identifying patterns, enriching thesauri,

Figure 8: Growing Importance of Extended Informational Assets (IDC)

Intelligent use of the EIA allows enrichment of dashboards of BI platforms

The necessary convergence between BI and CIapplications occurs by uniting the various datasources that compose the extended informationassets of the enterprise (EIA)

performing semantic analysis, dynamically creating content summaries, etc.

• The volume of unstructured data is potentially considerable. Although DBMSs are capable of generating very large quantities of data, the cost of hardware infrastructure required to operate warehouses capable of integrating the EIAs, which can soar to petabytes of data, could become exorbitant and limit the deployment of EBI applications.

tEcHnIcal annotatIonLimitations of DBMS

DBMSs must use sophisticated mechanisms to protect the integrity of data bases in a context of concurrent transactional access. To succeed, these mechanisms must preserve four essential transaction properties, known as ACID properties:• Atomicity - a transaction constitutes an atomic interchange

unit within the RDBMS and must be processed in its entirety (committed) or completely canceled (rolled back)

• Consistency - after executing a transaction, whether or not it is successful (i.e., committed or rolled back), the database must remain in a state consistent with system rules (i.e., only valid data will be written to the database)

• Isolation – two concurrent transactions must remain isolated from each other, and must not update the same data at the same time.

• Durability - the effects of one transaction must be durable and persistent, meaning once a transaction is committed, the database changes will be preserved even in case of a subsequent failure like a server crash. Reliable transactional processing is an essential DBMS function, but the demand such processing places on the DBMS limits the exploitation of the system by a wide user base for extended purposes.

Figure 9: Enriching the Dashboard with External Data


The purpose of these functions is to extract a limited amount of relevant structured data from an arbitrary volume of unstruc-tured data: brand names, product names, prices, notifications, and consumer comments, etc.

Let’s consider, for example, the case of a BI portal using information from telephone conversations between unsatis-fied customers and customer service agents. Each agent uses an application that allows him to select, from a previously established list, the reasons for the customer’s dissatisfaction, an open text field being available for comments or justification of their choice.

Incorporating analysis of this open text field in a DIS can indispensably facilitate the work of analysts, providing important insights into the reasons for returns, poor sales, lawsuits, malfunctions, and other trends that go far beyondthe very structured capacity of multiple choice responses.When restricted to an RDBMS, such information has to be anticipated and integrated, or “hard-coded”, into decision applications, but new technologies allow a dynamic, real-time enrichment of analytical applications.

Furthermore, DBMSs offer only rudimentary managementof unstructured or complex data types. Vendors (Ingres, Oracle, etc.) have attempted to extend the base of data types these systems can process, but DBMSs face inherent technical limtations in accommodating unstructured data. DBMSs were designed in the late 1980s, and primarily intendedto implement relational models and to support OLTP applications. The relational model requires representingthe structures of complex and undetermined data in advance,a task at odds with the practical need to autonomically treat voluminous unstructured data.

To enable a DIS to effectively exploit voluminous, mostly un-structured EIAs, it must offer capabilities that largely surpass the functionality of DBMS and other classic data manipulation tools.

EBI exploits new data collection, processing, indexing and access technologies that remove the existing technical and financial barriers associated with leveraging unstructured data in BI systems, without introducing noise pollution in the system.

These new technologies work by autonomically transformingnon-structured textual data into structured information usingsemantic analysis techniques drawn from artificial intelligence, including:• Automatic language detection• Lemmatization, or the intelligent recognition of the form

and variations of words from a language, i.e., feminine or masculine, plural, conjugation state, adjectival usage, etc.

• Advanced phonetic recognition to manage typographical errors, inversion of letters or letter groups, homonyms, etc.

• Personalized semantic filtering, adapted to the ontological context of each enterprise

• Semantic analysis, detection of lexical forms (patterns), recognition of named entities (people, places, times, dates, etc;), integration of business thesauri (ontologies), and creation of specific semantic rules

New solutions for data collection, enrichment, indexing and search effectively work around thelimitations of DBMS

The treatment of unstructured textual data re-quires the implementation of advanced semanticanalysis algorithms

Considering the volume of the EIA, it is a solid strategy to extract only targeted information from the impressive volume of unstructured data

EBI portals combine data from standard BI resources with unstructured data, possibly integrated with classic BI tools by the creationof new indicators

Figure 10: Association of Structured and Nonstructured Data in EBI PlatformsDus coria dolorem qui temporepelis prorpossin natusda


Given the volume of information to consider, the use of Web-derived, Web-scale technologies is indispensable. Because Web search engines are designed to process hundreds of millions of gigabytes of unstructured data, only they can meet these technological challenges.

4.2 Improved timeliness of data

The data warehouses that feed traditional BI systems are customarily updated by ETL tools operating in batch mode, which limits the timeliness of information. Operating at a layer above data-producing systems, Web search engine-derived technologies, can, on the other hand, handle real-time updates to staggering volumes of data without burdening underlying systems.

By transferring the majority of system queries from the data warehouse to the EBI index, it becomes possible to not only accelerate the frequency of core database updates, but to simultaneously expand data access while doing so. In addition, in some cases, sources that normally feed the warehouse may instead be directly indexed in real time by the EBI platform, further improving data freshness.

4.3 Better trend detection

According to IDC’s Susan Feldman, “Our recent research confirms that the mutual use of unstructured content and strongly structured databases shows that the two approaches complement each other and enable the discovery of unexpected information.”

In order to guarantee the best use of the enterprise’s extended informational assets (EIAs), the EBI platform must be able to dynamically combine all available data channels.

Let’s consider, for example, the case of an analyst studying fac-tors that may explain a marked decrease in subscription renew-als for a specific service. With an EBI platform, this user can:• First of all, locate the period in which he or she is interested• Search clients that have chosen not to renew their

subscriptions• Look at e-mails containing requests or complaints sent to

the company by these customers, and, if desired, view text transcripts of telephone conversations between these same clients and the company’s customer service representatives

• Search for the names of competitors possibly contained in these documents

• Seek out data on the Web containing special offers or promotions offered by these same competitors• Create an overall report for the sales and marketing

departments

Naturally, this association between structured and unstructured data also concerns navigation within a hypercube (drill, slice & dice, etc.), which must dynamically adapt to the context of the unstructured data search.

4.4 Easier Information access

4.4.1 Anticipating Needs and Improving the Relevance of PortalsFaced with the accumulation of documents and varied decision-al tools, users sometimes find it difficult to find the information or report containing the sought-after indicator. EBI is intended to meet this challenge by offering users transparent access to the metadata reference frame of the platform.

Let’s consider a simple question, for example, “What werethe consolidated sales figures from Division X last September?” In addition to producing the desired report, EBI portals go further, anticipating the user’s next queries. It will, for example, offer reports such as:• Consolidated and detailed sales figures per company divi-

sion• Analysis of sales figures per division, time, and sales execu-

tive• Customer service reports by product, division and time• Competitor analyses by region, time and product line

Hence EBI improves the relevance of the decision portal byhomogenizing the metadata reference frame and adaptingresults to the user’s profile.

The association of structured and unstructured data increases the relevance of the EBI platform,and simplifies its adoption by users

Figure 11: EBI Portal Benefiting from the Referential Framework of Metadata


4.4.2 Natural Language QueriesA development that users have long wished for, integrationof a natural language search function in decision portals helps analysts and decision makers ask questions that a traditional BI platform cannot handle, such as, “Which customers are satis-fied with product P?”

Such questions do not return 100% precise responses, but they can retrieve highly pertinent, consolidated information from myriad sources: email messages, call transcripts, feedback surveys, referral campaigns, etc. Even if traditional BI platforms could integrate the data necessary to handle such a query, how would one express it using BI tools and classic hypercube navigation mechanisms?

And even if, by successive attempts and with a profound knowl-edge of the portal, usersmanage to obtain a suitable answer, natural language, associated with new indexing and data retrieval methods, favors a more efficient interaction with the decision platform.

According to Dan Rottenberg, “The implementation of a natural language search interface increases the power of unstructured information, and that resulting from the combination thereofwith structured data, tenfold.”

Better exploitation of referential metadata, homogenization of metadata, and integration of information related to the user’s profile improves the relevance of EBI portals

The ability to express search queries in natural language is a recurring request from BI platform users

Natural language search can be implemented globally for the EBI portal (global search) or fora subset thereof (refined search)

Figure 12: EBI Portal Offering Natural Language Search

4.4.3 Navigation AssistanceThe EBI portal offers advanced navigation devices, adapted to each decision context, as well as to the user’s profile. This entails the creation of shortcuts (clickable links, for example) that facilitate navigation within the data: zooming in and out (drilling), filtering of data by keywords, permutation of the dimensions of the hypercube (slicing), etc.

Such shortcuts also enable filtering by document format: email, reports, text documents, presentations, etc. They constitute, finally, a simple and practical means to save (render persistent) a complete search or study context, in order to facilitate later analysis.

4.4.4 Unified Information Access & Collaboration

Nearly all decision platforms offer some type of portal inter-face in response to companies’ desire to simplify and centralize access to decision information for their employees. However, these are still too often simply gateways to myriad tools rather than truly unified information access platforms. It is crucial to

Assisted (guided) navigation of decision informa-tion and the ability to save complex navigation contexts are becoming critical to users

Users must be able to set - and share -preferences for navigation, search, and information analysis


banish the use of multiple access points for different functions: global reporting, ad hoc reporting, multidimensional analysis, interfaces for managing references or indicators, etc.

In addition to unifying access to system functions and data sources, EBI portals offer important personalization options. Once users are connected and signed in, the EBI portal offersa personalized home page that enables them to:• Modify the options for displaying dashboards and access-

ing their favorite tools• Choose the decision information sources most relevant to

them• Configure administrative interfaces, etc.

Sharing this personalization between user groups at the same level of ability, and/or with the same operational needs, has also become critical. Additionally, each user can share the decision information they choose, the EBI platform becoming, thus, a collaborative tool, integrating such functions as workflow in order to organize and supervise the distribution of decision information by taking into account the specific authorizations of each collaborator.

Finally, the growing maturity of tools for mashup applications and Service Oriented Architectures (SOA) facilitate integration of decision tools at the user’s work post. With the deployment of an EBI platform able to syndicate its services, it is possible to offer each company collaborator personalized access to decision information in order to help them increase their operational excellence, regardless of the technology implemented: widgets, rich client interfaces, CRM tools, etc.

Figure 13: Personalized Configuration and Assisted Navigation

4.4.5 Guaranteeing the Security and Confidentiality of Data

EBI expands the scope of extended information assets while democratizing access to these same assets. Naturally, this openness must be handled in compliance with security and confidentiality regulations for the data manipulated.

For example, only an explicitly identified group of users will have access to competitor analysis information. Likewise, security devices for access to reports, indicators, and other multidimensional analysis data must be installed with a highly refined granularity, reaching down to the level of individual data.

Because the stakes surrounding this issue are high, it is absolutely essential that an EBI solution leverages an enterprise ready search engine, one designed from the ground up to integrate fully with existing corporate security constraints—regardless of the format or source of the data manipulated.

4.5 Faster Implementation, Better Performance

EBI platforms offer higher performance and lower costs than traditional decision platforms:• They free data access from underlying RDBMS applica-

tions, reducing the load on these systems and therefore alleviating the heavy infrastructure and licensing costs associated with scaling these systems.

• They use distributed, service oriented architectures that offer highly efficient resource usage; unlimited, low cost scaling; and greater system availability.

• Leveraging the latest semantic technologies, they provide effective cost-efficient structuration of massive volumes of unstructured data, data heretofore inaccessible or acces-sible only at great cost.

• They offer easier management with truly unified access (and a truly unified platform) for managing all five decision platform functions:

The confidentiality of EIA data (structured and nonstructured) is a major stake for the company

The distribution of decision information must bestructured using a workflow tool adapted to the company’s organizational constraints, especially with regard to data confidentiality


• Data collection• Data integration• Data processing (transformation)• Data access• Platform administration

While it is only Web search engines that can meet the scaling needs of today’s extended corporate information environment, it is important to note that the performance and cost benefits detailed above can only be achieved with an EBI platform that uses a search engine that it is not only Web-ready, but enterprise-ready, i.e., it:• Integrates quickly and easily with existing security

systems• Can be rapidly deployed across a complex corporate IT

ecosystem• Provides semantic abilities that go far beyond keyword

indexing, offering the kind of deep text analytics that can finally merge unstructured and structured BI and CI data in a meaningful way

aBout logIca managEmEnt consultIngLogica Management Consulting is a division of Logica plc,a global IT and business services company employing 39,000 people in 36 countries. Logica’s activities include management counseling, systems integration, and outsourcing of business and IT processes.

Logica Management Consulting enables business transfor-mation through the innovative use of technology to improve operational performance. Their more than 1,000 consultants are noted for their profound sector knowledge as well as the balance in their business, operational and technical experience. Their pragmatic implementation of innovative technologies helps their clients position themselves at the forefront of their respective markets.

Logica is listed on the London and Amsterdam stock exchanges (LSE : LOG ; Euronext : LOG). For more information, visitwww.logica.com.

Information Intelligence

Collaborative Innovation

Dassault Systèmes, the 3dEXPERIENCE Company, provides business and people with virtual universes to imagine sustainable innovations. Its world-leading solutions transform the way products are designed, produced, and supported. Dassault Systèmes’ collaborative solutions foster social innovation, expanding possibilities for the virtual world to improve the real world. The group brings value to over 150,000 customers of all sizes, in all industries, in more than 80 countries. For more information, visit www.3ds.com.

CATIA, SOLIDWORKS, SIMULIA, DELMIA, ENOVIA, GEOVIA, EXALEAD, NETVIBES, 3DSWYM, 3DVIA are registered trademarks of Dassault Systèmes or its subsidiaries in the US and/or other countries.

Visit us at3ds.com/ExalEad

Delivering Best-in-Class Products

Europe/Middle East/Africa Dassault Systèmes 10, rue Marcel Dassault CS 40501 78946 Vélizy-Villacoublay Cedex France

Americas Dassault Systèmes 175 Wyman Street Waltham, Massachusetts 02451-1223 USA

Asia-Pacific Dassault Systèmes Pier City Shibaura Bldg 10F 3-18-1 Kaigan, Minato-Ku Tokyo 108-002 Japan

Virtual Products

3D Design

Realistic Simulation Dashboard Intelligence

Social Innovation

3D Communication

Model and Simulate our Planet

Digital Manufacturing and Production

EBI: Ext EndEd BusI nEss Int EllIgEncE REvolut IonIzEs BI · formation environment of today’s...

Documents

Transcript of EBI: Ext EndEd BusI nEss Int EllIgEncE REvolut IonIzEs BI · formation environment of today’s...