Framework Modeling Guidelines
-
Upload
manjeet-singh -
Category
Documents
-
view
31 -
download
1
description
Transcript of Framework Modeling Guidelines
Cognos 8 – Best Practices Transformer
Framework Manager Modeling Guidelines
Saravanan Vajjiravel
24-Sept-2011
COGNOS P r a c t i c e Learn. Adapt. Belive. Succeed with Proven Solutions
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 2
REVISION HISTORY ................................................................................................................................. 3
SUMMARY: ................................................................................................................................................. 4
AUDIENCE: ................................................................................................................................................. 4
INPUTS: ........................................................................................................................................................ 4
DELIVERABLES: ....................................................................................................................................... 4
STANDARDIZE USER INTERACTIONS AND REUSE CALCULATIONS ....................................................... 6 LEVERAGE EXISTING MODELS ............................................................................................................ 6 ENFORCE SOURCE CONTROL ............................................................................................................... 6 EMPLOY A STANDARD NAMING CONVENTION .................................................................................... 6 USE AN APPROPRIATE NUMBER OF MODELS ....................................................................................... 7 DATA SOURCE CONNECTIONS ............................................................................................................. 7 EFFICIENT PROCESSING ....................................................................................................................... 8 LIMITED FUNCTION SET ...................................................................................................................... 8 LINKING AND SEGMENTING ................................................................................................................. 8 VALIDATE THE MODEL.......................................................................................................................15 MODEL MODIFICATIONS ....................................................................................................................15 SYNCHRONIZE THE PROJECT ..............................................................................................................16 TASK AUTOMATION ...........................................................................................................................16 MODEL REPORT .................................................................................................................................16
DESIGN TASKS: ........................................................................................................................................17
BUILD TASKS: ...........................................................................................................................................19
CHECKLIST: ..............................................................................................................................................20
ADDITIONAL REFERENCE DOCUMENTS: .......................................................................................20
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 3
Revision History
Release Date By Description
1.0 01/06/2011 Saravanan Vajjiravel Base Version
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 4
Summary: This document is intended to provide guidelines and processes that should be used when
implementing Cognos 8 models. Framework Manager (FM) is a desktop tool used to model the metadata for all ad-hoc and standard SQL reporting in Cognos8. Framework Manager is also
used to create IQD files that feed PowerPlay Transformer. The information within this document
applies to versions Cognos 8.x.x, and should be used in conjunction with the standard Cognos documentation provided with Framework Manager.
Be aware that the metadata will have a significant influence on the queries that are generated. To ensure that predictable queries and results are returned, it is necessary to correctly model the
metadata.
Audience:
Cognos Developer (Framework Manager Modeler), Cognos Center of Excellence (Cognos Application Architect)
Inputs:
Approved Project
Detailed Report Specifications Document (to include Service Level/Business Requirement
expectations) Data Model
Naming Standards as Approved by Data Governance
Source to Target Map
Deliverables:
Framework Manager Model
Store deliverables within corporate source control or at a minimum on a shared network
drive that is part of a formal back up process Naming conventions for deliverables
Tools: Cognos 8 installation on development server
Framework Manager installed on developer workstation (Please note: The version must
match the version installed on the Cognos 8 Server.)
Database connectivity software (Oracle Client, SQL Client, etc.) for all required data
sources Access to the development gateway server (your network admin must make provisions to
have you set up within the Corporate LDAP and the Cognos enterprise environment.)
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 5
Overall Considerations: Framework Manager is the foundation to a successful Cognos 8 application. Fully establish user
requirements prior to building the Framework Manager model. Listed below are several items that should be taken into consideration before modeling efforts commence:
Will this be a high availability system that requires user access on 7x24 basis? How will
database updates be handled - perhaps synonyms or a database alias can be used to ensure
that source data is always available.
What is the expectation for query performance?
Note: This is not the same as report performance and should have been established as part
of the project approval process.
Will this application be integrated with another application?
How will data security be controlled?
Examples: Row level security, database signons, objects access through the interface.
Will the project support authoring across all of the studios? If the project requires delivery of
ad-hoc functionality, then extra effort must be made to ensure that users are presented
information in a meaningful way, and appropriate precautions have been taken to prevent
run-away, poor performing queries. Consideration may be given to the following options:
o Incorporate a prompt against an indexed value in the query subject. The idea is to
incorporate a filtering mechanism that prevents the ad-hoc user from performing
queries that will perform full table scans or transferring large amounts of extraneous
data from the data source to the report server for processing. Several of these could
be created (recommend one for every key) and listed as separate Query Subjects.
Each of the Query Subject contains all of the columns required for ad-hoc processing
and the only difference being the filter applied.
o Set up a new user connection to the database and put governance on the data
source that limits the returned result sets. Governors could also be placed on the
Cognos8 side to limit returned output.
o If summary data is required, create summarized material views to support common
access paths.
Does an FM model exist that contains the desired metadata?
What are the data sources? Are the data sources using conformed dimensions?
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 6
Recommended Practices:
The Design and Build Tasks outline the specific steps to develop a model; however, execution of these steps should be completed using the principals below:
Standardize User Interactions and Reuse Calculations
As early as possible in the process, standardize naming conventions, identify filters, and determine query items that will be commonly used as prompts. Calculations that are used
across multiple report deliverables should be modeled within Framework Manager.
Caution: Overuse of this technique could result in excessively long report specifications
and/or poor report performance.
Recommendation: Work closely with the appropriate DBA to ensure that the data sources
are optimized for reporting with necessary columns to be indexed.
Design Approach
Determine the modeling approach: relational versus dimensional.
Recommended: Sketch out the model on paper before entering the model into Framework
Manager and identify all ambiguous joins and data blind spots. (A blind spot is where a join
is ignored based on the selected query data.)
Reminder: This is not an entity relationship model (ERM): The goal is to provide effective queries, presented in a business-centric format.
Leverage Existing Models
A high return on investment can be gained for FM work if your firm capitalizes on previous modeling efforts. One of the best ways to accomplish this is by developing a Common
Dimension Model that incorporates calculations and filters that are common across the firm.
Developers should check with the Cognos Competency Center to validate whether or not the models already exist for their functional areas.
Enforce Source Control
When starting project make sure appropriate precautions have been taken to protect the work. All files located within the model’’ project folders should be source controlled and
checked in at least once a day.
Employ a Standard Naming Convention
Prior to creating a data source within Framework Manager, determine an appropriate naming convention. The model source should be named in a manner that clearly defines the
application and packages should be named with the application and business functional area to be deployed. Once project approval has been granted, the Cognos Center of Excellence
will provide a prefix that should be used for all model and package source.
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 7
Use an Appropriate Number of Models
As a rule, there will be one model created for a single application. However, the following
factors should be considered when determining whether to create a new model:
o The redundancy of information when compared to currently developed model.
o The number of tables and attributes.
o Consistency in business logic and presentation.
o Size of currently deployed models.
Data Source Connections
Once the appropriate name has been determined, submit a request to the Cognos Center of Excellence to create the data source connection within Cognos Connection. If the data source
information does not exist within Cognos Connection, the metadata will not be available to
the Framework Manager modeler.
Below are options to ensure that the data source is recognizable regardless of the deployed environment:
o Create a generic name such as DB2 that could be used regardless of the
environment. The client connection is configured to the appropriate environment
such as DB2Test, DB2QA, DB2Prod.
o Create a parameter map for the catalog and schema name. This macro will read
defaultName session parameter and will substitute the appropriate variables at run
time. Choosing this option gives you the greatest flexibility for deployment.
Syntax for the Data Source Catalog parameter map:
#$Datasource_Catalog {$account.defaultName}#
Syntax for the Data Source Schema parameter map:
#$Datasource_Schema {$account.defaultName}#
It is also possible to specify a connection string to an XML data source to include a
parameterized XML. However, using parameterized XML introduces the following restrictions:
o If the URL component is a prompt, it cannot contain other data.
o Data cannot be imported through Framework Manager
o Query validation does not work through Report Studio
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 8
Efficient Processing
Determine if all data processing can be accomplished at the database level. Abstraction of
data processing to the report application server level typically results in slower processing and higher resource consumption on the report server(s). A decision to perform local
processing should be based on the following:
o A clearly identifiable need for specific processing not available from the database
such as cross database joins and unsupported SQL99 functions.
o Anticipated data volume in the production environment.
Limited Function Set
Limit the function list in accordance with the native source.
Linking and Segmenting
Recommended: Linking and segmenting are recommended for large models. These techniques are also valuable when multiple FM developers are required.
o Links are created to help organize work across large projects, to maintain
consistency, and to reuse information.
For example, the project named Inventory contains the folder named Products. You can create a link from the Sales Products to Inventory Products. If any changes or
additions are made to the Inventory Products folder, you will see them in the Sales
Products folder.
A linked project is shared by other projects. It should not be created in a sub-directory within the master project directory.
o A segment is a project within a main project. A segment is owned by its main project.
A project segment is a complete project and changes to that project impact all
projects to which it is linked. If you want to open a segment as a separate project, it must be structured as a complete project. There must be a physical layer in each
segment that contains a subset of the data source query subjects on which they are
based. These data source query subjects provide access to the data and metadata and must be included in the appropriate segments.
Note: When changing the project structure, do not open the segments as individual
projects. Instead, check the main project and make changes from within it.
Tips:
Do not change the physical layer in a segment. Any change will be reflected in
the linked parent model and will impact all model segments that share data source query subjects. Changes may not be apparent outside the model in which
they are made until the model is closed and reopened.
Before a project is segmented, ensure that the folder and namespace are named
correctly. You cannot rename the folder or namespace after it has been
segmented.
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 9
Create Multiple Layers
At a minimum, a data layer and presentation layer should be created.
Recommended: If you are modeling multiple fact tables that are joined to common
dimensional information, create multiple namespaces under the presentation layer.
A typical look & feel of FM project explorer;
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 10
Layer 1 - Data:
The data layer will consist of the content of those tables required to process the work. If
effective data warehousing modeling techniques are employed, most of the data may be retrieved from the tables. However, if there is data that will NEVER be used, then include a
filter for the query subject. For the off-clarity, organize the tables by name (to include alias
tables).
1. Detecting Cardinality from the Data Source
When importing from a relational data source, cardinality is detected based on a set of
rules that you specify. The available options are: Use primary and foreign keys.
Use matching query item names that represent uniquely indexed columns. Use matching query item names.
The most common situation is to use primary and foreign keys as well as matching query items that represent uniquely indexed columns. The information is used to set some
properties of query items as well as to generate relationships.
To view the index and key information that was imported, right-click a query subject and
click Edit Definition.
2. Create relationships and ensure cardinality rules are appropriate. This is an extremely
important component of the modeling process. Framework Manager uses the cardinality
rules to assist the query engine in generating the appropriate SQL statements.
Different types of cardinalities are:
1. One-to-one (1..1) - One-to-one (1..1) relationships occur when one instance
of data in a query subject relates to exactly one instance of another. For
example, each student has one student number.
2. One-to-many (1..n) or zero-to-many (0..n) relationships occur when one
instance of data in a query subject relates to many instances of another. For
example, each teacher has many students.
3. Many-to-many (1..n) - Many-to-many (n..n) relationships occur when many
instances of data in a query subject relate to many instances of another. For
example, many students have many teachers.
Use 1..n or 0..1 for dimensionally aware queries
Identifies fact tables in star schemas
Identifies multi-fact queries and generated ‘stitched’ SQL statements.
Use 1..1 or 0..1 for default intersection queries (Non-stitched SQL).
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 11
Screenshot reference
3. Do not create fact-to-fact joins as these can create problems (such as query
performance due to data volume, query performance with outer joins, different grains of
detail). Fact tables should be separated by a dimension.
4. Validate that keys and other identifiers are correctly identified. This is important because
as automatic aggregation occurs for numeric data that may really be an attribute as
opposed to a fact.
Tip: Make sure the fact data that will be used in calculations is set to properly handle null values.
5. Resolve loop joins or ambiguous relationships. The most common type of ambiguous
relationships are where multiple valid relationships exist between two tables, reflexive
relationships (table joins to itself). This can be resolved by creation of alias tables;
however, it is not recommended to build deep hierarchies to resolve reflexive
relationships. This should be accomplished by flattening out the table.
6. Create star schema groupings when they are opportunities to build multiple SQL paths or
there is a need to create “factless” queries.
7. Set the SQL Generation type. (Minimized means that the generated SQL contains the
minimum number of tables and joins required to obtain values for the selected query
items.)
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 12
8. Validate the data being returned in the Query Subject.
Tip 1: Remember that calculated values are treated as facts. As such, these values may
also need to be tested outside of the Framework Manager Query Subject. Recommend
testing through one of the studios.
Tip 2: Frequently, development is performed against a database that represents a very
small percentage of the data that will later be queried. During the project, be sure to test
against data that is representative of production environment data.
Tip 3: Make sure that fact data that will be used in calculations is set to properly handle
null values.
9. If row-level data security has been applied using session variables, test by overriding the
session parameters.
10. Use parameterized SQL to provide dynamic filtering and grouping capability to the report
authors.
11. Add dimensional information to the Query Subject if multiple levels exist within a single
physical table. For instance, if a table has been de-normalized to contain both the
product and product type information (not just the key) data. Levels should be set to
ensure Framework Manager understands the appropriate grouping algorithms.
12. Use determinants to identify unique data within a de-normalized table.
Determinants reflect granularity by representing subsets or groups of data in a query
subject. They are used to ensure correct aggregation of this repeated data. Determinants are most closely related to the concept of keys and indexes in the data source; they are
imported based on unique key and index information in the data source.
An example of a unique determinant is Day in Time dimension. An example of a non
unique determinant is Month; the key in Month is repeated for the number of days in a particular month.
When you define a non-unique determinant, you should specify group by. The group by indicates that when the determinant’s keys or attributes are repeated in the data, Cognos
8 should apply aggregate functions and grouping to avoid double-counting.
If you have multiple determinants with Uniquely Identified check box selected, only the Group By setting of the first determinant is used. This is because the processing of
determinants stops at the first determinant with Uniquely Identified check box selected.
Below example illustrates some determinants and how you would define them as either
unique or non-unique with a group-by.
Year Key Month Key Month Name Day Key Day Name
2006 200601 January 06 20060101 Sunday January 1, 2006
2006 200601 January 06 20060102 Monday January 2, 2006
For this data set, you can define three determinants; two group-by determinants (Year
and Month) and one unique determinant (Day).
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 13
Determinants and Attributes
If a determinant defines attributes, all query items in the query subject must be
associated with the determinant and defined as either a key or an attribute. You can have a determinant with no attributes defined for it. Framework Manager uses this type
of determinant to indicate which query items are indexed.
In the snapshot below, you can see 3 determinants for year, month and day created for
the Time Dimension
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 14
When to Use Determinants
While determinants can be used to solve a variety of problems related to data
granularity, you should always use them in the following primary cases:
o A query subject that behaves as a dimension has multiple levels of granularity and
will be joined on different sets of keys to fact data. For example, Time has multiple
levels, and it is joined to Inventory Fact on the Month Key and to Sales Fact on the
Day Key.
o There is a need to count or perform other aggregate functions on a key or attribute
that is repeated. For example, Time has a Month Key and an attribute, Days in the
month that is repeated for each day. If you want to use Days in the month in a
report, you do not want the sum of Days in the month for each day in the month.
Instead, you want the unique value of Days in the month for the chosen Month Key.
In SQL, that is XMIN (Days in the month for Month Key).
13. Collapse hierarchical information into a single Query Subject.
14. Avoid incorporating query subjects that act as both facts and dimensions.
15. By default, Cognos 8 will aggregate all query items that are identified as a fact so ensure
that numeric items are properly identified.
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 15
Layer 2 - Presentation
The arrangement of the presentation layer will be largely driven by the required deliverables.
1. Create a layer for the Report Studio developers. In creating the developer view, include
the primary content from the data view. This provides the greatest flexibility for report
authors. They can look up reference information, validate their data, create calculations,
etc. Within this layer, create additional calculations, commonly used data filters, and (as
appropriate) additional query subjects that may be used for more complex authoring
needs.
Tip 1: Where a filter will be used exclusively against a query subject, test it through
Report/Query Studio to ensure that you are achieving the desired results.
Tip 2: If possible, use shortcuts.
2. Create a layer for Query Studio. This will be a simplified view intended for case-by-case
use by the end user. The layer intended for report authors will not be visible in Query
Studio. This layer must be organized and presented in a manner that is easily
understood.
Create a Package for Each Functional Area
Recommended: Create a separate package for each functional area. This ensures that the
package stays as trim as possible and limits the demand for regression testing. Coordinate the publication of all packages with the Cognos Center of Excellence to ensure appropriate
access has been granted.
Validate the Model
During the development of the Data Model, the author should be testing their query subjects to ensure that the appropriate data is being returned. Always validate the project before
publishing to the server. The tool requires that all errors be repaired prior to publication; however, warnings should be closely examined to ensure the future integrity of the model.
Tip: Development is frequently performed against a database that represents a very small percentage production data. At some point in time during the project, these tests should be
performed against data that is more representative of the production environment.
Model Modifications
Once the report authors begin development, modifications to the structure of the Framework Manager model may negatively impact the report authoring process.
Recommended: Through the Framework Manager model, find all report dependencies. Create a validation package using the Cognos Center of Excellence assigned naming
convention. The report authors will be notified that changes have been published to the
validation package and the dependent reports should be tested against the validation package. After the successful validation of the reports, the report authors will notify the
Framework Manager that the package can be republished.
Tip: Ensure that only one copy of the validation package is saved to Content Manager.
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 16
Synchronize the Project
Frequently, changes are made to the underlying data sources that need to be propagated to
the model. The preferred method of doing this is to synchronize the project. However, if the project is large and complex, this may be fairly time-consuming. You may choose to update
individual query subjects, by selecting the update option under the tools menu.
Task Automation
If there is a need to perform identical tasks against multiple models, perform repetitive tasks
or fully reconstitute a model, convert the log files to scripts to perform repetitive tasks or
fully reconstitute a new model, save the transaction logs as scripts and execute the script as appropriate.
Model Report
Create a model report to serve as modeling artifact.
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 17
Design Tasks:
1. Naming Conventions
a. Propose Data Source Naming Convention
b. Propose Model Naming Conventions
c. Propose Model Namespace and Folder Naming Conventions Folders are used to organize objects by subject or functional area. This makes it
easier for you to locate metadata, particularly in large projects.
Drawback: The main drawback of folders is that they require unique names for all query subjects, dimensions, and shortcuts. Therefore, they are not ideal for containing shared objects such as conformed dimensions.
d. Propose Query Subject/Query Item Naming Conventions
2. Data Preparation
a. Identify Source Data
b. Identify Method of Accessing Data (e.g., native, ODBC)
3. Model Design
a. Identify location to be used for repository control
b. Design Model Namespace/Folder Structure (e.g., star, transaction)
c. Design Model Join Strategies
d. Design Model Filters
e. Design Parameter Maps
f. Design Model Calculations
g. Develop Package Strategy
h. Identify Alias & Synonyms Issues
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 18
i. Identify attribute usage
Usage property identifies the intended use for the data represented by each query item. During importing, the Usage property is set according to the type of data that
the query items represent in the data source.
You need to verify that this property is set correctly. For example, if you import a numeric column that participates in a relationship, the Usage property is set to
identifier. You can change the property.
For relational query items, the value of the Usage property depends on the type of
database object the query item is based on.
Usage Property
Database Object Description
Identifier key, index, date, datetime
Represents a column that is used to group or summarize the data in a Fact column with which it has a relationship. Also represents an indexed column. Also represents a column that is of the date or time type.
Fact numeric, time interval
Represents a column that contains numeric data that
Attribute String Represents a column that is neither an identifier not fact such has descriptions
j. Identify formatting requirements
4. Validate, Create, and Review Prototype Model
5. Deliver Proposed Physical Cognos8 Data Flow Schematic
6. Deliver Proposed Model Table, Join, & Namespace/Folder Document
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 19
Build Tasks:
1. Create Physical View
a. Create Data source b. Add Tables
c. Setup Joins
d. Alias Tables (Synonyms) e. Modify attribute usage (i.e. differentiate attribute, facts and identifiers)
f. Modify formatting as needed ($, % …) g. Analyze Joins (minimize outer joins for performance and functionality)
h. Setup Namespaces, Folders & Query Subjects i. Identify Table Filters
j. Test Joins
2. Create Performance Views
a. Resolve ambiguous joins b. Set dimensional information
3. Create Business Views (Phase 1 – Simple) a. Organize Folders & Query Subjects (use Star Schema Groupings when possible)
b. Rename Columns with Business Names c. Validate Demo Phase 1 Package (Usability Test)
4. Create User Views (Phase 2 - Enhanced)
a. Re-Organize Folders & Query Subjects
b. Adjust Columns Based on Feedback c. Construct Model Calculations
d. Construct Conditions e. Validate Demo Phase 2 Package (Usability Test)
5. Create User Views (Phase 3 - Optimized) a. Finalize Folder, Query Subject, Calculation, Filter Changes
b. Test Joins With Representative Ad-Hoc Queries c. Validate Demo Phase 3 Package (Usability Test)
6. Apply Security to Model a. Create Object Security
b. Create Data Security
7. Cognos Connection a. Model Packaging & Publishing
b. Testing
i. Security ii. Package Functionality
c. Load Testing i. Database
ii. Server Performance
iii. Network Communications iv. Web
v. Determine Critical Mass
COGNOS P r a c t i c e
Learn. Adapt. Belive. Succeed with Proven Solutions
Page 20
Checklist:
Complete model design
Complete model testing (including functional and performance unit testing)
Publish model and apply security to package, objects, and data
Test Query Subjects changing sessions parameters
Perform a Checkpoint Review with appropriate Cognos Center of Excellence
representative
Create FM artifacts
Additional Reference Documents:
1. Framework Manager – User Guide (i.e. shipped along-with Cognos8 FM installation)
2. Framework Manager – Best Practices
For your quick reference, please find below embedded document
Framework_Manager_Best_Practices.pdf