Leveraging and Integrating Existing Data in your CMDB ... · PDF fileLeveraging and...

$: Leveraging and Integrating Existing Data in your CMDB ... · PDF fileLeveraging and Integrating Existing Data in your CMDB ... \Program Files\Remedy\Apps\EIE\servername\rl_sql\bin$
Leveraging and Integrating Existing Data in your CMDB ARSII-304

Alan AbramsXinify Technologies

The Ingredients To Success

› Data Structure Knowledge› External Data from Discovery/Inventory› Enterprise Integration Engine

– Remedy Link for Oracle– Remedy Link for MS-SQL– Flat File– Custom

› Atrium CMDB› Reconciliation Rules› Workflow Filters – if Needed

External Data Source

›Database data›Flat file data›Source Relationships can be stored as

– foreign key– relationship tables

Constantly changing/updating data may lead to inconsistent results.Plan your data transfers during a time when source data is in a steady state.

Enterprise Integration Engine

› EIE 7.0– Flat File– Custom– read/write AR System forms– CMDB Aware

• Configuration Items• Relationships

› Remedy Link for Oracle– Contains EIE 7.0 + service to read/write Oracle tables

› Remedy Link for MS-SQL– Contains EIE 7.0 + service to read/write MS-SQL tables

Remedy Link for MS-SQL Method

SQL Server

3rd Party Data

MS SQLServer

Remedy Link for MS SQL

Adapter

AR System Server

EIE

EIE Mapping Rules

Data ExchangeRule Helper

•MS-SQL Adapter•Pre-built adapter to SQL Databases

•Rule Helper•Uses the adapter to create vendor table and field name menus

•Data Exchange Application (DEA)•AR System based application at the heart of EIE

•EIE Service•DEA passes the service instructions on how to interact with the adaptor

Atrium CMDB

› Configuration Management Database– Store multiple datasets– Reconcile datasets– Store Relationships– Base data for OOTB Applications

› Data Sets– Data value in a field... but so much more– Data Sets you should use

• BMC.ASSET – The “Gold Standard”• BMC.SANDBOX – Temp location for GUI changes to Assets• VENDOR.PURPOSE.PRODUCT – custom datasets

– Examples: BMC.IMPORT.MARIMBA, MS.IMPORT.SMS, ACME.IMPORT.LEGACY

Using Sandbox

AR System Server

CMDB Master

Data Set

› Submit New/Update to Sandbox Data Set

› Run Reconciliation Job on Save– Data is Merged if allowed– Sandbox Entry is deleted

CMDB SandboxData Set

› Enable Use Sandbox Data Set

CMDB Reconciliation

Engine

CMDB Reconciliation

Rules

› Create Rule/Job to Reconcile Sandbox and BMC.ASSET

Remedy Asset Mgmt

› Query Data From Master Data Set

Optional Workflow

AR System Server

CMDB 3rd PartyDataset

Data Cleansing Workflow

› Do manual compare of data to determine need› Normalize \ Standardize

– KB vs MB vs GB– Tangerines vs Oranges – close but no prize

› Categorize› ...Get the square peg in the round hole› Fire on Merge and maybe Modify

Overview

AR System Server

SQL Server Example

3rd Party Data

MS SQLServer

Remedy Link for MS SQL

Adaptor

EIECMDB 3rd PartyDataset

CMDB SandboxDataset

Remedy Asset Mgmt

CMDB Reconciliation

Engine

CMDB Reconciliation

Rules

EIE Mapping Rules

Data Cleansing Workflow

CMDB MasterDataset

EIE Structure Simple

Data Exchange

Data Mapping

1-N

Relationship Exchange

Relationship Mapping

1-N

On Complete, Trigger Next

EIE Structure Complex

Data MappingComputer

Data Exchange


Relationship MappingComputer – O/S



Data Exchange

Data MappingOperating System

Data Exchange

Data MappingDrives

Data Exchange

Data MappingMonitor

Data Exchange

Data MappingSoftware

Trigger Next


Relationship MappingComputer - Drives


Relationship MappingComputer - Monitor


Relationship MappingComputer - Software

Trigger Next Trigger Next Trigger Next Trigger Next

How To – Rule Helper

› Populates the “vendor” table and column menus› Command Line driven

– <eie_install_dir>\rl_sql\bin\rulehelper.exe– example – C:\Program Files\Remedy\Apps\EIE\servername\rl_sql\bin\rulehelper.exe

› Run before you start any mapping work

Must Use Administrator

Login

How To – Rule Helper

1. Load 2. Add Desired Tables 3.Create Menus

How To – Data Mapping

VEN-CompSys to BMC:Comp

› Data Mapping Name = name for new mapping› Namespace = BMC.CORE› Class = Destination for data, pick from the list.› Vendor Application = SqlServer, Flatfile or Oracle› Vendor Table/View = Source of data (pick from list)

– if not in the list, run rulehelper.exe to create the menu items.› Direction = Vendor Data into CMDB.› Click Create Data Mapping.

Data Exchange

Data Mapping Relationship Exchange



If the vendor table/view does not show in the drop down list, it is because you have not used rulehelper.exe to populate the tables that the menus query.

Data Exchange




Data Exchange



› Keys must be unique on both sides– If multiple keys, stored as AND to define Unique

› Use your DatasetID as well


All field mappings must include the following CMDB Data Fields –› Field used to identify Key › DatasetId› Name = anything that identifies the entry› AssetLifecycleStatus (3 is the enumeration for “deployed”)› Short Description › Submitter › ReconciliationIdentity

Data Exchange




› Only works if you can write to the Vendor / Source Database› Can be any value› Usually is transfer status

– 0 = transfer with no error– 1 = error

Data Exchange




› If you “Update Row to Indicate Delete”– Set AssetLifecycleStatus to 6 (End of Life)

› If you choose to “Delete” Row– No field to set. EIE will do the deletion

Data Exchange




Use to limit amount of data in a given query.Important if you are experiencing performanceor memory issues

Data Exchange




› Only need to use if updating based on some data criteria› In most cases this is blank

– See update features on Options tab Data Exchange




› Usually the only response is status function– if using “function|status{}” nothing needed here

› Only works if EIE can write to the vendor tableData Exchange




› Use if a value in the vendor field determines need to delete› Not used if Delete is determined by absence of data in vendor table

– See Delete tab for what to do in this caseData Exchange




1. If set to Yes, no records with source data errors are transferred.2. Is it OK to transfer blank spaces.. not the same as NULL3. Say “Yes” if importing to AR – Gets rid of trailing blank spaces4. Stores change history in a field you designate5. If Yes, updates are only made when a change is detected in the vendor data

› You must select a field that can store a negative integer to store the checksum6. Allows you to retrieve new or change data since last transfer

› You must select a date/time field to store last transfer timeData Exchange



123456

How To - Data Exchange

› Schedule Only– Use Schedule tab

› Event Driven Only– run eiexfer.exe from command line – use in a filter or active link

Data Exchange




› Select the mapping(s) to include in the Exchange container.› Multiple Mappings

– Cuts down on queries for source or target› Main Mapping

– Need to designate if multiple mappings included– Key values only come from Main Mapping

Data Exchange




› IntervalData Exchange



› Time› To Be Triggered


Data Exchange



Alternate Method


› While Testing– Limit records so you have less to undo

› In Production – Set Enable Debug to No

Data Exchange




› Use if the AR Server involved in the exchange is NOT the server that EIE DEA is installed on.

› Advantage – 1 license can service Dev, Test & Production Data Exchange



How To – Relationship Mapping

Data Exchange



Data Mapping

Data Mapping

Vendor Relationship Mapping

V_Computer

V_DiskDrive

Vendor Relationship Table

BMC_ComputerSystem

BMC_DiskDrive

BMC_HostedSystemComponent

•Map using Vendor Relationship Mapping.•Trigger on Complete of Data Exchange.

How To – Relationship Mapping

Data Mapping

Class Relationship Mapping

*In this case, the relationship dataexchange only runs after the data transfer for the two configuration item classes has completed.

•Map using Class Relationship Mapping.•Put in an Event Driven Only Exchange*.

V_Computer

V_DiskDrive

BMC_ComputerSystem

BMC_DiskDrive

BMC_HostedSystemComponent

Data Exchange



Data Mapping

How To - Relationship Mapping

› Vendor Data into CMDB is the only valid Direction› Must Select Vendor or Class on Create Data Exchange




› Define matching criteria that defines the relationship› Class 1 = Source› Class 2 = Destination

Data Exchange




› Always Two Keys– Source.InstanceId– Destination.InstanceID

Data Exchange




› These 6 attributes are always required› For DatasetID, select either class to create the

constantData Exchange




› Use if you are doing Class based mappingData Exchange




› This is a useless tab› Relationship is always the target of the

mapping› There is nothing to query

Data Exchange




› Used only if a Vendor based mappingData Exchange



The Result of a Relationship Mapping

How To - Relationship Exchange

› Works the same as Data Exchange› Choose Relationship Exchange = Yes

Data Exchange



General EIE Hints

› When testing an exchange –– Test one exchange at a time (when done chain them to be triggered)– Set to Interval – 20 minutes– Stop and start the EIE service to cause it to run immediately– Set Limit = 1 (so that it only runs once)– Be sure you know your debug path

› When writing a Vendor Query –– Test it with Direct SQL – If it doesn’t work there, it won’t work here– Use Indexed fields if possible

This won’t work – MacAdress != NULLThis will – MacAdress IS NOT NULL

General EIE Hints

› When setting an AR System field in a mapping –– Doesn’t work like a setfield or pushfield operation– Must Use Rules to convert functions, keywords, etc.

Example of an Integer FieldThis won’t work – 0This will – Constant|0

Example of ConcatenationThis won’t work – $Manufacturer0$+”-”+$MediaType0$This will – function|concat($Manufacturer0$,”-”,$MediaType0$)

› Change ALL EIE Query Menus to On Open, not On Connect

Atrium CMDB

› Reconciliation Rules– Identify

• Look for potential duplicates– Compare– Merge– Cleanup / Purge

• Can delete entire data set from all tables• Can delete selected items

– Configuration Item Set field MarkedForDelete = Yes– Create Reconciliation Rule to Purge if field is set

» Define how to handle related items– Run as a Reconciliation Job

CMDB 3rd PartyDataset

CMDB Reconciliation

Engine

CMDB Reconciliation

Rules

CMDB MasterDataset

Leveraging and Integrating Existing Data in your CMDB ... · PDF fileLeveraging and...

Documents

Transcript of Leveraging and Integrating Existing Data in your CMDB ... · PDF fileLeveraging and...