Page 1 of 29
MDM Smart Search Configuration
DOCUMENT AUTHOR: Pawan Kumar DOCUMENT OWNER: Pawan Kumar
DATE CREATED: May 19, 2016 LAST UPDATED: June 6 , 2016
PROJECT: Producer/Broker Domain COMPANY: American Family Life Assurance Company
Page 2 of 29
Document Reviewer(s) Name and Title Signature Date Adam Bartolomeo, Informatica Solution Architect
Document Approver(s) Approver and Title Signature Date
Page 3 of 29
1. REFERENCE DOCUMENTS .......................................................................................................... 5
2. SMART SEARCH OVERVIEW ...................................................................................................... 5
3. WEBSPHERE PRE-CONFIGURATION ....................................................................................... 5
The following sections list WebSphere settings that must be set prior to beginning configuring Smart Search in MDM. .................................................................................................................................................................... 5
3.1 Registering Apache Solr MBeans ...................................................................................................................... 5
3.2 JVM generic arguments .................................................................................................................................... 6
4. CONFIGURING SMART SEARCH ................................................................................................ 6
4.1 Updating cmxcleanse.properties file ................................................................................................................ 7
4.2 Updating cmxserver.properties file .................................................................................................................. 8
4.3 Creating Zookeeper directory........................................................................................................................... 9
4.4 Configuring Process Server ............................................................................................................................. 10
4.5 Restart WebSphere..................................................................................................................................... 11
4.6 Checking solr admin console ....................................................................................................................... 11
5. RESETTING SMART SEARCH ................................................................................................... 12
5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table ...................................................................................... 13
5.2 Re-generating Business Entity Schema ........................................................................................................... 14
5.3 cmxcleanse.properties ................................................................................................................................... 17
5.4 cmxserver.properties ..................................................................................................................................... 18
5.5 Stopping WAS service .................................................................................................................................... 19
5.6 Cleaning solr/zkp folders in MDM .............................................................................................................. 20
5.7 Stating WAS ................................................................................................................................................... 22
5.8 Checking solr admin console. ......................................................................................................................... 22
5.9 Running Index Job in MDM Hub ..................................................................................................................... 23
6. TURNING OFF SMART SEARCH ............................................................................................... 26
Page 4 of 29
The following sections list Steps to turn off Smart Search if needed. ................................................................... 26
Page 5 of 29
1. REFERENCE DOCUMENTS
2. SMART SEARCH OVERVIEW Smart Search provides the ability to perform searching within a specific searchable business entity type or within all the searchable business entity types based on a string value. The Informatica Data Director application displays the matching searchable fields. The results are ranked based on how closely the results match the string value. Filters can be applied to identify more relevant results. Smart Search uses solr as its search platform, solr is based on Apache Lucene. In order to use smart search, configurations on the process server and running jobs for indexing the data must be done. The following sections outline these steps.
3. WEBSPHERE PRE-CONFIGURATION
The following sections list WebSphere settings that must be set prior to beginning configuring Smart Search in MDM.
3.1 Registering Apache Solr MBeans To support smart search in Informatica Data Director, register Apache Solr MBeans. In the WebSphere Console, check if following property is set as mentioned below in the custom properties of the Java Virtual Machine in WebSphere. javax.management.builder.initial=javax.management.MBeanServerBuilder
Document Description MDM_1010_ProvisioningToolGuide_en.pdf MDM Smart Search Guide
Page 6 of 29
3.2 JVM generic arguments The solr target directory needs to be added to the JVM generic arguments -Dsolr.solr.home=<INFAMDM root>\hub\cleanse\solr
For example: D:\infamdm\hub\cleanse\solr
The solr folder gets automatically created in the appropriate target when smart search is enabled.
4. CONFIGURING SMART SEARCH This section describes the steps to be followed for Smart Search Index configuration.
Page 7 of 29
4.1 Updating cmxcleanse.properties file This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #8.
1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources .
2. Using Notepad, open cmxcleanse.properties
3. Update the following existing Property values: • cmx.ss.enabled
Update the value from false to true • zookeeper.tickTime
Update the value from 2000 to 20000 • facet.infacount
Update the value from 1000 to 5000
4. Add the following new properties: • solr.allowAdminConsole=true • MAX_INITIAL_RESULT_SIZE_TO_CONSIDER =130
Page 8 of 29
5. Save the changes and close the file
4.2 Updating cmxserver.properties file This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 3, #2
1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources
2. Using Notepad, open cmxserver.properties.
3. Update the following existing Property values: • cmx.ss.enabled
Page 9 of 29
Update the value from false to true
4. Add the following new properties: • cmx.server.batch.smartsearch.initial.block_size =2000
4.3 Creating Zookeeper directory Create a folder for Zookeeper in Cleanse folder to store the in-memory snapshots of the data and the transactional log of updates to the database.
1. On the MDM application server, navigate to the cleanse folder: <MDM root>\hub\cleanse For example: D:\infamdm\hub\cleanse
2. Create new folder, zkp.
Page 10 of 29
Note: Please make sure that the name of the directory is not too long (greater than 7 letters) and doesn’t contain underscore/hyphens in its name. 4.4 Configuring Process Server This section follows the document “MDM_1010_ProvisioningToolGuide_en.pdf”, Chapter 8, Step 1, #4
1. In MDM Hub console acquire write lock. 2. Navigate to the Process Server workbench 3. Click on Edit for the Process Server 4. The following properties in Process Server must be enabled:
Enable Search Processing Enable ZooKeeper
5. When ZooKeeper is enabled, continue configuring the Process Server: ZooKeeper ID: 1 ZooKeeper Client Port: 2181 ZooKeeper Leader Port: 0
Page 11 of 29
ZooKeeper Follower Port : 0 ZooKeeper Directory : D:\infamdm\hub\cleanse\zkp ( created in step 4.3)
Note: Make sure that port assigned for ZooKeeper client port (2181) is open on server. This can be checked by netstat or telnet command by server admin.
4.5 Restart WebSphere Restart WebSphere to affect all changes for Smart Search.
4.6 Checking solr admin console To log into the solr admin console, go to URL : http://<MDMHOST:PORT>/MDESolr/#/~cloud http://dcamdm02:9080/MDESolr/#/~cloud
Navigate to cloud graph and you will see all subject area shard folder and its status. This takes some times to get created after WebSphere restart.
Page 12 of 29
Once the all shard folder are created, it can be validate at below location on server: D:\infamdm\hub\cleanse\solr
5. RESETTING SMART SEARCH It is best practice to disable smart search while running large data volumes in MDM, such as IDL.Also enabling Smart Search is inefficient prior to the completion of the IDL, as thousands of records will merge away and not need to be indexed for Smart Search
Page 13 of 29
While configuring Smart Search at Aflac, several issues were resolved by the IPS team and some required further assistance with Informatica Global Customer Services (GCS). A support ticket was opened, Informatica Support Case number 01915350 for Smart Search.
5.1 Cleaning C_REPOS_CO_CS_CONFIG Metadata table
1. Log into the CMX_ORS database 2. Check C_REPOS_CO_CS_CONFIG metadata table
3. If this table contains data, truncate it.
4. Confirm the table is empty.
Page 14 of 29
5.2 Re-generating Business Entity Schema In order to re-populate c_repos_co_cs_config table, we need to re-generate business entity schema from IDD config screen.
1. Login to IDD config screen , the URL will be something like http://dcamdm02:9080/config/config_login.jsf
2. Select the IDD application, Alfac IDD App,
Page 15 of 29
3. Open the Application State menu and select “Not Deployed”
4. Once selected, the application will get un-deployed and deployment status will say as “Not Deployed”.
5. Click “Generate Business entity Schema “
Page 16 of 29
6. After generateingBES. Select Close in the Informatica MDM Data Director pop-up window.
7. In the CMX_ORS database, confirm C_REPOS_CO_CS_CONFIG table has been repopulated
8. In the IDD Configuration URL, select Application State Full Deployment
Page 17 of 29
9. Once the Aflac IDD App is successfully deployed, the status will be “Full Deployment” and status will be valid (green check mark)
5.3 cmxcleanse.properties
1. Log into MDM application server 2. Go to <MDM INSTALL>/hub/cleanse/resources folder ((D:\infamdm\hub\cleanse\resources) 3. Using Notepad, open the cmxcleanse.properties file
Page 18 of 29
4. Check if smart search is enabled
5.4 cmxserver.properties
1. Log into the MDM application server 2. Go to <MDM INSTALL>/hub/server/resources folder (D:\infamdm\hub\server\resources) 3. Using Notepad, open cmxserver.properties
Page 19 of 29
4. Make sure that Smart Search properties are enabled (true):
5.5 Stopping WAS service
1. Log into the MDM application server 2. Open Control Panel > Services 3. Stop WAS service. DO NOT click Start, need to remove files from Smart Search folders
Page 20 of 29
5.6 Cleaning solr/zkp folders in MDM
1. On the MDM application server, go to D:/infamdm/hub/cleanse folder and take backup of the solr & zkp folders
2. Once the files are moved, , clean both of these folders (if nothing present in these folder , skip this process) Before cleaning the solrfolder:
Page 21 of 29
After cleaning all files are deleted from the solr folder:
Before cleaning the zkp folder:
Page 22 of 29
After cleaning zkp folder, all files are deleted:
5.7 Stating WAS 1. On the MDM application server, navigate to Control Panels > Services 2. Start WAS service
This would take some time to re-start WAS and MDM hub. 5.8 Checking solr admin console.
1. Log into the solr admin console, go to URL : <MDMHOST:PORT>/MDESolr/#/~cloud 2. Navigate to cloud graph and you will see all subject area folder status:
This shows that solr directory is connected to zookeeper and will get populated with shards. Once the all shard folder are created, it can be validate at below location on server: D:\infamdm\hub\cleanse\solr
Page 23 of 29
5.9 Running Index Job in MDM Hub
1. Log into MDM Hub Console 2. Go to Batch Viewer workbench
3. Connect to cmx_ors database
Page 24 of 29
4. Go to Batch Viewer Party Initially Index Smart Search data 5. Select Initial Index data for Search data Party Writing
6. To execute this job, select on “Initial Index data for Search data Party” and click on “Execute Batch”
Page 25 of 29
7. When executing, the status the current status will displays as “Processing”. Keep refreshing for current status by clicking on “Refresh Status”
8. When successfully finished, it will show below details:
9. Similarly run Index job for Party Writing number:
10. Go to Batch Viewer Party Writing Number Initially Index Smart Search data 11. Select Initial Index data for Search data Party Writing Number
12. To execute this job click on “Execute Batch”
Page 26 of 29
13. When executing, the status the current status will displays as “Processing”. Keep refreshing for current status by clicking on “Refresh Status”
14. When successfully finished, it will show below details:
15. This completes the Smart Search configuration. After enabling smart search, login to UI and test smart search.
6. TURNING OFF SMART SEARCH The following sections list Steps to turn off Smart Search if needed.
1. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\cleanse\resources
Page 27 of 29
2. Open this cmxcleanse.properties file and make sure that the below properties is set as:
cmx.ss.enabled=false
3. On the MDM Application server, navigate to the MDM Installation directory, D:\infamdm\hub\server\resources
Page 28 of 29
4. Open this cmxserver.properties file and make sure that the below properties is set as: cmx.ss.enabled=false
5. Disable ZooKeeper in MDM Hub :
Page 29 of 29
6. Restart WebSphere which will disable Smart Search.
Top Related