Implementing IBM DB2 Universal Database Enterprise Edition ...

Implementing IBM�� DB2�� Universal Database Enterprise Edition with Microsoft��

Cluster Server (TR-74.177)

June 15, 2001

Aslam Nomani

DB2 UDB System Verification Test

Lan Pham DB2 UDB Development

Implementing IBM DB2 Universal Database Enterprise Edtion with Microsoft Cluster Server (TR-74.177)

ii

This document contains proprietary information of IBM. It is provided under a license agreement and is protected by copyright law. The information contained in this publication does not include any product warranties, and any statements provided in this document should not be interpreted as such. © Copyright International Business Machines Corporation 2001. All rights reserved. U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

iii

Trademarks The following terms are trademarks or registered trademarks of International Business Machines Corporation in the United States, or other countries, or both: IBM DB2 DB2 Universal Database Netfinity Microsoft, Windows, and Windows NT are trademarks or registered trademarks of Microsoft, Inc. Other products or service names may be trademarks or registered trademarks of their respective owners.


iv

Contents Abstract ........................................................................................................................................ vi

About the Authors ....................................................................................................................... vii

ITIRC Keywords ........................................................................................................................ viii

Introduction ................................................................................................................................... 1

Conceptual Overview.................................................................................................................... 2

Failover and Failback .................................................................................................................... 5

DB2MSCS Utility ......................................................................................................................... 7

Hot Standby Configuration ........................................................................................................... 9

Mutual Takeover Configuration.................................................................................................. 15

Configuring the DB2 Administration Server for Hot Standby.................................................... 20

Remote Client Connections......................................................................................................... 26

User Scripts ................................................................................................................................. 28

Testing the Configuration............................................................................................................ 31

Rolling Upgrade .......................................................................................................................... 32

Repairing a Cluster after Catastrophic Machine Failure ............................................................. 33

Configuring Other DB2 Services for Hot Standby Configuration .............................................. 35

Using Generic Service Resource Type.................................................................................... 35

Using Post-Online Script to Start Other DB2 Services........................................................... 36

Managing Security....................................................................................................................... 38

DB2 Authentication................................................................................................................. 38

Appendix A – Limitations and Restrictions................................................................................ 39

Appendix B – Frequently Asked Questions ................................................................................ 40

Appendix C – Message Reference .............................................................................................. 42

Appendix D – DB2 UDB Software Levels ................................................................................ 51

v

Appendix E - Declustering an Instance ....................................................................................... 52

Appendix F - Manually Performing Steps Done by the DB2MSCS Utility ............................... 53


vi

Abstract IBM(R) DB2(R) Universal DatabaseTM (UDB) is the industry's first multimedia, Web-ready relational database management system, strong enough to meet the demands of large corporations and flexible enough to serve medium-sized and small e-businesses. DB2 Universal Database combines integrated power for business intelligence, content management, and e-business with industry-leading performance and reliability. This, coupled with MSCS (Microsoft Cluster Server), strengthens the solution by providing a highly available computing environment.

MSCS facilitates the automatic failover of applications and data from one system to another in the cluster after a hardware or software failure.

A complete high availability (HA) setup includes many parts, one of which is the MSCS software. A good HA solution includes planning, design, customization, and change control. An HA solution reduces the amount of time that an application is unavailable by removing single points of failure.

This document takes you through a sample configuration utilizing DB2 UDB Enterprise Edition (EE) V7.2. You can use similar steps to configure DB2 UDB Workgroup Edition and DB2 UDB Connect Enterprise Edition.

This paper is not intended to provide a detailed understanding of MSCS or DB2 UDB. We assume you are already familiar with both MSCS and DB2 UDB. It is our intent in this paper to provide an understanding of how DB2 UDB integrates into the MSCS environment and how to configure DB2 within that environment.

vii

About the Authors Aslam Nomani has been working with the IBM Database Technology team in the Toronto Laboratory for five years. For the past four years, he has worked in the DB2 UDB System Verification Test department. Aslam has worked extensively in testing DB2 UDB EE and DB2 UDB EEE in an MSCS environment. He is currently one of the team leads within the DB2 UDB test organization.

Lan Pham has been working with the IBM Database Technology team in the Toronto Laboratory for over nine years. In the past six years, he has worked in the DB2 for Windows Development team and was responsible for the porting of DB2 Extended Enterprise Edition to Windows, as well as implementing key Windows integration features such as Active Directory, Kerberos security, and MSCS integration. He is currently the architect for DB2 on Windows NT and Windows 2000 environment.


viii

ITIRC Keywords HA DB2 DB2 Universal Database Netfinity Availability Windows Windows NT Windows 2000 Microsoft Cluster Server

Introduction

1

Introduction DB2 is dependent on a core group of resources to ensure successful execution of database applications. If one of these resources were to become unavailable, DB2 would no longer be fully functional. Within a high availability (HA) environment, it is very important to understand the resources required and then to plan a strategy to ensure that these resources are continuously available to the application. Clustering software can be very beneficial in an HA environment as it provides a mechanism to ensure that all resources are continuously available to an application. The clustering software can also go one step further and ensure the application is continuously available.

Microsoft Cluster Server (MSCS) provides the ability to failover resources between multiple machines. These resources include such items as disks, IP addresses, files shares, and network names. DB2 has used MSCS’s ability to create additional resource types to develop a resource type called DB2. By grouping various resources together utilizing the MSCS group feature, a virtual machine is created that can float among all nodes in the cluster. Thus, if one machine in the cluster fails, the group of resources will failover and start on another machine.

Figure 1: DB2 Group Floating between Machines

Tip: Within an HA environment, it is important that the administrator try to alleviate any single points of failure because any system is only as reliable as its weakest link (software applications, disks, networks, processors, etc.).


2

Conceptual Overview Since an MSCS environment allows multiple machines to take ownership of the same disk resource, this disk must have shared direct access from all machines in the cluster. The cluster also maintains a heart beat between the nodes to determine which nodes in the cluster are available. The heart beat communication usually flows through an internal private network while remote clients access the cluster via a public network. Thus, a typical cluster topology may appear as follows:

Figure 2: Typical Cluster Configuration

Upon successfull installation of MSCS, Cluster Administrator will show the machines in the cluster, the resources, the resource types, the groups, and the networks available to the cluster. Cluster Administrator is a graphical administration tool that is part of MSCS.

Conceptual Overview 3

Figure 3: Cluster Administrator after Initial Install of MSCS

The screenshot above is an initial snapshot of the cluster immediately after MSCS has been installed. This cluster will be used as the starting point for examples used throughout this paper. Here are some of the notable items about the cluster:

- The name of the cluster is MYCLUSTER. - The two machines in the cluster are WA26 and WA27. - A Cluster Group and five other groups labelled Disk Group 1 through 5 exist. - Each Disk Group contains one Physical Disk resource. - Two networks exist; they are labelled “Private Network” and “Public Network.” - Currently the Cluster Group is on machine WA26 and all the Disk Groups are on WA27.

Tip: Ensure all groups within the MSCS configuration can failover successfully between all machines in the cluster before proceeding with configuring DB2 within the MSCS environment.

Within the default available resource types available to the cluster, their is no resource type that corresponds to a DB2 instance. DB2 UDB creates a resource type called DB2, and each resource of type DB2 corresponds to a DB2 single partition instance. Since DB2 now integrates into the MSCS environment with a resource type that can be monitored by MSCS, MSCS can now ensure that DB2 stays online, along with all the resources required by DB2.


4

The DB2 resource type is automatically created when the DB2MSCS utility is executed. The DB2MSCS utility will be discussed in a later section.

Since instances and all databases within an instance require disks to store information as well as an IP address to allow for remote connections, DB2 utilizes the group feature within MSCS to group multiple resources into a single logical entity. Thus, the combination of a DB2 resource, disks, and a TCP/IP address represent almost all the resources required to succesfully run a DB2 instance. The other resources that are required are processors and memory. These last two resources are obtained from the machine on which the group is currently active and do not failover between machines.

As already noted, the group that represents the instance contains a DB2 resource, disks, and an IP address. However, the order that these resources come online is critical when the group is brought online. If the DB2 resource starts first, it may encounter failures because the instance and the database may require access to files on a disk that is not online yet. Thus, the dependency feature within MSCS is utilized. The dependency feature allows the ability to define which resources must be completely started before attempting to start another resource. The DB2 resource is automatically configured to be dependent on the disks as well as the IP address. This dependency also applies to the stopping process. MSCS will now ensure the DB2 resource is completely stopped before any attempt to bring the disks and IP address offline.

Failover and Failback 5

Failover and Failback MSCS is responsible for deciding whether resources and groups are restarted on the current machine or whether they should failover to another machine in the cluster. It is very important that the cluster is aware of which resources have been started so it knows which resources it must try to keep online. If you bring resources or groups online via Cluster Administrator, then MSCS is aware that these resources have been started and will attempt to ensure they stay available in the case of a failure. If DB2 is started using a non-cluster interface (DB2START, NET START, or an automatic start from Service Manager), then MSCS is not aware that the DB2 instance has been started and will make no attempt to keep the DB2 instance up and running.

MSCS will monitor all resources and groups that are brought online via Cluster Administrator. If a machine in the cluster fails, MSCS will move all resources and groups to another cluster machine and ensure that any resources that were online will be brought back online. When a resource fails, MSCS will attempt to bring that resource back online on the current machine first and if this continues to fail, it will move the whole group associated with the resource to another cluster machine and try to bring it online. The number of times MSCS will retry bringing the resource online is configurable within Cluster Administrator. So is the machine preference in regards to where the group will failover if their are more than two machines in the cluster. Failures of the DB2 resource could occur because of exceptions within DB2 or because an operating system resource has run low. Since failure detection of DB2 is triggered by termination of the DB2 process, a hang situation will not automatically trigger a restart of the DB2 resource.

When a failover occurs due to a machine failure or other cause, databases on the failing node may be in a transactionally inconsistent state. When the database starts up on the surviving machine, it must go through a crash recovery phase to bring the database back to a consistent state. To maintain data integrity and transactional consistency, the database will not be completely available until crash recovery has completed.

In a mutual takeover environment, it is very important to plan for the highest potential machine requirements if all MSCS groups are online on a single machine at any given time. If the machine is not capable of handling the workload, the result may range from performance degradatations to further abnormal terminations. When a failover occurs, it is possible to reduce the resource utilization of the failing database. The section “User Scripts” will provide an example of reducing configuration parameters.

Failback is the ability for an MSCS group to move back to its preferred machine once that machine is back online within the cluster. The term fallback is also commonly used when referring to failback. The failback involves taking the group offline on its current machine, moving the group over to its preferred machine, and then finally bringing the group online on the preferred machine. One of the disadvantages of automatic failback is that every time the group containing the DB2 instance is brought offline, all database connections will be forced off the database. MSCS drives the failback based on configurations within Cluster Administrator, with the default behavior being not to failback.


6

Since taking the group with the DB2 instance offline forces off all database connections, DB2 by default prohibits this behavior to avoid the unintentional forcing of users. Thus, any attempt to take a DB2 group offline will fail if there are database connections or if the database has been activated. However, by enabling the DB2 profile variable DB2_FALLBACK, the behavior of DB2 will be modifed so that when the group is brought offline, all database connections will be automatically forced off, and the database will be deactivated. Thus, DB2_FALLBACK should be utilized if there is a need to take the DB2 group offline while the database is in use.

DB2MSCS Utility 7

DB2MSCS Utility The DB2MSCS utility is a standalone command line utility used to transform a non-HA instance into an HA instance. The utility will create all MSCS groups, resources, and resource dependencies. It will also copy all DB2 information stored in the Windows registry to the cluster portion of the registry as well as moving the instance directory to a shared cluster disk. The DB2MSCS utility takes as input a configuration file provided by the user specifying how the cluster should be set up. The fields within the configuration file that are needed for DB2 EE are as follows:

DB2_INSTANCE The name of the DB2 instance. If the instance name is not specified, the default instance (the value specified by the DB2INSTANCE environment variable) is used. This parameter has a global scope and should be specified only once in the DB2MSCS.CFG file. CLUSTER_NAME The name of the MSCS cluster. All the resources specified following this line are created in this cluster until another CLUSTER_NAME parameter is specified. GROUP_NAME The name of the MSCS group. If this parameter is specified, a new MSCS group is created if it does not exist. If the group already exists, it is used as the target group. Any MSCS resource specified after this parameter is created in this group or moved into this group until another GROUP_NAME parameter is specified. Specify this parameter once for each group. IP_NAME The name of the IP Address resource. The value for the IP_NAME is arbitrary, but it must be unique in the cluster. When this parameter is specified, an MSCS resource of type IP Address is created. This parameter is required for remote TCP/IP connections. This parameter is optional in a single partition environment. A recommended name is the hostname that corresponds to the IP address. IP_ADDRESS The TCP/IP address for the IP resource specified by the preceding IP_NAME parameter. This parameter is required if the IP_NAME parameter is specified. This is a new IP address that is not used by any machine in the network. IP_SUBNET The TCP/IP subnet mask for the IP resource specified by the preceding IP_NAME parameter. This parameter is required if the IP_NAME parameter is specified. IP_NETWORK The name of the MSCS network to which the preceding IP Address resource belongs. This parameter is optional. If it is not specified, the first MSCS network detected by the system is used. The name of the MSCS network must be entered exactly as seen under the Networks branch in Cluster Administrator. Note: The previous four IP keywords are used to create an IP Address resource.


8

DISK_NAME The name of the physical disk resource to be moved to the current group. Specify as many disk resources as you need. The disk resources must already exist. When the DB2MSCS utility configures the DB2 instance for failover support, the instance directory is copied to the first MSCS disk in the group. To specify a different MSCS disk for the instance directory, use the INSTPROF_DISK parameter. The disk name used should be entered exactly as seen in Cluster Administrator. INSTPROF_DISK An optional parameter to specify an MSCS disk to contain the DB2 instance directory. If this parameter is not specified the DB2MSCS utility uses the first disk that belongs to the same group INSTPROF_PATH An optional parameter to specify the exact path where the instance directory will be copied. This parameter MUST be specified when using IPSHAdisks, a ServerRAID Netfinity disk resource (i.e. INSTPROF_PATH=p:\db2profs). INSTPROF_PATH will take precedence over INSTPROF_DISK if both are specified. Additional fields exist for the DB2MSCS input file; however, only the fields listed above are required for a DB2 EE instance. Example configuration files will be shown in subsequent sections of this paper. Tip: Ensure the IP address used for IP_ADDRESS is a new IP address that does not already belong to any machine on the network. Also ensure all values used for DISK_NAME and IP_NETWORK are entered exactly as seen in Cluster Administrator.

Hot Standby Configuration 9

Hot Standby Configuration In a hot standby configuration, at least one machine in the MSCS cluster is idle and dedicated as a backup in the event of a failure. The standby machine can act as the backup for one or more instances, depending on the configuration. Figure 4: Hot Standby Configuration The following example will detail the steps required to set up a hot standby configuration using a single DB2 instance. The DB2MSCS utility should be executed from a domain user that has local Administrator authority on each machine in the cluster. After DB2 UDB has been installed on each machine in the cluster, each machine must be rebooted before proceeding with the DB2MSCS configuration. 1. Configure the MSCS cluster and ensure it is healthy. 2. Install DB2 UDB on every machine in the cluster. The installation of DB2 must be on a

local drive. (Steps 1 and 2 can be done in reverse order if necessary). The installation will create an instance called DB2 on each machine. The DB2 instance of the machine where the DB2MSCS utility is executed will be the instance that is transformed to an HA instance. The DB2 instances on the remaining cluster machines will now point to the newly transformed HA instance.


10

Since the sqllib directory exists locally on each machine, ensure any programs such as stored procedures or scripts exist on each cluster machine in the appropriate path. For all programs where a path name can be specified, place the program on the shared cluster disk so only one copy of the program is required.

3. From the Windows Services dialog box, ensure the DB2 service representing the instance is configured to start manually. The name of the service representing the instance has the form “DB2 - <instance name>”. Alternatively, you can create a new instance for use with the cluster. You only need to create this instance on the machine where the DB2MSCS utility is being executed. If you are using multiple instances, take care that the current session points to the correct instance by setting the DB2INSTANCE environment variable.

4. Ensure the instance is stopped using the DB2STOP command. Instance DB2 will be used to configure the HA instance. An input configuration file must be configured for use with the DB2MSCS utility. Before you transform the instance to become an HA instance, note that the instance directory will be currently stored on the local drive where DB2 was installed: c:>db2set db2instprof c:\SQLLIB The instance directory is currently located under c:\sqllib\db2 [c:\sqllib\<instance_name>]. The input configuration file for the DB2MSCS utility will appear in the following form: DB2_INSTANCE=DB2 CLUSTER_NAME=MYCLUSTER GROUP_NAME=DB2 Group IP_NAME=mscs5 IP_ADDRESS=9.21.85.54 IP_SUBNET=255.255.255.0 IP_NETWORK=Public Network DISK_NAME=Disk O: DISK_NAME=Disk P: INSTPROF_DISK=Disk P: To run the DB2MSCS utility, specify the -f option for the file name and ensure the command returns successfully: C:\>db2mscs –f:db2mscs.cfg.db2 DB21500I The DB2MSCS command completed successfully. Tip: Back up all existing databases within an instance before transforming it to an HA clustered instance.


The execution of the DB2MSCS utility transforms the instance DB2 to a clustered instance that resides within MYCLUSTER. A group is created called “DB2 Group”. Within DB2 Group a new IP address is created called “mscs5” with an address of 9.21.85.54, Disk O and Disk P are placed in DB2 Group, and the instance directory is moved to Disk P. The db2ilist command also now indicates that the instance is clustered by showing C:<cluster name> after the instance name. c:>db2set db2instprof P:\DB2PROFS c:\>db2ilist DB2 C : MYCLUSTER

Figure 5: Cluster Adminstrator with Instance DB2 in DB2 Group The new screen shot of Cluster Admnistrator shows the following: - A group called “DB2 Group” is created. - Disk O and Disk P have been moved into DB2 Group. - An IP address called “mscsc5” has been created and resides in DB2 Group. - A resource called “DB2” of type DB2 has been created and resides in DB2 Group. - DB2 Group currently resides on machine WA26.

5. The next step is to determine which machine is intended as the primary machine and which machine is intended as the standby machine. In our case we will pick WA26 as


12

the primary machine and WA27 as the secondary machine. The properties for DB2 Group initially is configured with no preferred owners. Modify this configuration if you want to specify a preferred owner.

Figure 6: Cluster Administrator Properties Box for DB2 Group

6. After the preferred owner is specificed, the user must determine if they want DB2 Group to automatically failback to machine WA26 after it recovers from a failure. If automatic failback is not desired, then no further actions are required because the default behavior is to prevent failback. If automatic failback is desired, then the properties box for DB2 Group must be modified. On the Failback tab, select Allow failback. If you want to enable failback when WA26 is back online, set Allow failback to Immediately. If you only want to enable failback for a particular period of the day, for example between 1 A.M. and 8 A.M., configure that in the Failback between options. If WA26 comes back online during this time window, then the failback will occur. If machine WA26 comes back online at 12:59 A.M., the group will not automatically failback at 1 A.M.


Figure 7: Failback Settings for DB2 Group For failback to automatically occur, we must be able to bring DB2 Group offline on the machine it is currently running. If the database has been activated or has any connections, taking the DB2 resource offline will fail. Therefore, you must set DB2_FALLBACK to ensure DB2 automatically forces all applications and deactivates the database if necessary. c:\>db2set DB2_FALLBACK=ON

7. Now that the instance DB2 has been configured, all databases within the instance should be placed on the the disk drives that exist within the same group as instance DB2. For instance DB2, all data should reside on Disk O and Disk P. If existing databases are being used and they currently do not reside on Disk O or Disk P, use a redirected restore to move the data to the shared drives. For information on how to perform the redirected restore, refer to the DB2 Universal Database documentation. If you create a new database, make sure you create the database and its tablespaces on either Disk O or Disk P. If the DB2 instance fails over to another machine and does not have access to all of its data files, expect database errors.


14

c:\>db2 create db samplea on p: DB20000I The CREATE DATABASE command completed successfully. c:\>db2 create tablespace tsa managed by database using(file 'p:\container1' 50000, file 'o:\container2' 50000) DB20000I The SQL command completed successfully.

Mutual Takeover Configuration 15

Mutual Takeover Configuration A mutual takeover configuration has a DB2 instance running on each machine in the cluster. If a machine fails in the cluster, the result will be multiple DB2 instances running on the same machine. Careful planning must be taken to ensure the surviving machine in the cluster is capable of handling the workload that will be placed on it. WA26 WA27

Figure 8: Mutual Takeover Configuration Configuring a mutual takeover environment is very similar to configuring a hot standby environment, except now a second instance will be configured. The following example will detail the steps required to set up a mutual takeover configuration. 1. Perform all the steps in the prior section for configuring a hot standby configuration.

This will create an instance DB2 with its preferred owner being machine WA26 2. Create the instance DB3 on machine WA27.

c:\>db2icrt DB3 c:\>db2ilist DB3 DB2 C : MYCLUSTER


16

The output from db2ilist shows that there are two instances, DB3 and DB2. The information following the output for DB2 states that it is a clustered instance and it is in the cluster called MYCLUSTER. Currently, DB3 does not exist on machine WA26.

3. Create the input file for the DB2MSCS utility so it can transform DB3 to a clustered instance. The resources used for this group will have to be different than the resources used for instance DB2, because instance DB2 and instance DB3 will not necessarily reside on the same machine. The input configuration file for the DB2MSCS utility will look like the following example: DB2_INSTANCE=DB3 CLUSTER_NAME=MYCLUSTER GROUP_NAME=DB3 Group IP_NAME=mscs6 IP_ADDRESS=9.21.85.57 IP_SUBNET=255.255.255.0 IP_NETWORK=Public Network DISK_NAME=Disk Q: DISK_NAME=Disk R: To run the DB2MSCS utility, specify the -f option for the input file name and ensure the command executes successfully. Execute the DB2MSCS utility on machine WA27, because that is where the instance DB3 currently exists. c:\>db2mscs –f:db2mscs.cfg.db3 DB21500I The DB2MSCS command completed successfully. The configuration of instance DB3 creates a group called “DB3 Group” with a different IP address than was created for instance DB2 and with Disk Q and Disk R that are also not being utilized by instance DB2. The field INSTPROF_DISK was not used for configuring DB3 like it was used for instance DB2. Thus for DB3, INSTPROF_DISK will default to the first disk identified, which will be Disk Q. The output of db2ilist will also show that both instances are now clustered c:\>set db2instance=DB3 c:\>db2set db2instprof Q:\DB2PROFS c:\>db2ilist DB3 C : MYCLUSTER DB2 C : MYCLUSTER


Figure 9: Cluster Administrator with Instance DB3 in DB3 Group

The new screen shot of Cluster Administrator shows the following additions to the initial hot standby setup: - A group called “DB3 Group” is created. - Disk Q and Disk R have been moved into DB3 Group. - An IP address called “mscs6” has been created and resides in DB3 Group. - A resource called “DB3” of type DB2 has been created and resides in DB3 Group. - DB3 Group currently resides on machine WA27.

4. Since this configuration is a mutual takeover configuration and instance DB2 is

configured to use WA26 as its primary machine, instance DB3 should be configured to use WA27 as its primary machine. The properties for DB3 Group should be modified to represent WA27 as the primary machine and WA26 as the secondary machine.


18

Figure 10: Properties Box for DB3 Group 5. After the preferred owner is configured for DB3 Group, the user must determine if

automatic failback is desired. If automatic failback is desired, the failback options must be set under the Failback tab for DB3 Group. DB2_FALLBACK should also be set to ON to ensure the group can be successfully taken offline. c:\>set db2instance=DB3 c:\>db2set DB2_FALLBACK=ON

6. The final step ensures that all databases and corresponding tablespaces associated with DB3 reside on either Disk Q or Disk R. If an old database is being used and does not already exist on Disk Q or Disk R, a redirected restore will be needed. Refer to the DB2 UDB documentation for details regarding the redirected restore. If a new database is being created, ensure the database and its tablespaces are created on either Disk Q or Disk R. c:\>db2 create db sampleb on q: DB20000I The CREATE DATABASE command completed successfully.


c:\>db2 create tablespace tsb managed by database using(file ‘q:\container1' 50000, file 'r:\container2' 50000) DB20000I The SQL command completed successfully.

The previous command was executed on WA27 and DB2INSTANCE was set to point to DB3. DB3 does not currently reside on WA26, so any attempts to execute DB2 commands against DB3 on WA26 will fail.


20

Configuring the DB2 Administration Server for Hot Standby The DB2 Administration Server is used by the DB2 Control Center to administer DB2 instances and databases. The Administration Server is a single-partition instance and similar in nature to any other DB2 EE instance. Thus, the Administration Server will face the same availability issues as a regular instance. If high availability of the Administration Server is desired, this instance must be transformed to become a clustered instance. The steps to transform the Administration Server instance are identical to the steps used to transform any other regular instance. In the example below, you will use the Administration Server to administer instance DB2. The example below will also reuse the shared disk and IP address that was configured for instance DB2, resulting in instance DB2 and the Administration Server sharing these resources. Since these two instances are sharing the same resources and the Administration Server is used for administering instance DB2, it is mandatory to place the Administration Server in the same group as instance DB2. Below are the instructions for configuring the DB2 Administration Server for instance DB2: 1. Stop the Administration Server on all machines.

C:\>db2admin stop SQL4407W The DB2 Administration Server was stopped successfully. If an Administration Server does not exist, use “db2admin create” to create this instance.

2. Drop the Administration Server on all cluster nodes except the first node by performing the following command on WA27. C:\>db2admin drop SQL4402W The DB2ADMIN command was successful.

3. On the first node (WA26) where the Administration Server resides, go into the Windows Services dialog box and modify the Administration Server instance so it is set to start manually. The name of the Administration server is DB2DAS00, so it will show up as DB2-DB2DAS00 in the Services dialog box.

4. Create a configuration input file to be used with the DB2MSCS utility to transform the Administartion Server to a clustered instance. DB2_INSTANCE=DB2DAS00 CLUSTER_NAME=MYCLUSTER DB2_LOGON_USERNAME=db2admin DB2_LOGON_PASSWORD=db2admin GROUP_NAME=DB2 Group DISK_NAME=Disk O:

Configuring the DB2 Administration Server for Hot Standby 21

INSTPROF_DISK=Disk O: Note that the group name “DB2 Group” is the same group that you used for configuring instance DB2; thus, all resources will be created in the same group as instance DB2. Also, you are reusing Disk O and not configuring an IP address, because the IP address already configured for instance DB2 can be reused. Even though Disk O was already configured for DB2 Group, it must be specified again, because that is where all instance information associated with the Administration Server will be placed.

5. Execute the DB2MSCS utility from the first node. c:\>db2mscs –f:db2mscs.cfg.admin DB21500I The DB2MSCS command completed successfully.

6. Just like a regular instance, the DB2DAS00 instance cannot be taken offline while connections to the instance exist. If you want to be able to bring the Administration Server offline while there are still active connections, set DB2_FALLBACK to ON: set DB2INSTANCE=DB2DAS00 db2set DB2_FALLBACK=ON

7. On all clients used for DB2 administration, remove any references to the Administration Server using the DB2 Control Center. Then, use the DB2 Control Center to recatalog a reference to the Administration Server utilizing the highly available cluster IP address. To recatalog a reference to the Administration Server from the Control Center, right click on Systems and select Add: - System name – MSCS5 (an arbitrary name to identify the cluster system) - Remote instance – DB2DAS00 (the name of the Administration Server) - Operating system – Windows NT - Protocol – TCP/IP - Host name – (The highly available IP address used by the Administration Server) - Service name – 523 (the port automatically configued for the Administration Server)


22

Figure 11: DB2 Control Center Adding a System The reference to the Administration Server has been created. Now the instance to be managed should be added. From the Control Center, right click on the new reference to MSCS5 and select Add: - Remote instance – DB2 ( the name of the instance to be administered) - Protocol – TCP/IP - Host name – (The highly available IP address configured for the instance to be administered) - Service name – (The value specified for the database manager configuration paramater SVCENAME for the remote instance to be managed. Further discussion of SVCENAME exists in the Remote Client Connections section.)


Figure 12: DB2 Control Center Adding an Instance After the instance DB2 has been added, right click on Databases under the DB2 instance and select Add. Specify the database name, samplea, that was created under the DB2 instance and click OK. The samplea database can now be remotely administered regardless of whether the instance resides on machine WA26 or WA27.


24

Figure 13: DB2 Control Center Adding a Database

The Administration Server is now integrated within the cluster. Care should be taken to ensure any remote client connections go through the highly available IP address and any scripts or data that are needed by the Administration Server are placed on the highly available disks associated with the DB2DAS00 instance. Cluster Administrator now shows instance DB2 and instance DB3 configured in the cluster with their own groups, and the group containing instance DB2 also contains the Administration Server.


Figure 14: Cluster Adminstrator with DB2DAS00 Added to DB2 Group Note: In a mutual takeover configuration, the Administration Server cannot be clustered for more than one instance, because only one Administration Server can run on a single machine at one time.


26

Remote Client Connections MSCS provides the ability to create a highly available TCP/IP resource. Remote clients should utilize the highly availabe IP address when connecting to the server. When cataloging a TCP/IP connection from the client to the DB2 server, you must use the IP address configured for the cluster that is located in the same group as the instance. Thus, when configuring a client connection to a database within instance DB2, you must use the IP address associated with mscs5. Similarly, any connections cataloged to a database within instance DB3 must use the IP address associated with mscs6. If an IP address is used that is associated with either a physical machine or an IP address in a different group than the instance, there is no guarantee the IP address will point to the machine where the database is actually residing. The other factor that needs to be taken into consideration is that the client will connect to the server using a specified port. That identical port must be available to the instance on all machines within the cluster, and should be defined in the services file on each machine in the cluster. The following is an example of how to catalog connections from a remote client to databases samplea and sampleb, which were created in prior examples. Assume instance DB2 is currently on WA26 and instance DB3 is currently on WA27. 1. Add the following entries to the services file for both WA26 and WA27 to reserve

unused ports for both DB2 and DB3. db2cDB2 50000/tcp #connection port for the DB2 instance DB2 db2iDB2 50001/tcp #interrupt port for the DB2 instance DB2 db2cDB3 50002/tcp #connection port for the DB2 instance DB3 db2iDB3 50003/tcp #interrupt port for the DB2 instance DB3

2. Ensure DB2COMM is set to TCPIP for each instance: set DB2INSTANCE=DB2 db2set DB2COMM=TCPIP set DB2INSTANCE=DB3 db2set DB2COMM=TCPIP

3. Update the database manager configuration for each instance so the instance knows which port to use: On WA26 for instance DB2, c:\>db2 update dbm cfg using svcename db2cdb2 On WA27 for instance DB3,

Remote Client Connections 27

c:\>db2 update dbm cfg using svcename db2cdb3

4. For the changes in steps 2 and 3 to take effect, the instance must be brought offline and then back online. From Cluster Administrator, right click on the the resource DB2 and select Take Offline. After the resource is in an offline state, right click on the resource and select Bring Online. Perform the same actions for DB3.

5. Now that both instances are ready to receive incoming remote requests, the databases on these instances must be cataloged from a remote machine. The services file on the remote client must be updated with the same entries made on the server machines. To catalog database SAMPLEA that resides in instance DB2: c:\>db2 catalog tcpip node nodea remote mscs5 server db2cdb2 c:\>db2 catalog db samplea at node nodea c:\>db2 connect to samplea user db2adm using xxxxx To catalog database SAMPLEB that resides in instance DB3: c:\>db2 catalog tcpip node nodeb remote mscs6 server db2cdb3 c:\>db2 catalog db sampleb at node nodeb c:\>db2 connect to sampleb user db2adm using xxxxx For cataloging the tcpip node, mscs5 can be replaced with the IP address corresponding to mscs5 in the cluster, if this does not already exist on the domain name server. Similar substitutions are also valid for cataloging the node corresponding to instance DB3.


28

User Scripts DB2 provides the ability to execute a batch script before and after each DB2 resource is brought online. This batch script resides in the instance directory of each instance, so each instance will have a separate copy of these script files. To determine the instance directory, issue the db2set command shown below and then append the instance name to the results. c:>set DB2INSTANCE=DB2 c:\>db2set db2instprof P:\DB2PROFS In this particular case, the instance directory for instance DB2 will be located under p:\db2profs\db2. The scripts that execute before and after the DB2 resource is brought online are called db2cpre.bat and db2cpost.bat, respectively. These scripts are also referred to as the pre-online and post-online scripts, respectively. These batch files are optional; they do not exist by default and will only get executed if they exist. These batch files are launched by the MSCS Cluster Service and are run in the background. The script files must redirect standard output to record any output as a result of commands run from within the script file. The pre-online script will execute and complete before any attempt to bring the DB2 resource online is made. Thus, it is important that the commands in the pre-online script execute in a reasonable amount of time so MSCS will not timeout on its attempt to bring the DB2 resource online. The example below will reduce configuration parameters for the instance DB2 and the database samplea if the instance is not on its primary machine. From earlier configurations, WA26 was set as the primary machine and WA27 as the secondary machine. To ensure these changes take effect, use the actual database name for doing administration work and assume all remote clients are connecting to the database with an alias . On the server, create an alias called aliasa that corresponds to the database samplea that resides within instance DB2. [p:\db2profs\db2\db2cpre.bat]

@echo off if "%COMPUTERNAME%" == "WA26" goto WA26 if "%COMPUTERNAME%" == "WA27" goto WA27 echo Error ! No script for this computer >> p:\db2profs\db2\pre.out goto end :WA26 db2cmd -c db2 -z p:\db2profs\db2\pre.out -vf p:\db2profs\db2\prewa26.clp goto end :WA27 db2cmd -c db2 -z p:\db2profs\db2\pre.out -vf p:\db2profs\db2\prewa27.clp goto end

User Scripts 29

:end The db2cpre.bat will call prewa26.clp if the the instance DB2 is starting on machine WA26 or it will call prewa27.clp if the instance is starting on machine WA27. The reason for using two different script files is because the desired behavior is different depending on which machine the instance is starting on. If the instance DB2 is starting on its primary machine (WA26), it will utilize more machine resources than if it starts on the secondary machine (WA27). [p:\db2profs\db2\prewa26.clp]

uncatalog db aliasa update dbm cfg using INTRA_PARALLEL YES

[p:\db2profs\db2\prewa27.clp]

uncatalog db aliasa update dbm cfg using INTRA_PARALLEL NO

The prewa26.clp and prewa27.clp scripts will uncatalog the alias before the instance starts so remote clients cannot access the database when the instance has been started. Intra-partition parallelism will be enabled for instance DB2 if it is on its primary machine (WA26) and disabled if it is on the failover machine (WA27). The post-online script will execute after the DB2 resource has been brought online and will be executed asynchronously by MSCS. The initial batch file will call one of two scripts depending on which machine the instance is starting on. [p:\db2profs\db2\db2cpost.bat]

if "%COMPUTERNAME%" == "WA26" goto WA26 if "%COMPUTERNAME%" == "WA27" goto WA27 echo Error ! No script for this compute >> p:\db2profs\db2\post.out goto end :WA26 db2cmd -c db2 -z p:\db2profs\db2\post.out -vf p:\db2profs\db2\postwa26.clp goto end :WA27 db2cmd -c db2 -z p:\db2profs\db2\post.out -vf p:\db2profs\db2\postwa27.clp goto end :end

The db2cpost.bat will call postwa26.clp if the the instance DB2 is starting on machine WA26 or it will call postwa27.clp if the instance is starting on machine WA27.


30

[p:\db2profs\db2\postwa26.clp]

restart db samplea update db cfg for samplea using dbheap 12000 alter bufferpool IBMDEFAULTBP size 10000 connect reset activate db samplea catalog db samplea as aliasa

[p:\db2profs\db2\postwa27.clp]

restart db samplea update db cfg for samplea using dbheap 6000 alter bufferpool IBMDEFAULTBP size 6000 connect reset activate db samplea catalog db samplea as aliasa

The postwa26.clp and postwa27.clp will restart the database samplea so it will be in a consistent state . The scripts will then set higher configuration parameters if the instance is on its primary machine (WA26) and then perform a connect reset. Since the connection within these scripts is the only connection to the database at this time, the subsequent activate database command will utilize the new settings. Finally, the alias called aliasa will be recataloged and clients will be able to connect to the database. You can create similar scripts for instance DB3 in its instance directory.

Testing the Configuration 31

Testing the Configuration When testing a high availability configuration, you ideally want to test all points of failure to ensure a redundant path is used. For example, such things as disk failures, network failures, machine failures, and software failures should be tested. The objective of this section is not to provide detail on how to test the whole system, but it will show you how to test the DB2 portion of the system. 1. From the remote client, connect to the highly available database and issue a query. 2. Move the group containing the instance to another machine. If DB2_FALLBACK is not

enabled, ensure the client disconnects from the database. 3. From the remote client, attemp to reissue the query from step 1. This attempt should fail,

because the database connection has been lost. 4. From the remote client, reconnect to the same highly available database and issue the

query from step 1. This attempt should succeed. 5. Move the group containing the instance back to the primary machine. If

DB2_FALLBACK is not enabled, ensure the client disconnects from the database. 6. From the remote client, attempt to reissue the query from step 1. This attempt should

fail, because the database connection has been lost. 7. From the remote client, reconnect to the same highly available database and issue the

query from step 1. This attempt should succeed. 8. Reissue steps 1-7 using various simulated hardware and software failures and use a client

workload that more closely simulates the actual workload expected in the production environment.


32

Rolling Upgrade A rolling upgrade is the ability to upgrade software on the cluster while still keeping the application online. An MSCS environment ideally lends itself for performing a rolling upgrade of DB2, because the instance can be online on one machine while the other machine is being upgraded. The current release of DB2 requires the instance to be shut down during a DB2 fixpak upgrade. Scheduled for September 2001, DB2 V7.2 FixPak 4 and above can be applied to a machine in the cluster while the instance is online on another machine in the cluster. Current instructions for upgrading WA27 and WA26: 1. Move all groups containing DB2 to machine WA26. 2. Stop the cluster service on WA27 (net stop clussvc). 3. In the SQLLIB\BIN directory on WA27, rename db2wolf.dll to db2wolf.bak. 4. Start the cluster service on WA27 (net start clussvc). 5. Move all groups containing DB2 to machine WA27. Ensure all resources of type DB2

are offline before moving them to WA27. All other resources within the group containing DB2 should be online.

6. Upgrade DB2 on machine WA27. 7. In the SQLLIB\BIN directory on WA27, if a new db2wolf.dll does not exist from the

upgrade, rename db2wolf.bak to db2wolf.dll. 8. The instance can now be successfully brought online on WA27. 9. Upgrade machine WA26 by rerunning steps 1-7, but on the opposite machine Upgrade instructions for DB2 V7 FixPak 4 and above (WA27 will be upgraded first): 1. Move all groups containing DB2 to machine WA26. DB2 will be online on machine

WA26. 2. Upgrade DB2 on machine WA27. 3. Move all groups containing DB2 to machine WA27. DB2 will be online on machine

WA27. 4. Upgrade DB2 on machine WA26.

Repairing a Cluster after Catastrophic Machine Failure 33

Repairing a Cluster after Catastrophic Machine Failure When a machine is damaged beyond repair, the operating system must be reloaded or the machine replaced. Consider machine WA27 damaged beyond repair so that the operating system has to be reinstalled. The following example depicts the steps to repair instance DB2, DB3, and the Administration Server in the configuration. Currently DB2, DB3, and the Administration Server are active on machine WA26, and clients can still fully access the databases. 1. From Cluster Administrator on WA26, right click on WA27 and select Evict Node to

remove WA27 from the cluster. 2. DB2 uses a DB2 profile variable called DB2CLUSTERLIST, which is utilized by DB2

to determine which machines are in the cluster. Initially, DB2CLUSTERLIST will show two machines in the cluster, as demonstrated by the following example using the instance DB2: c:\>db2set DB2CLUSTERLIST WA26 WA27 On machine WA26, remove machine WA27 from the cluster list for each instance: set DB2INSTANCE=DB2 db2set DB2CLUSTERLIST=wa26 set DB2INSTANCE=DB3 db2set DB2CLUSTERLIST=wa26 set DB2INSTANCE=DB2DAS00 db2set DB2CLUSTERLIST=wa26 If the cluster consists of 4 machines (macha, machb, machc, and machd), and machd is damaged beyond repair, the cluster list has to be set to the remaining 3 machines: db2set DB2CLUSTERLIST=macha machb machc

3. On machine WA27 reinstall the operating system. 4. On machine WA27 reinstall MSCS adding it back to the initial cluster. 5. On machine WA27, reinstall DB2 UDB. 6. On machine WA26, add machine WA27 to the cluster list for each instance:

set DB2INSTANCE=DB2 db2iclus ADD /m:wa27 set DB2INSTNACE=DB3 db2iclus ADD /m:wa27 set DB2INSTANCE=DB2DAS00


34

db2iclus ADD /m:wa27

7. The groups containing DB2, DB3, and the Administration Server are now ready to failover to machine WA27.

8. At an appropriate time, use the Move Group option in Cluster Administrator to ensure the groups containing the DB2 instances can successfully start on machine WA27.

If upgrading machine WA27 with a new machine, first move all groups to machine WA26, and then follow steps 1-8.

Configuring Other DB2 Services for Hot Standby Configuration 35

Configuring Other DB2 Services for Hot Standby Configuration During installation, beside the main DB2 server services, DB2 creates several Windows services, which are listed below:

DB2 Security Server - authenticates database users when authentication is performed at the client machine DB2 Governor - collects statistics about the applications running against a database DB2 License Server - checks for a license when clients connect to the database DB2 - DB2CTLSV - supports centralized administration of remote DB2 servers DB2 JDBC Applet Server - provides JDBC server support DB2 JDBC Applet Server - Control Center - provides JDBC server support for the Control Center applet Warehouse Logger - provides warehouse logging service Warehouse Server - controls the interaction of warehouse components, logs messages, controls and monitors warehouse agents, and provides scheduling functions

These services can be configured for failover in a hot standby configuration by using either the MSCS Generic Service resource or using the DB2 post-online scripts. In either case, from the Windows Service dialog box, ensure these services are configured to start manually. These services should not be configured for failover in a mutual takeover configuration.

Using Generic Service Resource Type The MSCS Generic Service resource type can be used to manage Windows services as cluster resources. The Generic Service resource type provides only the most basic functionality. For example, the failure of a Generic Service resource is determined by a query of the Microsoft Windows NT or Windows 2000 Service Controller. If the service is running, it is presumed to be online. To enable any of the above DB2 services for failover, use Cluster Administrator to define a Generic Service resource for each server and place it in the same group as the DB2 resource. When creating the Generic Service resource, ensure that the dependancy should be set to depend on the DB2 resource. For example, to enable the DB2 JDBC Applet Server for failover, a Generic Service resource is created in DB2 Group, as in Figure 15.


36

Figure 15: Cluster Adminstrator New Resource Box When creating the Generic Service resource, a service name must be specified. Refer to the following table for the service name for each DB2 service:

Service Display Name Service Name DB2 Security Server DB2NTSECSERVER DB2 Governor DB2GOVERNOR DB2 License Server DB2LICD DB2 JDBC Applet Server DB2JDS DB2 JDBC Applet Server - Control Center DB2ControlCenterServer Warehouse Server vwkernel Warehouse Logger vwlogger Any application that depends on DB2 may also be configured either as Generic Service or Generic Application resource so that it also fails over or fails back along with the DB2 server.

Using Post-Online Script to Start Other DB2 Services As an alternative to using Generic Service, it is possible to use the post-online script to start other DB2 services after the DB2 server is brought online. The example illustrates scripts for instance DB2. The scripts will reside in p:\db2profs\db2 directory. For example, to use the post-online

Configuring Other DB2 Services for Hot Standby Configuration 37

script to start the DB2 JDBC Applet Server and DB2 JDBC Applet Server - Control Center, use the following procedure: 1. Go the "p:\db2profs\db2" directory and create the DB2CPOST.BAT and STARTSVC.BAT

as follows: [DB2CPOST.BAT] p:\db2profs\db2\startsvc.bat > p:\db2profs\db2\startsvc.out 2>&1 [STARTSVC.BAT] net start db2jds net start db2ControlCenterServer

2. Test the post-online script by moving DB2 Group to the other node. The output file p:\db2profs\db2\startsvc.out should appear in the following form: If the services have already been started:

C:\WINNT\system32>net start db2jds The requested service has already been started. More help is available by typing NET HELPMSG 2182. C:\WINNT\system32>net start db2ControlCenterServer The requested service has already been started. More help is available by typing NET HELPMSG 2182. If the services have not been started:

C:\WINNT\system32>net start db2jds The DB2 JDBC Applet Server service is starting. The DB2 JDBC Applet Server service was started successfully.

C:\WINNT\system32>net start db2ControlCenterServer The DB2 JDBC Applet Server - Control Center service is starting. The DB2 JDBC Applet Server - Control Center service was started successfully.


38

Managing Security After DB2 is working properly under MSCS, you may want to specify which users can administer the cluster. To administer a cluster, users must have either administrative permissions on both nodes or specific permissions to administer the cluster. By default, the local Administrators group on both nodes have permissions to administer the cluster. To give a user permissions to administer a cluster without giving the user Administrative permissions on both nodes:

1. Bring up Cluster Administrator. 2. Right-click on the cluster name, and then click Properties. 3. Click Security or Permissions. 4. Specify which users and groups may administer the cluster.

The users also must have access to the DB2 registry variables that are stored in the cluster registry under the HKEY_LOCAL_MACHINE\Cluster\IBM\DB2\PROFILES registry key. By default, the local Administrators group on both nodes have full control to the cluster registry. To give a user permission to access the DB2 registry variables:

1. Run regedt32.exe. 2. Expand the Cluster key until the

HKEY_LOCAL_MACHINE\Cluster\IBM\DB2\PROFILES key is displayed. 3. Select the key, then click Security. 4. Click Permissions. 5. Specify which users and groups may need to run DB2.

DB2 Authentication We recommend that you use domain security (domain users and domain groups) so that when DB2 fails over to another machine, the same (domain) user can connect to the database with the same authority. By default, domain administrators have full access to the database. To restrict SYSADM authority to domain users and groups :

1. Create a domain group. The group name must conform to the DB2 naming convention, using 8 characters or less.

2. Add any domain users who will have DB2 SYSADM authority for this domain group. 3. From the machine that runs DB2, update the database manager configuration parameter

SYSADM_GROUP to the name of the domain group. 4. Restart DB2.

Appendix A – Limitations and Restrictions 39

Appendix A – Limitations and Restrictions When running DB2 in an MSCS environment, the following limitations and restrictions apply: 1. Since MSCS uses the drive letter when referring to Physical Disk resources, we

recommend that you use drive letters when determining the path for DB2 databases and tablespace containers.

2. Since MSCS cannot manage raw disk devices, DB2 should not be configured to use raw disk devices.

3. The DB2 instance must be managed from an MSCS interface such as Cluster Administrator. MSCS will not monitor resources started outside of its control, because it will not be aware the resource has been started. MSCS will also treat any attempts to stop a DB2 instance outside of its control as a resource failure, if that resource was initially brought online by Cluster Administrator. Thus, mechanisms such as DB2STOP, DB2START, and the DB2 Control Center should not be used to start and stop a DB2 instance within this environment.


40

Appendix B – Frequently Asked Questions 1. Q. Must I create my EE instance on each machine in the cluster before executing the

DB2MSCS utility? A. The EE instance only needs to reside on the machine where the DB2MSCS utility is being executed. If an instance of the same name resides on the other machines in the cluster, those instances will now point to the newly transformed HA instance.

2. Q. I ran the DB2MSCS utility and I got the following error: c:\>db2mscs -f:db2mscs.cfg.db2 Warning: The message file is missing DB2MSCS processing complete, rc = 126 A. After DB2 has been installed, each machine in the cluster must be rebooted before proceeding with running the DB2MSCS utility.

3. Q. I ran the DB2MSCS utility and I got the following error:

c:\>db2mscs -f:db2mscs.cfg.badip DB21524E Failed to create the resource "mscs5". Win32 error: "" A. The resource mscs5 has a corresponding IP address that is already in use on the network. Ensure the IP address specified is not already in use.

4. Q. I ran the DB2MSCS utility and I got the following error:

c:\>db2mscs -f:db2mscs.cfg.baddisk DB21526E Failed to move resource “Diiiisk O:". Win32 error: "The cluster resource could not be found.” A. The Physical Disk resource specified does not exist within the cluster. Ensure the name of the disk in the input configuration file is exactly identical to the name specifed within Cluster Administrator.

5. Q. I run the DB2MSCS utility and it does not execute successfully. A. Refer to the message reference in Appendix C of this paper to determine appropriate actions.

6. Q. My instance does not failover.

A. The instance must be started through a cluster interface, such as Cluster Administrator, or the cluster will not be aware it must try to keep this DB2 instance online.

7. Q. I do a db2stop and the instance automatically restarts. A. If the DB2 instance was started through Cluster Administrator, it must be stopped

Appendix B – Frequently Asked Questions 41

through a cluster interface. MSCS treats db2stop as a resource failure and will attempt to bring it back online.

8. Q. I try to take the group containing the DB2 instance offline, but the DB2 resource does

not successfully come offline. A. Ensure DB2_FALLBACK is set to ON for that instance (db2set DB2_FALLBACK=ON). To successfully bring the group containing the DB2 instance offline at this point, stop the cluster service on the machine where the DB2 instance is trying to come offline.

9. Q. My remote client successfully connects to the database, but when I failover the

instance, I cannot successfully reconnect. A. Ensure that the client is cataloged to connect to the highly available IP Address resource that is configured in the same group as the instance that contains the database. Also, ensure that the port defined by the SVCENAME paramater in the database manager configuration is not already in use.

10. Q. When I issue DB2 commands locally from the DB2 command line processor, I get

the following error: C:\>db2 connect to samplea SQL5005C System Error. A. The instance is not on the current machine, so the instance directory is not accessible. Issue the commands on the machine where the instance resides.

11. Q. The commands in the pre-online and post-online script fail with authentication errors.

A. The pre-online script and post-online script are run under the owner of the Cluster Server service. Ensure the owner of the Cluster Server has sufficient privileges to execute the commands in the pre-online and post-online scripts.

Note: A trace of the execution of the DB2MSCS utility can be envoked by executing the following command: db2mscs –f:db2mscs.cfg.db2 –d:trace.out This trace will be beneficial to IBM support for problem determination if necessary.


42

Appendix C – Message Reference DB21500I The DB2MSCS command completed successfully. Explanation: The user request was successfully processed. User Response: No action required. DB21501E Invalid option "<option-name>" was specified for the command. Explanation: An invalid argument was entered for the DB2MSCS command. Valid arguments for the command are:

-f:InputFileName Specifies the DB2MSCS.CFG input file to be used by the MSCS utility. If this parameter is not specified, the DB2MSCS utility reads the DB2MSCS.CFG file that is in the current directory.

-d:TraceFileName Turns on debug tracing and specifies the name of the trace output file.

User Response: For more information about this command, refer to the Administration Guide. Enter the command again as follows:

DB2MSCS -f InputFileName

DB21502E Cannot open the configuration file "<file-name>". Explanation: A configuration file could not be opened. Depending on the filename in the message text, this error can be explained as follows:

If the file name in the message text was the name of the input file specified for the DB2MSCS command, the input file cannot be found. If the file name was db2systm, then the database manager configuration file for the target instance is missing. If the file name was db2mscs.bak, then the backup configuration file could not be created in the instance directory. If the file name was db2mscs.bak and an undo operation was performed, then the backup configuration file from the instance directory could not be opened when performing the undo operation.

User Response: Depending on the file that was in error, the problem may be corrected as follows:

If the file name in the message text was the name of the input file specified for the DB2MSCS command, ensure that the file exists in the current directory, or that a fully qualified file name is specified for the command. If the database manager configuration file for the target instance is missing, then drop and re-create the instance. If the backup configuration file could not be created in the instance directory, ensure that the instance profile directory exists and that the current logon account has write access to the directory. If the backup configuration file from the instance directory could not be opened when performing the undo operation, ensure that the MSCS disk that contains the instance profile directory is online on the current machine and retry the operation.

DB21503E Not enough memory is available to process this command. Explanation: There was not enough memory to continue processing the command.

Appendix C – Message Reference 43

User Response: Ensure that the system has sufficient real and vitual memory. Close all applications that are not in use to free up additional memory for the system. "DB21504E The input parameter is too long, keyword = "<keyword-name>" value = "<keyword-value>". Explanation: The value %2 specified for the keyword %1 exceeded the maximum allowable limit for that parameter. User Response: Specify a value that conforms to the following maximum length restrictions:

Maximum length for a group or resource name is 64 Maximum length for an IP address or subnet mask is 15 Maximum length for a DB2 instance name is 8 Maximum length for a network name, cluster name, or computer name is 64 Maximum length for a username or password is 256

DB21505E You must specify the "<keyword-name>" keyword before the "<keyword-name>" keyword. Explanation: The sequence of parameters specified in the DB2MSCS configuration file is not valid. The group name must be specified before any other resource parameter can be specified. For each resource, the resource name parameter must be specified before any resource parameter can be specified. User Response: Modify the DB2MSCS configuration file so that the sequence of parameters is correct. DB21506E The cluster "<cluster-name>" cannot be accessed. Ensure that the cluster name is correct and that the cluster service is running. Explanation: The DB2MSCS utility could not open the cluster because either the cluster name was incorrect or the cluster service on the current machine has not been started. User Response: If the cluster service has not been started on the current machine, then start the cluster service by running the command "net start clussvc" or by starting the Cluster Service from the Services dialog. If the cluster name was specified incorrectly in the DB2MSCS configuration file, modify the cluster name and resubmit the command. DB21507E The instance name "<instance-name>" is not valid. Explanation: The instance name specified in the DB2MSCS configuration file is not valid, or the DB2INSTANCE environment variable was not set to a valid instance name. User Response: If the instance name was specified in the DB2MSCS configuration file, verify that the instance name is valid and resubmit the command. If the instance name was not specified in the configuration file, ensure the DB2INSTANCE environment variable is set to the name of a valid DB2 instance. DB21509E The keyword "<keyword-name>" is only valid for the partitioned database instance. Explanation: The keyword specified is only valid if the target instance is a partitioned database instance. For example, the DB2_NODE keyword should only be specified for the partitioned database instance. User Response: Comment out the invalid keyword in the configuration file and resubmit the command.


44

DB21510E "<address-value>" is not a valid Internet address. Explanation: The value specified for either the IP address or the subnet mask does not conform with the Internet address format. A valid Internet address format has the form: "<nnn>.<nnn>.<nnn>.<nnn>", where <nnn> is a number from 0 to 255. User Response: Correct the invalid address in the configuration file and resubmit the command. DB21511E Node "<node-number>" does not exist. Explanation: The node number specified in the DB2_NODE keyword does not correspond to a valid database partition number. User Response: Correct the DB2_NODE parameter to specify an existing node number. DB21512E The keyword "<keyword-name>" is not valid. Explanation: The keyword specified was not a valid DB2MSCS keyword. User Response: Use a valid DB2MSCS keyword. For more information about keywords, refer to the Administration Guide. DB21513E Cannot create MSCS group "<group-name>". Win32 error: "<error-number>" Explanation: The DB2MSCS utility could not create the group because of the Windows system error. User Response: Refer to the Windows system error message for further information. DB21514E Win32 error message: "<error-number>" Explanation: The DB2MSCS utility failed to complete because of a Windows system error. User Response: Refer to the Windows system error message for further information. DB21515E The required resource property specified by keyword "<keyword-name>" is missing for the resource "<resource-name>". Explanation: A resource could not be created because one of its required parameters was not specified. For example, for IP Address resource, the IP address and the subnet mask must be specified. For the Network Name resource, the network name must be specified. User Response: Ensure that the required parameter is specified and resubmit the command. DB21516E Cannot bring the resource "<resource-name>" online. Ensure that the properties of the resource are set correctly.


Explanation: After a resource is created, the DB2MSCS utility validates the resource by attempting to bring the resource online. Failing to bring a resource online indicates that either the resource property was not specified correctly or that the cluster network was not functioning properly. User Response: If a disk resource was in error, ensure that the disk subsystem and the disk device driver are functioning properly. Use the Event Viewer to examine if any disk device driver problem was recorded in the Event logs. If an IP Address resource was in error, ensure that the IP parameters are correct and that the network(s) where the IP address resides is functioning properly. Also, the IP address that is specified for DB2 must NOT be used by any other machine in the network. If you are not sure what parameters to use for the IP address, consult with your network administrator. If a Network Name resource was in error, ensure that the network is functioning properly and that the value specified for the Netname parameter has not been used by any machine in the network. Note that the Network Name parameter is not required for an EE instance. As a workaround, you may want to comment out the Network Name parameter and proceed. If a DB2 resource was in error, you can find a record of that error in the db2diag.log. DB21517E MSCS Network "<network-name>" is not active. Explanation: The network parameter specified for the IP address is not active. User Response: From the Cluster Administration view, activate or enable the target network and resubmit the command. DB21518E There is no active MSCS network. Explanation: The network parameter was not specified for the IP address, and there was no network available to be used. User Response: A valid MSCS network must be configured. Refer to your cluster documentation for how to add and configure an MSCS network. DB21519E Cannot bring the resource offline. Ensure that the properties of the resource are set correctly. Explanation: The DB2MSCS could not bring a resource offline. The resource may be in use by the cluster software. User Response: Retry the operation. If the problem persists, run with the trace option and contact your IBM Service Representative for further assistance. DB21520E The DB2PATH profile variable is not defined. Explanation: The DB2PATH registry profile variable is not defined for the current machine. The DB2PATH must be set to the path where DB2 is installed.


46

User Response: Set the DB2PATH to the directory where DB2 is installed using the db2set command. For example, db2set DB2PATH=D:\SQLLIB. DB21521E Cannot read from file "<filename>". Explanation: The DB2MSCS utility cannot read data from the indicated file. User Response: Ensure that the file is not locked and that the current logon users <that the users currently logged on?> have sufficient authority to read the file. DB21522E Cannot open machine registry for machine "<machine-name>". Ensure that the machine is active and that the current logon account has Local Administrator authority. Explanation: The DB2MSCS could not open the registry for the remote machine for read and write access. By default, only users that belong to the Local Administrator groups on that machine have read and write access to the machine registry. This error is also returned if the remote machine is not active. User Response: Ensure that the target machine is active, then log on to a domain account that belongs to the Local Administrator group on the target machine and resubmit the command. DB21523E Cannot close machine registry for machine "<machine-name>". Ensure that the machine is active and that the current logon account has Local Administrator authority. Explanation: After opening the remote registry, the DB2MSCS utility failed to close the handle to the remote registry because of an internal error. User Response: Run with the trace option and contact your IBM Service Representative for further assistance. DB21524E Failed to create the resource "<resource-name>". Win32 error: "<error-number>" Explanation: The command failed to create the target resource because of a Windows system error. User Response: Refer to the Windows system error message for additional information. DB21525E Failed to add dependency to the resource "<resource-name>". Win32 error: "<error-number>" Explanation: The command failed to add dependency for the target resource because of a Windows system error. User Response: Refer to the Windows system error message for additional information. DB21526E Failed to move resource "<resource-name>". Win32 error: "<error-number>" Explanation: The command failed to move a resource because of a Windows system error. User Response: Refer to the Windows system error message for additional information.


DB21527E No disk resource is specified for the group "<group-name>". Explanation: At least one disk resource must be specified for each group. User Response: Assign one or more disk resources to the group indicated in the error message. DB21528E The value "<keyword-value>" specified for the INSTPROF_DISK keyword does not match any disk in the same group. Explanation: The DB2INSTPROF_DISK keyword is used to specify the location where the content of the instance profile directory will be copied. The value for the DB2INSTPROF_DISK keyword must match one name of a disk resource in the same group. User Response: Set the DB2INSTPROF_DISK to the name of one of the disk resources in the same group. DB21529E The DB2MSCS utility cannot access the registry of machine "<machine-name>". Ensure that the machine is active and that the current logon account has Local Administrator authority. Explanation: The DB2MSCS utility cannot access the registry of the target machine. User Response: Log on to a domain account that belongs to the Local Administrator group on the target machine and resubmit the command. DB21530E The DB2MSCS utility cannot access the cluster registry for the cluster "<cluster-name>". Ensure that the cluster is active and the current logon account has Local Administrator authority. Explanation: To administer a cluster, users must have either administrative permissions on both nodes or specific permissions to administer the cluster. By default, the Local Administrators group on both nodes has permissions to administer the cluster. User Response: Log on to an account that has sufficient access to the cluster. To give a user permissions to administer a cluster without giving the user Administrative permissions on both nodes:

1. Run the Cluster Administration GUI. 2. Right-click the cluster name, and then click Properties. 3. Click Security or Permissions. 4. Specify which users and groups may administer the cluster.

DB21531E Cannot obtain property for MSCS disk. Win32 error: "<error-number>" Explanation: The DB2MSCS utility cannot obtain the drive letter from the MSCS disk resource. This problem usually occurs when the disk resource specified by the INSTPROF_DISK keyword is an IBM Netfinity disk resource, "IPSHA Disk". User Response: Do not use the INSTPROF_DISK keywork. Instead, use the INSTPROF_PATH keyword to explicitly specify the target location where the instance profile directory will be copied. DB21532E A internal error occurred. File: "<file-name>", Line "<line-number>". Please contact your IBM Service Representative.


48

Explanation: The DB2MSCS failed because of an internal error. User Response: Run with the trace option and contact your IBM Service Representative for further assistance. DB21533E An error occurred during the migration of the DB2 instance, rc = "<error-code>". Explanation: After all the required MSCS resources had been created, the DB2MSCS utility failed to migrate the DB2 instance to run in a clustered environment because of an internal error. During an instance migration, the utility performs the following steps:

1. Copy the instance directory to the location specified by the INSTPROF_DISK or INSTPROF_PATH keyword.

2. Move the DB2 registry profile variables from the machine registry to the cluster registry. 3. Set the DB2INSTPROF registry variable to point to the new instance profile location. 4. Set the DB2CLUSTERLIST to the name of the current machine.

User Response: Before running the DB2MSCS utility, ensure that:

On the current machine, the instance can be started and stopped successfully from the command line. On other cluster node(s), the same instance must be stopped and optionally dropped. All the disk resources are active on the current machine and can be moved back and forth successfully between the cluster nodes. The current logon user has sufficient access to the local machine registry and the cluster registry.

If the problem persists, contact your IBM Service Representative and provide both the DB2MSCS traces and DB2 traces. DB21534E An error occurred during addition of MSCS node to the DB2 instance, rc = "<error-code>". Explanation: The utility failed to add the other MSCS node to the DB2 instance. During this operation, the utility will do the followings:

1. Update the DB2 cluster machine list by adding the name of the target machine to the DB2CLUSTERLIST registry variable.

2. Create the DB2 service and the registry instance profile for the current DB2 instance on the target node. User Response: Before running the DB2MSCS utility, ensure that:

On the current machine, the instance can be started and stopped successfully from the command line. On other cluster node(s), the same instance must be stopped and optionally dropped. All the disk resources are active on the current machine and can be moved back and forth successfully between the cluster nodes. The current logon user has sufficient access to the target machine registry and the cluster registry.

If the problem persists, contact your IBM Service Representative and provide both the DB2MSCS traces and DB2 traces. DB21535E The instance-owning database partition server is not on the current machine. Explanation: When migrating a partitioned database instance, the DB2MSCS utility must be run on the instance-owning machine. User Response: Run the DB2MSCS utility from the instance-owning machine. DB21536E The user name "<userid>" is not valid.


Explanation: The user name specified is not valid. User Response: Specify a valid user name. DB21537E The password "<password>" is not valid. Explanation: The password specified is not valid. User Response: Specify the correct password. DB21538E The password for the account "<account-name>" has expired. Explanation: The password for the target account has expired. User Response: Reset the password and resubmit the command. DB21540E Group "<group-name>" requires at least one Network Name resource. Explanation: When migrating a partitioned database instance, a Network Name resource must be created for the group that contains the instance-owning node. User Response: Specify a Network Name resource in the group indicated. DB21541E An error occurred when removing the MSCS node from the DB2 instance, rc = "<error-code>". Explanation: During an "undo" operation, the utility failed to remove an MSCS node from the DB2 instance because of an internal error. User Response: Manual clean up is required. To manually clean up the instance, do the following:

1. Stop and drop the DB2 instance. 2. Remove all DB2 resources and their dependent resources from the Cluster Administrator window.

DB21542E An error occurrred while attempting to remove failover support for the instance. Failover support is still active for this instance, rc = "<error-code>"." Explanation: During an "undo" operation, the utility failed to uncluster the DB2 instance because of an internal error. User Response: Manual clean up is required. To manually clean up the instance, do the following:

1. Stop and drop the instance. 2. Remove all DB2 resources and their dependent resources from the Cluster Administrator window.

DB21543E The resource name "<resource-name>" does not match any IP resource in the same group. Explanation: A Network Name resource must be configured to depend on an IP Address resource in the same resource group.


50

User Response: Specify the name of an IP Address resource that resides in the same group as a dependency for the Network Name resource. DB21544E The MSCS resource "<resource-name>" already exists. Explanation: The resouce name specified already exists in the cluster. User Response: Specify a different resource name. DB21545E The module "<file-name>" was loaded, but the function "<function-name>" is not found. Ensure that you are running on the version of DB2 that supports MSCS. Explanation: The utility failed to obtain the address of a required function because the version of DB2 is not compatible with the version of the DB2MSCS utility. User Response: Use the version of the DB2MSCS utility that is shipped with the DB2 product. DB21546E The module "<file-name>" could not be loaded. Explanation: The utility failed to load the required DLL. User Response: Reinstall the DB2 product. DB21547E Error occurred while moving group "<group-name>" to node "<node-number>". Win32 error: "<error-code>" Explanation: The utility failed to move the group to the target node because one or more resources cannot be moved. User Response: Ensure that all cluster nodes are active and that all disk resources can be moved back and forth between cluster nodes. If the problem persists, contact your IBM Service Representative. DB21548E A logon account for the DB2 service must be specified for a partitioned database system. Specify a valid logon account using the DB2_LOGON_USERNAME and DB2_LOGON_PASSWORD keywords. Explanation: The DB2 service for a partitioned database system must be configured to be run under a valid domain account. User Response: Specify a valid domain account using the DB2_LOGON_USERNAME and DB2_LOGON_PASSWORD keywords.

Appendix D – DB2 UDB Software Levels 51

Appendix D – DB2 UDB Software Levels The following defines the minimal level of DB2 that should be used when DB2 is configured within the MSCS environment: Windows NT – DB2 UDB V5.2 or greater Windows 2000 – DB2 UDB V6 FixPak 5 or greater


52

Appendix E - Declustering an Instance If you no longer want an HA instance, back up all databases within the clustered instance and then drop the instance. You can then re-create the instance and restore all databases to the appropriate directories. Assume you have instance DB2 clustered. 1. Back up the database samplea that resides within instance DB2. 2. Bring instance DB2 offline from Cluster Administrator. 3. Drop the instance DB2 from one of the machines in the cluster. This will drop the

instance from all machines. db2idrop db2 Tip: When dropping the instance, all instance information will be lost. Ensure that you save any instance information that may be required in the future, such as database manager configuration parameters and DB2 profile variable settings: db2 get dbm cfg > db2cfg.out db2set –all > db2set.out

4. From Cluster Administrator, drop the DB2 resource corresponding to instance DB2 as well as the IP address used for instance DB2. Move all Physical Disk resources back to their initial groups. Then drop the group associated with instance DB2, because no resources will exist within this group.

5. If no other DB2 instances are configured for the MSCS cluster, the DB2 resource type can be dropped. The following command only needs to be run from one machine in the cluster: db2wolfi u

6. Recreate the instance DB2. db2icrt db2

7. Restore the databases that were backed up in step 1.

Appendix F - Manually Performing Steps Done by the DB2MSCS Utility 53

Appendix F - Manually Performing Steps Done by the DB2MSCS Utility The example presented below will manually configure the DB2 instance identically to the way it was configured in the “Hot Standby Configuration” section of this paper. We recommend that you use the DB2MSCS utility, however, decomposing the steps may help during problem determination if errors occur using the DB2MSCS utility. 1. Initially MSCS and DB2 UDB EE are installed on both machines. 2. Create a new instance called DB2 on machine WA26.

db2icrt DB2

3. Ensure from the Windows Services dialog box that the instance is configured to start manually.

4. Stop the instance DB2 with the DB2STOP command if the instance is running 5. Install the DB2 resource type from WA26.

c:>db2wolfi i ok If the db2wolfi command comes back with an “Error : 183”, then it is already installed. To confirm, the resource type can be dropped and added again. Also, the resource type will not show up in Cluster Administrator if it does not exist. c:>db2wolfi u ok c:>db2wolfi i ok

6. From WA26 use the db2iclus command to transfrom the DB2 instance into a clustered instance. c:\>db2iclus migrate /i:db2 /c:mycluster /m:wa26 /p:p:\db2profs DBI1912I The DB2 Cluster command was successful. Explanation: The user request was successfully processed. User Response: No action required. Note: The directory p:\db2profs should be on a clustered drive and must already exist. This drive should also be currently owned by machine WA26.

7. From WA26 use the db2iclus command to add other machines to the DB2 cluster list: c:\>db2iclus add /i:db2 /c:mycluster /m:wa27 DBI1912I The DB2 Cluster command was successful.


54

Explanation: The user request was successfully processed. User Response: No action required. This command should be executed for each subsequent machine in the cluster.

8. From Cluster Administrator, create a new group called “DB2 Group”. 9. From Cluster Administrator, move the Physical Disk resources Disk O and Disk P into

DB2 Group. 10. From Cluster Administrator, create a new resource type of type “IP Address” called

“mscs5”, that resides on the Public Network. This resource should also belong to DB2 Group. This will be a highly available IP address, and this address should not correspond to any machine on the network. Bring the IP Address resource type online and ensure that the address can be pinged from a remote machine.

11. From Cluster Administrator, create a new resource of type “DB2” that will belong to DB2 Group. The name of this resource must be exactly identical to the instance name, so it is called DB2 for this case. When Cluster Administrator prompts for dependencies associated with the DB2 resource, ensure it is dependent on Disk O, Disk P and mscs5.

12. Configure DB2 Group for failback if desired via Cluster Administrator and using the DB2_FALLBACK profile variable.

13. Create or restore all databases putting all data on Disk O and Disk P 14. Test the failover configuration

Implementing IBM DB2 Universal Database Enterprise Edition ...

Documents

Transcript of Implementing IBM DB2 Universal Database Enterprise Edition ...