o as Clustering

JD Edwards EnterpriseOne 8.97 and

OAS 10.1.3.1 Clustering

An Oracle JD Edwards EnterpriseOne

Best Practices Paper

July 2008

EnterpriseOne 8.97 and Multiple JVMs on OAS 10.1.3.1 �� Page i

PURPOSE STATEMENT

This document is intended to help you understand the features, functionality, and option available for EnterpriseOne HTML Server clustering in Oracle Application Server 10.1.3.1.0 conjunction with 8.97 Server Manager. There are many ways to configure multiple types of clustering and your results may differ depending on your custom setup.

This document is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Updates and enhancements are provided in accordance with Oracle’s Technical Support Policies at:

www.oracle.com/support/collateral/oracle-technical-support-policies.pdf

EnterpriseOne 8.97 and OAS 10.1.3.1 Clustering �� Page ii

JD Edwards EnterpriseOne 8.97 and Multiple JVMs on OAS 10.1.3.1

PURPOSE STATEMENT ______________________________________________________ i

Overview____________________________________________________________________ 1

About OAS 10.1.3.1 ___________________________________________________________ 1

10.1.3.1.0 vs. previous releases ______________________________________________________ 1

OAS 10.1.3.1.0 and JD Edwards EnterpriseOne __________________________________________ 1

Clustering ___________________________________________________________________ 2

OAS 10.1.3.1.0 Cluster and 8.97 Server Manager ________________________________________ 2

Multiple JVMs ____________________________________________________________________ 2

Vertical Clustering _________________________________________________________________ 2

Horizontal Clustering _______________________________________________________________ 3

Application Server level clustering ____________________________________________________ 3

Application level clustering __________________________________________________________ 3

OAS 10.1.3.1.0 with JD Edwards EnterpriseOne 8.97 Installation Overview ______________ 4

Prerequisites _____________________________________________________________________ 4

Architectural Diagram ______________________________________________________________ 5

Standalone Installation ________________________________________________________ 6

Creating OAS 10.1.3.1 Managed Instance ______________________________________________ 6

Creating OC4J Container ____________________________________________________________ 8

Distributing EnterpriseOne HTML Server code__________________________________________ 10

Installing EnterpriseOne HTML Server ________________________________________________ 12

Installing JDBC driver______________________________________________________________ 15

Testing our E1HTML Server Install ___________________________________________________ 17

Vertical Clustering ___________________________________________________________ 20

Second OC4J Install on the Same Node________________________________________________ 20

EnterpriseOne 8.97 and OAS 10.1.3.1 Clustering �� Page iii

Testing Second OC4J Install _________________________________________________________ 21

Configuring Vertical Cluster_________________________________________________________ 22

Configuring HTTP Server for load balancing ____________________________________________ 27

Testing the vertical cluster and request load balancing __________________________________ 28

Horizontal Clustering _________________________________________________________ 30

Standalone Installation on Second Server _____________________________________________ 30

Configuring Dynamic Node Discovery_________________________________________________ 32

Configuring Horizontal Clustering ____________________________________________________ 37

Modify httpd.conf file for horizontal cluster ___________________________________________ 40

Testing Horizontal Cluster and Load Balancing _________________________________________ 42

Vertical and Horizontal Cluster with Web Cache ___________________________________ 42

Configuring New Web Cache Listen Port ______________________________________________ 43

Configuring Origin Servers__________________________________________________________ 45

Configuring Site Server ____________________________________________________________ 47

Modify HTTP Server Configuration files _______________________________________________ 49

Test Web Cache setup _____________________________________________________________ 49

Multiple JVMs for OC4J instance ________________________________________________ 50

Trobleshooting ______________________________________________________________ 52

Internal Server Error ______________________________________________________________ 52

Page cannot be displayed __________________________________________________________ 52

Server Manager; Considerations and Limitations __________________________________ 53

Considerations ___________________________________________________________________ 53

Limitation _______________________________________________________________________ 53

©Copyright Oracle 2006. All rights reserved. EnterpriseOne 8.97 and Multiple JVMs on OAS 10.1.3.1 �� Page 1

JD Edwards EnterpriseOne 8.97 and Multiple JVMs on

OAS 10.1.3.1 Clustering

Overview There are multiple types of clustering available in Oracle Application Server (OAS) 10.1.3.1. We will

describe types of clustering that can be utilized in conjunction with E1 HTML Server. This paper

will focus mainly on how to setup, interact, monitor, and manage vertical clustering, horizontal

clustering and multiple JVM configurations in OAS 10.1.3.1 with 8.97 Server Manager.

About OAS 10.1.3.1 Oracle Application Server 10gR3 is a newer release of the Service-Oriented Architecture facilities

provided by Oracle Fusion Middleware. It includes comprehensive J2EE 1.4 support, enhanced

Web Services support, and enhanced support for standards EJB 3.0, Java Server Faces, Java 1.5 and

much more.

10.1.3.1.0 vs. previous releases The major difference between Oracle Application Server 10g Release 3 ( OAS 10.1.3.1.0) and

previous releases is that OAS 10.1.3.1.0 does not include an OracleAS Metadata Repository or the

DCM software. As a result, there is no concept of an OracleAS Farm. Instead, in OAS 10.1.3.1.0,

you configure your application server instances to communicate via Oracle Process Manager and

Notification Server (OPMN). When you configure two or more OAS 10.1.3.1.0 instances in this

manner, the instances can be managed from the Cluster Topology page in the OAS 10.1.3.1.0

Application Server Control Console.

OAS 10.1.3.1.0 and JD Edwards EnterpriseOne JD Edwards EnterpriseOne Tools Release 8.97 supports OAS 10.1.3.1 and previous release of OAS,

such as 10.1.2.0.0 and 10.1.2.0.2 are not supported. One important part to keep in mind, OAS

10.1.3.1.0 is not a full product release. It only contains Oracle Service-Oriented Architecture (SOA)


suite and Oracle Containers for J2EE (OC4J). There are no Web Cache, Infrastructure, and Portal

components in OAS 10.1.3.1.0. If you want these additional components to work with 10.1.3.1.0

you will have to obtain them from 10.1.2.0.2.

Clustering Oracle defines clustering as a set of processes running on single or multiple computers that share the same workload. To simplify, we can classify this into two major types of clustering, vertical and horizontal (described below). OAS 10.1.3.1.0 provides multiple methods/mechanism to create these types of clustering, such as Application Server level clustering, and OC4J level clustering. In addition OAS 10.1.3.1.01 also provides option call “Multiple JVMs”, simple method for increasing performance and level of fault tolerance for deployed application on hardware with addition resources.

OAS 10.1.3.1.0 Cluster and 8.97 Server Manager While all these options are available in OAS 10.1.3.1.0 and can be used in conjunction with JD Edwards EnterpriseOne Tools release 8.97, only Multiple JVM option is currently supported with 8.97 Server Manager for monitoring and management of EnterpriseOne HTML Servers. Customers wanting to implement options other than Multiple JVMs, such as vertical and/or horizontal clustering must do so outside of Server Manager utilizing Oracle Enterprise Manager 10g Application Server Control Console.

Multiple JVMs By default, OC4J instance executes on and uses single Java Virtual Machine (JVM). However, in

OAS, you can configure an OC4J instance to run on multiple JVMs. On a physical machine with

additional hardware resources this will provide increase performance, by having single OC4J

instance run on multiple processes, and provide a level of fault tolerance for deploy application via

session information in HTTP. Any OC4J instance, except for OC4J instance that is hosting an

active Application Server Control (ascontrol application), can be configured to use multiple JVMs

whether or not it is part of a cluster or defined as a part of a group.

Vertical Clustering In typical vertical clustering, multiple OC4J containers that are identically configured are grouped

together in a single OAS 10.1.3.1.0 installation, single Application Server (single node), on one

physical machine. This with or without using multiple JVM option will give user option to better

optimize hardware resources.


It is possible to have multiple installations of OAS 10.1.3.1.0, multiple Application Server (multiple

nodes), on one physical machine and cluster can be achieved either at Application Server (node)

level or at Application level. Step-by-step instruction on how to set this up will be discuss late in

this paper.

Horizontal Clustering In horizontal cluster, installation of OAS 10.1.3.1.0 is done on multiple physical machines. With

multiple nodes in hand we can cluster these nodes at Application Server level and/or at Application

level. We will first configure Dynamic Node Discovery so that ONS on each node can

communicate with each other then we will create and/or add our horizontal instance (E1HTML

Server application) to a cluster.

One distinct advantage of horizontal cluster is better failover protection along with a better High

Availability solution. In this scenario, even if hardware failures occur on one physical machine and

that machine is unavailable other node residing on other physical machine will be able to accept and

process user requests. Within vertical clustering, although you might be optimizing hardware

resources, if that physical machine goes down it is single point of failure.

Application Server level clustering Application Server level clustering is better described as multiple OAS 10.1.3.1.0 server instances configured to communicate via Oracle Notification Servers (ONS), which is part of Oracle process Manager and Notification Server (OPMN). In doing so, all the instances could be managed from the Cluster Topology page in Application Server Control Consol. This can be done for both vertical and horizontal cluster and OC4J instances within these cluster can be further optimized for hardware resource utilization by configuring multiple JVMs. This can be accomplished via 4 methods, Dynamic node discovery, Static Discovery Server, Cross-Topology Gateway, and Static Node-to-Node Communication, all these methods require Oracle Application Server Standard Edition and above. This is different from previous release with OracleAS Cluster where Application Server instances were configured to use common OracleAS Metadata Repository and management of the farm was conducted via Distributed Configuration Management (DCM) software. This provides additional layer of flexibility which is discuss in ‘OC4J Level Clustering’ below.

Application level clustering Application level clustering, better defined as an OC4J “Grouping”, is collection of OC4J instances in a cluster topology. This is more of management concept as certain management tasks can be performed simultaneously on all OC4J instance in a group, such as deploying application, stop and starting services, configuring Server Properties, etc… As mentioned above, this provides an additional layer of flexibilities in managing OC4J clusters. Unlike OarcleAS Clusters in previous


releases, where any configuration change made to individual instance get automatically propagated all other OC4J instances in that cluster, in OAS 10.1.3.1.0 changes can be made to a group and all of its OC4J instances via Group page in Application Server Control Console or to a single OC4J instance in a group. This is useful if you need to change JVM heapsize to match different hardware configuration or need to collect additional debug information by enabling verbose for garbage collection for specific instance in cluster.

OAS 10.1.3.1.0 with JD Edwards EnterpriseOne 8.97

Installation Overview

The configuration shown below will be used throughout remainder of this paper. Your

configuration may vary depending options and type of installation you are performing. As

previously mentioned, there are many types and methods you can exercise to create clustering within

OAS 10.1.3.1.0. In this red paper we will discuss one possible way. However, currently 8.97 Server

Manager is not OAS 10.1.3.1.0 cluster aware. Therefore, only Multiple JVM option for monitoring

and managing EnterpriseOne HTML Server with 8.97 Server Manager is supported at this time and

will be sole focus from here on.

The following steps can be described as proof-of-concept; customers wanting to implement these

options do so at their own risk. Furthermore, currently all installation, configuration, and

maintenance utilizing Application Server level clustering or Application level clustering can only be

done via Oracle Enterprise Manager 10g Application Server Control Console.

There is no DMZ or Web Cache consideration in this paper. Customers who wish to implement

these options can do so but this paper will not cover those topics.

Prerequisites Following prerequisites must be completed in its entirety prior to proceeding with rest of this red paper.

1. Complete installation of JD Edwards EnterpriseOne Tools release 8.97 with latest update. This includes Deployment Server and Enterprise Server.

a. Verify installation is functional by completing PORTTEST. NOTE: Please refer to 8.97 Tools Release installation guide for more detail on Porttest.(note: do you need to define or point to doc on Porttest?)

2. Complete installation of 8.97 Server Manager (To maximize usage, Oracle recommends that Server Manager be installed on EnterpriseOne Deployment Server).

a. Make sure Server Manger is running and functional by logging into Server Management Console.


3. Complete the installation of Server Management Agent on the machine you will be using for OAS install.

a. Verify that Server Manager Agent is up and running and that it can communicate with Server Management Console.

4. Complete upload of EnterpriseOne 8.97 HTML code that matches Enterprise Server and Deployment Server to Server Manager via Server Management Console.

5. Complete JDBC driver installation via Server Management Console so that it is available during EnterpriseOne HTML server install.

6. Complete installation of Oracle Application Server 10g Release3 (OAS 10.1.3.1.0). a. Test to make sure that you have functioning OAS install

Please refer to JD Edwards EnterpriseOne Documentation for detail step-by-step installation instruction for Server Manager, Deployment Server, Enterprise Server and OAS 10.1.3.1.0 install for JD Edwards EnterpriseOne software.

Architectural Diagram Following naming convention will be used.


Standalone Installation Begin by logging into Server Manager Management Console installed on the Deployment Server. If

Server Manger (SM) Agent is installed correctly on the OAS machine and is able to communicate to

SM Management Console located on Deployment Server via JMX port it will start auto registration

process of managed home.

Creating OAS 10.1.3.1 Managed Instance

1. Click on the link for OAS Server under Managed Home Location

(DENGSSWIN01.mlab.jdedwards.com).

2. Create new managed instance by clicking ‘Create New Managed Instance’ box.


3. Select ‘Oracle Application Server 10.1.3.1’ for Instance Type.

4. Complete the instance properties fields with correct information that correspond to your

installation. For this installation we will use OAS10131 as our instance name.

5. Click ‘Continue’ button on Confirmation page.


6. Click ‘Create Instance’ button on Finish page to initiate creation of managed instance.

Once managed instance have been created successfully you will see page similar to one

shown below.

Creating OC4J Container

Next, we will create OC4J container that will host E1HTML Server application.


1. From Oracle Application Server OAS1 page locate “J2EE Containers” section enter

OC4J container name (we are using EA_JS_81), select Group from drop down box if you have more than one group. NOTE: group can only be created from OAS ASC and must be created prior to creating OC4J container from SM) and click ‘Create’ button.

2. Once OC4J creation process has completed, from the same page you will now see newly created OC4J container under “J2EE Containers” section. If “State” shows Undetermined refresh the screen.


Distributing EnterpriseOne HTML Server code Next, we will distribute EnterpriseOne HTML Server code to OAS 10.1.3.1 server

(DENGSSWIN01).

1. Click on the link for ‘Managed Software’ under the Quick Link menu.

2. This brings up list of all managed software that has been uploaded to SM.

NOTE: If you have not already uploaded E1HTML Server code to SM please review

8.97 SM guide for step-by-step instruction on how to complete this task.

3. Select the software to distribute (EnterpriseOne HTML Server Version 8.97.x.x) and

click ‘Distribute’ button.


4. Select the server (DENGSSWIN01) from the list and click ‘Distribute Software’ button to

start the distribution of EnterpriseOne HTML Server code to OAS 10.1.3.1 Server

(DENGSSWIN01).

NOTE: In our case, software was previously distributed so you do have option to select the

machine.

5. Behind the scene, the process takes E1HTML Server par file from SM and copy it to component directory under SM Agent home directory (ours is in C:\JDE_HOME\component)


6. Once EnterpriseOne HTML Server software have been distributed go back to ‘Managed

Homes and Managed Instance’ page by clicking on ‘Management Dashboard’ breadcrumb

and clicking link for OAS server (DENGSSWIN01.mlab.jdedwards.com). From there you

can verify whether software have been distributed properly under ‘Managed Software

Components’ section.

Installing EnterpriseOne HTML Server Next, we will install EnterpriseOne HTML server to OAS 10.1.3.1 Server under EA_JS_81 container which we just under DENGSSWIN01 Managed Instance.


1. From previous screen where we verified that EnterpriseOne HTML Server code has been

properly distributed, click on ‘Create New Managed Instance’ button under ‘Managed

Instance’ section.

2. Select ‘EnterpriseOne HTML Server for Instance Type and click ‘Continue’ button.

3. On the Instance Properties page fill-in all the required field and click ‘Continue’. Server Group – this is to group all the servers that SM is managing, for more detail please review 8.97 SM Guide. For this paper server group ‘oasc’ was created prior. Instance Name – this is what will show up as a link for Instance Name under Managed Home page, like OAS1. J2EE Server – this is OC4J container that we created previously. If there is more than one OC4J container then you can select them from drop down box. Software Components – This will list all software components that have been distributed to your machine under <JDE_HOME>\component directory.


4. Enter valid entries for Path Code, Default Environment, Default Role, and Bootstrap

Environment and click ‘Continue’.

5. On the Finish page, click on ‘Create Instance’ to start the EnterpriseOne HTML Server

installation process.

This process can take up to few minutes. Once the installation is complete, SM management console will display a page similar to one shown below. NOTE: Status might show that E1HTML Server is currently running initially. Refresh the screen to get true status.


Installing JDBC driver Next, we will install JDBC driver to enable communication between EnterpriseOne HTML Server and our EnterpriseOne database.

1. Click on link for ‘JDBC Drivers’ under the ‘Quick Links’ menu.


2. In this example we are using SQL Server as our database. Therefore we will be

installing JDBC driver for SQL Server. Select the server that you want JDBC driver

install to (DENGSSWIN01.mlab.jdedwards.com). Notice that this time JDBC driver is

not installed under ‘Status’ column. Click ‘Install Driver’ button to start the installation

process.

3. JDBC Installation process is quick so after clicking ‘Install Driver’ button refresh the

screen and verify the Status again.


4. JDBC Driver install process copies the JDBC driver from SM to <ORACLE_HOME>\j2ee\<oc4j container name>\applib directory. NOTE: (troubleshooting) if you are having issue installing the driver first check the agent log. If agent log indicate issue with copying driver files check this directory. If you only see partial driver files delete them and re-install the driver by repeating steps above.

Testing our E1HTML Server Install Now, start the E1HTML Server and make sure you can login.

1. Click on ‘Management Dashboard’ link then EnterpriseOne HTML Server link

(EA_JS_81) under Managed Instance for OAS server (DENGSSWIN01).

2. Click ‘Start’ under General Section and click ‘OK’ on the popup box.


3. Verify E1HTML Server has started. Status should show that it is running.

NOTE: if ‘Instance Information’ menu shows 0 Active JVM Count then hit F5 on the browser window to refresh the screen.

4. Open a new browser window and type in the URL for E1HTML Server or click on the

link for the URL under ‘Instance Properties’ (http://DENGSSWIN01.mlab.jdedwards.com:81/jde/owhtml), which will launch new Internet browser window with E1HTML login screen.


5. Login using valid EnterpriseOne user.

6. Test an application to make sure that everything is working properly.

7. Verify user session from SM by clicking on ‘User Sessions’ link under Runtime Metrics

Menu on the left from EnterpriseOne HTML Server: EA_JS_81 page.


Vertical Clustering In typical vertical clustering, multiple OC4J containers that are identically configured are grouped together in a single OAS 10.1.3.1.0 installation, single Application Server (single node), on one physical machine. This with or without using multiple JVM option will give user option to better optimize hardware resources.

Second OC4J Install on the Same Node Repeat the steps outlined in the following sections to create another OC4J container and E1HTML Server to be use for vertical Cluster. NOTE: Don’t forget to install JDBC driver for that OC4J instance.

Creating OC4J Container _____________________________________ Error! Bookmark not defined.

Installing EnterpriseOne HTML Server __________________________ Error! Bookmark not defined.

Installing JDBC driver________________________________________ Error! Bookmark not defined.

Following parameter was used for our second OC4J instance.

1. Under creating OC4J Container a. OC4J Container Name: VC_JS_82

2. Under Installing EnterpriseOne HTML Server a. Instance Properties Page


Once install of second OC4J instance with E1HTML Server is completed you can go back to Managed Homes and Managed Instance page. There you should be able to see both instances (EA_JS_81 and VC_JS_82) for our Oracle Application Server node (DENGSSWIN01)

Testing Second OC4J Install Next, we need to test the second install to make sure it is functioning correctly before we cluster these two instances.

1. Same as before when we tested our first install, either click the link from EnterpriseOne

HTML Server: VC_JS_82 page or open a new internet browser and enter URL to access the login screen. Make sure you are using port 82 since this is port we used for the second install. Enter valid EnterpriseOne userid / password and click ‘Sign In’.


2. Test an application to make sure it is working properly.

3. From SM -> EnterpriseOne HTML Server: VC_JS-82 page click on ‘User Sessions’ link

under Runtime Metrics menu to verify that user is accessing VC_JS-82 instance.

Configuring Vertical Cluster Now, we need to cluster these two OC4J instances that are installed on DENGSSWIN01, Oracle Application Server 10.1.3.1.0. These steps are done via Application Server Control Console (ASC).


1. Login to ASC on DENGSSWIN01. Click on the link to our first OC4J instance

(EA_JS_81)

2. From OC4J: EA_JS_81 page click on ‘Application’ link either from top or bottom of

the page


3. From Application: EA_JS_81 page, click on ‘Administration’ link.

4. Next, click on icon under ‘Go to Task’ column next to ‘Clustering Properties’ task name.


5. Select ‘Override parent application clustering settings’ and from drop down box next to

Clustering select ‘Enable’. On Replication Protocol section select Multicast IP Replication and enter valid values for Multicast IP Address and Multicast Port. For this example, following values were used; Multicast IP Address – 230.230.0.102 Multicast Port – 45566 Click ‘OK’ NOTE: The Multicast IP address must be in the range 224.0.1.0 – 239.255.255.255 and port must be a port that is not being used on this system.

6. Once the process is completed, you should get a confirmation page.


7. Behind scene, this process updates orion-application.xml file located in

<ORACLE_HOME>\j2ee\<OC4J container (EA_JS_81)>\application-deployment\<Application Name (EA_JS_81)\ with cluster directive.

8. Repeat steps 1 thru 7 for second OC4J instance (VC_JS_82)

NOTE: Multicast IP Address and Port must be identical to our first OC4J instance.


Configuring HTTP Server for load balancing Now that we have 2 OC4J instances on the same node clustered (vertical clustering), we need to create single entry point for the vertical cluster and identify OC4J mount point so that HTTP Server can load balance the request between them. NOTE: This section is not required if you are configuring Vertical Cluster with Web Cache. Please refer to ‘Vertical and Horizontal Cluster with Web Cache’ section below for more detail.

1. Open up httpd.conf file using text editor. Httpd.conf file is located under

<ORACLE_HOME>\Apache\Apache\conf directory. Append follow info to bottom of httpd.conf file and then save it.

Listen 90 NameVirtualHost dengsswin01.mlab.jdedwards.com:90 <VirtualHost *:90> Oc4jMount /jde instance://E1JAS.DENGSSWIN01.mlab.jdedwards.com:EA_JS_81,E1JAS.DENGSSWIN01.mlab.jdedwards.com:VC_JS_82 Oc4jMount /jde/* instance://E1JAS.DENGSSWIN01.mlab.jdedwards.com:EA_JS_81, E1JAS.DENGSSWIN01.mlab.jdedwards.com:VC_JS_82 </VirtualHost>

NOTE: Listen and VirtualHost directive creates single entry point. Although we install our E1HTML Severs on port 81 and 82 we are setting single entry point to port 90. Oc4jMount point for /jde and /jde/* are then identified by point them to both OC4J instances (containers) preceded by node which they are installed on.


2. Once you have saved httpd.conf file with modification from step 1 restart the HTTP

Server.

Testing the vertical cluster and request load balancing Now that we have cluster 2 E1HTML Server instance, created single entry point and configured HTTP Server to load balance request for these two instances we need to test to make sure everything is working properly.


1. Open 2 internet browser windows and type in URL to our single entry point (port 90). Example (http://dengsswin01.mlab.jdedwards.com:90/jde/E1Menu.maf). If you get the login screen then our entry point was configured correctly. Login using valid EnterpriseOne user id and password and launch different application on each browser.

2. Next we need to make verify that requests are being balanced between 2 OC4J instances.

To do that, login to SM and go to ‘Managed Home and Managed Instance’ page. In the drop down box for Select View choose Search for User Resources’.

3. From ‘Search For User Activity’ page, type in user id that you used to access E1HTML

and click ‘Search’.


4. Verify that both OC4J instance is hosting a user session.

Horizontal Clustering In a horizontal cluster, installation of OAS 10.1.3.1.0 is done on multiple physical machines. With multiple nodes in hand we can cluster these nodes at Application Server level and/or at Application level.

Standalone Installation on Second Server If you have followed our example from the beginning, you should now have Oracle Application Server 10.1.3.1.0 (DENGSSWIN01) with 2 OC4J instances (EA_JS_81 and VC_JS_82) that are vertically clustered. Next, we need to complete another OC4J install on second physical machine (DENGSSWIN02) so we can configure horizontal cluster. For the interest of time and paper (if you were to print this documentation) I will not repeat those steps here. Instead I’ll ask you to go back and follow ‘Standalone Installation’ section of this paper to complete the install on the second physical server. For our example, following values were used during the installation on the second Server

DENGSSWIN01 DENGSSWIN02 Oracle Application Server Managed Instance Name

OAS1 OAS2

OC4J Instance/Container Name

EA_JS_81 VC_JS_82

HC_JS_81

E1HTML Server Managed Instance Name

EA_JS_81 VC_JS-82

HC_JS_81


1. Once install have been successfully complete, you should see your new Oracle

Application Server instance and E1HTML Server instance from ‘Managed Homes and Managed Instances’ page in SM.

2. Test to make sure you are able to login and that everything is working properly.

Furthermore, verify via SM EnterpriseOne HTML Server: HC_JS_81 page that ‘Instance Information’ menu show valid information. NOTE: At this point, from either Oracle Application Server OAS1 or Oracle Application Server OAS2 page in SM will only display J2EE containers that are installed within each physical server. Furthermore, you will have to access Oracle Application Control Console of each install separately (see screenshot below) since they are each standalone install of Oracle Application Server 10.1.3.1.0.


Configuring Dynamic Node Discovery Multicast discovery is a basic cluster configuration, in which each Oracle Notification Server (ONS) node broadcasts a simple multicast message announcing its presence, thereby enabling nodes within the cluster to dynamically "discover" one another. In this model, each ONS maintains its own map of the cluster topology. When a new ONS is added to the cluster, each existing ONS adds the new node and its connection information to its map. At the same time, the new ONS adds all of the existing nodes to its map. Alternately, when an ONS is removed from the cluster, the maps for the remaining nodes are updated with this change. The dynamic discovery configuration is set within a <discover> subelement of the <topology> element in the opmn.xml file on each Oracle Application Server instance in the topology. Dynamic nodes can also be configured with either Application Server Control Console (ASC) ‘Topology Network Configuration’ page or with opmnctl. In 10.1.3 opmnctl supports a new config topology command that allows you to specify, update, or delete the multicast <discovery> entry within this file. The update command inserts or updates the <discover> element with the specified value. We will set up Dynamic Node discovery using ASC ‘Topology Network Configuration’ page for DENGSSWIN01 and opmnctl command for DENGSSWIN02.


1. Login to ASC on DENGSSWIN01 and click the ‘Topology Network Configuration’ link

located on the bottom of the page.


2. From ‘Topology Network Configuration: E1JAS.DENGSWIN01.mlab.jdedwards.com page, select radio button for ‘Configuring Dynamic Node Discovery Using Multicast’ and enter valid value in the Discover field and click ‘Apply’. For our example, follow value was used; 225.0.20.8001. NOTE: The Multicast IP address must be in the range 224.0.1.0 – 239.255.255.255 and port must be a port that is not being used on this system.

3. Once the change has been applied screen will refresh to display confirmation message.

4. Behind the scene, this process will <topology> directive in opmn.xml file located under

<ORACLE_HOME>\opmn\conf directory.

5. To make sure that the latest change has been applied reload the opmn.xml file.

Open a Command Prompt from DENGSSWIN01 and navigate to <ORACLE_HOME>\opmn\bin directory and issue following command; opmnctl reload


6. Next, we need to configure our second Oracle Application Server (DENGSSWIN02).

Open a Command Prompt from DENGSSWIN02 and navigate to <ORACLE_HOME>\opmn\bin directory.

7. Issue following command to update the opmn.xml file on DENGSSWIN02

opmnctl config topology update discover=*225.0.0.20:8001


opmnctl reload

8. Login to ASC on either DENGSSWIN01 or DENGSSWIN02 and you should be able

see both Nodes under ‘Members’ section. If you do not see both Nodes under Member section verify that you have entered same Multicast IP Address and Port for both Nodes in ‘Topology Network Configuration’ page.


9. Subsequently, now from either Oracle Application Server OAS1 or Oracle Application Server OAS2 page you can view all the OC4J containers on both node and you can manage them (stop, start, and/or restart) from single location within SM.

Configuring Horizontal Clustering Now that we have working standalone installation (HC_JS_81) on our second Oracle Application Server and that Dynamic Node Discovery is properly configured to cluster these 2 physical servers (DENGSSWIN01 and DENGSSWIN02), we need to add our standalone install to working vertical cluster to form a horizontal cluster. This is same procedure that we used for vertical clustering. Only difference is that we are doing this for HC_JS_81, our E1HTML Server application server on DENGSSWIN02.

1. Login to ASC, it doesn’t matter which one. You can use ASC on DENGSSWIN01 or

DENGSSWIN02 since we have configured dynamic node discovery. Click on the link (HC_JS_81) for OC4J container that is hosting our E1HTML Server instance.


2. Click on the ‘Application’ link.

3. Click on the ‘HC_JS_81’ link for E1HMTL Server Application.


4. Click on the ‘Administration’ link.

5. From Application: HC_JS_81 page, click on the icon under Go to Task column for

Clustering Properties.


6. Select ‘Override parent application clustering settings’ and select Enable from drop

down box for Clustering. Under Replication Protocol, select Multicast IP Replication and fill-in same IP and port information (IP address 230.230.0.102 and Port number 45566) that we used during our vertical cluster setup. Click ‘OK’ to apply the change.

7. Your screen will refresh to display confirmation message when the process is completed.

Modify httpd.conf file for horizontal cluster Now that horizontal clustering has been configured, we need to update HTTP Server and add our next cluster instance for single entry point and request load balancing. NOTE: This section is not required if you are configuring Horizontal Cluster with Web Cache. Please refer to ‘Vertical and Horizontal Cluster with Web Cache’ section below for more detail.


1. Open httpd.conf file from DENGSSWIN01 under

<ORACLE_HOME>\Apache\Apache\conf and add our horizontal instance to the Oc4jMount point and Save the file. E1JASC.DENGSSWIN02.mlab.jdedwards.com:HC_JS_81

2. Restart HTTP Server on DENGSSWIN01.

NOTE: we are only using HTTP Server from DENGSSWIN01. You can also use DENGSSWIN02 instead of DENGSSWIN0 if you like; just make same modification to the httpd.conf for the HTTP Server on DENGSSWIN02. Alternatively, if you are using 3rd party clustering product you can also cluster HTTP Server for further redundancy.


Testing Horizontal Cluster and Load Balancing Now that we have horizontal clustering configured we need to test them to make sure that everything is working properly.

Bottom half of the screenshot above shows 3 accesses to E1HTML server using 3 separate browser windows. Top half show user resource information and you can see that load is evenly distributed among the entire cluster members.

Vertical and Horizontal Cluster with Web Cache The Oracle Web Cache can be used to cache static and dynamically generated web pages. Web Cache also provides load balancing and can route non-cached requests to a set of web servers providing protection when some of the servers are overloaded or become inaccessible. Web Cache component is not included in Oracle Application Server 10.1.3.x distribution and will have to be installed from Oracle Application Server 10.1.2.x distribution. Please check Minimum Technical Requirement site for supported release.


Please refer to Chapter 5 of JD Edwards EnterpriseOne Tools Release 8.97 HTML Web Server Reference Guides for Oracle Application Server 10.1.3.1 (http://www4.peoplesoft.com/contdoc.nsf/0/96337DD5940C20DA8825738B001BB6E9?OpenDocument) for detail instruction on how to install and configure Web Cache with JD Edwards EnterpriseOne HTML Server.

Configuring New Web Cache Listen Port By default Web Cache receives browser requests on Listen Port which is port 80. This will be entry point for your end user. For our example, we will create new Listen Port, port 90.

Login to ASC for OAS 10.1.2.x installation and verify that Web Cache component is enabled. (If not already enabled click on ‘Enable/Disable Components’ button and move Web Cache component to enable side.) Next, click on link for ‘Web Cache’.


From ‘Web Cache’ page, click the link for ‘Administration’

Next, click the link for ‘Ports’


This will bring up ‘Ports’ page. Click the ‘Add a row’ button and add required information for new Listen Port. For our example we created new Listen Port, port 90 and used * for IP address.

Configuring Origin Servers Origin servers are either application Web servers or proxy Web servers. Oracle Application Sever Web Cache typically sends cache misses to application Web servers for internal sites and proxy servers for external sites protected by a firewall. In our test case Origin Servers are all instances of EnterpriseOne HTML Servers that belongs to a cluster (vertical and/or horizontal).

To configure Origin Servers, from Administration page for ‘Web Cache’ click the link for ‘Origin Servers’.


From ‘Origin Server’ page, click the ‘Create’ button to add Origin Server.

Complete all require field. For our example we used following values which pointed to our first EnterpriseOne HTML Server, EA_JS_81. Host: dengsswin01.mlab.jdedwards.com Capacity: 100 Port: 81 Protocol: HTTP Failover Threshold: 5 Ping URL: /jde/E1Menu.maf Ping Frequency: 10 Click ‘OK’ button to create your Origin Server.


Repeat the steps to add all the instances in the cluster.

Configuring Site Server Site server is Web Cache Server. Site Server definition is comprised of Host and Port and this determines how requests are forwards to Origin Servers. This is also what end users will use as entry point to EnterpriseOne HTML Server. To create new Site Server login to ASC of Web Cache server and go to Administration page for ‘Web Cache’

From ‘Web Cache’ Administration page click on the link for ‘Sites’.


Create new Named Sites Definitions by click on the ‘Create’ button under ‘Named Sites Definitions’ section.

Enter Host name and Port number you would like to use to define entry point to Web Cache Server. For our example, we created new site definition using port 90. This was added using ‘Ports’ link under ‘Web Cache’ Administration page.


Next, select all the Origin Server that you would like to associate with this site. Web Cache will load balance incoming request among all Origin Servers associated to this site. NOTE: You have two options for Origin Server, Application Web Server and Proxy Web Server. Available Origin Server will be listed depending on how it was create during ‘Origin Server’ section.

Modify HTTP Server Configuration files After completing our Web Cache server configuration we need to modify the configuration files for our HTTP servers so that it can function properly with our Web Cache server. For each EnterpriseOne HTML Server install new scf_xxxx.conf gets created under <OAS_Home>\Apache\Apache\conf directory and is loaded during startup of Oracle HTTP Server. We need to add the port number that we defined for our Web Cache server, port 90.

We added few additional settings. These ‘Expires’ directives are not features of Web Cache but is highly recommended for EnterpriseOne HTML Server to increase performance.

Test Web Cache setup


Multiple JVMs for OC4J instance By default, OC4J instance execute on and uses single Java Virtual Machine (JVM). However, in OAS, you can configure an OC4J instance to run on multiple JVMs. On physical machine with additional hardware resources this will provide increase performance, by having single OC4J instance run on multiple processes. This option can be implemented on top of vertical and/or horizontal clustering.

Number of JVMs you specify should be based on hardware resources available. Miss configuration

here will have negative impact in regards to performance.

1. From ‘Managed Homes and Managed Instance’ page in SM click on the link for OAS

Server Instance (OAS1).


2. Locate OC4J instance name that you want to work with under J2EE Container section.

Under the column labeled JVM Processes specify number of JVMs you would like to

utilize for this OC4J instance and click ‘Save’. This process is dynamic and additional

process will start immediately.

3. Refresh the screen to verify start of additional JVM process and Process ID that are

associated to each of them.

4. You can also verify multiple JVM processes via ASC.


Trobleshooting

Internal Server Error

Receive Internal Server error when accessing my vertical or horizontal cluster after configuring httpd.conf file for my single entry point without using Web Cache. Answer: Make sure that upper and lower case of the application server name matches between httpd.conf and ASC

Page cannot be displayed

Received ‘Page cannot be displayed’ error message when accessing horizontal cluster using Web Cache Answer: First, check and see if you can get to that horizontal cluster instance from same browser outside of Web Cache by using URL.


If you cannot access via URL and you are using non-default port, such as 90, you might have to add the port to httpd.conf file of that Oracle HTTP Server (OHS) and restart.

Server Manager; Considerations and Limitations Currently Server Manager is not cluster aware. Therefore, when it comes to clustering certain administration task must be completed outside of Server Manager, such as utilizing Application Server Control (ASC).

Considerations Following task must be completed outside of Server Manager using Oracle Application Server Control console (ASC).

• Create new OC4J group

• Cluster Oracle Application Server install on multiple nodes

• Cluster E1 HTML server Following task should be completed using Server Manager.

• All other functionalities

• Start and stop E1 HMTL Server

• Start and stop Oracle HTTP server

• Update SM Agent

• Update E1 HTML Server tools release

• Utilize SAW functionalities

• Make ini changes

• Set number of JVM process for OC4J container

• Use Multiple JVM option instead of Vertical Clustering

Limitation List below describes current limitation within Server Manager.

• Cannot change Multiple JVM settings for horizontal instance from single Managed Instance

• After new E1 HTML tools release cluster is broken and will have to be manually configured again.

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

oracle.com

Copyright © 2006, Oracle. All rights reserved.

This document is provided for information purposes only and the

contents hereof are subject to change without notice.

This document is not warranted to be error-free, nor subject to any

other warranties or conditions, whether expressed orally or implied

in law, including implied warranties and conditions of merchantability

or fitness for a particular purpose. We specifically disclaim any

liability with respect to this document and no contractual obligations

are formed either directly or indirectly by this document. This document

may not be reproduced or transmitted in any form or by any means,

electronic or mechanical, for any purpose, without our prior written permission.

Oracle, JD Edwards, PeopleSoft, and Retek are registered trademarks of

Oracle Corporation and/or its affiliates. Other names may be trademarks

of their respective owners.

This document is for informational purposes only and may not be incorporated

into a contract or agreement.

o as Clustering

Documents

Transcript of o as Clustering