Data Load utility

Contents

1. Customizing the Data Load utility to encrypt data1.1. Creating a CSV file with user password information1.2. Configuring the Data Load utility1.3. Extending the generic table loader to encrypt user passwords1.4. Verifying your customization

2. Customizing the Data Load utility for XML input2.1. Creating an XML input file2.2. Creating a custom data reader2.3. Configuring the Data Load utility2.4. Verifying your customization2.5. Deploying your customized code

3. Loading data into custom tables3.1. Create the custom tables3.2. Creating an input file3.3. Configuring the environment settings3.4. Configuring the business object configuration file3.5. Configuring the load order configuration file3.6. Run the Data Load utility

Data Load utility tutorialsThe Data Load utility tutorials introduce the various tasks that are related to customizing the Data Load utility for WebSphere Commerce applications.

Tutorial Learning objectives Description

Tutorial: Customizing the Data Load utility toencrypt data After completing this tutorial, you should be able to:

Configure the Data Load utility.Extend the generic table loader.Encrypt data by using Java classes.

This tutorial demonstrates how to customize theData Load utility to load user and passwordinformation into the WebSphere Commerce databaseseparately.

Customizing the Data Load utility for XML inputAfter completing this tutorial, you should be able tocomplete the following tasks.

Create your XML data readerConfigure the Data Load utility to use your datareader and mediator

This tutorial demonstrates how to customize thedata load utility to load data from an XML file.

Tutorial: Loading data into custom tables with theData Load utility After completing this tutorial, you should be able to

complete the following tasks:

Configure the Data Load utility to use the tableobject builder and table object mediatorConfigure the table object builder to load datainto custom tables

This tutorial demonstrates how to use a table-basedbusiness object builder and mediator to load datainto extended or custom database tables.

1. Next >

Tutorial: Customizing the Data Load utility to encrypt data This tutorial demonstrates how to customize the Data Load utility to load user and password information into the WebSphere Commerce database

separately.

For security reasons, the Data Load utility does not load user password information from CSV files into the WebSphere Commerce database by default. After userinformation is loaded into the database from a CSV file, the value for the user passwords is set to NULL.

After user data is loaded to the WebSphere Commerce database, each user password is automatically reset the first time that a user attempts to logon to thestore. The password is reset regardless of the password that was entered at logon time, since the password in the database is null.

This automatic reset of the null password at logon time can be disabled by setting the following flag in the instance configuration file to false:

MemberSubSystem/ResetPassword/resetNullPasswordEnabled

This flag is enabled by default when the Foundation feature of WebSphere Commerce 7 Feature Pack 3 is enabled. An email is sent to the user with a temporarypassword that they can use to log on to the store and then change the password.

This tutorial shows you how to handle user password information by customizing the Data Load utility to load user passwords separately from the users.

Learning objectives

After completing this tutorial, you should be able to:

Configure the Data Load utility.Extend the generic table loader.Encrypt data by using Java classes.

Time requiredExpect this tutorial to take 1 hour to complete.

Skill levelIntermediate

AudienceThis tutorial is intended for WebSphere Commerce developers who want to extend the generic table loader to load customized data.

System requirementsBefore you begin this tutorial, ensure that the following tasks are complete:

Member data is loaded into the database.Password input (plain message text) is obtained from the external system by using data readers.Passwords are stored in a CSV file.

PrerequisitesTo complete this tutorial, ensure that you have a basic understanding of the following concepts:

Data Load utility architecture.Password encryption.Generic table loader with extension table or column handler.

Next >

< Previous | 1.1. Next >

Creating a CSV file with user password information In this lesson, you create a new CSV file that contains the user password information to load into the WebSphere Commerce database.

Note: This tutorial demonstrates how to write a custom column handler and encrypt a clear text password by using the WCPasswordEncrypter class. It assumesthat customers obtain password input (plain message text) from the external system by using their own encryption algorithm for password encryption anddecryption. If you do not want to define clear text password information in a CSV file, you can use an alternative method to define user password information:

Export the user logon ID and encrypted password in the CSV file. Modify the PasswordHandler tutorial code to decrypt the encrypted user password beforethe WCPasswordEncrypter class is called to encrypt the user password information.Create a custom data reader to read the user logon ID and password directly from the external system. By reading the ID and password directly you canensure that no clear text or encrypted passwords are saved in a CSV file. Use the custom data reader to decrypt the user password by using your ownalgorithm. By combining the custom data reader with the customized PasswordHandler, you can securely load the user password into the database.

Sample data is provided in the WCDE_installdir\components\foundation\samples\Dataload\Member\Person directory. In this step, you load the sample

customer data into the USERREG database table. Then, load the corresponding password information into the database table separately after the customizationis complete.

Procedure

1. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server,see Starting and stopping WebSphere Commerce Test Server.

2. To load the sample customer data into the USERREG database table, enter the following command:

./dataload.sh ../components/foundation/samples/DataLoad/Member/Person/wc-dataload.xml

dataload ..\components\foundation\samples\DataLoad\Member\Person\wc-dataload.xml

3. Verify the results of the data load:

a. Review the Data Load utility summary report. For more information, see Verifying the results of the data load.

b. Query the USERREG database table to verify that the data for the registered customers is loaded into the table.

Note: The values for the following columns are set to null:

LOGONPASSWORDSALT

After the customization is complete, the encrypted data for these columns is loaded into the database table.

4. Create the WCDE_installdir\components\foundation\samples\Dataload\Member\Password directory.

5. In this directory, create the MemberDataload_Password.csv file.

6. Copy the following content into your MemberDataload_Password.csv file:

logonId logonPassword

jacobsmith shopper1

ryanjohnson shopper2

olivialee shopper3

This CSV file provides the user passwords for three different customers. The encrypted data for these passwords is loaded into the USERREG databasetable after the customization is complete.

7. Save and close the file.

< Previous | Next >

< Previous | 1.2. Next >

Configuring the Data Load utility In this lesson, you configure the Data Load utility environment, business object, and load order configuration files.

Environment configuration fileUse to specify your business context information, database connection property, ID Resolver, and data writer class. For more information about this

file, see wc-dataload-env.xsd.

Business object configuration file

Use to specify your data reader, business object builder, and business object mediator. For more information about this file, see wc-dataload-

businessobject.xsd.

Load order configuration fileUse to specify a pointer to your CSV input file, environment variable settings file, and business object configuration file. You can also specify which

data load mode you want to perform. For more information about this file, see wc-dataload.xsd.

Procedure

1. Configure the environment variable settings that are used by the Data Load utility within an environment configuration file:

a. Create an XML file named wc-dataload-env.xml in the WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

b. Open the wc-dataload-env.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadEnvConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../xml/config/xsd/wc-dataload-env.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

<!-- Madisons Store in Toolkit -->

1 <_config:BusinessContext storeIdentifier="Madisons"

languageId="-1" currency="USD">

</_config:BusinessContext>

<!-- database setting for derby in Toolkit -->

2 <_config:Database type="derby" name="..\db\mall" schema="APP"/>

<!— IDResolver class -->

<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="10000000"/>

<!— DataWriter class -->

3 <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

</_config:DataLoadEnvConfiguration>

c. Update the attribute values in the following elements within the file:

Element Required update Example

1

<_config:BusinessContext>

Specify your storesettings. <_config:BusinessContext storeIdentifier="Madisons" catalogIdentifier="Madisons" />

2 <_config:Database> Specify your databaseenvironment properties. <_config:Database name="..\db\mall" type="derby" />

3 <_config:DataWriter> Specify your data writerclass. <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

For more information about data load environment settings, see Configuring the data load environment settings.

2. Specify your data reader, business object builder, and business object mediator that is used by the Data Load utility:

a. Create a business object configuration file named wc-Passwordobject-loader.xml in the

WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

b. Open the wc-Passwordobject-loader.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataloadBusinessObjectConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload-businessobject.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

<_config:DataLoader className="com.ibm.commerce.foundation.dataload.BusinessObjectLoader">

<!-- Define data reader class to read the input from CSV file for loading -->

1 <_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" >

</_config:DataReader>

2 <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

<!-- exclude the columns whose values are loaded previously from USERREG table except LOGONPASSWORD & SALT -->

<_config:Table name="USERREG" excludeUnListedColumns="true">

<_config:Column name="USERS_ID" value="USERS_ID" valueFrom="IDResolve">

<_config:IDResolve tableName="USERREG" generateNewKey="false" >

<_config:UniqueIndexColumn name="LOGONID" value="logonId" />

</_config:IDResolve>

</_config:Column>

<_config:Column name="LOGONID" value="logonId" />

<!-- The value of the SALT will be generated and added back to the userreg table later -->

<_config:Column name="SALT" value="" />

<!-- Plain password read from CSV file will be encrypted by the customized class PasswordEncryptionhandler -->

<_config:Column name="LOGONPASSWORD" value="LOGONPASSWORD" valueFrom="ColumnHandler" >

<_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler">

<!-- The following parameters will be passed to the class PasswordEncryptionhandler -->

<_config:Parameter name="logonPassword" value="logonPassword" />

<!-- WC-server.xml is the file to store the merchant key -->

<_config:Parameter name="instanceConfigLocation" value="c:\IBM\WCDE_ENT70\workspace\WC\xml\config\wc-server.xml" valueFrom="Fixed" />

<!-- Customer may choose the merchant key in customKeyConfigLocation

<_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" /> -->

</_config:ColumnHandler>

</_config:Column>

</_config:Table>

3 <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" componentId="com.ibm.commerce.member" >

</_config:BusinessObjectMediator>

</_config:BusinessObjectBuilder>

</_config:DataLoader>

</_config:DataloadBusinessObjectConfiguration>

c. Update the attribute values in the following elements within the file:

Element Required update Example

1 <_config:DataReader> Specify the datareader class thatyou implement.

<_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" >

</_config:DataReader>

2

<_config:BusinessObjectBuilder>

Specify the businessobject builder classthat you implement.

Note: Specify theTableObjectBuilderclass in this tutorialsince you extendthe generic tableloader to load thecustomized datainto the database.

The table objectbuilder is anextension of abusiness objectbuilder for dataload. It acceptsdata that is read bythe data reader andconstructs valueobjects to representphysical tables inthe database.

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

3

<_config:BusinessObjectMediator>

Specify the businessobject mediatorclass.

Note: Specify theTableObjectMediatorclass in this tutorialsince you extendthe generic tableloader to load thecustomized datainto the databaseinstead of extendingthe business objectlogic.

<_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" componentId="com.ibm.commerce.member" >

</_config:BusinessObjectMediator>

d. Set the excludeUnListedColumns parameter to true to exclude database columns that were previously loaded to the USERREG database table during

the initial load:

<_config:Table name="USERREG" excludeUnListedColumns="true">

When a column is excluded, the value in that column is not modified during the data load.

The following code snippet shows the parameter values for the columns from the USERREG database table. Any columns that are defined in thedatabase table (such as STATUS and PLCYACCT_ID) that are not included in this snippet were already loaded into the database during the initialload.

<_config:Column name="USERS_ID" 1 value="USERS_ID" valueFrom="IDResolve">

<_config:IDResolve tableName="USERREG" generateNewKey="false" >

<_config:UniqueIndexColumn name="LOGONID" value="logonId" />

</_config:IDResolve>

</_config:Column>

<_config:Column name="LOGONID" 2 value="logonId" />

<!-- The value of the SALT will be generated and added back to the userreg table later -->

<_config:Column name="SALT" 3 value="" />

<!-- Plain password read from CSV file will be encrypted by the customized class PasswordEncryptionhandler -->

<_config:Column name="LOGONPASSWORD" 4 value="LOGONPASSWORD" valueFrom="ColumnHandler" >

<_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler">

<!-- The following parameters will be passed to the class PasswordEncryptionhandler -->

<_config:Parameter name="logonPassword" value="logonPassword" />

<!-- WC-server.xml is the file to store the merchant key -->

<_config:Parameter name="instanceConfigLocation" value="c:\IBM\WCDE_ENT70\workspace\WC\xml\config\wc-server.xml" valueFrom="Fixed" />

</_config:ColumnHandler>

</_config:Column>

Note: Customer also can choose to store their merchant key in their own customKeyConfigLocation, the usage example is provided below as well.

<!-- Customer may choose the merchant key in customKeyConfigLocation:

<_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" />

1 USERS_IDThe value for this parameter is retrieved from the IDRESOLVE column of the USERREG database table, according to the LOGONID input.

2 LOGONIDThe value for this parameter is read from the LOGONID column that is defined in the CSV input file.

3 SALTThe value for this parameter is generated by the generateSalt method and then added to the USERREG database table later. Specify thecolumn for SALT here so that the corresponding salt value is written into the database later. The excludeUnlistedColumns parameter is setto true to specify that all of the columns that are not listed in the preceding code snippet are excluded and are not written to thedatabase.

4 LOGONPASSWORDThe value for this parameter is read from the ColumnHandler interface. A new class that is called PasswordEncryptionhandler will becreated as an extension class of AbstractColumnHandler in the next step of this tutorial. This class is used to generate the value for theSALT parameter and encrypt the user password. This value is achieved by taking the user password from the CSV input file and the

merchant key from the wc-server.xml file. Parameters such as logonPassword and instanceConfigLocation are passed to the new

PasswordEncryptionhandler class.

3. Specify the data load order and mode that the Data Load utility uses to load data:

a. Create an XML file named wc-dataload.xml in the WCDE_installdir\components\foundation\samples\DataLoad\Member\Password directory.

b. Open the wc-dataload.xml file and paste the following content into the file:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

1 <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

2 <_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace">

<!-- Dataload password -->

<_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml">

<!--Indicates the location of input csv files, can be overrided at runtime.-->

<_config:DataSourceLocation location="MemberDataload_Password.csv" />

</_config:LoadItem>

</_config:LoadOrder>

</_config:DataLoadConfiguration>

c. Update the attribute values in the following elements within the file:

Element Required update Example

1

<_config:DataLoadEnvironment>

Specify your environmentvariable file. <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

2 <_config:LoadOrder> Specify your CSV inputfile, business objectconfiguration file, and dataload mode.

<_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace">

<!-- Dataload password -->

<_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml">

<!--Indicates the location of input csv files, can be overrided at runtime.-->

<_config:DataSourceLocation location="MemberDataload_Password.csv" />

</_config:LoadItem>

</_config:LoadOrder>

For more information about configuring your data load order, see Configuring the data load order.

< Previous | Next >

< Previous | 1.3. Next >

Extending the generic table loader to encrypt user passwords In this lesson, you create a new Java class to implement the ColumnHandler interface to encrypt user password data.

Create a custom column handler to resolve data from database columns that are based on input values that cannot be mapped to the column value through theconfiguration. For example, if your input to populate the password column in the database is a clear text password, but the column in the database table

expects an encrypted value, create a custom column handler to encrypt the clear text password and return an encrypted password from the resolveColumnValue()

method.

Procedure

1. Open WebSphere Commerce Developer.

2. Switch to the Java EE perspective.

3. In the Enterprise Explorer view, navigate to WebSphereCommerceServerExtensionsLogic > src.

4. Right-click src; then click New > Package.

5. In the Name field, type com.mycompany.commerce.dataload.password. Click Finish to create a new package.

6. Right-click com.mycompany.commerce.dataload.password; then click New > Class.

7. In the Name field, type PasswordEncryptionHandler.

8. From the Superclass field, click Browse to browse for the AbstractColumnHandler abstract class; then click OK.

9. Click Finish to create the class. A new file opens for editing.

10. Paste the following code snippet into the new file:

package com.mycompany.commerce.dataload.password;

import java.util.Map;

import com.ibm.commerce.foundation.common.util.logging.LoggingHelper;

import com.ibm.commerce.foundation.dataload.config.AbstractColumnHandler;

import com.ibm.commerce.foundation.dataload.exception.DataLoadException;

import com.ibm.commerce.foundation.dataload.object.ExtendedTableDataObject;

import com.ibm.commerce.member.dataload.helper.MemberDataLoadHelper;

import com.ibm.commerce.util.WCPasswordEncrypter;

/**

*The main purpose of the PasswordEncryptionHandler is to resolve the column value based

* on the input values which cannot be simply mapped to the column value

* through the configuration. For example, to populate the password column in the

* database, your input is a clear text password, but the column in the database expects

* an encrypted value, you need to write a custom handler to encrypt the clear

* password and return an encrypted password from the method resolveColumnValue().

*/

public class PasswordEncryptionHandler extends AbstractColumnHandler {

private final static String CLASSNAME = PasswordEncryptionHandler.class.getName();

private static final java.util.logging.Logger LOGGER = LoggingHelper.getLogger(PasswordEncryptionHandler.class);

private static final String CUSTOM_KEY_CONFIG_LOCATION = "customKeyConfigLocation";

private static final String INSTANCE_CONFIG_LOCATION = "instanceConfigLocation";

/**

* Resolve the column value based on the input parameters passed in.

* The parameter map will contain the map from LOGONID to the value of logonId.

* The extenededTableDataObject represents the row of the table.

* @param parameterMap a parameter map which are name-value pairs

* defined in the configuration.

* @param extenededTableDataObject the current table data object which contains

* the table configuration and some column values which have been

* resolved so far.

* @return resolved new column value for the encrypted password

* @throws DataLoadException if there are some errors to resolve the value.

*/

public String resolveColumnValue(Map<String, String> parameterMap,

ExtendedTableDataObject extendedTableDataObject) throws DataLoadException {

// TODO Auto-generated method stub

final String METHODNAME = "resolveColumnValue";

/* Enable the log trace for the method */

if (LoggingHelper.isEntryExitTraceEnabled(LOGGER)){

LOGGER.entering(CLASSNAME, METHODNAME);

}

/* Call API to get the salt value */

String salt = MemberDataLoadHelper.generateSalt(); 1

/* Store the salt value back into the USREEG table */

extendedTableDataObject.addColumnData 2 ("SALT", salt);

String password = parameterMap.get("logonPassword");

String InstanceconfigLocation = parameterMap.get(INSTANCE_CONFIG_LOCATION);

String customKeyConfigLocation = parameterMap.get(CUSTOM_KEY_CONFIG_LOCATION);

String encryptedPWD = null;

try {

WCPasswordEncrypter encrypter = new WCPasswordEncrypter();

/* Provide the file location which the merchant key is stored in so that to initialize the Merchant key registration */

encrypter.initializeKeyRegistry 3 (customKeyConfigLocation, InstanceconfigLocation);

/*Encrypt the plain text password */

encryptedPWD = encrypter.encryptPassword 4 (password, salt);

} catch (Exception e) {

// TODO Auto-generated catch block

e.printStackTrace();

}

if (LoggingHelper.isEntryExitTraceEnabled(LOGGER)) {

LOGGER.exiting(CLASSNAME, METHODNAME);

}

/* return the encrypted password */

return encryptedPWD ;

}

}

1 generateSalt()Generates a random salt string with a length of 12 characters.

2 extendedTableDataObject.addColumnDataExtends TableDataObject to include more ID resolver information for certain database table columns. This class represents a row in a physicaldatabase table such as USERREG. addColumnData takes the column name and value as input for Salt and adds the value back to the SALTcolumn in the USERREG database table.

3 initializeKeyRegistryInitializes the key registry by providing one of the following parameters:

customKeyConfigFilenameSpecifies the full path to the custom key configuration file when the merchant key is stored in an external medium.

instanceXMLFilenameSpecifies the full path to the instance configuration XML file when the merchant key is stored within the instance configuration file.

If both parameters are specified, an initial attempt is made to initialize the key registry with the custom key configuration file. If that fails, anattempt is made to initialize the key registry with the instance configuration file.

4 encryptPasswordReturns the WebSphere Commerce encrypted password with the given clear text password and salt. The caller of this method must first initialize

the key registry by calling the initializeKeyRegistry(String instanceXMLFilename, String customKeyConfigFilename) method.

< Previous | Next >

< Previous

Verifying your customization In this step of the tutorial, you verify your customization by running the Data Load utility in WebSphere Commerce Developer.

Procedure

1. Open WebSphere Commerce Developer.

2. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server,see Starting and stopping WebSphere Commerce Test Server.

3. Click Run > Run Configurations. The Run Configurations dialog displays.

4. Create a new Java Application configuration. Double-click the Java Application on the left panel; then on the Main tab, enter the following information:

a. In the Name field, type a name for your configuration.

b. In the Project field, enter the project WebSphereCommerceServerExtensionsLogic.

c. In the Main class field, enter the class name com.ibm.commerce.foundation.dataload.DataLoaderMain.

5. Click the Arguments tab; then enter the following information:

a. In the Program arguments field, enter ..\samples\DataLoad\Member\Password\wc-dataload.xml.

b. In the VM arguments field, set the following parameters

-Dj2se

Sets the Java platform type to J2SE. Set the value for the parameter to true.

-Djava.util.logging.config.file

Sets the JVM parameter. Set the value of the parameter to ..\workspace\WC\xml\config\dataload\logging.properties.

c. In the Working directory field, enter the WCDE_installdir/bin directory.

6. Click the Classpath tab; then click User Entries and add the following class path information:

a. Click Add External JARs, then browse to the location of the JDBC driver JAR file for the specific database you are using. Click Open.

WCDE_installdir\lib\derby.jar

db2_installdir\java\db2jcc4.jar

oracle_installdir\jdbc\lib\ojdbc6.jar

b. Click Advanced > Add Folder, then click OK. Navigate to WC > xml, then click OK.

7. Click Run.

8. Verify your results by running the following SQL statement:

Select * from userreg

The USERREG database table displays the content that you specified in your CSV file.

9. Open your store in an internet browser.

10. Click Sign In and log on to the store with the following user credentials:

User name Password

jacobsmith shopper1

This user information is defined in the MemberDataload_Password.csv file and loaded into the WebSphere Commerce database.

< Previous

2. Next >

Customizing the Data Load utility for XML inputThis tutorial demonstrates how to customize the data load utility to load data from an XML file.

Note: Support for loading XML formatted files with the Data Load utility is provided by default. You do not have to complete this customization to useXML formatted input files with the Data Load utility. For more information about the supported format for Data Load utility input files, see File format for DataLoad input files.

If your XML files use a different format than the format that is supported by default, you can use this tutorial to help you create an XML reader that supportsthe XML format of your files.

In the tutorial "Adding new properties to a WebSphere Commerce service by using the data service layer", you customize the catalog service. The customizationadds support warranty information and care instructions for catalog entries. Now, you load the warranty and care instruction information into your targetdatabase by using the Data Load utility.

The following diagram shows the data load process flow.

1. The data reader reads the input file, and transforms the input into a data object. Data objects are typically Map objects.2. The business object builder instantiates the business object and populates it from the data in the data object.3. The business object mediator converts the business object into physical object.4. The data writer writes the physical object into the database.

The provided CSV reader reads the input files and creates Map objects. XML files are difficult to represent in a Map object because they have complexstructures. In this scenario, you create a reader that builds the logical object directly from the input XML file. Since this reader is creating a logical object, thedefault business object builder is used. The default business object builder returns the input data object without changing it.

Learning objectives

After completing this tutorial, you should be able to complete the following tasks.

Create your XML data readerConfigure the Data Load utility to use your data reader and mediator

Time requiredThis tutorial is expected to take 1 hour to complete.

Skill levelIntermediate

AudienceThis tutorial is intended for WebSphere Commerce developers who want to customize the data load utility to read from XML files.

System requirementsBefore you begin this tutorial, ensure that you complete the following prerequisites:

Completed the following steps in:Tutorial: Extending a BOD service to manage UserData with the Data Service layer

Customizing the WebSphere Commerce schemaCustomizing the physical layer to include the new information

Prerequisites

Saving the sample code

Before you begin this tutorial, download the DataLoadSample.zip. Save this file onto your development machine. As an example, you might save it into the

WCDE_installdir directory.

Knowledge prerequisites

Before you begin this tutorial, ensure that you have:

A basic understanding of the Data Load utility architectural overviewA basic understanding of DOM

An understanding of Service Data Objects (SDO)

Next >

< Previous | 2.1. Next >

Creating an XML input file

In this lesson, you create an XML file to contain the catalog entry, warranty, and care instruction data to load with the Data Load utility.

The XML file is based on the data that is created in the Tutorial: Extending a BOD service to manage UserData with the Data Service layer. This file containscatalog entry base and description information, warranty information, and care instruction information, which extends the catalog entry object.

Procedure

1. Create the WCDE_installdir\samples\Dataload\warranty\ directory.

2. Create a warranty.xml file in the WCDE_installdir\samples\Dataload\warranty\ directory.

3. Copy the following content into your warranty.xml file.

<?xml version="1.0" encoding="utf-8"?>

<customInfo>

<customProductInfo partNumber="MW-0003" Type="ProductBean">

<warranty warrantyTerm="30" warrantyType="LIMITED"/>

<attributes>

<attribute name="manufacturerPartNumber" value="" />

<attribute name="manufacturer" value="MapleWear" />

<attribute name="url" value="" />

<attribute name="field1" value="1" />

<attribute name="field2" value="1" />

<attribute name="field3" value="1" />

<attribute name="field4" value="" />

<attribute name="field5" value="" />

<attribute name="onSpecial" value="1" />

<attribute name="onAuction" value="1" />

<attribute name="buyable" value="1" />

</attributes>

<description language="-1" Name="Striped turtleneck"

ShortDescription="Soft, stretchy rib-knit turtleneck."

LongDescription="This soft, stretchy rib-knit turtleneck is perfect either on its own or layered under a jumper."

Thumbnail="images/catalog/apparel/apparel_70x70/IMG_0003_e.jpg"

FullImage="images/catalog/apparel/apparel_160x160/IMG_0003_e.jpg"

Keyword="">

<attribute name="careinstruction" value="Avoid soap and water"/>

<attribute name="auxDescription1" value=""/>

<attribute name="auxDescription2" value=""/>

<attribute name="available" value="1"/>

<attribute name="availabilityDate" value="2008-09-03 16:00:00"/>

<attribute name="published" value="1"/>

</description>

</customProductInfo>

</customInfo>

The customProductInfo element maps to a catalog entry noun, while the description element maps to a catalog entry description noun part.

The data load utility supports creating, replacing, or deleting data objects in the database table. To replace data objects in the database, ensure that yousupply all attributes. In the preceding example, all catalog entry information is provided.

4. Save and close the file.

< Previous | Next >

< Previous | 2.2. Next >

Creating a custom data reader

In this lesson, you create a custom data reader to read your XML input file and then transform the input data into business objects.

The Data Load utility already provides you with a CSV data reader, which is implemented by the CSVReader class. It is necessary to create custom data readersin order to support data source of types other than CSV files. This tutorial shows you how to create a custom data reader to support the XML file that you

created (warranty.xml). A DOM parser is used to parse the XML file to a DOM tree, and then build a commerce business object.

To create a custom data reader, you need to implement the DataReader interface. The AbstractDataReader class implements most methods in the DataReader

interface. The custom data reader class can extend this class and provide the additional implementation for the next(), getSourcePosition(), and init() methods.

The following Java classes are provided:

WarrantyReader.java: This class implements the default behavior of the DataReader interface. It reads the XML input file, and then creates the business

object.

WarrantyConstants.java: This class defines the constants that are used in parsing the XML file.

Procedure

1. Extract the DataLoadSample.zip file to a temporary directory on your machine.

2. Open WebSphere Commerce Developer.

3. In the Enterprise explorer view, right-click the WebSphereCommerceServerExtensionsLogic folder and click New > Package.

4. In the Name field, enter com.mycompany.commerce.dataload.reader and click Finish.

5. Right-click the com.mycompany.commerce.dataload.reader package, and click Import > General > File system.

6. Click Next; then click Browse and navigate to the Dataload_Temp directory, where Dataload_Temp is the location where you extracted the sample Java

files. Select the WarrantyReader.java file and WarrantyConstants.java file. Click Finish to import the files.

7. Open the WarrantyReader.java file and examine its content.

The following methods are implemented:

init(): This method initializes the data reader. It gets the input XML file location from the configuration file, and then reads the XML file into the DOM

tree object.

next(): This method parses the XML DOM tree object, builds the next logical object, and encapsulates it within a DataLoadBusinessObject. It then

returns the DataLoadBusinessObject.

The returned data is passed to the business object builder to generate business objects. Each business object is passed to the correspondingbusiness object mediator to process the data.

In this tutorial, a business object builder is not needed because the data reader is creating the business objects, which are encapsulated in the

DataLoadBusinessObject.

For example, the following code snippets generate SDO for the attribute part number:

// Get partNumber

attrvalue = ((Element) node)

.getAttribute(WarrantyConstants.PART_NUMBER);

if (attrvalue != null) {

attrvalue = attrvalue.trim();

}

if (LOGGER.isLoggable(Level.FINER)) {

LOGGER.logp(Level.FINER, CLASSNAME, METHODNAME,

WarrantyConstants.PART_NUMBER + ":" + attrvalue);

}

CatalogEntryType catentry = CatalogFactory.eINSTANCE

.createCatalogEntryType();

CatalogEntryIdentifierType identifier = CommerceFoundationFactory.eINSTANCE

.createCatalogEntryIdentifierType();

PartNumberIdentifierType partNumber = CommerceFoundationFactory.eINSTANCE

.createPartNumberIdentifierType();

identifier.setExternalIdentifier(partNumber);

catentry.setCatalogEntryIdentifier(identifier);

partNumber.setPartNumber(attrvalue);

If you want to know more about generating SDOs, see Service Data Objects (SDO).

getSourcePosition(): This method returns the source position of the data object returned by the next() method last called.

parseWarranty(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the warranty element and its

subelements. It then populates the appropriate attributes of the logical object.

parseDescription(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the description element and its

subelements. It then populates the appropriate attributes of the logical object.

parseCatentryAttribute(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the CatentryAttribute

element and its subelements. It then populates the appropriate attributes of the logical object.

< Previous | Next >

< Previous | 2.3. Next >

Configuring the Data Load utility

In this step of the tutorial, you configure the environment settings, load order, and the business object configuration file.

Procedure

1. Create your data load environment configuration file. In the environment configuration file, specify the business context information, database connectionproperty, ID Resolver, and data writer class. For more information about creating this file, see Configuring the data load environment settings. Theenvironment variables that are used by the Data Load utility must be configured in this XML file.

a. Copy the sample wc-dataload-env.xml environment configuration file into the WCDE_installdir\samples\DataLoad\warranty directory. The sample

environment configuration file is located within the DataLoadSample.zip compressed file.

b. Open the wc-dataload-env.xml file for editing.

Change the storeIdentifier, and catalogIdentifier attribute values for the <_config:BusinessContext> configuration element to match your store

settings. For example,

<_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="Aurora" />

Change the name, and type attribute values for the <_config:Database> configuration element to match your database settings. For example,

<_config:Database name="..\db\mall" type="derby" />

Change the classname attribute value for the <_config:DataWriter> configuration element to specify the data writer class. By default the Data

Load utility uses the com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter data writer class. For example,

<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

c. Save and close the file.

2. Create your business object configuration file. In the business object configuration file, you specify your data reader, business object builder, and businessobject mediator. For more information, see Configuring the business object configuration file.

a. Copy the sample wc-warrantyobject-loader.xml business object configuration file into the WCDE_installdir\samples\DataLoad\warranty directory. The

sample configuration file is located within the DataLoadSample.zip compressed file.

b. Open the wc-warrantyobject-loader.xml for editing.

Change the classname attribute value for the <_config:DataReader> element to specify the data reader class that you implemented. For example,

<_config:DataReader className="com.mycompany.commerce.dataload.reader.WarrantyReader" />

Change the classname attribute value for the <_config:BusinessObjectBuilder> element to specify the business object builder class that you

implemented. For example,

<_config:BusinessObjectBuilder className="" >

Change the className and componentId attribute values for the <_config:BusinessObjectMediator> element to specify the business object

mediator. In this tutorial, the warranty and care instruction data are extension of the catalog entry business object. Specify theCatalogEntryMediator class and componentId. For example,

<_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator" componentId="com.ibm.commerce.catalog"/>

c. Save and close the file.

3. Configure the data load order configuration file. In the load order configuration file, you specify a pointer to your XML input file, environment settings file,and business object configuration file. You also specify which data load mode that the utility is to run. For more information, see Configuring the data loadorder.

a. Copy the sample wc-dataload-warranty.xml load order configuration file into the WCDE_installdir\samples\DataLoad\warranty directory. The sample

configuration file is located within the DataLoadSample.zip compressed file.

b. Open the wc-dataload-warranty.xml file for editing

Change the configFile attribute value for the <_config:DataLoadEnvironment> element to specify the environment configuration file. For example,

<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

Within the <_config:LoadOrder > element, configure the <_config:LoadItem> and <_config:DataSourceLocation elements. Change the commitCount,

batchSize, dataLoadMode, name, and businessObjectConfigFile attribute values. For example,

<_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantyobject-loader.xml">

</_config:LoadItem>

Within the <_config:DataSourceLocation element, change the location attribute value to specify the input file. For example, <_config:LoadOrder >

<_config:LoadOrder >

<_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantyobject-loader.xml">

<_config:DataSourceLocation location="warranty.xml"/>

</_config:LoadItem>

</_config:LoadOrder>

c. Save and close the file.

In the workspace_dir\WC\xml\config\dataload\logging.properties, you can specify the trace level, and enable console and file logging. The following settings are

used by default:

Logging level is set to both console and file.

Trace level is set to INFO.

Log file location is set to WCDE_installdir\logs\wc-dataload.log.

You do not need to modify the log file location. However, you might want to change the trace level when you are debugging a data load problem. The Data Load

utility provides some command line options to change the trace level. For more information, see Data Load utility.

If any errors occur, an error log file is generated per load item. The error file name and location is specified in the summary report on the console. The error log

file is generated in the same directory as the wc-dataload.log file.

< Previous | Next >

< Previous | 2.4. Next >

Verifying your customization

In this lesson, you verify your customization by running the Data Load utility in the development environment.

Procedure

1. Open WebSphere Commerce Developer.

2. Ensure that the WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping the server,see Starting and stopping WebSphere Commerce Test Server.

3. Click Run > Run Configurations. The Run Configurations dialog displays.

4. Create a new Java Application configuration. Double-click the Java Application on the left panel; then on the Main tab, enter the following information:

a. In the Name field, type a name for your configuration.

b. In the Project field, enter the project WebSphereCommerceServerExtensionsLogic.

c. In the Main class field, enter the class name com.ibm.commerce.foundation.dataload.DataLoaderMain.

5. Click the Arguments tab; then enter the following information:

a. In the Program arguments field, enter ..\samples\DataLoad\warranty\wc-dataload-warranty.xml.

b. In the VM arguments field, set the following parameter

-Dj2se

Sets the Java platform type to J2SE. Set the value of the parameter to true.

-Djava.util.logging.config.file

Sets the JVM parameter. Set the value of the parameter to ..\workspace\WC\xml\config\dataload\logging.properties.

c. In the Working directory field, enter the WCDE_installdir/bin directory.

6. Click the Classpath tab; click User Entries, then add the following class path information:

a. Click Add External JARs, then browse to the location of the JDBC driver JAR file for the specific database you areusing. Click Open.

WCDE_installdir\lib\derby.jar

db2_installdir\java\db2jcc4.jar

oracle_installdir\jdbc\lib\ojdbc6.jar

b. Click Add JARs, then browse to the location of the Inventory-Server.jar and Price-Server.jar files in the WCDE_installdir/WC directory.

c. Click Advanced > Add Folder, then click OK. Navigate to WC > xml, then click OK.

7. Click Run.

8. Verify your results by running the following SQL statements:

SELECT * FROM XWARRANTY;

SELECT * FROM XCAREINSTRUCTION;

You see the warranty and care information content you specified in your XML file.

< Previous | Next >

< Previous

Deploying your customized code

In this lesson, you deploy your custom code to a target WebSphere Commerce Server. Deploying to the server involves transferring the code from theWebSphere Commerce development environment to the target WebSphere Commerce Server.

Procedure

1. Create the XWARRANTY and XCAREINSTRUCTION tables in the database that is used by your target WebSphere Commerce Server:

a. Connect to your DB2 database.b. Run the SQL statement to create the tables.

CREATE TABLE XWARRANTY

(

CATENTRY_ID BIGINT NOT NULL,

WARTERM INTEGER,

WARTYPE VARCHAR(32),

OPTCOUNTER SMALLINT,

CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID),

CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID)

);

CREATE TABLE XCAREINSTRUCTION

(

CATENTRY_ID BIGINT NOT NULL,

LANGUAGE_ID INTEGER NOT NULL,

CAREINSTRUCTION VARCHAR(254),

OPTCOUNTER SMALLINT,

CONSTRAINT XCAREINST_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINST_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINST_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID)

);

a. Connect to your Oracle database.b. Run the SQL statement to create the tables.

CREATE TABLE XWARRANTY

(

CATENTRY_ID NUMBER NOT NULL,

WARTERM INTEGER,

WARTYPE VARCHAR(32),

OPTCOUNTER SMALLINT,

CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID),

CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID)

);

CREATE TABLE XCAREINSTRUCTION (

CATENTRY_ID NUMBER NOT NULL,

LANGUAGE_ID INTEGER NOT NULL,

CAREINSTRUCTION VARCHAR(254),

OPTCOUNTER SMALLINT,

CONSTRAINT XCAREINSTRUCTION_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINSTRUCTION_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINSTRUCTION_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID)

);

2. Create the WC_installdir\instances\instance_name\xml\config\com.ibm.commerce.catalog-ext directory if the directory does not exist.

3. Copy all the custom Data Service Layer configuration files from the WebSphere Commerce development environment to the WebSphere Commerce serverinstance directory.

a. Navigate to the workspace_dir\WC\xml\config\com.ibm.commerce.catalog-ext directory.

b. Copy all of the custom Data Service Layer configuration files within this directory.

c. Navigate to the WC_installdir\instances\instance_name\xml\config\com.ibm.commerce.catalog-ext directory.

d. Paste all of the files into this directory.

4. Package your custom XML reader class and the custom physical SDO classes. Package these classes in the WebSphereCommerceServerExtensionsLogic

project to a JAR file.

5. Create the WC_installdir\ext\lib directory if the directory does not exist. Copy the JAR file to the WC_installdir\ext\lib directory. Your custom JAR files must

be placed within the WC_installdir\ext\lib directory.

6. Copy all the custom data load configuration files from the WebSphere Commerce development environment to the WebSphere Commerce serverenvironment.

a. Navigate to the WCDE_installdir\samples\DataLoad\warranty directory.

b. Copy all of the custom data load configuration files within this directory.

c. Navigate to the WC_installdir\samples\DataLoad\warranty directory.

d. Paste all of the files into this directory.

7. Modify the data load environment configuration file, wc-dataload-env.xml, to match your WebSphere Commerce Server instance information:

Change the attribute values for the <_config:BusinessContext> configuration element to match your store settings.

Change the attribute values for the <_config:Database> configuration element to match your database environment properties.

For more information about configuring the data load environment file, see Configuring the data load environment settings.

8. Test your customization on the WebSphere Commerce server by running the Data Load utility. Ensure that you include the parameter -

Dinstance=instance_name when you run the utility For example,

./dataload.sh ..samples/DataLoad/warranty/wc-dataload-warranty.xml -Dinstance=instance_name

dataload ..\samples\DataLoad\warranty\wc-dataload-warranty.xml -Dinstance=instance_name

9. Verify the results of the data load. You can verify your results by reviewing the summary and by checking that the data exists in the appropriate databasetables. To check the database tables for the sample custom tables, run the following SQL statements.

SELECT * FROM XWARRANTY;

SELECT * FROM XCAREINSTRUCTION;

If you see the warranty and care information that is included in the csv file that you loaded, your data load is successful.

< Previous

3. Next >

Tutorial: Loading data into custom tables with the Data Load utilityThis tutorial demonstrates how to use a table-based business object builder and mediator to load data into extended or custom database tables.

By default the Data Load utility supports a table object builder and table object mediator class for loading data directly into a database table. This table-basedbuilder and mediator can be used to load data when there is no component-based business object mediator available that supports the loaded objects. Thistable-based builder and mediator can also be used to load custom data into custom database tables when physical SDOs are not defined for the data that isbeing loaded. The table-based builder and mediator is used by the Data Load utility to map and then load the data to the appropriate database tables. To usethe table object builder and table object mediator, you do not have to map the data to a logical noun, create a custom business object mediator or builder, orextend an existing mediator or builder. The table object builder and mediator are specified and configured within the data load business object configuration file.

For more information about the table object builder and mediator, see Data load table-based mediator and builder.

The following diagram shows the data load process flow.

1. The data reader reads the input file, and transforms the input into a data object. Data objects are typically map objects. In this tutorial, the default CSVdata reader is used to read a provided sample CSV input file.

2. The business object builder instantiates the business object and populates it from the data in the data object. The table object builder is used in thistutorial as the business object builder for instantiating and populating the business objects.

3. The business object mediator converts the business object into physical object. The table object mediator is always used with the table object builder.This table-based mediator is used as the business object mediator in this tutorial to convert the business objects.

4. The data writer writes the physical object into the database. This tutorial uses the default data writer to write the generated physical objects into thedatabase.

Learning objectives

After completing this tutorial, you should be able to complete the following tasks:

Configure the Data Load utility to use the table object builder and table object mediatorConfigure the table object builder to load data into custom tables

Time requiredThis tutorial is expected to take 1 hour to complete.

Skill levelIntermediate

AudienceThis tutorial is intended for WebSphere Commerce administrators who want to configure and run Data Load utility to load data into custom tables.

PrerequisitesBefore you begin this tutorial, ensure that you complete the following tasks.

Install WebSphere Commerce Developer version 7. This tutorial uses a WebSphere Commerce development environment to create custom database tables,and run the Data Load utility to populate the tables. You can choose to configure the Data Load utility and run this utility in an authoring or productionenvironment.

Download the sample WarrantyTableLoad.csv data load input file to a temporary directory.

Ensure that you have an understanding of the following concepts and tasks.

Overview of the Data Load utilityConfiguring and running the Data Load utility

Next >

< Previous | 3.1. Next >

Create the custom tables

In this lesson, you create custom extension tables that you can use the Data Load utility table-based builder and mediator to load data into.

The following diagram outlines the changes to the WebSphere Commerce schema:

The preceding diagram shows the new XWARRANTY and XCAREINSTRUCTION tables, and how they relate to the existing CATENTRY and CATENTDESC databasetables.

The XWARRANTY table has a foreign key to the CATENTRY table.The XCAREINSTRUCTION table has a foreign key to the CATENTDESC table.The XCAREINSTRUCTION table has a foreign key to the CATENTRY table.

Procedure

1. Connect to the development database..

2. Create the customization tables by running the following SQL statements:

CREATE TABLE XWARRANTY (

CATENTRY_ID BIGINT NOT NULL,

WARTERM INTEGER,

WARTYPE VARCHAR(32),

OPTCOUNTER SMALLINT,

CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID),

CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID) ON DELETE CASCADE

);

CREATE TABLE XCAREINSTRUCTION (

CATENTRY_ID BIGINT NOT NULL,

LANGUAGE_ID INTEGER NOT NULL,

CAREINSTRUCTION VARCHAR(254),

OPTCOUNTER SMALLINT,

CONSTRAINT XCAREINST_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINST_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID) ON DELETE CASCADE,

CONSTRAINT XCAREINST_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID) ON DELETE CASCADE

);

CREATE TABLE XWARRANTY (

CATENTRY_ID NUMBER NOT NULL,

WARTERM INTEGER,

WARTYPE VARCHAR(32),

OPTCOUNTER SMALLINT,

CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID),

CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID));

CREATE TABLE XCAREINSTRUCTION (

CATENTRY_ID NUMBER NOT NULL,

LANGUAGE_ID INTEGER NOT NULL,

CAREINSTRUCTION VARCHAR(254),

OPTCOUNTER SMALLINT,

CONSTRAINT XCAREINSTRUCTION_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINSTRUCTION_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID),

CONSTRAINT XCAREINSTRUCTION_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID)

);

< Previous | Next >

< Previous | 3.2. Next >

Creating an input file to load data into multiple tables

In this lesson, you create an input file that contains data for multiple database tables. The data can be loaded in a single data load operation to populatemultiple tables.

Procedure

1. Create the WCDE_installdir\samples\Dataload\warranty\ directory.

2. Copy the following WarrantyTableLoad.csv file into the WCDE_installdir\samples\Dataload\warranty\ directory. The following sample data is included within this

file. This data is loaded into both the XWARRANTY and XCAREINSTRUCTION database tables from the single CSV file.

PartNumber Type ParentPartNumber Sequence Name ShortDescription WarrentyTerm WarrantyType CareInstruction

Cords Product 1 Men's corduroypants

Men's 8-walecorduroy pants

30 Limited Tumble Dry,Normal, LowHeat

Cords - Black - 29W x 28 L

Item Cords 1 Men's corduroypants - Black29 W x 28 L

Men's corduroy pants- Black 29 Width x 28Length

30 Limited Tumble Dry,Normal, LowHeat

Cords - Brown - 29W x 28 L

Item Cords 2 Men's corduroypants - Brown29 W x 28 L

Men's corduroy pants- Brown 29 Width x28 Length

30 Limited Tumble Dry,Normal, LowHeat

Classic pleateddress pant

Product 2 Classic pleateddress pants

Wrinkle-free 5 pocketclassic pleated dresspant

60 Limited Tumble Dry,Normal, No Heat

Classic pleateddress pant - Black- 29 W x 32 L

Item Classic pleateddress pant

1 Classic pleateddress pants -Black - 29 W x32 L

Classic pleated dresspants - Black 29Width x 32 Length

60 Limited Tumble Dry,Normal, No Heat

Dress shirt Product 1 Dress shirt Solid color button-down dress shirt

30 Comprehensive 100% cotton.Dry clean only.

Dress shirt - White- 15

Item Dress shirt 1 Dress shirt -White - 15

Dress shirt - White -15 collar - 32 Sleeve

30 Comprehensive 100% cotton.Dry clean only.

Dress shirt - White- 16

Item Dress shirt 2 Dress shirt -White - 16

Dress shirt - White -16 collar - 35 Sleeve

30 Comprehensive 100% cotton.Dry clean only.

Casual shirt Product 2 Casual shirt Solid color buttondown casual shirt

90 Comprehensive 100% cotton.Machinewashable.

Casual shirt -White-Small

Item Casual shirt 1 Casual shirt -White - Small

Casual shirt - White -Small. Wrinkle free

90 Comprehensive 100% cotton.Machinewashable.

Casual shirt - Blue- Medium

Item Casual shirt 2 Casual shirt -Blue - Medium

Casual shirt - Blue -Medium. Wrinkle free

90 Comprehensive 100% cotton.Machinewashable.

< Previous | Next >

< Previous | 3.3. Next >

Configuring the environment settings

In this lesson, you configure the Data Load utility environment settings.

In the data load environment configuration file, you specify your business context information, database connection property, ID Resolver, and data writer class.The environment variables that are used by the Data Load utility are configured in this XML file. For more information, see Configuring the data load environmentsettings.

Procedure

1. Go to the directory, WCDE_installdir\samples\DataLoad\.

2. Create a folder Warranty.

3. Copy the sample wc-dataload-env.xml file from the WCDE_installdir\samples\DataLoad\Catalog directory into the Warranty directory.

4. Open the new wc-dataload-env.xml copy for editing. Change the values in this file to match your system values.

5. Find the <_config:BusinessContext> element. Change the value for the attributes of this element to match the values for your store. For example,

<_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="AuroraCatalog" />

Where Aurora is your store identifier, and AuroraCatalog is the identifier for you store catalog.

6. Find the <_config:Database> element. Change the value for the type, name, and schema attributes of this element to match the values for your database

environment. For example,

<_config:Database type="derby" name="..\db\mydb" schema="mydbschema"/>

7. Find the <_config:DataWriter> element. This element identifies the data writer class that is used during the data load process. Ensure that the default data

writer is specified.

<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

8. Save your changes. Your file can resemble the following code:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadEnvConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../xml/config/xsd/wc-dataload-env.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

<_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="AuroraCatalog" languageId="-1" currency="USD">

</_config:BusinessContext>

<_config:Database type="derby" name="..\db\mydb" schema="mydbschema"/>

<_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="10000000"/>

<_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" />

</_config:DataLoadEnvConfiguration>

< Previous | Next >

< Previous | 3.4. Next >

Configuring the business object configuration file

In this lesson, you configure the data load business object configuration file.

In the business object configuration file, you specify your data reader, business object builder, and business object mediator. For more information, seeConfiguring the business object configuration file.

This configuration file contains the information that is required by the Data Load utility to resolve the column and table identifiers for the input data. This file isused by the Data Load utility to populate a business object with the input data and any default or specified fixed values. To load the data included in the sample

WarrantyTableLoad.csv input file, this configuration file identifies two builders and two mediators for the Data Load utility to use. To load the input data, the

configuration file specifies that the utility is to use the default base business object builder and a component-based business object mediator to load data intothe CATENTRY table. This configuration file utility species that the utility is to use the table-based business object builder and mediator to load data into theXWARRANTY AND XCAREINSTRUCTION tables. You can also use the table-based builder and mediator to load data into the CATENTRY table, but you arerecommended to use the component-based mediator.

Procedure

1. Copy a sample business object configuration file from the WCDE_installdir\samples\DataLoad\Catalog directory into the

WCDE_installdir\samples\DataLoad\warranty directory. For example, wc-loader-catalog-entry.xml. Rename the file. For example, wc-warrantytableobject-

loader.xml.

2. Open the new business object configuration file for editing.

3. Find the <_config:BusinessObjectBuilder> element. If you are creating a business object configuration instead of editing an existing file, create this

<_config:BusinessObjectBuilder> element within the element.

4. Set the attributes for the business object builder element.

a. Set the className attribute value to be the com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder object builder class.

b. Set the value for the packageName attribute to be com.ibm.commerce.catalog.facade.datatypes.CatalogPackage.

c. Set the value for the dataObjectType attribute to be CatalogEntryType.

For example, the following snippet specifies the component-based object builder class for catalog entries.

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder"

packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" >

</_config:BusinessObjectBuilder>

5. Configure the data mappings for the catalog entry business object. For example, the following code snippet includes data mappings for catalog entryobjects.

<_config:DataMapping>

<_config:mapping xpath="CatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="PartNumber" />

<_config:mapping xpath="catalogEntryTypeCode" value="Type" />

<_config:mapping xpath="ParentCatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="ParentPartNumber" />

<_config:mapping xpath="displaySequence" value="Sequence" />

<_config:mapping xpath="Description[0]/Name" value="Name" />

<_config:mapping xpath="Description[0]/ShortDescription" value="ShortDescription" />

</_config:DataMapping>

6. Specify the business object mediator className to be the CatalogEntryMediator class. This class is the mediator for loading catalog entry data. Set the

value for the componentId attribute for the mediator to be com.ibm.commerce.catalog. For example, the following snippet specifies the component-based

mediator and component identifier.

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder"

packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" >

...

<_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator"

componentId="com.ibm.commerce.catalog" >

</_config:BusinessObjectMediator>

</_config:BusinessObjectBuilder>

7. Create a second <_config:BusinessObjectBuilder> element. Set the className attribute value to be the TableObjectBuilder table object builder class.

For example, the following snippet specifies the table object builder class

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

</_config:BusinessObjectBuilder>

8. Add elements to include the configurations for the XWARRANTY, and XCAREINSTRUCTION tables that the data is to be loaded into. Remove any other

existing table and column configurations within this <_config:BusinessObjectBuilder> element. Specify each of the tables that the Data Load utility is to load

data into within a separate <_config:Table> element. You do not need to include the CATENTRY table. The data for this table is loaded by using the

component-based builder and mediator that you previously configured. For example, the following code snippet includes separate table configurationelements for the XWARRANTY and XCAREINSTRUCTION tables.

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

<_config:Table name="XWARRANTY">

</_config:Table>

<_config:Table name="XCAREINSTRUCTION">

</_config:Table>

</_config:BusinessObjectBuilder>

9. Configure the columns for each of the tables that are to be loaded with the table-based builder and mediator. Specify the configuration for each column

separately within the appropriate <_config:Table>. Use the <_config:Column> element to configure a column. Specify the attributes for each column

configuration. The following snippet specifies the column configuration for loading data in the two extension tables:

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

<_config:Table name="XWARRANTY">

<_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve"

<_config:IDResolve tableName="CATENTRY" generateNewKey="false" >

<_config:UniqueIndexColumn name="PARTNUMBER" value="PartNumber" />

<_config:UniqueIndexColumn name="MEMBER_ID" value="storeOwnerId" valueFrom="BusinessContext" />

</_config:IDResolve>

</_config:Column>

<_config:Column name="WARTERM" value="WarrantyTerm" />

<_config:Column name="WARTYPE" value="WarrentyType" />

</_config:Table>

<_config:Table name="XCAREINSTRUCTION">

<_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" />

<_config:Column name="LANGUAGE_ID" value="langId" valueFrom="BusinessContext" />

<_config:Column name="CAREINSTRUCTION" value="CareInstruction" />

</_config:Table>

</_config:BusinessObjectBuilder>

10. Specify the business object mediator to be the TableObjectMediator class. If you are using the TableObjectBuilder class as the builder, always use thistable object mediator as the business object mediator. This class is the mediator that the Data Load utility uses for all table-based data load. Thefollowing snippet specifies the table object mediator:

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

...

<_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" >

</_config:BusinessObjectMediator>

</_config:BusinessObjectBuilder>

11. Save your file. Your file can resemble the following code:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataloadBusinessObjectConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload-businessobject.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

<_config:DataLoader className="com.ibm.commerce.foundation.dataload.BusinessObjectLoader" >

<_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" useHeaderAsColumnName="true" >

<_config:property name="keyColumns" value="PartNumber" />

</_config:DataReader>

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder"

packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" >

<_config:DataMapping>

<_config:mapping xpath="CatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="PartNumber" />

<_config:mapping xpath="catalogEntryTypeCode" value="Type" />

<_config:mapping xpath="ParentCatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="ParentPartNumber" />

<_config:mapping xpath="displaySequence" value="Sequence" />

<_config:mapping xpath="Description[0]/Name" value="Name" />

<_config:mapping xpath="Description[0]/ShortDescription" value="ShortDescription" />

</_config:DataMapping>

<_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator"

componentId="com.ibm.commerce.catalog" >

</_config:BusinessObjectMediator>

</_config:BusinessObjectBuilder>

<_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >

<_config:Table name="XWARRANTY">

<_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve"

<_config:IDResolve tableName="CATENTRY" generateNewKey="false" >

<_config:UniqueIndexColumn name="PARTNUMBER" value="PartNumber" />

<_config:UniqueIndexColumn name="MEMBER_ID" value="storeOwnerId" valueFrom="BusinessContext" />

</_config:IDResolve>

</_config:Column>

<_config:Column name="WARTERM" value="WarrantyTerm" />

<_config:Column name="WARTYPE" value="WarrentyType" />

</_config:Table>

<_config:Table name="XCAREINSTRUCTION">

<_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" />

<_config:Column name="LANGUAGE_ID" value="langId" valueFrom="BusinessContext" />

<_config:Column name="CAREINSTRUCTION" value="CareInstruction" />

</_config:Table>

<_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" >

</_config:BusinessObjectMediator>

</_config:BusinessObjectBuilder>

</_config:DataLoader>

</_config:DataloadBusinessObjectConfiguration>

< Previous | Next >

< Previous | 3.5. Next >

Configuring the load order configuration file

In this lesson, you configure the Data Load utility load order configuration file.

In the load order configuration file, you specify a pointer to your XML input file, environment settings file, and business configuration file. The load orderconfiguration file defines the mode in which the Data Load utility uses to load data. You also specify the order in which Data Load utility loads the data. For moreinformation, see Configuring the data load order.

Procedure

1. Copy the sample wc-dataload.xml file from the WCDE_installdir\samples\DataLoad\Catalog directory into the WCDE_installdir\samples\DataLoad\Warranty

directory. Rename the file wc-tableobjectdataload.xml.

2. Open the new wc-tableobjectdataload.xmldata load order configuration file for editing.

3. Find the <_config:DataLoadEnvironment> element. Change the value for the configFile attribute to be your data load environment configuration file. For

example,

<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

4. Find the <_config:LoadOrder > element. Change the value for the businessObjectConfigFile attribute to be your business object configuration file. Change the

value for the location element of the <_config:DataSourceLocation> configuration property to be the path for your input file. For example,

<_config:LoadOrder >

<_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantytableobject-loader.xml">

<_config:DataSourceLocation location="WarrantyTableLoad.csv"/>

</_config:LoadItem>

</_config:LoadOrder>

5. Save your changes. Your file can resemble the following code:

<?xml version="1.0" encoding="UTF-8"?>

<_config:DataLoadConfiguration

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload.xsd"

xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config">

<_config:DataLoadEnvironment configFile="wc-dataload-env.xml" />

<_config:LoadOrder >

<_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantytableobject-loader.xml">

<_config:DataSourceLocation location="WarrantyTableLoad.csv"/>

</_config:LoadItem>

</_config:LoadOrder>

</_config:DataLoadConfiguration>

Note: If you want to know more information on configuring your data load order, see Configuring the data load order.

< Previous | Next >

< Previous

Run the Data Load utility

In this lesson, you verify the configuration by running the Data Load utility to load data into the custom tables.

Procedure

1. Optional: Specify the trace level, and enable console and file logging.

a. Navigate to the workspace_dir\WC\xml\config\dataload directory

b. Open the logging.properties file for editing.

c. Change the logging settings for running the Data Load utility. The following settings are used by default:

Logging level is set to both console and file.

Trace level is set to INFO.

Log file location is set to WCDE_installdir\logs\wc-dataload.log.

You do not need to modify the log file location. However, you might want to change the trace level when you are debugging a data load problem.The Data Load utility provides some command-line options to change the trace level. For more information, see the Data Load utility.

An error log file is generated per load item when there are any errors. The error file name and location is specified in the summary report on the

console. The error log file is in the same directory as the wc-dataload.log file.

2. Open WebSphere Commerce Developer.

3. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server,see Starting and stopping WebSphere Commerce Test Server.

4. Open a command-line interface and navigate to the appropriate directory:

WC_installdir/bin

WCDE_installdir\bin

5. Enter the command to run the Data Load utility to load your data load configuration file:

./dataload.sh ../samples/DataLoad/Warranty/wc-tableobjectdataload.xml

dataload ..\samples\DataLoad\Warranty\wc-tableobjectdataload.xml

Where wc-tableobjectdataload.xml is the name of your data load configuration file.

6. Verify the results of the data load. You can verify your results by reviewing the summary and by checking that the data exists in the appropriate databasetables. To check the database tables for the sample custom tables, run the following SQL statements.

SELECT * FROM XWARRANTY;

SELECT * FROM XCAREINSTRUCTION;

If you see the warranty and care information that is included in the csv file that you loaded, your data load is successful.

< Previous