Data Transfer Mechanisation_DST001CEn

7/29/2019 Data Transfer Mechanisation_DST001CEn

1/22

UNSD - OECD Joint System

Data Transfer Mechanism - Technical SpecificationsUN Comtrade Version

Ref : 135664293.doc


2/22

UNSD - OECD Joint SystemData Transfer Mechanism - Technical Specifications UN Comtrade Version

Page ii

1 Feb 2006

INFOOECD Ref: 135664293.doc


3/22


Page iii

UPDATE FORM

Name and Versionnumber

Sent By Signed Approved By Signed

DST001C 1 Feb 06 OECD/STD

DISTRIBUTIONAndreas Lindner OECDTrevor Fletcher OECDGrgory Legoff OECD

Lynda Hawe OECDRonald Jansen UN

Markie Muryawan UNHenri Kayali Info

VERSION DATE OPERATION SPECIFICS

A 19 Oct05

Creation of the document

B 12 Dec05

Add-ons: Comtrade Distribution details (operating mode)

C 1 Feb 06 Cover page and Introduction edited



4/22


Page iv

TABLE OF CONTENTS

1. INTRODUCTION .................................................................................................................................................... ......1

1.1. WHAT IS MEANTBY JOINT SYSTEM? .......................................................... ............. ............. ............. ............. ..........11.2. ABOUTTHIS DOCUMENT............................................................................................................................................. 21.3. ASSOCIATED DOCUMENTS........................................................................................................................................... 2

2. OVERVIEW ...................................................................................................................................................... ......... ....3

3. TECHNICAL SPECIFICATIONS ............................................................................................................................ ...4

3.1. REQUIREMENTS......................................................................................................................................................... 43.2. DATA STORAGE FORMATS........................................................................................................................................... 4

3.2.1. Data Capture ........................................................................................................................................ ......... .......43.2.2. Data Processing and Dissemination ............................................................................................................. .......4

3.3. DATA FLOW FORMATS............................................................................................................................................... 53.4. DATA TRANSFER PROTOCOLS...................................................................................................................................... 63.5. SECURITY................................................................................................................................................................ 73.6. SYNCHRONIZATION ................................................................................................................................................... 7

4. OPERATING MODE .......................................................................................................................................... ......... .8

4.1. DOWNLOAD COMTRADETOOLS SETUP FILES................................................................................................................... 84.2. INSTALL COMTRADETOOLS......................................................................................................................................... 94.3. TEST WEB SERVICES............................................................................................................................................... 114.4. DOWNLOAD DATATOA CSV FILE............................................................................................................................. 124.5. USINGTHE CONFIGURATION FILE................................................................................................................................ 14

4.5.1. Change Column Mapping ...................................................................................................................................154.5.2. Use ProjectID ...................................................................................................................................... ......... ......15

4.6. IMPORT DATAINTOA SQL SERVER DATABASE............................................................................................................ 164.6.1. Check SQL Server Updates .......................................................................................................................... ......164.6.2. Create the Destination Table .................................................................................................................... ........ .16

4.6.3. Define the Connection String ........................................................................................................................... ..164.6.4. Define the Column Mapping Between CSV File and Database Table ......................................................... .....174.6.5. Import a Dataset into the Table Debug Mode ......................................................................................... .......174.6.6. Import Datasets ...................................................................................................................................................174.6.7. Use the New Resume Parameter ................................................................................................................... ...17

FIGURES ................................................................................. ............. ............. ............. ............. ............ ............. ...18



5/22


Page 1

1. Introduction

1.1. What Is Meant by Joint System?

Since 2000, UNSD1 and OECD2 have joined forces to reduce the response burden for national data

providers in the field of merchandise trade statistics and to agree on one unique and transparent set ofdata instead of having parallel (and often not harmonized) data collections. This considerable amount ofvery detailed investigation has led to the signing of an UNSD/OECD Memorandum of Understanding inJanuary 2001 andfurther extensions, where UNSD and OECD jointly developed

(i) an agreement on data collection and data sharing,(ii) common trade data processing standards, and(iii) computer applications for use in processing of trade data.

This set of agreements, standards and applications is referenced by both organizations as:

UNOECD Joint Trade Data Collection and Processing System (Joint System)

Both organizations agreed to use the above name in any documents describing their cooperative

activities. It is also to be used when referring to the respective databases.

The Joint System is the foundation of the trade databases maintained by both organizations. Bothorganizations continue to keep separate (institutionally and legally) trade databases to ensure rights andability of each organization to pursue their own policies with regard to

(i) storing various data in addition to those agreed in Joint System [e,g., historical data,memorandum items, derived data etc],

(ii) developing additional analytical and presentational functionalities and controlling user access tothem, and

(iii) dissemination practices including pricing of user access to the database services, entering intocontracts with users, etc.

Each organization maintains its own brand name for use in data dissemination activities. The names ofthose databases are:

For UNSD: UN Comtrade3

For OECD: OECD ITCS4 Database

To ensure user awareness of the joint efforts of both organizations in the area of international merchandisetrade statistics, both organizations agreed to make data responsibility very clear.

Consequently, the UN Comtrade disclaimer contains the following statement:

Beginning 2005, trade data of OECD member countries are compiled by OECD in accordance withUN-OECD Common Trade Data Processing Standards and are available in UN Comtrade inaccordance with UN/OECD Data Sharing Agreement. Any deviation from this practice is reflected

in the country/period metadata.

Similarly, the OECD trade database disclaimer contains the following statement:

The trade data for the selected non-OECD member countries are compiled by UNSD inaccordance with UN/OECD Common Trade Data Processing Standards and are available in OECDITCS Database in accordance with UN-OECD Data Sharing Agreement. Any deviation from thispractice is reflected in the country/period

1 United Nations Statistics Division

2 Organization for Economic Co-operation and Development3 Commodity Trade Statistics Database4 International Trade by Commodity Statistics

INFO/OECD Ref: 135664293.doc


6/22


Page 2

1.2. About this Document

This document makes up the technical specifications of the Joint System data transfer mechanism(DTM) between UNSD and OECD databases. DTM is introduced in Section 2. below.

Data transfers between UNSD and OECD are the responsibility of specialized OECD statistics database

Administrators.

Word links are available throughout this document like for instance, at the beginning of the section.

Pictures include detailed comments in callouts or in their captions. These comments contribute totrimming down the text and making it clearer. This layout is generally appreciated by non-expert readers.

1.3. Associated Documents

The UNSD OECD Joint System package also includes the following documents:

Data Capture / Data Processing Manual Web Browser Client User Manual Database Administrator Client Manual Database Dissemination Model Guide



7/22


Page 3

2. Overview

As mentioned in the previous section, UNSD and OECD will use a unique logical storage to make theirstatistical data available through the Web.

In fact, the Joint System will still use two physical SQL Serverdatabases, one for each organization, tostore the collected and processed commodity trade statistical data.

These two databases will be continuously updated and synchronized through a data exchange processexplained later in this document. Thus, the databases will stay identical regarding their logical contents. Aspecific flavor will also be added to each database contents to reflect OECD and UNSD standards,particular additions and habits.

This process is illustrated below:

ITS Meeting Sept 20055

Common

Processing

Module

Common

Processing

Module

Common

Processing

Module

Common

Processing

Module

Working

Database

Working

Database

OECD UNSD

Working

Database

Working

Database

Data capture Data capture

Process Process

Disseminate Disseminate

Dissemination

Database

Dissemination

DatabaseDissemination

Database

Dissemination

Database

Data Exchange &

Synchronisation

Data Access

Figure 1: statistical data flow for UNSD and OECD in Joint System

Data transfers and conversions are physically carried out by an UNSD software package, namedComtradeTools, which makes use of Web Services. ComtradeTools functions, Configuration file andcommands are discussed in theOperating Mode below.

The purpose of this document is to describe, step by step, the technical specifications of data transfer asrepresented above:

Transfer protocols Data flow formats Data storage formats Security OECD UNSD synchronization Operating mode

Some parts of this document are closely related to the Database Administrator ClientManual.



8/22


Page 4

3. Technical Specifications

3.1. Requirements

The OECD Client who wishes to import updated data from the UNSD database should be equipped withthe following software components:

Microsoft Windows 2000 or a higher version; Microsoft .NET Framework version 1.1; Microsoft SQL Server 2000; WinZip program (note: some versions of Windows have a built-in unzip capability)

In addition, the OECD Client should have a basic knowledge of XML files and SQL Server QueryAnalyzer.

3.2. Data Storage Formats

3.2.1. Data Capture

Contributing countries generally send their annual statistical data as plain text, Excel or .DBF (FoxPro)files. In fact, any format is acceptable.

MS Access is used as an intermediate storage for all these files before loading them up to the SQL Serverdatabase. Importation into Access and SQL Server database are carried out through the Get External Data(Access) and Import Data (SQL Server) functions.

Note: data in Access format is kept for archival purposes.

3.2.2. Data Processing and Dissemination

As mentioned before, SQL Server is used to store the statistical data to be processed (WorkingDatabases) and distributed (Dissemination Databases)

Data processing (importation, checking, conversion, aggregation) is described in the Data Capture /ProcessingManual.



9/22


Page 5

3.3. Data Flow Formats

The OECD and UNSD databases are kept synchronized through bi-directional data transfers. Thesynchronization process is explained in this document and in the Database Administrator ClientManual.During these transfers, the XML5format is used, with SDMX6 cross-sectional schema. The incoming datais finally converted into CSV7 format and loaded into the destination database.

The following insertion is an example of SDMX cross-sectional schema:

-

-

UN626573874

truefalse

2005-11-10T04:34:38

-

United Nations

UN_COMTRADE_BE

01

Update

2005-11-10T04:34:38

-

--

Figure 2: example of SDMX cross-sectional schema

Link tohttp://www.sdmx.org

for more detail about SDMX.

The same data format is used for transfers between working and dissemination databases (see fig. 1).

The SDMX data is compressed by 70% to 80% before the data transfer.

5 Extensible Markup Language

6 Statistical Data and Metadata Exchange7 Comma Separated Values; e.g., the contents of a plain text file. Such a file can be used by Notepad orExcel

http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://www.sdmx.org/http://www.sdmx.org/http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://c/Documents%20and%20Settings/simon/Local%20Settings/Temporary%20Internet%20Files/Content.IE5/HXMZPR5L/comtrade_trade_data[1].sdmx##http://www.sdmx.org/


10/22


Page 6

3.4. Data Transfer Protocols

The common HTTP8is used all over the Internet via Web Services. Web Services is a remote library (orClass, from a language point of view), published on the Web via XML standards, with properties andmethods, some of which devoted to Web data transfers. In our case, due to data size, REST Web serviceis selected because of its ability to stream and use on-the-fly compression.

The picture below illustrates the Joint System data transfer mechanism between UNSD and OECD:

Figure 3: transfer mechanism between UNSD and OECD

Transfers are Pulled by OECD: data is sent on OECD Users requests to ComtradeTools see OperatingMode below.

8 Hypertext Transfer Protocol



11/22


Page 7

3.5. Security

It is based on IP Addresses of Users who wish to connect to the Web Server and on passwords to thedissemination database. The IP address seen by ComtradeTools is the Web access providers, not thelocal network address of the connecting station.

An example ofdenied access is given in section 4. Web Services is also used via the Access Control Listfunction. In addition, the User ID / password combination used when logging on to Windows is used toaccess the CoprAapplication (see Data Capture / Data ProcessingManual).

The secured (encrypted) HTTP, HTTPS, could possibly be used.

3.6. Synchronization

The OECD and UNSD databases are synchronized through a time stamp mechanism insideComtradeTools: each time a data set is imported from reporting countries to the UNSD database, its timestamp is updated to the current date, and a comparison with the previous time stamp tells whether thisdata item must be downloaded to the OECD database to update it.

The format of a time stamp is YYYY-MM-DDTHH:MM:SS (labels in italics areplaceholders)

From a general point of view, the updating data can be filtered by country, year or commodity, in additionto being selected by the time stamp. Examples of such selection parameters are given in the next section.

Selection criteria can be defined directly in the command line or by Keys in the ComtradeToolsConfiguration file. The example below is the definition of such a key:


12/22


Page 8

4. Operating Mode

This section is an illustration of the operating procedure used to import UN data into the OECD database.The conversion and synchronization phases are also reproduced. An example of column mapping and

SQL Server data creation is provided. In fact, all the useful operations are discussed. Depending on thestatus of the operating workstation and the data to be transferred, some of the operations described herewont be necessary to carry out an import.

As mentioned above, a software package, ComtradeTools, is used to initiate the various transfer /conversion functions. ComtradeTools is a command line program: it is launched from the Run window,like, for instance, the Registry Editor.

The first thing to do is to download the ComtradeTools package, if not already present on the Usersworkstation.

4.1. Download ComtradeTools Setup Files

Get ComtradeTools installation files by linking tohttp://unstats.un.org/unsd/comtradedist/comtradeToolsSetup.zip

The usual Open/Save interface is displayed. The easiest way to proceed is to choose Open; WinZip will beactivated and will display the three expanded installation files (.exe, .ini, .msi):

Figure 5: the three ComtradeTools setup files unpacked by WinZip

Transfer the installation files to a proper local directory(suggested: :\Program Files\UNSD\ComtradeTools; System Drive is usually C).


Installation package (compressed file)
http://unstats.un.org/unsd/comtradedist/comtradeToolsSetup.ziphttp://unstats.un.org/unsd/comtradedist/comtradeToolsSetup.zip


13/22


Page 9

4.2. Install ComtradeTools

Double-click9the .exe file. A wizard will help you install the Application; in fact, you just have to choose theApplication folder (the installation folder defined during the previous step is the default). The installationprocess takes a few seconds. The following files are created:

Figure 6: the ComtradeTools files

(See next page)

9 Depending on the System options, a single click may be enough.


ComtradeTools task

Configuration file

Installation / Application folder


14/22


Page 10

The presence of the Application can also be checked through the usual Add/Remove Programs interface:

Figure 7: Add/Remove Programs interface

This utility should be used to uninstall ComtradeTools, as it is for any other application.



15/22


Page 11

4.3. Test Web Services

To make sure that everything is in order especially regarding access rights, a request to ComtradeToolscan be sent through the Web Browser. For instance:

http://unstats.un.org/unsd/comtradedist/getSdmxV1.aspx?px=H1&r=381&y=2003,2002&cc=TOTAL&p=0&comp=false

This request asks for the transfer of a selected data set from the UN database to the OECD Clientworkstation. The selection parameters are identified by keywords like px or r. The meanings of thesekeywords are explained below, in section 4.4. If the transfer operation works fine, the destination is a XMLfile with contents like:

Figure 8: contents of the destination XML file after the transfer (partial view, may need enlargement)

The access control concerns the sender IP address as seen by the UNSD Server, as mentioned in the

Securitysection. If access is denied, the following message is sent:

Figure 9: access to ComtradeTools function denied

http://unstats.un.org/unsd/comtradedist/getSdmxV1.aspx?px=H1&r=381&y=2003,2002&cc=TOTAL&p=0&comp=falsehttp://unstats.un.org/unsd/comtradedist/getSdmxV1.aspx?px=H1&r=381&y=2003,2002&cc=TOTAL&p=0&comp=false


16/22


Page 12

4.4. Download Data to a CSV File

The next requests will be sent from the Command Line (aka DOS) window. This window pops up whenselecting Start > Run. The following command is an example of download request:

ComtradeTools /r:360 /y:2003 /px:H1 /action:DownloadAndConvertToCSV

The program name, ComtradeTools, is followed by a list of slash-separated parameters. Theseparameters are:

r: reporting country code. Here, 360 stands for Indonesiay: reporting yearpx: classification used. H1 means HS1996action: here, a CSV file will be created with the converted imported data.

The Command Line window with the command (leftmost part) in it is reproduced below:

Figure 10: command line window

Clicking OK submits the transfer request to ComtradeTools. The resulting process can be followed on aDOS pane that pops up:

Figure 11: command line execution error(s) occurred

We see that the transfer failed. Submitting the URL that appears in this DOS pane would return the accessdenied message in fig. 9.

(See next page)



17/22


Page 13

Had the access been granted, the transfer would have been successful, with the following messages onthe DOS pane:

Figure 12: command line execution successful transfer

The code of the reporting country is now 8 (Albania): Indonesia would have returned too large a data set.

We notice an extra parameter in the command line: outputDirectorywith C:\Temp as value. This is thelocal directory the CSV file is created in. The name of this file is Temph120018_CSV.txt, with theselected dataset values converted to CSV in it. The destination file name is built from the selectioncriteria: classification, year and country, prefixed by the destination folder.

The picture below shows the first records of the destination CSV file:

Figure 13: dataset downloaded into a CSV file first records

The first line is made of column headers, with, for instance, 8 as reporting country. The next records are

the extracted data values.


Columnheaders

Another

command line


18/22


Page 14

4.5. Using the Configuration File

This file has the XML format and contains the definition of Keys that are used by ComtradeTools asparameters. It is part of the ComtradeTools package (seefig. 6)

Its original contents (for the example we use) are displayed below:

Figure 14: the original ComtradeTools Configuration file notice the comments (HTML format)

The keys in this file are defined by their names and values within an add key tag. The 2000Datakey hasbeen introduced in the Synchronization section. In the rest of this section, well give examples of how theConfiguration file can be used.



19/22


Page 15

4.5.1. Change Column Mapping

When the original UNSD data (SDMX format) is transferred to a CSV file, and later when the CSV data isimported into the OECD SQL Server database, a column mapping must take place between the sourceand the destination. The correspondence is defined by the two following keys in the Configuration file:

AttributeMapping: SDMXto CSVSQLColumnMapping: CSV to SQL Server table

In the Configuration file, well first change the AttributeMapping entry:

to:

We notice that:1. two correspondences have been removed: RPT=reporter; time=year.

2. two other correspondences have been added:REPORTED_CURRENCY=cur; CONVERSION_FACTOR=cf.

We can then run the transfer request again:

ComtradeTools /r:8 /y:2001 /px:H1 /action:DownloadAndConvertToCSV OutputDirectory:C:\temp

And compare the new CSV file we get with the previous one (fig. 13).

Finally, we restore the two removed correspondences (reporter and year) in the AttributeMapping key.

SQLColumnMapping will be used later.

4.5.2. Use ProjectID

Keys can be added to the Configuration file to replace a parameter string in the ComtradeTools command.These keys (and therefore the parameter string) can be submitted via the ProjectID label in the commandline. For instance, we can insert the following definition in the Configuration file:

The name of the key is IndonesiaH1 and its contents make a selection by reporting country(360=Indonesia) and classification (H1=HS1996). LastSynch is reset and TIMESTAMP is used assynchronization option. The command line can now be:

ComtradeTools /y:2003 /action:DownloadAndConvertToCSV /outputDirectory:c:\temp\sdmx\/ProjectID:IndonesiaH1

The transfer and conversion to CSV will be carried out, according to the parameters in IndonesianH1 andin the command line. When the process is completed, the timestamp will be updated to keep thesynchronization process in order.



20/22


Page 16

4.6. Import Data into a SQL Server Database

4.6.1. Check SQL Server Updates

The User should first make sure that the sql2ksp3.exe Client component is installed on their workstation bylinking to:

http://www.microsoft.com/sql/downloads/2000/sp3.asp file

If this component is missing, the following error will occur:

"Unhandled Exception: System.InvalidCastException: QueryInterface for interface DTS.CustomTask failed."

Of course, this operation should be carried out only once for a given system.

4.6.2. Create the Destination Table

The following commands and source data should be submitted to SQL Server Query Analyzer, either fromthe keyboard or from a script file, to create the destination table in a database (for this example, anyexisting database managed by SQL Server will do):

if exists (select * from dbo.sysobjects where id =object_id(N'[dbo].[tempLoad]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)drop table [dbo].[tempLoad]GO

CREATE TABLE [dbo].[tempLoad] ([pfCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[yr] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[rgCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[rtCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[ptCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[cmdCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[cmdID] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[qtCode] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,

[TradeQuantity] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[NetWeight] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[TradeValue] [varchar] (255) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[source] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[currency] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,[conversion_factor] [varchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL) ON [PRIMARY]

GO

Figure 15: SQL Query Analyzer input to create the destination table in a database

This SQL sequence first checks for table tempLoad existence; if the table exists, it is removed from thedatabase. tempLoad is then (re-)created and its columns are defined with their names, type and length.Here, type=Character and length=50 or 255.

4.6.3. Define the Connection String

The following key should be added to the Configuration fi le:

This is the path to the SQL Server database and the target table.

In this example, Northwind, the standard Windows demonstration database, is used. Optional valueparameters make it possible to customize the destination table according to Year, Reporting Country orClassification.



21/22


Page 17

4.6.4. Define the Column Mapping Between CSV File and Database Table

Key SQLColumnMapping in the Configuration file is used. Its value should be:

Compared to the original value in the Configuration file, the new columns currency andconversion_factor have been added.

Constant values (e.g., labels) can also be mapped through the SQLConstValueMapping key. In theoriginal Configuration File, we have:

4.6.5. Import a Dataset into the Table Debug Mode

The ComtradeTools command line is used again to transfer data from the UN database and import theresulting CSV file into tempLoad table in Northwind database, as defined in the Configuration File:

ComtradeTools /r:8 /y:2001 /px:H1 /action:DownloadAndImportToSQLDB

/removeTemp:false /DTSDebug:true

With the DownloadAndImportToSQLDB action, the intermediate step that results in the creation of the CSVfile is transparent to the User its name is still built from the selection parameters. This action is carriedout through SQL Server DTS (Data Transformation Services). The DTSDebug option makes it possible tocreate and examine the DTS package; this is useful for troubleshooting.

Refer to SQL Server documentation for details on DTS.

Once the action is completed the User should check the contents of table tempLoad in the database.

4.6.6. Import Datasets

The IndonesianH1 key has already been appended to the Configuration file:

This key includes the reset of LastSynch (see section 4.5.2)

We can now re-run the direct import with the synchronization option:

ComtradeTools /action:DownloadAndImportToSQLDB /outputDirectory:c:\temp\sdmx\/projectID:IndonesiaH1 /synchOption:Timestamp

4.6.7. Use the New Resume Parameter

As from 15 Aug 2005, a new parameter in the command line makes it possible to resume the selectedComtradeTools action up to n times if errors are encountered. This parameter is used in the command

below:ComtradeTools /action:DownloadAndImportToSQLDB /outputDirectory:c:\temp\sdmx\/projectID:IndonesiaH1 /synchOption:Timestamp /resume:3

The operation will be resumed up to three times until no error occurs.



22/22


Page 18

FIGURES

FIGURE 1: STATISTICAL DATA FLOW FOR UNSD AND OECD IN JOINT SYSTEM....................................3

FIGURE 2: EXAMPLE OF SDMX CROSS-SECTIONAL SCHEMA........................................................................5

FIGURE 3: TRANSFER MECHANISM BETWEEN UNSD AND OECD.................................................................6

FIGURE 4: THE IMPORT - SYNCHRONIZATION PROCESS................................................................................7

FIGURE 5: THE THREE COMTRADETOOLS SETUP FILES UNPACKED BY WINZIP..............................8

FIGURE 6: THE COMTRADETOOLS FILES.............................................................................................................9

FIGURE 7: ADD/REMOVE PROGRAMS INTERFACE..........................................................................................10

FIGURE 8: CONTENTS OF THE DESTINATION XML FILE AFTER THE TRANSFER (PARTIAL VIEW,MAY NEED ENLARGEMENT)....................................................................................................................................11

FIGURE 9: ACCESS TO COMTRADETOOLS FUNCTION DENIED...................................................................11

FIGURE 10: COMMAND LINE WINDOW................................................................................................................12

FIGURE 11: COMMAND LINE EXECUTION ERROR(S) OCCURRED...........................................................12

FIGURE 12: COMMAND LINE EXECUTION SUCCESSFUL TRANSFER......................................................13

FIGURE 13: DATASET DOWNLOADED INTO A CSV FILE FIRST RECORDS............................................13

FIGURE 14: THE ORIGINAL COMTRADETOOLS CONFIGURATION FILE NOTICE THE

COMMENTS (HTML FORMAT).................................................................................................................................14

FIGURE 15: SQL QUERY ANALYZER INPUT TO CREATE THE DESTINATION TABLE IN ADATABASE......................................................................................................................................................................16

Data Transfer Mechanisation_DST001CEn

Documents

Transcript of Data Transfer Mechanisation_DST001CEn