Amazon S3 Connector User Guide V1 - Informatica ETL · PDF fileAmazon S3 Connector User Guide...
Transcript of Amazon S3 Connector User Guide V1 - Informatica ETL · PDF fileAmazon S3 Connector User Guide...
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Amazon S3 Connector User Guide V1.2
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Contents
Amazon S3 Connector User Guide V1.2 .................................................................................. 3
1. Introduction ....................................................................................................................... 5
2. Technologies ...................................................................................................................... 6
3. High level Architectural Diagram ....................................................................................... 6
4. AmazonS3 Assumptions/Prerequisites ............................................................................... 7
5. Different Types optional documents ................................................................................... 7
6. Creating AmazonS3 Connection ......................................................................................... 8
7. Source/Target object dropdown notes ............................................................................. 10
8. Connector Read notes ...................................................................................................... 12
9. Connector Write notes ..................................................................................................... 12
10. Assumptions & Limitations ............................................................................................. 14
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
1. Introduction
Informatica Cloud Connector SDKs are off-cycle, off-release “add-ins” that provide connectivity to SaaS and on-premise
applications not supported natively in Informatica Cloud. Connector SDKs can be enabled anytime for your cloud org Id, and are
designed to address common use cases – including getting data into, and out of, the supported application.
Connector SDKs do not require any separate software installation on the machine running the Informatica Cloud Secure Agent.
The Connector uses an internal Java application to communicate with the Secure Agent, as shown in the architecture diagram
below:
Once you have enabled the Connector Package software for your org Id, you need to create a connection in Informatica Cloud to
access to the AmazonS3.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
2. Technologies
Connector is developed using - JDK 1.7, AWS SDK 1.7.3
3. High level Architectural Diagram
Compatibility: This connector supports Amazon S3 document service.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
4. AmazonS3 Assumptions/Prerequisites
AmazonS3 connector requires an account with Amazon cloud web service enabled
Access Key and secret Key also required connecting the AmazonS3
5. Different Types optional documents
Document name
ApplicationState.CSV
Customer.CSV
DocAdvanceShippingNotice.DAT
DocBudgetTransaction.TXT
DocCashDiscuountbatch.POW
DocSaleQuotation.CSV
Item
Accounts
Position.DAT
This list is just example of document in S3 bucket, user can have any number of csv document on the s3 bucket
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
6. Creating AmazonS3 Connection
Before you use the “AmazonS3 connector” in a data synchronization task, you must follow these steps to create a
connection in Informatica Cloud: 1. Click on the Configuration tab.
2. Click on the Connections page.
3. Click the New button.
Connection Property Description
Connection Name (Mandatory) Provide the Name of the connection
Description (optional) Provide the description
Type (Mandatory) Select “AmazonS3 (Informatica Cloud)”
Secure Agent (Mandatory) Secure Agent that Informatica Cloud uses to access the Web Service.
Access Key (Mandatory) Provide the Access Key of Amazon S3 for your account
Secret Key (Mandatory) Provide secret key of Amazon S3 for your account
Region (optional) Select the region
Folder Path (Mandatory) Provide folder path of Amazon S3 bucket
Delimiter(Mandatory) Provide delimiter of file of S3 bucket (single character, default is ‘,’ comma)
Note: The folder path should not starts or ends with “/” (forward slash) or “\” (backward slash) Root bucket name in folder path is case sensitive.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
7. Source/Target object dropdown notes
When user select source/target connection as AmazonS3 type connection,
1. The source object dropdown displays the list of Amazon S3files (object) by given folder path in the connection property.
2. If object has file extension then that Object name (file name) displayed in the dropdown suffixed with __<file extension> (Example: account__csv, department__DAT)
3. If object doesn’t has the extension then that object name only will displayed in the dropdown (Example: BusinessHours, Item, etc)
Refer the screen shot for source & target object dropdown
Source Object Dropdown (AmazonS3):
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Target Object Dropdown (AmazonS3):
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
8. Connector Read notes
4. Amazon S3 “read” methods returns entire records in the selected file of given folder path in connection properties.
5. Data Filter feature is not supported
9. Connector Write notes
1. User should perform create target operation in DSS task step-3 (refer screen shot) to create the document in S3 bucket. Refer the screen shot
2. If source file does not have any data record (empty file) then it will not be created in the Amazon S3.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA Phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com
© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
10. Assumptions & Limitations
1. Connector supports and compatible only for Amazon S3
2. An Amazon web service user account with Access Key and Secret Key are required to make the connection and
perform the Read & Create operation.
3. User has to provide correct folder path of Amazon S3 bucket when create S3 connection (Root bucket name in folder
path is case sensitive)
4. User has to provide correct delimiter of file of S3 bucket when create S3 connection (Default is comma ‘,’)
5. User has to select appropriate file which has delimiter as given in connection properties
6. Filed mapping & data should be proper and appropriate to perform Read & Create operation.
7. Connector not supports Data Filter
8. Connector not supports Update operation
9. Connector not supports Delete operation
10. Connector not supports Upsert operation
11. Connector not supports Mid Stream feature