1

SQL Pass-Through and the ODBC Interface:Extract and Transform Your Data FASTER – For PC SAS Users

2

Overview

Definitions

• ODBC

• RDBMS

• SQL

• Pass-Through Query

Syntax

• ODBC libname statement

• Explicit SQL pass-through query using ODBC interface

Why Use Explicit Pass-Through?

• Increased processing speed

• Differences between Oracle, DB2, and SAS

• Work around SAS limitations

• Implicit pass-through doesn’t always work (ex: unconvertable SAS functions, complex joins, outer joins)

3Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Definitions

4

Definitions

ODBC: Open Database Connectivity

RDBMS: Relational Database Management System

• DB2

• Access

• Oracle

• SQL Server

SQL: Structured Query Language

• Each RDBMS has its own dialect of SQL

• Use SQL within SAS by invoking Proc SQL

Pass-Through Query: Instead of having SAS do the work, process the data in its native environment using native SQL functions (implicit vs. explicit)

5

ODBC Connection Setup

6Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Syntax

7

ODBC Libname Statement (Implicit)

libname CDB ODBC datasrc=db2p user=uid password=yourpwd schema=GWY1;

proc sql;

create table WORK.ACCOUNT as

select ACCT_NUM,

OPEN_ENRLM_BEG_DT,

OPEN_ENRLM_END_DT

FROM CDB.ACCOUNT;

quit;

8

Explicit Pass-Through Syntax

proc sql;

connect to odbc(datasrc = "db2p" user=uid password=yourpwd);

create table WORK.ACCOUNT as

select * from connection to odbc

(select ACCT_NUM,

OPEN_ENRLM_BEG_DT,

OPEN_ENRLM_END_DT

FROM GWY1.ACCOUNT);

disconnect from odbc;

quit;

9Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Why Use Explicit Pass-Through?

10Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Processing Location Affects Processing Speed"No one will believe you solved this problem in one day! We've been working on it for months. Now, go act busy for a few weeks and I'll let you know when it's time to tell them." --submission from a real-life “Dilbert Quotes” contest

11

Processing Location...

PC SASData in RDBMS

(Oracle, DB2, etc.)ODBC

Your PC Server

Regular Query does processing work in SAS on your PC using SAS

functions

Pass-Through Query does processing work in RDBMS where the data is stored using “native” Oracle/DB2 functions

Faster!!Faster!!

12

...Affects Processing Speed

Avoids large data movement

Query is done in the database which is optimized to handle the queries

Only results are returned to SAS

Multi-table, multi-field, complex joins are handled faster

Entire tables are brought into SAS (slow, takes up a lot of resources and space)

SAS does the processing work using SAS functions

The more tables being referenced, the larger the tables, and the more complex the joins, the greater the slowdown

Pass-through query:Non pass-through query:

13Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Differences in Numeric Precision – Oracle example

14

A Mysterious Problem

Oracle database

Claim UID field

• Primary key

• Character field in integrated claims table: length 16

• Numeric field in source system claims table: decimal, precision 16

Task:

• Use SAS put function to convert numeric to character

• Then join tables on Claim UID

Result:

• Only a 10% match between tables on Claim UID

• Each match has 10 different member numbers associated with it

• All claim IDs end in 0

15

Original Query

proc sql;

create table work.s1_claim_row as

select

put(claim_uid,best16.) as claim_uid length= 16

from prodj.s1_claim

;

quit;

16

What Happened?

Differences in Numeric Precision between SAS and Oracle database

• SAS can only handle precision to 12-14 digits

• Oracle can store precision up to 38 digits

When I convert numeric to character in SAS, loss of precision in the last digit

Last digit gets stored as 0

Use Oracle to do the conversion using pass-through query and Oracle to_char function

Remember! SAS SQL is different than Oracle SQL – Put function won’t work in a pass-through query

17

Solution (SAS/Access Interface to ODBC)

proc sql;

connect to odbc(datasrc = "PRODJ" user=xxxxxxxxxxx password=xxxxxxxxxxxx);

create table work.s1_claim_row as

select * from connection to odbc

(select to_char(claim_uid,'0000000000000000') as claim_uid

from onesource_o.s1_claim

)

;

disconnect from odbc;

quit;

18

Solution (SAS/Access Interface to Oracle)

proc sql;

connect to oracle(path = "prodj.cigna.com" user=xxxxxxx password=xxxxxxx);

create table work.s1_claim_row as

select * from connection to oracle

(select to_char(claim_uid,'0000000000000000') as claim_uid

from onesource_o.s1_claim

)

;

disconnect from oracle;

quit;

19Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Differences in Naming Conventions – DB2 example

20

Original Query

libname CDB ODBC datasrc=db2p user=uid password=yourpwd schema=GWY1;

proc sql;

create table WORK.ACCOUNT as

select ACCT_NUM,

OPEN_ENRLM_BEG_DT,

OPEN_ENRLM_END_DT

FROM CDB.ACCOUNT;

quit;

21

Log Error

17 LIBNAME CDB ODBC datasrc=db2p user=xxxx password=XXXXXXXX schema=GWY1;

NOTE: Libref CDB was successfully assigned as follows:

Engine: ODBC

Physical Name: db2p

18 proc sql;

19 create table WORK.ACCOUNT as

20 select ACCT_NUM,

21 OPEN_ENRLM_BEG_DT,

22 OPEN_ENRLM_END_DT

23 FROM CDB.ACCOUNT

24 ;

ERROR: This DBMS table or view cannot be accessed by the SAS System because it contains column names that are not unique when a SAS normalized (uppercased) compare is performed. See "Naming Conventions" in the SAS/ACCESS documentation.

22

Problem

ERROR: This DBMS table or view cannot be accessed by the SAS System because it contains column names that are not unique when a SAS normalized (uppercased) compare is performed. See "Naming Conventions" in the SAS/ACCESS documentation.

DB2 database has different naming conventions than SAS

The issue here is case, but table/column name lengths and special characters can also be a problem

Additional Example:

• ERROR: TABLE NAME 'INTERVENTION_TRACKING_INTERVENTION' is too long for a SAS name in this context

• Table names in SAS must be less than 32 characters

• DB2 table names can be up to 128 characters

23

Solutionproc sql;

connect to odbc(datasrc = "db2p" user=uid password=yourpwd);

create table WORK.ACCOUNT as

select * from connection to odbc

(select ACCT_NUM,

OPEN_ENRLM_BEG_DT,

OPEN_ENRLM_END_DT

FROM GWY1.ACCOUNT);

disconnect from odbc;

quit;

24Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © 2010 CIGNA.

Conclusion

25

Why Use Pass-Through Queries?

Decrease Processing Time

• The closer you move the processing operations to where the data is located, the more efficient your query is

• Pass-through queries process the data in its native environment

Differences Between Oracle, DB2, and SAS

• Numerical precision

• Table/field naming conventions

Work Around SAS Limitations

• Use Oracle or DB2 functions to do the work if SAS can’t

• Particularly useful for initial data extractions

• Good when you are familiar with Oracle or DB2 SQL dialects

26

Contact Information

Presentation author: Jessica Hampton

For more information about SQL pass-through queries, email Jessica.Hampton@cigna.com

Useful links:

• SAS ODBC Driver: User’s Guide and Programmer’s Reference: http://support.sas.com/documentation/onlinedoc/91pdf/sasdoc_91/odbc_ugref_6971.pdf

• SAS/Access for Relational Databases Reference: http://support.sas.com/documentation/onlinedoc/91pdf/sasdoc_913/access_rdbref_9297.pdf

• Numeric Precision in SAS http://support.sas.com/techsup/technote/ts230.html http://support.sas.com/techsup/technote/ts654.pdf

• DB2 Naming Conventions http://support.sas.com/documentation/cdl/en/acreldb/63647/HTML/default/viewer.htm#a001383772.htm