A Seat Belt for Data - s3.amazonaws.com€¦ · A Seat Belt for Data ... FastLoad, MultiLoad,...

of 60/60
#TDPARTNERS16 Sept 11,2016 GEORGIA WORLD CONGRESS CENTER A Seat Belt for Data Practical Use of Encryption in Teradata Systems Jim Browning Enterprise Security Architect Teradata Labs
  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of A Seat Belt for Data - s3.amazonaws.com€¦ · A Seat Belt for Data ... FastLoad, MultiLoad,...


    A Seat Belt for Data

    Practical Use of Encryption in

    Teradata Systems

    Jim Browning Enterprise Security Architect Teradata Labs

  • 2

    Encryption is the equivalent of a seat belt for data. -- Andy Cordial, Managing Director, Origin Storage

  • Key Encryption Drivers

    Cryptographic Technology Primer

    Important Considerations

    Use of Encryption in Teradata Systems



  • 4

    Drivers for the Use of Encryption Types of Data Requiring Protection

    Credit Card Information

    > Credit Card Numbers (PAN)

    > Service Codes

    > Expiration Dates

    Personal Identifying Information

    > Social Security Numbers

    > Tax Identifiers

    > Drivers License Numbers

    > Date of Birth

    Consumer Financial Data

    > Account Numbers

    > PINs

    Protected Health Information

    > Identifiable Patient Data

    > Medical Record Numbers

    Corporate Financial Data

    > Non-public Information

    Human Resources Data

    > Payroll Information

    > Performance Ratings

    Customer and Prospect Data

    Trade Secrets and Intellectual Property

  • 5

    Drivers for the Use of Encryption Standards and Regulations

    Payment Card Industry (PCI) Data Security Standard

    HIPAA Privacy Rule

    U.S. State Security Breach Notification Laws

    EU General Data Protection Regulation (GDPR)

  • Key Encryption Drivers

    Cryptographic Technology Primer

    Important Considerations

    Use of Encryption in Teradata Systems



  • 7

    Any sufficiently advanced technology is indistinguishable from magic. -- Sir Arthur C. Clarke

  • 8

    Symmetric Key Cryptography

    Symmetric Key Cryptography

    Cryptography in which the same key is used for encryption and decryption

    Single Key Cryptography

    Secret Key Cryptography

    Shared Key Cryptography

    Hello World! B$s70x2G0&vC1lZA Hello World!

  • 9

    Symmetric Key Cryptography

    Symmetric Key Algorithms

    Data Encryption Standard (DES)

    Triple DES (3DES)


    Advanced Encryption Standard (AES)


    International Data Encryption Standard (IDEA)


  • 10

    Symmetric Key Cryptography

    Advanced Encryption Standard (AES)

    Standardized by FIPS in 2001 (FIPS Pub 197)

    Uses 128-bit, 196-bit, or 256-bit keys

    Operates on 16-byte data blocks

    De facto standard for commercial and government applications

    Hello World! Hello World! Js7%[email protected]

  • 11

    Asymmetric Key Cryptography

    Asymmetric Key Cryptography

    Cryptography in which a pair of mathematically related keys are used for encryption and decryption

    Public Key Cryptography

    Data encrypted using one key (e.g., public key) can only be decrypted using the other key (e.g., private key) in the pair - and vice versa

    Hello World! 9vDf4$1j&Fqo*cR1 Hello World!

  • 12

    Asymmetric Key Cryptography

    Asymmetric Key Algorithms

    RSA (Rivest Shamir Adleman)

    Digital Signature Algorithm (DSA)

    Elliptic Curve Cryptography (ECC)

    RSA and DSA are used to provide confidentiality for secure communications protocols such as Transport Layer Security (TLS) and Secure Shell (SSH)

  • 13

    Hash Cryptography

    Hash Cryptography

    Cryptography in which a mathematical algorithm is applied to produce a condensed representation of data

    No key is used

    Hash output is a fixed length - regardless of the size of input data

    It is computationally infeasible to find the data which corresponds to a given hash, or to find different data that produces the same hash

    Hash is one-way; plain text cannot be recreated from the hash

    Hello World!

    Js7%[email protected]



  • 14

    Hash Cryptography

    Hash Cryptographic Algorithms

    Message Digest Algorithm 5 (MD5)

    Produces 128-bit hash

    Secure Hash Algorithm (SHA)

    SHA-1: Produces 160-bit hash

    SHA-256: Produces 256-bit hash

    SHA-512: Produces 512-bit hash

    Applications for hashing

    Password storage

    Message integrity

    Digital signatures

  • 15

    Secret Key Negotiation

    Diffie-Hellman Key Negotiation Protocol

    Allows client and server to agree on a secret key over an insecure communication channel

    Protocol uses two public values

    p is a prime number

    g is an integer less than p with the property that for every number n between 1 and p-1 inclusive, there is a power k of g such that n = gk mod p

    Security is provided by the discrete logarithm problem

    It is computationally infeasible to calculate the shared secret key when the prime p is sufficiently large

  • 16

    Secure Communications Protocols

    Transport Layer Security (TLS) An industry standard protocol for transmitting data in a secure manner over a network

    Supersedes older Secure Sockets Layer (SSL) protocol

    Defines methods for authentication, data encryption, and message integrity

    Underlies protocols such as https, ldaps, ftps, pop3s, etc.

    Secure Shell (SSH) Establishes a cryptographic tunnel

    between two network hosts

    Secures remote logons, file transfers and remote command executions

    SSH Client SSH Server

  • Key Encryption Drivers

    Cryptographic Technology Primer

    Important Considerations

    Use of Encryption in Teradata Systems



  • 18

    The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with armed guards.

    -- Dr. Eugene Spafford, Professor of Computer Science, Purdue University

  • Important Considerations



    Table Size Expansion

    Encryption Key Security

    User/Application Transparency

  • 20


    Cryptographic operations add performance overhead

    Function of hardware generation and speed of processors

    Cryptographic operations are CPU-intensive

    Function of cryptographic algorithm and key strength

    Function of size of data and/or number of columns encrypted

    Function of the characteristics of queries

    Encrypted indexes, range searches, joins on encrypted columns

    Function of the frequency of access

    Function of table skew (affects parallel execution of cryptographic functions)

  • 21








    Crypto Operations

    per Second

    Size of Data

  • 22

    Table Size Expansion

    Encrypted data (ciphertext) requires more storage space than

    plain text data

    Function of the size of original columns

    Padding required for the selected cryptographic algorithm (e.g., 8-byte block, 16-byte block)

    Function of the number of columns encrypted

    Loss of multi-value compression (MVC) benefits


    (14 bytes)

    Original Data

    (2 bytes)


    (4 bytes)

    Original Data

    (12 bytes)


    (16 bytes)

  • 23

    Encryption Key Security

    Encryption keys must be securely managed

    Strong keys should be securely and randomly generated

    Keys must be protected wherever stored (disk or memory)

    Keys should be distributed in a secure manner

    Access to keys should be restricted

    Keys should be periodically rotated

    Keys should be archived with encrypted data

    Random numbers should not be generated with a method chosen at random. -- Donald E. Knuth, The Art of Computer Programming, Volume 2: Seminumerical Algorithms

  • 24

    User/Application Transparency

    Use of encryption should be largely transparent to users and


    No transparency issues with encryption of network traffic

    Cryptographic operations on table data can be largely transparent through the use of views and triggers

    Flexibility needed to directly invoke functions as required to optimize queries

    Use of encrypted data types further improves transparency

    Limits on type functionality often restrict use

    e.g., no statistics collection, cannot be part of an index

  • Key Encryption Drivers

    Cryptographic Technology Primer

    Important Considerations

    Use of Encryption in Teradata Systems



  • 26

    Security doesn't matter until all of a sudden it does - and then it *really* matters. -- Ben Adida, Mozilla Architect

  • Use of Encryption within Teradata Systems


    Securing Remote Support Connectivity

    Securing Network Connections

    Securing Stored Passwords

    Securing Stored Data

    Securing Backups and Archives

  • 28

    Secure Remote Support Connectivity Teradata ServiceConnectTM

    No Virtual Private Network (VPN) or public IP address required

    Firewall-friendly all connections initiated from Service

    Workstation (SWS) on customer side

    Connections secured using HTTPS and 128-bit TLS encryption


    Teradata Customer



    Policy Server




  • 29

    Secure Remote Support Connectivity Teradata ServiceConnectTM

    Outbound Connections TLS port 443

    Remote connectivity Teradata Vital Infrastructure (TVI) alerts, events, and reports Crashdump uploads

    ServiceConnectTM Enhanced Axeda Policy Server

    Enforce restrictions on remote support activities performed by Teradata Logon access File upload/download Command execution

    Audit remote support activities performed by Teradata

  • 30

    Network Traffic Encryption

    Network Traffic Encryption Provides confidentiality for sensitive data when transmitted over untrusted networks

    Protects against compromise by network sniffers


    HTTP over TLS to secure communications to web services

    LDAP over TLS to secure communications to authentication services

    Teradata Generic Security Services (TDGSS) to secure communications between Teradata Clients and Database

  • 31

    Network Traffic Encryption HTTP over TLS (https)

    Secures browser-based connections to web-based services

    TDput port 8443

    Use TDput AllowedCiphers file to manage encryption algorithms allowed for connections

    Viewpoint port 443 or 9443

    Use Certificates portlet to create and install TLS certificate

    Create and install self-signed certificate

    Create certificate signing request and install CA-signed certificate

    Use General portlet to enable Require Access via HTTPS setting

    Aster AppCenter port 443 or 444

    Teradata REST Services Gateway port 1443

  • 32

    Network Traffic Encryption LDAP over TLS (ldaps)

    Secures connections to LDAP authentication and authorization services

    Required to ensure protection of enterprise users and passwords


    Use the LDAP Servers portlet to add and delete ldaps configurations as required

    Use the Certificates portlet to import the Certification Authority (CA) certificates of the LDAP servers into a custom keystore on the Teradata Viewpoint server

    Teradata Gateway

    Configure Teradata Generic Security Services (TDGSS) LDAP method to use one or more ldaps services

    Import the Certification Authority (CA) certificates of the LDAP servers into a custom directory on the Teradata node

    Re-build TDGSS globally distributed object (GDO)

  • 33

    Network Traffic Encryption Teradata Generic Security Services (TDGSS)

    Teradata Generic Security Services (TDGSS)

    Encryption built into the Teradata client/server communications protocol

    Strong symmetric encryption

    AES configurable to use 128-bit (default), 192-bit, or 256-bit keys

    Secure key negotiation algorithm

    Diffie-Hellman key negotiation

    Keys unique to each database session

    Message integrity

    Logon String Encryption

    Logon string (including password) is always encrypted

    Functionality cannot be disabled

  • 34

    Teradata Tools and Utilities (TTU)

    Teradata Tools and Utilities client interfaces are encryption


    Configuration Controls

    Allows for a client interface or tool to be configured to encrypt database sessions

    Programmatic Controls

    Allows for dynamic enabling and disabling of encryption within a database session

  • 35

    Teradata Tools and Utilities (TTU) Configuration Options

    CLIv2 System Parameter Block

    ODBC Driver Options

  • Teradata Tools and Utilities (TTU) Programmatic Options


    TPT Load Operator

  • 37

    Network Traffic Encryption Enforcement Options

    Gateway Control

    Set RequireConfidentiality to enforce network traffic encryption by host group for all sessions

    # gtwcontrol [-g HGID] x yes

    Network Security Policy

    Enforce network traffic encryption and quality of protection (QOP) levels by user, profile, or IP address

  • 38

    SQL Response without/with Encryption

    SELECT * FROM Customer WHERE CCN = '4021884221972216';

  • 39

    Database Password Security

    Database Passwords

    Passwords stored in two data dictionary tables



    Stored passwords secured using a salted SHA-256 cryptographic hash

    Salt is randomly generated data added to the password before it is hashed

    Provides greater protection against pre-computed password attacks

    On logon, user-supplied password is salted, hashed and compared to password in DBC.Dbase table

  • 40

    Database Password Security Example

  • 41

    Teradata Wallet

    Teradata Wallet Provides a mechanism to securely store and protect passwords (or other

    credential information) on client systems for use when connecting to the Teradata Database Passwords encrypted using strong cryptography (AES-256) Includes an API used by Teradata Client tools and utilities for retrieving logon


    Enables compliance with security policies and best practices that prohibit storing of credential information in clear text

    Use of imbedded passwords in BTEQ scripts, FastLoad, MultiLoad, TPump, FastExport scripts, etc.

    Storage of passwords in TPT operator definitions Storage of passwords in ODBC DSNs

  • 42

    Teradata Wallet

    Key Concepts Wallet information is segregated by Client user operating system user

    A given user can only access information from his or her own Wallet

    Wallet contains strings with name-value pairs

    name (used to identify the string) Names are arbitrary and are determined by the user Names are not treated as sensitive/confidential

    value (the secret information) Values are treated as sensitive/confidential

    Encrypted when passed to any system call Encrypted when saved on disk

  • 43

    Teradata Wallet Usage Example

    $ tdwallet add sdlc9000-jb132500

    Enter desired value for the string named sdlc9000-jb132500:

    String named sdlc9000-jb132500 added.

    $ cat logontest.btq


  • 44

    Stored Data Encryption

    Full Disk Encryption (FDE)

    Column-level Encryption/Tokenization

    HDFS File/Volume Encryption

    HIVE Column-level Tokenization

  • 45

    Full Disk Encryption (FDE)

    Full Disk Encryption Hardware-based encryption occurs on self-encrypting disk drives

    Encryption performed using AES with 256-bit keys No impact to performance of disk read/write operations

    Authentication key generated internally and managed on the disk array controller Authentication key is supplied to the disk drive

    during device discovery and is used to unlock the encryption key

    Encryption key is generated by the disk drive and is maintained on the drive

    Protects data if disk drive is lost, stolen, or de-commissioned

    Available for newer generations of Teradata data warehouse systems and appliances

  • 46

    Column-level Data Encryption/Tokenization

    Column-level Data Encryption/Tokenization

    Supports multiple data protection options including strong encryption and tokenization

    Includes external key management systems for secure key generation and protection of keys when stored

    Supports multiple cryptographic algorithms and key strengths

    Provides additional access controls to protect sensitive information (even DBC can't see unencrypted data unless specifically authorized)

    Designed to fully exploit Teradata Database parallelism and scalability

    Enterprise-wide solutions that work with most major databases and operating systems (not just Teradata)

  • 47

    Protegrity Data Protector for Teradata Provides additional separation of duties through a separate Security Manager interface

    for creation and maintenance of security policies Includes a patented key management system for secure key generation and protection

    of keys when stored Supports multiple protection options (strong cryptography, hashing, tokenization)

    Includes additional auditing separate from database audit logs (such as the Access Log)

    File Protector support for Teradata Aster and Hadoop HDFS

    Application Protector support for Hive tokenization

  • 48

    HP Enterprise SecureData Stateless Key Management

    Encryption keys tied directly to identities that map to data in Identity Management Systems Keys dynamically generated only after authentication

    NIST-Standard FF1 AES Hyper Format Preserving Encryption Enables encryption of data without loss of formatting Allows encrypted data to be used as database indices Preserves referential integrity

    Hyper Secure Stateless Tokenization

    Teradata UDA Integration Apply data protection during Teradata import

    processing (e.g., SQL, Sqoop, MapReduce) Use de-identified data within Teradata Use and export re-identified data from Teradata (SQL,

    Hive, MapReduce) Export data and re-identify outside Teradata (ETL

    process) Use storage-level encryption within Hadoop

  • 49

    Vormetric Data Security Platform Addresses industry compliance mandates and government regulations globally by

    securing data in physical, virtual and cloud infrastructures, through Data Encryption, Key Management, Access Policies, Privileged User Control, and Security Intelligence

    Vormetric Protection for Teradata Database Centrally manages encryption across Teradata

    environments - including the Teradata Database and Hortonworks big data nodes

    Enforce granular controls to enable administrators to perform operational tasks, without accessing sensitive data in the clear

    Standards-based application programming interfaces (APIs) and user-defined functions (UDFs) that can be used to perform cryptographic and key management operations

    Hardened, FIPS-certified appliance for administration and key storage


  • Source: Dataguise 50

    Dataguise DgSECURE Discovers and detects sensitive data in structured, semi-structured, and unstructured

    content (including Hadoop and Teradata) Pre-defined templates for sensitive data types to quickly build security policies

    Protects sensitive data at the element level Encryption

    AES and NIST-recognized format-preserving encryption options Encrypt engines run as fully automated processes (agent)

    across standard Hadoop ingest methods (Flume, Sqoop, and FTP) as well as for data at-rest in certified MapReduce and HIVE agents for Cloudera and Hortonworks

    Industry standard Key Management Interface Protocol (KMIP) client support for leading key management facilities

    Masking Obfuscation (one-way operation) Retains statistical distribution of data

    Continuous auditing and monitoring of all attempts to access sensitive data

  • 51

    Teradata Tape Encryption

    Teradata Tape Encryption Hardware-based encryption occurs on tape drives

    Encryption performed using AES with 256-bit keys

    No impact to performance of archive/restore operations

    Encryption supported for backup-to-tape or copy-from-disk/tape

    Encryption managed by two Scalar Key Manager appliances

    Built to comply with the FIPS 140-2 Level 1 federal security standard

    Supported for LTO-5/LTO-6 drives on Quantum Scalar i80, i500 and i6000 libraries

  • 52

    EMC Data Domain Encryption

    EMC Data Domain Encryption

    Inline data encryption with compression

    Encryption performed using AES with 128-bit or 256-bit keys

    Implemented using FIPS 140-2 validated RSA BSafe cryptographic libraries

    Moderate impact to performance of archive/restore operations

    Single system encryption key for all data

    System key encrypted via an access passphrase

    Supported for EMC Data Domain DD4200 Deduplication Storage Systems

  • 53


    Encryption is a powerful tool for protecting sensitive data that

    is transmitted over network or stored on disk or tape

    The use of encryption should be considered as part of a

    comprehensive defense-in-depth strategy for implementation

    of multiple layers of security controls to protect Teradata UDA assets

  • Questions?


  • Thank You



    Follow Me

    Twitter @

    Rate This Session #

    with the PARTNERS Mobile App

    Remember To Share Your Virtual Passes

    [email protected]



  • Backup

  • 57

    Drivers for the Use of Encryption Standards

    Payment Card Industry (PCI) Data Security Standard 3.4 Render PAN, at minimum, unreadable anywhere it is stored (including on portable digital media, backup media, in logs) by using any of the following approaches:

    One-way hashes based on strong cryptography, (hash must be of the entire PAN)

    Truncation (hashing cannot be used to replace the truncated segment of PAN)

    Index tokens and pads (pads must be securely stored)

    Strong cryptography with associated key-management processes and procedures

    3.5 Document and implement procedures to protect keys used to secure stored cardholder data against disclosure and misuse:

    3.5.1 Restrict access to cryptographic keys to the fewest number of custodians necessary

    3.5.2 Store secret and private keys used to encrypt/decrypt cardholder data in one (or more) of the following forms at all times:

    Encrypted with a key-encrypting key that is at least as strong as the data-encrypting key, and that is stored separately from the data-encrypting key

    Within a secure cryptographic device (such as a hardware (host) security module (HSM) or PTS-approved point-of-interaction device)

    As at least two full-length key components or key shares, in accordance with an industry-accepted method

    4.1 Use strong cryptography and security protocols (for example, TLS, IPSEC, SSH, etc.) to safeguard sensitive cardholder data during transmission over open, public networks

  • 58

    Drivers for the Use of Encryption Standards

    HIPAA Privacy Rule 164.312 (a)(2)(iv) Encryption and decryption (Addressable). Implement a mechanism to encrypt and decrypt electronic protected health information

    164.312 (e)(1) Standard: Transmission security. Implement technical security measures to guard against unauthorized access to electronic protected health information that is being transmitted over an electronic communications network

    164.312 (e)(2)(i) Integrity controls (Addressable). Implement security measures to ensure that electronically transmitted electronic protected health information is not improperly modified without detection

    164.312 (e)(2)(ii) Encryption (Addressable). Implement a mechanism to encrypt electronic protected health information whenever deemed appropriate

  • 59

    Drivers for the Use of Encryption Security Breach Notification Laws

    State Legislation

    47 of 50 U.S. states have Security Breach Notification laws

    26 states enforce civil or criminal penalties for failure to promptly disclose breaches

    4 states mandate the use of encryption

    Most states exempt notification if data is encrypted

    AZ SB 1338 1. "Breach", "breach of the security of the system", "breach of the security system" or "security breach" means an unauthorized acquisition of and access to unencrypted or unredacted computerized data that materially compromises the security or confidentiality of personal information maintained by a person

  • 60

    Drivers for the Use of Encryption Regulations

    EU General Data Protection Regulation (GDPR) Applies to the processing of personal data in the context of the activities of an establishment of a

    controller or a processor in the Union, regardless of whether the processing takes place in the Union or not Effective May 25, 2018 Repeals EU Data Protection Directive 95/46/EC

    Article 32 Security of processing

    1. Taking into account the state of the art, the costs of implementation and the nature, scope, context and purposes of processing as well as the risk of varying likelihood and severity for the rights and freedoms of natural persons, the controller and the processor shall implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk, including inter alia as appropriate:

    (a) the pseudonymisation and encryption of personal data;

    Article 34 Communication of a personal data breach to the data subject

    3. The communication to the data subject referred to in paragraph 1 shall not be required if any of the following conditions are met:

    (a) the controller has implemented appropriate technical and organisational protection measures, and that those measures were applied to the personal data affected by the personal data breach, in particular those that render the personal data unintelligible to any person who is not authorised to access it, such as encryption;