Is Your Hadoop Environment Secure?

© 2012 Datameer, Inc. All rights reserved.© 2012 Datameer, Inc. All rights reserved.

Building Secure Hadoop Environments

© 2012 Datameer, Inc. All rights reserved.

View the full recording

You can view the full recording of this on-demand webinar with slides at:

http://info.datameer.com/Slideshare-Building-Secure-Hadoop-Environments.html


About our Speaker

Karen HsuWith over 15 years of experience in enterprise software, Karen Hsu has co-authored 4 patents and worked in a variety of engineering, marketing and sales roles.

Most recently she came from Informatica where she worked with the start-ups Informatica purchased to bring data quality, master data management, B2B and data security solutions to market.

Karen has a Bachelors of Science degree in Management Science and Engineering from Stanford University.


About our Speaker

Filip SluneckoFilip is part of the Customer support team at Datameer.

He is a Linux professional and Python enthusiast. Before joining Datameer, he was on the Hadoop team at AVG, an antivirus/security company.

Filip now uses his 8 years experience with Linux servers and Hadoop security to help Datameer customers.

© 2012 Datameer, Inc. All rights reserved.© 2012 Datameer, Inc. All rights reserved.

Building Secure Hadoop Environments


Agenda

Challenges and use cases

Hadoop security landscape

Components for building successful Hadoop environments

Call to Action


Hadoop Data Security Challenges

Architectural issues

Hadoop security is developing

Vendors offer bolt-on solutions

Securosis, Oct 12, 2012

To add security capabilities into a big data environment, the capabilities need to scale with the data… Most security tools fail

to scale and perform with big data environments.- Adrian Lane, Securosis


Hadoop Security Use Cases

Use Case Requirement Example Description

Role based access

Data access is restricted through the abstraction layer

Users have a view of data in Hadoop they can manipulate

Transformation of sensitive values during load

Data is transformed, masked, or encrypted.

Cluster is copied and then masked/transformed so that analysts work on anonymized data


Role Based Access

Data Access

HDFSRestrict View

Map-Reduce

Pig / Hive


Transformation of Sensitive Values

Data Access

HDFS

Map-Reduce

Transform Data

Load


Load

Hybrid of Role Based Access and Transformation of Sensitive Values

Data Access

HDFS

Map-Reduce

Transform Restrict View


Hadoop Security Offerings

Type Description Example vendorsRole based access control Use LDAP / Active Directory (AD)

authentication to identify and manage users. Leveraging Kerberos to provide mutual authentication

Encryption • File encryption • Disk encryption• Format preserving encryption

Masking Data Masking performed before load

Block level encryption Linux directory level encryption with external key store


Components for Building Secure Hadoop Environment

Secure access – SSL

Access controls

Secure authentication

Kerberos

Logging – auditing

File Encryption

Disk encryption


Secure access


Access ControlsDatameer Example

Impersonation

Kerberos

LDAP

Roles

Object permission


Object PermissionDatameer Example

Info graphics

Export job

Workbooks

Data links

Import jobs

Object types


RolesDatameer Example


Remote AuthenticatorDatameer Example

Integrating into an existing infrastructure

Active directory support

Import groups and users to Datameer

Centralized user management


Kerberos


Impersonation


Demonstration


Disk Encryption

Why it’s important• 1 year - 2%

• 2 year - 6-8%

Criteria for success• Encryption per process

• Key management

• Safe and in full compliance with HIPAA, PCI-DSS, FERPA


File EncryptionEmerging Technology

Intel Hadoop

Project Rhino• Encryption and key management.

• A common authorization framework.

• Token based authentication and single sign on.

• Improve audit logging.


Logging and Auditing

Datameer

UI Access Job execution

Hadoop

File access Job runs


Logging and Auditing

Centralized logging

Collectors Storage Real Time Search Visualization

Datameer Datameer* Katta Datameer

Splunk Splunk Elasticsearch Splunk

Flume Elasticsearch Solr Greylog

Greylog Solr Graphite

Hive


Recap

Challenges and use cases

Hadoop security landscape

Components for building successful Hadoop environments• Secure access – SSL

• Access controls

• Secure authentication

• Kerberos

• Logging – auditing

• File Encryption

• Disk encryption


Call to Action

Contact• Filip Slunecko

[email protected]• Karen Hsu khsu

@datameer.com

Meet us atDiscover Big Data 8 City Workshop near you!http://info.datameer.com/Discover-Big-Data-RoadShow.html

Implementing Hadoop Security Workshop• Contact

[email protected] for more details

www.datameer.com

mailto:[email protected]





Online Resources

Try Datameer: www.datameer.com Follow us on Twitter @datameer

http://www.datameer.com/

Is Your Hadoop Environment Secure?

Technology

Transcript of Is Your Hadoop Environment Secure?