© 2012 Datameer, Inc. All rights reserved.© 2012 Datameer, Inc. All rights reserved.
Building Secure Hadoop Environments
© 2012 Datameer, Inc. All rights reserved.
View the full recording
You can view the full recording of this on-demand webinar with slides at:
http://info.datameer.com/Slideshare-Building-Secure-Hadoop-Environments.html
© 2012 Datameer, Inc. All rights reserved.
About our Speaker
Karen HsuWith over 15 years of experience in enterprise software, Karen Hsu has co-authored 4 patents and worked in a variety of engineering, marketing and sales roles.
Most recently she came from Informatica where she worked with the start-ups Informatica purchased to bring data quality, master data management, B2B and data security solutions to market.
Karen has a Bachelors of Science degree in Management Science and Engineering from Stanford University.
© 2012 Datameer, Inc. All rights reserved.
About our Speaker
Filip SluneckoFilip is part of the Customer support team at Datameer.
He is a Linux professional and Python enthusiast. Before joining Datameer, he was on the Hadoop team at AVG, an antivirus/security company.
Filip now uses his 8 years experience with Linux servers and Hadoop security to help Datameer customers.
© 2012 Datameer, Inc. All rights reserved.© 2012 Datameer, Inc. All rights reserved.
Building Secure Hadoop Environments
© 2012 Datameer, Inc. All rights reserved.
Agenda
Challenges and use cases
Hadoop security landscape
Components for building successful Hadoop environments
Call to Action
© 2012 Datameer, Inc. All rights reserved.
Hadoop Data Security Challenges
Architectural issues
Hadoop security is developing
Vendors offer bolt-on solutions
Securosis, Oct 12, 2012
To add security capabilities into a big data environment, the capabilities need to scale with the data… Most security tools fail
to scale and perform with big data environments.- Adrian Lane, Securosis
© 2012 Datameer, Inc. All rights reserved.
Hadoop Security Use Cases
Use Case Requirement Example Description
Role based access
Data access is restricted through the abstraction layer
Users have a view of data in Hadoop they can manipulate
Transformation of sensitive values during load
Data is transformed, masked, or encrypted.
Cluster is copied and then masked/transformed so that analysts work on anonymized data
© 2012 Datameer, Inc. All rights reserved.
Role Based Access
Data Access
HDFSRestrict View
Map-Reduce
Pig / Hive
© 2012 Datameer, Inc. All rights reserved.
Transformation of Sensitive Values
Data Access
HDFS
Map-Reduce
Transform Data
Load
© 2012 Datameer, Inc. All rights reserved.
Load
Hybrid of Role Based Access and Transformation of Sensitive Values
Data Access
HDFS
Map-Reduce
Transform Restrict View
© 2012 Datameer, Inc. All rights reserved.
Hadoop Security Offerings
Type Description Example vendorsRole based access control Use LDAP / Active Directory (AD)
authentication to identify and manage users. Leveraging Kerberos to provide mutual authentication
Encryption • File encryption • Disk encryption• Format preserving encryption
Masking Data Masking performed before load
Block level encryption Linux directory level encryption with external key store
© 2012 Datameer, Inc. All rights reserved.
Components for Building Secure Hadoop Environment
Secure access – SSL
Access controls
Secure authentication
Kerberos
Logging – auditing
File Encryption
Disk encryption
© 2012 Datameer, Inc. All rights reserved.
Secure access
© 2012 Datameer, Inc. All rights reserved.
Access ControlsDatameer Example
Impersonation
Kerberos
LDAP
Roles
Object permission
© 2012 Datameer, Inc. All rights reserved.
Object PermissionDatameer Example
Info graphics
Export job
Workbooks
Data links
Import jobs
Object types
© 2012 Datameer, Inc. All rights reserved.
RolesDatameer Example
© 2012 Datameer, Inc. All rights reserved.
Remote AuthenticatorDatameer Example
Integrating into an existing infrastructure
Active directory support
Import groups and users to Datameer
Centralized user management
© 2012 Datameer, Inc. All rights reserved.
Kerberos
© 2012 Datameer, Inc. All rights reserved.
Impersonation
© 2012 Datameer, Inc. All rights reserved.
Demonstration
© 2012 Datameer, Inc. All rights reserved.
Disk Encryption
Why it’s important• 1 year - 2%
• 2 year - 6-8%
Criteria for success• Encryption per process
• Key management
• Safe and in full compliance with HIPAA, PCI-DSS, FERPA
© 2012 Datameer, Inc. All rights reserved.
File EncryptionEmerging Technology
Intel Hadoop
Project Rhino• Encryption and key management.
• A common authorization framework.
• Token based authentication and single sign on.
• Improve audit logging.
© 2012 Datameer, Inc. All rights reserved.
Logging and Auditing
Datameer
UI Access Job execution
Hadoop
File access Job runs
© 2012 Datameer, Inc. All rights reserved.
Logging and Auditing
Centralized logging
Collectors Storage Real Time Search Visualization
Datameer Datameer* Katta Datameer
Splunk Splunk Elasticsearch Splunk
Flume Elasticsearch Solr Greylog
Greylog Solr Graphite
Hive
© 2012 Datameer, Inc. All rights reserved.
Recap
Challenges and use cases
Hadoop security landscape
Components for building successful Hadoop environments• Secure access – SSL
• Access controls
• Secure authentication
• Kerberos
• Logging – auditing
• File Encryption
• Disk encryption
© 2012 Datameer, Inc. All rights reserved.
Call to Action
Contact• Filip Slunecko
[email protected]• Karen Hsu khsu
@datameer.com
Meet us atDiscover Big Data 8 City Workshop near you!http://info.datameer.com/Discover-Big-Data-RoadShow.html
Implementing Hadoop Security Workshop• Contact
[email protected] for more details
www.datameer.com
© 2012 Datameer, Inc. All rights reserved.
Online Resources
Try Datameer: www.datameer.com Follow us on Twitter @datameer
Top Related