Is Your Hadoop Environment Secure?

Building Secure Hadoop Environments

View the full recording

You can view the full recording of this on-demand webinar with slides at:

http://info.datameer.com/Slideshare-Building-Secure-Hadoop-Environments.html

About our Speaker

Karen HsuWith over 15 years of experience in enterprise software, Karen Hsu has co-authored 4 patents and worked in a variety of engineering, marketing and sales roles.

Most recently she came from Informatica where she worked with the start-ups Informatica purchased to bring data quality, master data management, B2B and data security solutions to market.

Karen has a Bachelors of Science degree in Management Science and Engineering from Stanford University.

About our Speaker

Filip SluneckoFilip is part of the Customer support team at Datameer.

He is a Linux professional and Python enthusiast. Before joining Datameer, he was on the Hadoop team at AVG, an antivirus/security company.

Filip now uses his 8 years experience with Linux servers and Hadoop security to help Datameer customers.

Building Secure Hadoop Environments

Agenda

Challenges and use cases

Hadoop security landscape

Components for building successful Hadoop environments

Call to Action

Hadoop Data Security Challenges

Architectural issues

Hadoop security is developing

Vendors offer bolt-on solutions

Securosis, Oct 12, 2012

To add security capabilities into a big data environment, the capabilities need to scale with the data… Most security tools fail

to scale and perform with big data environments.- Adrian Lane, Securosis

Hadoop Security Use Cases

Use Case Requirement Example Description

Role based access

Data access is restricted through the abstraction layer

Users have a view of data in Hadoop they can manipulate

Transformation of sensitive values during load

Data is transformed, masked, or encrypted.

Cluster is copied and then masked/transformed so that analysts work on anonymized data

Role Based Access

Data Access

HDFSRestrict View

Map-Reduce

Pig / Hive

Transformation of Sensitive Values

Data Access

Map-Reduce

Transform Data

Hybrid of Role Based Access and Transformation of Sensitive Values

Data Access

Map-Reduce

Transform Restrict View

Hadoop Security Offerings

Type Description Example vendorsRole based access control Use LDAP / Active Directory (AD)

authentication to identify and manage users. Leveraging Kerberos to provide mutual authentication

Encryption • File encryption • Disk encryption• Format preserving encryption

Masking Data Masking performed before load

Block level encryption Linux directory level encryption with external key store

Components for Building Secure Hadoop Environment

Secure access – SSL

Access controls

Secure authentication

Kerberos

Logging – auditing

File Encryption

Disk encryption

Secure access

Access ControlsDatameer Example

Impersonation

Kerberos

Object permission

Object PermissionDatameer Example

Info graphics

Export job

Workbooks

Data links

Import jobs

Object types

RolesDatameer Example

Remote AuthenticatorDatameer Example

Integrating into an existing infrastructure

Active directory support

Import groups and users to Datameer

Centralized user management

Kerberos

Impersonation

Demonstration

Disk Encryption

Why it’s important• 1 year - 2%

• 2 year - 6-8%

Criteria for success• Encryption per process

• Key management

• Safe and in full compliance with HIPAA, PCI-DSS, FERPA

File EncryptionEmerging Technology

Intel Hadoop

Project Rhino• Encryption and key management.

• A common authorization framework.

• Token based authentication and single sign on.

• Improve audit logging.

Logging and Auditing

Datameer

UI Access Job execution

Hadoop

File access Job runs

Logging and Auditing

Centralized logging

Collectors Storage Real Time Search Visualization

Datameer Datameer* Katta Datameer

Splunk Splunk Elasticsearch Splunk

Flume Elasticsearch Solr Greylog

Greylog Solr Graphite

Challenges and use cases

Hadoop security landscape

Components for building successful Hadoop environments• Secure access – SSL

• Access controls

• Secure authentication

• Kerberos

• Logging – auditing

• File Encryption

• Disk encryption

Call to Action

Contact• Filip Slunecko

fslunecko@datameer.com• Karen Hsu khsu

@datameer.com

Meet us atDiscover Big Data 8 City Workshop near you!http://info.datameer.com/Discover-Big-Data-RoadShow.html

Implementing Hadoop Security Workshop• Contact

marketing@datameer.com for more details

www.datameer.com

Online Resources

Try Datameer: www.datameer.com Follow us on Twitter @datameer

Is Your Hadoop Environment Secure?

Technology

Transcript of Is Your Hadoop Environment Secure?

BIG Data & Hadoop Applications in Environment

Big Data Security Analysis And Secure Hadoop Server

Secure your Hadoop clusters with BlueTalon SecureAccess for WebHDFS

Cache Security- Configuring a Secure Environment

Scalable, Secure Storage Biobank...Hadoop has been used by many organizations without demand for strong security [15]. Only few companies have deployed secure Hadoop environments such

Hadoop ecosystem framework n hadoop in live environment

oneM2M Technical Specification Secure Environment Abstraction

Security Management Providing a Secure Environment.

Secure Vehicle Communication 1 February 2006 Secure Execution Environment Secure Execution Environment for V2V and V2I Communication Antonio Kung SEVECOM.

Secure LVC Advanced Training Environment (SLATE)

CTD Enabling Secure Hadoop Environments - …cdn.govexec.com/media/ctd_enabling_secure_hadoop_environments.pdfCTD Enabling Secure Hadoop Environments Author: Kate Tong Created Date:

The Secure Environment

Best Practices for Enterprise User Management in Hadoop Environment

Apache Argus - How do I secure my entire Hadoop cluster? Olivier Renault @ Hortonworks

Hadoop Operations: How to Secure and Control Cluster Access

Enterprise User Management - DOAG Deutsche ORACLE ... · we will see how hadoop admin maintains EUM in a fully secure environment. EUM ... Enterprise User Management On A Kerberos

Secure PHP environment

Centralized HADOOP Secure Data Store with Data Integration ......Hadoop environment using a security key provided by the customer. The Customer The customer is a mobile security provider

Societys Role Towards a Secure Environment

Security 0 The Secure Environment. Security 1 The Secure Environment Security goals (C.I.A.) and threats.