1. 1 Hadoop Customer Operations Engineer

2. 20129 Cloudera email: daisuke@cloudera.com twitter: d1ce_ 2 3. Hadoop Sentry 3 4. 5. - RDBMSHadoop Hive(MapReduce), Pig, etc 5 6. - + Hive, Impala, etc SQL () 6 7. - 7 8. 1. 2. 1 3. 8 9. 1. YARN 9 10. 1. YARN 2. 10 11. 11 ... ... 12. 12 ... 13. 13 14. 1. YARN 2. 14 15. AA ( = ) A 15 16. 16 Hadoop (YARN) () 17. 18. Kerberos Hadoop, Hive/Impala HDFS ACL(HDFS-4685) Access Control List Apache Sentry Hive/Impala 18 19. Kerberos Hadoop, Hive/Impala HDFS ACL(HDFS-4685) Access Control List Apache Sentry Hive/Impala 19 20. Kerberos Hadoop, Hive/Impala HDFS ACL(HDFS-4685) Access Control List Apache Sentry Hive/Impala 20 21. - Kerberos Hadoop 21 22. - Kerberos 1 (Kerberos) 22 [daisuke@dice2 ~]$ klist -ef klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_2002) [daisuke@dice2 ~]$ hdfs dfs -ls /user/daisuke 14/07/06 08:19:10 ERROR security.UserGroupInformation: PriviledgedActionException as:daisuke (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 14/07/06 08:19:10 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] ... 23. - Kerberos 2 (Kerberos) 23 [daisuke@dice2 ~]$ kinit Password for daisuke@CLOUDERA: [daisuke@dice2 ~]$ klist -ef Ticket cache: FILE:/tmp/krb5cc_2002 Default principal: daisuke@CLOUDERA ... () ... [daisuke@dice2 ~]$ hdfs dfs -ls /user/daisuke Found 109 items drwx------ - daisuke daisuke 0 2014-07-05 17:00 /user/daisuke/.Trash -rw-r--r-- 1 daisuke daisuke 2307 2014-06-06 03:55 /user/daisuke/TestUDF ... () ... [daisuke@dice2 ~]$ hdfs dfs -put data /user/daisuke [daisuke@dice2 ~]$ 24. - Kerberos Hive HiveServer2(HS2)Kerberos JDBCHS2 Impala Kerberos 24 25. Kerberos Hadoop, Hive/Impala HDFS ACL(HDFS-4685) Access Control List Apache Sentry Hive/Impala 25 26. HS2(Impersonation) Kerberos hive HDFS/(rwx) HiveImpala() 26 27. 27 - Apache Sentry http://sentry.incubator.apache.org/ 28. 28 - Apache Sentry Apache Incubator OracleCloudera / HiveImpalaHadoop 29. Apache Sentry 30. Sentry action SALESCUSTOMERSINSERT 30 server=server1->db=sales->table=customers->action=insert server ----------------------- Hive(Impala) _database _table(view) ----------- /view _URI --------------------- UDF 31. () actionINSERT, SELECT, ALL3 31 action CREATE TABLE ALL database SHOW TABLES SELECT or INSERT table DROP TABLE ALL server REFRESH ALL table COMPUTE STATS ALL table .... .... .... http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/cdh5sg_sentry.html http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_authorization.html 32. () : (HR)HR_TABLEINSERT (*.ini) HDFS() 32 manager=server=server1->db=hr_db->table=hr_table->action=insert 33. Sentry 34. () 34 35. 35 (DB) 36. 36 (DB)PO_DB HR_DB 37. 37 (DB) / 38. 38 (DB) SELECT... HR_.. INSERT...HR_ SELECT...PO_... INSERT...PO_... 39. Sentry 40. Impala Hive Policy Engine Future Policy Provider File Database Sentry 40 Impala HiveServer 2 , Local FS/ HDFS 41. Sentry 41 QueryMR SQL Parse Build Plan SQL Check Sentry 42. 43. Sentry 43 GRANT/REVOKE Sentry1.4 CDH5.1 44. 45. 45 Hadoop (YARN) () => YARN Apache Sentry OSS 46. 46