Apache Ambari - What's New in 1.6.1
23
© Hortonworks Inc. 2014 Apache Ambari 1.6.1 July 2014
-
Upload
hortonworks -
Category
Technology
-
view
1.076 -
download
3
description
Learn about the new features of Apache Ambari 1.6.1. http://ambari.apache.org/
Transcript of Apache Ambari - What's New in 1.6.1
© Hortonworks Inc. 2014
Apache Ambari1.6.1
July 2014
© Hortonworks Inc. 2014
What’s New in 1.6.1
• New Host Checks• External Database Connection Testing• Stack Repository Mgmt Enhancements• Performance Improvements
• For a complete list of changeshttps://issues.apache.org/jira/browse/AMBARI/fixforversion/12326545
Page 2
© Hortonworks Inc. 2014
New Host ChecksCustom JDKHostname resolutionReverse hostname
Page 3
© Hortonworks Inc. 2014
Background: Current JDK Options
• Ambari Server setup provides three JDK options
• Use options [1] or [2] for Ambari to automatically download + install the JDK on all hosts–Requires Internet access
• Use option [3] to specify an already installed JDK–This is a must when performing a no-Internet / local install–Ambari expects this JDK to already be installed on ALL hosts
Page 4
© Hortonworks Inc. 2014
Custom JDK Host Check
• Problem: when selecting option [3], if a host did not already have the JDK installed, service and component starts would fail late in the Cluster Install Wizard process– Difficult to troubleshoot
• Solution: New “Custom JDK” Host Check performed during Host Registration– Confirms the Custom JDK is installed on each host
Page 5
© Hortonworks Inc. 2014
Hostname Resolution Check
• Checks that each host can resolve hostname for other hosts in the cluster
Page 6
© Hortonworks Inc. 2014
Reverse Hostname Check
• On each host, check hostname matches reverse lookup
Page 7
© Hortonworks Inc. 2014
External Database Connections
• Simplified JDBC Driver Setup• Ability to “test” Database Connections
Page 8
© Hortonworks Inc. 2014
Background: External Database Drivers
Page 9
• Ambari supports using an external database for Hive Metastore and Oozie
• Ambari 1.6.0 (or earlier) experience required user to put the JDBC Driver ON the PLANNED host for Hive Metastore or Oozie Server PRIOR to starting cluster install
• Potential chances for error– Wrong host– JDBC Driver in wrong location– Miss this step completely
© Hortonworks Inc. 2014
Simplified JDBC Driver Setup
• User provides driver during ambari-server setup…ambari-server setup --jdbc-db=oracle --jdbc-driver=/path/to/odjcb6.jar
• During Cluster Install Wizard…–Ambari checks that user provided the driver during setup–Ambari copies the driver to the correct destination host
Page 10
AMBARISERVER
HiveMetastore
OozieJDBC Driver JAR
© Hortonworks Inc. 2014
JDBC Driver Setup Reminder
Page 11
© Hortonworks Inc. 2014
Test Database Connection
• Get JDBC driver from Ambari Server• Distribute to target machine (Oozie or Hive Metastore)• Test the database connection (show success/fail)• Popup warning if user proceeds without testing
Page 12
1
2
3
© Hortonworks Inc. 2014
Stack Repository MgmtHDP and HDP-UTILS managed per StackTemplate-driven .repo filesManage Base URLs from Ambari Web
Page 13
© Hortonworks Inc. 2014
Background: Repositories
Page 14
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
Ambari Server
and Agent
Packages
Hadoop Stack
Packages
Stack Utils
LZO, ExtJS,
Snappy,
Nagios, Ganglia
© Hortonworks Inc. 2014
Background: .repo Files
Page 15
AMBARI Repo HDP-UTILS Repo
HDP Repo
AMBARI HDP STACK
ambari.repo
HDP.repo
BUT
HDP-UTILS is
Stack specific
© Hortonworks Inc. 2014
ambari.repo Changes
Page 16
The ambari.repo file no longer includes HDP-UTILS, only AMBARI
[root@c6402 ~]# more /etc/yum.repos.d/ambari.repo
[AMBARI.1.6.1-1.x]name=Ambari 1.xbaseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/gpgcheck=1gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1
1
© Hortonworks Inc. 2014
HDP + HDP-UTILS Stack Repositories
Page 17
Prompted to provide HDP and HDP-UTILS Stack repository Base URLs2
© Hortonworks Inc. 2014
HDP.repo and HDP-UTILS.repo
Page 18
Ambari Server Ambari Agent
4) Server sends HDP.repo & HDP-UTILS.repo
2) Reads Stack Base URL values
6) Agent uses HDP.repo HDP-UTILS.repo when installing packages
1) Agent reports
3) Creates HDP.repo & HDP-UTILS.repo **
5) Agent puts HDP.repo & HDP-UTILS.repo in place
** Generated from Stack template file
© Hortonworks Inc. 2014
Stack .repo Template File
• HDP.repo and HDP-UTILS.repo template file
/var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/templates/repo_suse_rhel.j2
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/templates/repo_suse_rhel.j2
Page 19
Ambari generates the Stack .repo files from a template3
© Hortonworks Inc. 2014
Managing Repositories
Page 20
Manage repository Base URLs from Ambari Web4
© Hortonworks Inc. 2014
Performance + Scale
Page 21
© Hortonworks Inc. 2014
2,000 Node Clusters
• Testing Ambari with 2,000 node clusters• Continued to move Web UI logic to Server• Big focus on Hosts page (filtering, sorting, paging)
• Ambari Session at Hadoop Summit 2014http://www.slideshare.net/Hadoop_Summit/managing-2000-node-cluster-with-ambari
http://www.youtube.com/watch?v=sVsHfmZ4s2M
Page 22
© Hortonworks Inc. 2014
Learn More
Page 23
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI
