Discover HDP 2.1: Using Apache Ambari to Manage Hadoop Clusters
Manage Hadoop Cluster with Ambari
-
Upload
tek-charnsilp-chinprasert -
Category
Technology
-
view
169 -
download
1
Transcript of Manage Hadoop Cluster with Ambari
Mr. Charnsilp Chinprasert Email: [email protected]
Apache Ambari A Hadoop management simpler
Apache Hadoop
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.
Hadoop Cluster
Install Cluster ยังไง?
แก้ไข Config ทั้ง Cluster ยังไง?
Hadoop Ecosystem
Install แต่ละ Service ยังไง?
Config ยังไง
แต่ละ Service ไม่เหมือนกัน?
จัดการแต่ละ Service ยังไง?
จะรู้ได้ยังไงว่า Service ยังปกติ?
What is Apache Ambari?
The Apache Ambari a 100% open source platform provisioning, managing and monitoring Apache Hadoop Cluster
Hadoop Ecosystem
Apache Ambari
Provisioning
Monitoring Management
Apache Ambari• Agent Installation• Service Installation
• Service Management• Cluster Configuration
• OverAll Dashboard• By Host• By Service• Alert Trigger
Provisioning
Monitoring Management
Welcome Ambari!
Install Cluster ยังไง?
System Architecture
Ambari Server
Linux Server
Linux Server
Ambari AgentLinux
Server
Ambari Agent
Linux Server
Ambari Agent
Ambari Agent
Port: 8440/8441(SSL)
Ambari Web
Ambari Api
Installation Agent❖ Provisioning by remote Secure Shell (SSH)
❖ OS Support
❖ RHEL (Redhat Enterprise Linux)
❖ SLES (SuSE Linux Enterprise Server)
❖ Ubuntu
❖ Debian
Ambari Server
Linux Server
Ambari Server
Linux Server
Ambari Agent
SSH SSH
ambari-server start
Setup Cluster
Installation Agent
• Cluster name• Stacks version
• Remote Repos• Local Repos
Installation Agent• Target Hosts
• FQDN• SSH private key
Installation AgentHost Checks
• มีกระบวนการตรวจสอบและconfig OS ให้พร้อมกัน Environment ของHadoop
• ทำให้ Linux ที่นำมาติดตั้งไม่จำเป็นต้อง ลงApplication ใดๆเลย • ถ้ายังไม่ได้ Install หรือ Config Ambari จะจัดการให้• ใช้ Puppet
Install & Configure ServicesSelect
Services
Assign Masters
Assign Slaves & Clients
Customize Services
Review & Confirm
Install & Configure ServicesSelect
Services
Assign Masters
Assign Slaves & Clients
Customize Services
Review & Confirm
Install & Configure ServicesSelect
Services
Assign Masters
Assign Slaves & Clients
Customize Services
Review & Confirm
Install & Configure ServicesSelect
Services
Assign Masters
Assign Slaves & Clients
Customize Services
Review & Confirm
Install & Configure ServicesSelect
Services
Assign Masters
Assign Slaves & Clients
Customize Services
Review & Confirm
Installing
Installing
Installing
Welcome Ambari!
จัดการแต่ละ Service ยังไง?
Service ManagementHDFS Yarn
Service Management
Cluster Configuration
Move namenode ได้
Move namenodeService
Management
Cluster Configuration
Move namenodeService
Management
Cluster Configuration
Move namenodeService
Management
Cluster Configuration
Move namenodeService
Management
Cluster Configuration
Move namenodeService
Management
Cluster Configuration
แก้ไขConfig ทั้ง Cluster ยังไง?
Cluster Configuration
Config ที่เดียว มีผลทั้งCluster
Basic View สำหรับ Configส่วนที่จำเป็นและสำคัญ!!
Service Management
Cluster Configuration
Cluster Configuration
ถ้ามีแก้ไขConfig ระบบจะแนะนำConfigตัวอื่นๆที่ตวรแก้ไขตาม
Service Management
Cluster Configuration
Cluster Configuration
Compare Code ระหว่าง version ได้!!!
Service Management
Cluster Configuration
Configuration History LogService
Management
Cluster Configuration
จะรู้ได้ไงว่าที่เราConfigไปแล้วมันดี?
จะรู้ได้ยังไงว่า Service ยังปกติ?
Monitoring
DashboardDashboard
By Service
By Host
Alert Trigger
By ServiceDashboard
By Service
By Host
Alert Trigger
By HostDashboard
By Service
By Host
Alert Trigger
Alert TriggerDashboard
By Service
By Host
Alert Trigger
Apache Ambari• Agent Installation• Service Installation
• Service Management• Cluster Configuration
• OverAll Dashboard• By Host• By Service• Alert Trigger
Provisioning
Monitoring Management
Q&A
• https://ambari.apache.org• https://issues.apache.org/jira/secure/attachment/12559939/Ambari_Architecture.pdf• http://www.slideshare.net/hortonworks/ambari-agentregistrationflow-17041261• https://cwiki.apache.org/confluence/display/AMBARI/Technology+Stack• http://www.throughwave.co.th/2011/11/10/enterprise-apache-hadoop/• http://www.slideshare.net/hortonworks/apache-ambari-whats-new-in-200
Thank you