Ambari Meetup: NameNode HA

© Hortonworks Inc. 2013

HDFS NameNode HA

Jing Zhao

Hortonworks


Background and Motivation

• Prior to Hadoop 2.0– One NameNode for metadata management– Single point of failure

• HDFS High Availability– Two NameNodes in the same cluster– Active NameNode: responsive for all client operations– Standby NameNode: maintain enough state to provide a fast failover

• Shared storage – Active NN writes edit log – Standby NN reads edit log and applies to its own namespace– During failover, Standby NN reads all the edits and transitions to Active state

Architecting the Future of Big Data


Quorum Journal Management

• Before QJM– NFS directory as shared storage– Limitation: custom hardware, complicated deployment…

• QJM– Quorum commits of edit log to separate daemons (JournalNodes) – Paxos-like protocol to guarantee correctness

• JournalNodes– A successful edit log commit must write to a majority of JNs– Lightweight, can be co-located on machines for NN, JT, or RM– Recommended number of JNs: 3 or 5




DataNode DataNode DataNode

. . . . .

Standby

NameNode

Active

NameNode

Shared Edits

JournalNode JournalNode JournalNode

Block Reports to Active & Standby

Write Read


Automatic Failover

• Manual failover– Administrative commands

• Use ZKFailoverController for Automatic Failover– ZKFC: a ZooKeeper client that monitors and manages the NN state– Health monitoring– ZooKeeper session management– ZooKeeper-base election




DataNode DataNode DataNode

. . . . .

Standby

NameNode

Write

Active

NameNode

Shared Edits

JournalNode JournalNode JournalNode

ZKFC ZKFC

ZooKeeper Service

Cluster

Monitor and try to take active lock

Monitor liveness and

health. Manage HA

state.

WriteRead

Monitor liveness and

health. Manage HA

state.

Monitor and maintain active lock

Ambari Meetup: NameNode HA

Technology

Transcript of Ambari Meetup: NameNode HA