Moving JournalNode service from one machine to another
In case you would like to move JournalNode service to another host, here are the steps to do so:
- Put HDFS in safemode
su hdfs c 'hdfs dfsadmin fs hdfs://<active node>:8020 safemode enter'
- Execute a save namespace of the Active NameNode
su hdfs c 'hdfs dfsadmin fs hdfs://<active node>:8020 saveNamespace'
- Stop all services via Ambari
- Add journal node to the Ambari database through the API call:
curl v u admin:admin H "XRequestedBy:ambari" i X POST d '{"host_components" : [{"HostRoles":{"component_name":"JOURNALNODE"}}] }' http://<ambariserver>:8080/api/v1/clusters/<clustername>/hosts? Hosts/host_name=<new journal node hostname>
- Within the Ambari web UI, click on the “Host” tab to display all the hosts for the cluster. Click on the host in which you just added the JournalNode service to in step 4. On this page you should see the JournalNode component in the “Components” section. Next to the JournalNode label is a drop down button, click it and select the “install” option. A progress bar should show up displaying the progress of the installation.
- Once the installation finishes, update the configuration property below within in the Ambari UI within the HDFS section. <dfs.namenode.shared.edits.dir> to include the new journal node and exclude the old one
- Start the new installed journal node
- Start the rest of the journal nodes
- Start the active NameNode to refresh the configuration and then stop it.
- On the previous active NN node (from step 9), run “hdfs namenode initializeSharedEdits force”
- Restart the active NameNode
- On the standby NameNode run “hdfs namenode bootstrapStandby” (it will prompt you to “reformat filesystem in storage directory /hadoop/hdfs/namenode? (Y or N) , answer Y “ )
- Start DataNodes
- Start standby NameNode
- Remove journal node by the following api call
curl u admin:admin H "XRequestedBy: ambari" X DELETE http://<ambari server>8080/api/v1/clusters/<cluster name>/hosts/<old journal node>/host_components/JOURNALNODE
- Start all services (ENSURE THIS IS DONE AND ZOOKEEPER IS UP AND RUNNING OR NEITHER NAMENODE WILL BE ACTIVE)