Hadoop mapreduce not working on mac local setup

Yarn jobs hangs on local mac environment

Missing node in Resource manager UI

There are no nodes in the node list http://192.168.68.145:8001:8088/cluster/nodes

File: yarn-site.xml
Location: /usr/local/Cellar/hadoop/2.7.3/libexec/etc/hadoop/yarn-site.xml

<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>localhost</value>
</property>

 

Resource Manager UI displaying ipv6 for local address

Hadoop does not support ipv6 addresses at this moment – https://wiki.apache.org/hadoop/HadoopIPv6

But unfortunately yarn picks up ipv6 address when network is connected to an ipv6 network or disconnected from all internet interfaces. In the latter scenario it picks up ipv6 address for local loopback.

To fix that, we need to make yarn to stick to ipv4 stack. Append below line to yarn-env.sh

YARN_OPTS="$YARN_OPTS -Djava.net.preferIPv4Stack=true"