Knowledge is Power
Source: http://www.techbite.in/2013/05/debug-hadoop-source-code-using.html If you are someone who wants to dive into Hadoop source code and get a feel of the implementation details of all the abstracted out nitty-gritties of Hadoop’s architectural overview, and want to get your hands dirty by...
Are you seeing something similar to – mkdir: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /user/robin/sort-job-in. Name node is in safe mode. Here is what I did, issued following command to take namenode out of safe mode. >hadoop dfsadmin -safemode leave Safe mode...
Is the cluster set up correctly? The best way to answer this question is empirically: run some jobs and confirm that you get the expected results. Benchmarks make good tests, as you also get numbers that you can compare with...
Hadoop API for .NET ===================== Introduction Hadoop Streaming is a facility for writing map-reduce jobs in the language of you choice. Hadoop API for .NET is a wrapper to Streaming that provides a convenient experience for .NET developers. An understanding...
Are you seeing this ? org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in hadoop/dfs/data: namenode namespaceID = X; datanode namespaceID = Y The fix (in the order) Delete VERSION files in the dfs directory $ find /hadoop/dfs -name VERSION -exec rm -rf “{}”...
Is it 50030 or 50300 for that JobTracker UI? I can never remember! Hadoop’s daemons expose a handful of ports over TCP. Some of these ports are used by Hadoop’s daemons to communicate amongst themselves (to schedule jobs, replicate blocks,...
Source: http://bradhedlund.com/2011/09/10/understanding-hadoop-clusters-and-the-network/ This article is Part 1 in series that will take a closer look at the architecture and methods of a Hadoop cluster, and how it relates to the network and server infrastructure. The content presented here is largely...
More
