Common Hadoop Troubleshooting Tips


1. One of the common problems with new installation is connecting datanode to a different namenode

2012-04-04 18:50:38,863 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /var/lib/hadoop-0.20/cache/hdfs/dfs/data: namenode namespaceID = 1635219806; datanode namespaceID = 976537351
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:238)
    at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:153)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:410)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:305)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1627)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1567)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1585)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1711)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1728)

2012-04-04 18:50:38,864 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:


Solution:
Delete old data from datanode and connect to namenode again.