Saturday, 3 November 2012

Debugging hadoop mapreduce jobs using eclipse in local setup

Add the following line to 'conf/hadoop-env.sh':

export HADOOP_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5009"


Then setup eclipse to connect to the above port (5009) using remote debugging configuration.

Detailed steps: http://code.google.com/p/hadoop-clusternet/wiki/DebuggingJobsUsingEclipse

I tried running this is in the Stand alone configuration of Hadoop.
Have to try this in the pseudo-distributed mode as well, does it work ?

Here is an article that claims to debug the daemon processes:
http://srinathsview.blogspot.in/2012/05/debugging-hadoop-task-tracker-job.html


2 comments:

  1. There are lots of information about latest technology and how to get trained in them, like Big Data Hadoop Training in Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(Hadoop Course in Chennai). By the way you are running a great blog. Thanks for sharing this.

    Best Hadoop Training in Chennai
    | Best hadoop training institute in chennai

    ReplyDelete
  2. Informative post on Android mobile application development!!! As the usage of iPhones keep on increasing, there is massive demand for best performing Apps and Games for iOS platform. Best IOS Training in Chennai

    ReplyDelete