Saturday 3 November 2012

Debugging hadoop mapreduce jobs using eclipse in local setup

Add the following line to 'conf/hadoop-env.sh':

export HADOOP_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5009"


Then setup eclipse to connect to the above port (5009) using remote debugging configuration.

Detailed steps: http://code.google.com/p/hadoop-clusternet/wiki/DebuggingJobsUsingEclipse

I tried running this is in the Stand alone configuration of Hadoop.
Have to try this in the pseudo-distributed mode as well, does it work ?

Here is an article that claims to debug the daemon processes:
http://srinathsview.blogspot.in/2012/05/debugging-hadoop-task-tracker-job.html


4 comments:

  1. Thanks for making me this article. You have done a great job by sharing this content in here. Keep writing article like this.

    Cloud Training
    Cloud Training in Chennai
    Big Data Hadoop Admin Training

    ReplyDelete