Saturday, 3 November 2012

Debugging hadoop mapreduce jobs using eclipse in local setup

Add the following line to 'conf/hadoop-env.sh':

export HADOOP_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5009"


Then setup eclipse to connect to the above port (5009) using remote debugging configuration.

Detailed steps: http://code.google.com/p/hadoop-clusternet/wiki/DebuggingJobsUsingEclipse

I tried running this is in the Stand alone configuration of Hadoop.
Have to try this in the pseudo-distributed mode as well, does it work ?

Here is an article that claims to debug the daemon processes:
http://srinathsview.blogspot.in/2012/05/debugging-hadoop-task-tracker-job.html


5 comments:

  1. There are lots of information about latest technology and how to get trained in them, like Big Data Hadoop Training in Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(Hadoop Course in Chennai). By the way you are running a great blog. Thanks for sharing this.

    Best Hadoop Training in Chennai
    | Best hadoop training institute in chennai

    ReplyDelete
  2. Informative post on Android mobile application development!!! As the usage of iPhones keep on increasing, there is massive demand for best performing Apps and Games for iOS platform. Best IOS Training in Chennai

    ReplyDelete
  3. Thanks for making me this article. You have done a great job by sharing this content in here. Keep writing article like this.

    Cloud Training
    Cloud Training in Chennai
    Big Data Hadoop Admin Training

    ReplyDelete
  4. Amazing Article ! I have bookmarked this article page as i received good information from this. All the best for the upcoming articles. I will be waiting for your new articles. Thank You ! Kindly Visit Us @ Coimbatore Travels | Ooty Travels | Coimbatore Airport Taxi | Coimbatore taxi

    ReplyDelete
  5. I and my friends were going through the nice, helpful tips from the blog then the sudden came up with an awful suspicion I never expressed respect to the website owner for those secrets.
    safety course in chennai

    ReplyDelete