Showing posts with label nutch. Show all posts
Showing posts with label nutch. Show all posts

Sunday, 30 December 2012

Playing around with Solr

The following article shows how to add a custom field (for ISBN no.s) to be indexed by Solr:

http://robotlibrarian.billdueber.com/solr-field-type-for-numericish-ids/

Would love to write a custom one myself soon !

Saturday, 15 December 2012

Setting up Nutch on MAC - issues [Solved]

I setup nutch following the official tutorial : http://wiki.apache.org/nutch/NutchTutorial

However, I am facing this error when i try to run the command "bin/nutch crawl urls -dir crawl -depth 3 -topN 5" :


2012-12-15 14:43:13.028 java[82636:c07] *** Terminating app due to uncaught exception 'JavaNativeException', reason: 'KrbException: Could not load configuration from SCDynamicStore'
*** First throw call stack:
(
0   CoreFoundation                      0x00007fff8b94cfc6 __exceptionPreprocess + 198
1   libobjc.A.dylib                     0x00007fff87621d5e objc_exception_throw + 43
2   CoreFoundation                      0x00007fff8b9d72a9 -[NSException raise] + 9
3   JavaNativeFoundation                0x0000000108aa1c3f JNFCallStaticVoidMethod + 213
4   libjava.jnilib                      0x0000000108ac1169 Java_sun_security_krb5_SCDynamicStoreConfig_installNotificationCallback + 450
5   JavaNativeFoundation                0x0000000108aa4182 JNFPerformEnvBlock + 86
6   SystemConfiguration                 0x00007fff8703d3b8 rlsPerform + 119
7   CoreFoundation                      0x00007fff8b8bb6e1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
8   CoreFoundation                      0x00007fff8b8baf4d __CFRunLoopDoSources0 + 253
9   CoreFoundation                      0x00007fff8b8e1d39 __CFRunLoopRun + 905
10  CoreFoundation                      0x00007fff8b8e1676 CFRunLoopRunSpecific + 230
11  java                                0x00000001081c0843 java + 18499
12  java                                0x00000001081c029a java + 17050
13  java                                0x00000001081bda98 java + 6808
)
terminate called throwing an exceptionAbort trap: 6

About  SCDynamicStore : http://developer.apple.com/library/mac/#documentation/Networking/Reference/SCDynamicStore/Reference/reference.html

SOLUTION:

Put the following line in the bin/nutch file :

NUTCH_OPTS="$NUTCH_OPTS -Djava.security.krb5.realm= -Djava.security.krb5.kdc="

based on https://issues.apache.org/jira/browse/HADOOP-7489