i run flume on hadoop command
flume-ng agent -c conf -f conf/flume-twitter.conf -n twitteragent
and not able save twitter data. shows error. posted error below. can have idea ??
2016-01-29 15:48:56,979 (twitter4j async dispatcher[0]) [debug - com.cloudera.flume.source.twittersource$1.onstatus(twittersource.java:108)] artistlr: rt ar tsuk1 unlikely duo: homeland's nina hoss plots art joker christian janko wski #arts https://t.co/hjycldcgk0 2016-01-29 15:48:57,136 (sinkrunner-pollingrunner-defaultsinkprocessor) [info - org.apache.flume.sink.hdfs.bucketwriter.open(bucketwriter.java:234)] creatin g hdfs://l27.0.0.1:9000/tweets/movies/2016/01/29/15//flumedata.1454062721600.tmp 2016-01-29 15:48:57,139 (sinkrunner-pollingrunner-defaultsinkprocessor) [warn - org.apache.flume.sink.hdfs.hdfseventsink.process(hdfseventsink.java:455)] hd fs io error java.io.ioexception: incomplete hdfs uri, no host: hdfs://l27.0.0.1:9000/tweets/movies/2016/01/29/15/flumedata.1454062721600.tmp @ org.apache.hadoop.hdfs.distributedfilesystem.initialize(distributedfilesystem.java:136) @ org.apache.hadoop.fs.filesystem.createfilesystem(filesystem.java:2591) @ org.apache.hadoop.fs.filesystem.access$200(filesystem.java:89) @ org.apache.hadoop.fs.filesystem$cache.getinternal(filesystem.java:2625) @ org.apache.hadoop.fs.filesystem$cache.get(filesystem.java:2607) @ org.apache.hadoop.fs.filesystem.get(filesystem.java:368) @ org.apache.hadoop.fs.path.getfilesystem(path.java:296) @ org.apache.flume.sink.hdfs.bucketwriter$1.call(bucketwriter.java:243) @ org.apache.flume.sink.hdfs.bucketwriter$1.call(bucketwriter.java:235) @ org.apache.flume.sink.hdfs.bucketwriter$9$1.run(bucketwriter.java:679) @ org.apache.flume.auth.simpleauthenticator.execute(simpleauthenticator.java:50) @ org.apache.flume.sink.hdfs.bucketwriter$9.call(bucketwriter.java:676) @ java.util.concurrent.futuretask.run(futuretask.java:262) @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145) @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615) @ java.lang.thread.run(thread.java:745) 2016-01-29 15:48:57,334 (agent-shutdown-hook) [info - org.apache.flume.lifecycle.lifecyclesupervisor.stop(lifecyclesupervisor.java:79)] stopping lifecycle s upervisor 11
look closely @ url: l27.0.0.1
has small l instead of digit 1. typo in configuration?
Comments
Post a Comment