hadoop集群跑wordcount出错:Child Error Caused by Creation of symlink

WisonWang 发布于 2014/01/16 15:58
阅读 1K+
收藏 0

大家好!

我的运行环境:ubuntu12.04LTS+Apache hadoop1.2.1

                    一台master&一台slave,二者配置完全一样:JDK1.7

在第一次跑wordcount程序时总是出错,详细如下:

hadoop@Master :~$ hadoop jar ./hadoop121/hadoop-examples-1.2.1.jar wordcount input output7
Warning: $HADOOP_HOME is deprecated.

14/01/16 15:39:04 INFO input.FileInputFormat: Total input paths to process : 2
14/01/16 15:39:04 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/01/16 15:39:04 WARN snappy.LoadSnappy: Snappy native library not loaded
14/01/16 15:39:04 INFO mapred.JobClient: Running job: job_201401161451_0002
14/01/16 15:39:05 INFO mapred.JobClient:  map 0% reduce 0%
14/01/16 15:39:08 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000003_0, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_0 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_0 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:08 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_0&filter=stdout
14/01/16 15:39:08 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_0&filter=stderr
14/01/16 15:39:11 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000003_1, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_1 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_1 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:11 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_1&filter=stdout
14/01/16 15:39:11 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_1&filter=stderr
14/01/16 15:39:15 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000003_2, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_2 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000003_2 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:16 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_2&filter=stdout
14/01/16 15:39:16 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000003_2&filter=stderr
14/01/16 15:39:22 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000002_0, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_0 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_0 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:22 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_0&filter=stdout
14/01/16 15:39:22 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_0&filter=stderr
14/01/16 15:39:25 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000002_1, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_1 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_1 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:25 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_1&filter=stdout
14/01/16 15:39:25 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_1&filter=stderr
14/01/16 15:39:28 INFO mapred.JobClient: Task Id : attempt_201401161451_0002_m_000002_2, Status : FAILED
java.lang.Throwable: Child Error
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Creation of symlink from /home/hadoop/hadoop121/libexec/../logs/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_2 to /home/hadoop/tmp/ /mapred/local/userlogs/job_201401161451_0002/attempt_201401161451_0002_m_000002_2 failed.
    at org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:110)
    at org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
    at org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
    at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)

14/01/16 15:39:28 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_2&filter=stdout
14/01/16 15:39:28 WARN mapred.JobClient: Error reading task outputhttp://Slave1.Hadoop:50060/tasklog?plaintext=true&attemptid=attempt_201401161451_0002_m_000002_2&filter=stderr
14/01/16 15:39:31 INFO mapred.JobClient: Job complete: job_201401161451_0002
14/01/16 15:39:31 INFO mapred.JobClient: Counters: 4
14/01/16 15:39:31 INFO mapred.JobClient:   Job Counters
14/01/16 15:39:31 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=24035
14/01/16 15:39:31 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
14/01/16 15:39:31 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
14/01/16 15:39:31 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0

在网上找不到解决方案啊!有谁遇过同样的问题?求解答

加载中
0
WisonWang
WisonWang
有木有人给点建议呀?
0
defool
defool
无法创建Log的目录?
0
WeirdBIrd
WeirdBIrd
没玩过  Hadoop , 初步看来,你检查下权限问题
0
刘伟
如下文章参考下, http://blog.sina.com.cn/s/blog_b88e09dd01014x2g.html
0
j
jeanlyn
你jps一下你的slave机子,看下进程启动都正常不?
返回顶部
顶部