hadoop:1.0,虚拟机三个节点.在执行测试的列子时,发生如下异常
hadoop jar hadoop-examples-1.0.0.jar pi 10 10
Number of Maps = 10
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
13/05/17 10:25:31 INFO mapred.FileInputFormat: Total input paths to process : 10
13/05/17 10:25:31 INFO mapred.JobClient: Running job: job_201305170927_0002
13/05/17 10:25:32 INFO mapred.JobClient: map 0% reduce 0%
13/05/17 10:25:53 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000008_0, Status : FAILED
Error: Java heap space
13/05/17 10:25:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000009_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:10 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:16 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:21 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000008_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:25 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:26 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000009_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:31 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:32 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:34 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:42 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000000_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:43 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:45 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:46 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000006_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000000_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:58 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:59 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000006_1, Status : FAILED
Error: Java heap space
13/05/17 10:27:04 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:11 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_2, Status : FAILED
Error: Java heap space
attempt_201305170927_0002_m_000005_2: log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Task).
attempt_201305170927_0002_m_000005_2: log4j:WARN Please initialize the log4j system properly.
13/05/17 10:27:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:18 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:27 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:27 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:32 INFO mapred.JobClient: Job complete: job_201305170927_0002
13/05/17 10:27:32 INFO mapred.JobClient: Counters: 8
13/05/17 10:27:32 INFO mapred.JobClient: Job Counters
13/05/17 10:27:32 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=251291
13/05/17 10:27:32 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
13/05/17 10:27:32 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
13/05/17 10:27:32 INFO mapred.JobClient: Rack-local map tasks=14
13/05/17 10:27:32 INFO mapred.JobClient: Launched map tasks=36
13/05/17 10:27:32 INFO mapred.JobClient: Data-local map tasks=22
13/05/17 10:27:32 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
13/05/17 10:27:32 INFO mapred.JobClient: Failed map tasks=1
13/05/17 10:27:32 INFO mapred.JobClient: Job Failed: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201305170927_0002_m_000007
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
求高手解答,指点指点
我今天也遇到这个问题,我用的是0.20.2版本。在mapred-site.xml中设置:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx200m</value>
</property>
值改大一些 1024m。
希望有用!
谢谢2位的回答,按照一楼的方法,我修改了一下内存大小.同时也修改了一下 hadoop-env.sh中
export HADOOP_HEAPSIZE=2000
这个的大小,开始装的时候我把这个值设置的太小了.
单机虚拟环境没敢设置太多,原来问题就出在这里了.
一语点醒梦中人啊.