hadoop异常 Error: Java heap space

爱生活_爱JAVA 发布于 2013/05/17 11:17
阅读 9K+
收藏 1

hadoop:1.0,虚拟机三个节点.在执行测试的列子时,发生如下异常

hadoop jar hadoop-examples-1.0.0.jar pi 10 10

Number of Maps  = 10
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
13/05/17 10:25:31 INFO mapred.FileInputFormat: Total input paths to process : 10
13/05/17 10:25:31 INFO mapred.JobClient: Running job: job_201305170927_0002
13/05/17 10:25:32 INFO mapred.JobClient:  map 0% reduce 0%
13/05/17 10:25:53 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000008_0, Status : FAILED
Error: Java heap space
13/05/17 10:25:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000009_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:10 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:16 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:21 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000008_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:25 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:26 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000009_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:31 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:32 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:34 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:42 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000000_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:43 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:45 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:46 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000006_0, Status : FAILED
Error: Java heap space
13/05/17 10:26:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000000_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:56 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:58 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_1, Status : FAILED
Error: Java heap space
13/05/17 10:26:59 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000006_1, Status : FAILED
Error: Java heap space
13/05/17 10:27:04 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000003_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:11 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000005_2, Status : FAILED
Error: Java heap space
attempt_201305170927_0002_m_000005_2: log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Task).
attempt_201305170927_0002_m_000005_2: log4j:WARN Please initialize the log4j system properly.
13/05/17 10:27:13 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000007_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:18 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000001_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:27 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000002_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:27 INFO mapred.JobClient: Task Id : attempt_201305170927_0002_m_000004_2, Status : FAILED
Error: Java heap space
13/05/17 10:27:32 INFO mapred.JobClient: Job complete: job_201305170927_0002
13/05/17 10:27:32 INFO mapred.JobClient: Counters: 8
13/05/17 10:27:32 INFO mapred.JobClient:   Job Counters 
13/05/17 10:27:32 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=251291
13/05/17 10:27:32 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/05/17 10:27:32 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/05/17 10:27:32 INFO mapred.JobClient:     Rack-local map tasks=14
13/05/17 10:27:32 INFO mapred.JobClient:     Launched map tasks=36
13/05/17 10:27:32 INFO mapred.JobClient:     Data-local map tasks=22
13/05/17 10:27:32 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/05/17 10:27:32 INFO mapred.JobClient:     Failed map tasks=1
13/05/17 10:27:32 INFO mapred.JobClient: Job Failed: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201305170927_0002_m_000007
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
        at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
        at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

求高手解答,指点指点

加载中
0
l
lwtbn1

我今天也遇到这个问题,我用的是0.20.2版本。在mapred-site.xml中设置:

<property>
  <name>mapred.child.java.opts</name>
  <value>-Xmx200m</value>
</property>

值改大一些  1024m。

希望有用!

0
爱生活_爱JAVA
爱生活_爱JAVA

谢谢2位的回答,按照一楼的方法,我修改了一下内存大小.同时也修改了一下 hadoop-env.sh中

export HADOOP_HEAPSIZE=2000

这个的大小,开始装的时候我把这个值设置的太小了.

单机虚拟环境没敢设置太多,原来问题就出在这里了.

一语点醒梦中人啊.

返回顶部
顶部