hive+hbase整合的时候遇到问题。

天南海北 发布于 2011/12/21 11:45
阅读 2K+
收藏 0

我按照http://www.javabloger.com/article/apache-hadoop-hive-hbase-integration.html这里操作的时候,执行到下一句出现问题,而且并没有插入到hbase里,请问这是为什么?谢谢了!

hive> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=1;

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201112211741_0005, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201112211741_0005
Kill Command = /usr/local/hadoop/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201112211741_0005
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2011-12-21 19:42:55,442 Stage-0 map = 0%,  reduce = 0%
2011-12-21 19:43:40,312 Stage-0 map = 100%,  reduce = 100%
Ended Job = job_201112211741_0005 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201112211741_0005_m_000002 (and more) from job job_201112211741_0005
Exception in thread "Thread-169" java.lang.RuntimeException: Error while reading from task log url
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: http://localhost.localdomain:50060/tasklog?taskid=attempt_201112211741_0005_m_000000_1&start=-8193
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
        at java.net.URL.openStream(URL.java:1010)
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
        ... 3 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
加载中
0
linan
linan
后来解决问题了吗? 我也遇到同样的问题了。 
0
linan
linan

找到问题了,

看错误日志,url是:

http://localhost.localdomain:50060/tasklog?taskid=attempt_201112211741_0005_m_000000_1&start=-8193 

        at


再看看HADOOP的源码:TaskLogServlet 

 

    String attemptIdStr = request.getParameter("attemptid");

 if (attemptIdStr == null) {

      response.sendError(HttpServletResponse.SC_BAD_REQUEST, 
                         "Argument attemptid is required");
      return;
    }
所以taskid应该是attemptid ,hadoop的版本不一致导致的。

你可以在浏览器里访问:http://localhost.localdomain:50060/attemptid?taskid=attempt_201112211741_0005_m_000000_1&start=-8193 

获得真正的错误信息。

返回顶部
顶部