sqoop 从mysql数据迁移到hdfs中

神是到着念 发布于 2014/08/11 23:41
阅读 723
收藏 0

我在连接数据库的时候显示都能连接上,list表的时候也能显示所有的表。但是我在导入数据的时候走到:

guoyachao@master :~/software/hbase-0.94.11$ sqoop list-tables --connect  jdbc:mysql://192.168.1.100:3306/test --username root --password root
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
14/08/11 06:59:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/11 06:59:34 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
test_hbase
guoyachao@master :~/software/hbase-0.94.11$ sqoop import --connect  jdbc:mysql://192.168.1.100:3306/test --username root --password root --table test_hbase -m 1
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
14/08/11 07:02:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/11 07:02:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/08/11 07:02:11 INFO tool.CodeGenTool: Beginning code generation
14/08/11 07:02:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_hbase` AS t LIMIT 1
14/08/11 07:02:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_hbase` AS t LIMIT 1
14/08/11 07:02:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/guoyachao/software/hadoop-1.0.4
注意:/tmp/sqoop-guoyachao/compile/2acdd51525ffead97c25d28580b4b077/test_hbase.java 使用或覆盖了已过时的 API。
注意:要了解详细信息,请使用 -Xlint:deprecation 重新编译。
14/08/11 07:02:19 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-guoyachao/compile/2acdd51525ffead97c25d28580b4b077/test_hbase.jar
14/08/11 07:02:20 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/08/11 07:02:20 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/08/11 07:02:20 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/08/11 07:02:20 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/08/11 07:02:20 INFO mapreduce.ImportJobBase: Beginning import of test_hbase

这一步的时候就卡住啦。我搭的环境是分布式的,这个会有影响吗? 我在伪分布式的环境中就可以导入数据,这是为什么呢?求大神指教。


加载中
0
神是到着念
神是到着念

查看拉hadoop的日志,发现我的hadoop处于安全模式,造成不能修改和新增,所以停到这里不走啦,最后在hdfs-site.xml文件下配置

        <property>

                <name>dfs.safemode.threshold.pct</name>
                <value>0</value>
<final>true</final>
        </property>就OK啦
0
humeifang
humeifang
你好,你用的是那个版本的,我从mysql导入数据时出错SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@6a78afa0 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@6a78afa0 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
请问你可以分享你的环境搭建的步骤吗?
0
humeifang
humeifang
可以连接数据库,但是导入的时候出错,求解。。。谢谢
返回顶部
顶部