spark1.6.2集群搭建启动问题

fshjq 发布于 2016/12/21 10:57
阅读 396
收藏 0

3台ubuntu虚拟机搭建spark完成后,执行start-all.sh启动spark集群后,

master提示两个worker>>failed to launch org.apache.spark.deploy.worker.Worker。然后我去查看worker的日志,内容如下:

Spark Command: /software/jdk1.8.0_101/bin/java -cp /software/spark-1.6.2-bin-hadoop2.6/conf/:/software/spark-1.6.2-bin-hadoop2.6/lib/spark-assembly-1.6.2-hadoop2.6.0.jar:/software/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/software/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/software/spark-1.6.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/software/hadoop-2.6.4/etc/hadoop/ -Xms1g -Xmx1g org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://spark1:7077
========================================
16/12/20 18:10:10 INFO worker.Worker: Registered signal handlers for [TERM, HUP, INT]
16/12/20 18:10:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/12/20 18:10:15 INFO spark.SecurityManager: Changing view acls to: root
16/12/20 18:10:15 INFO spark.SecurityManager: Changing modify acls to: root
16/12/20 18:10:15 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
16/12/20 18:10:20 INFO util.Utils: Successfully started service 'sparkWorker' on port 40931.
16/12/20 18:10:21 INFO worker.Worker: Starting Spark worker 192.168.40.131:40931 with 2 cores, 512.0 MB RAM
16/12/20 18:10:21 INFO worker.Worker: Running Spark version 1.6.2
16/12/20 18:10:21 INFO worker.Worker: Spark home: /software/spark-1.6.2-bin-hadoop2.6
16/12/20 18:10:31 INFO server.Server: jetty-8.y.z-SNAPSHOT
16/12/20 18:10:31 INFO server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:8081
16/12/20 18:10:31 INFO util.Utils: Successfully started service 'WorkerUI' on port 8081.
16/12/20 18:10:31 INFO ui.WorkerWebUI: Started WorkerWebUI at http://192.168.40.131:8081
16/12/20 18:10:31 INFO worker.Worker: Connecting to master spark1:7077...
16/12/20 18:10:46 INFO worker.Worker: Retrying connection to master (attempt # 1)
16/12/20 18:10:46 INFO worker.Worker: Connecting to master spark1:7077...
16/12/20 18:10:46 INFO worker.Worker: Successfully registered with master spark://spark1:7077

我感觉这就是成功创建worker吧,我运行spark-shell和web UI都正常,为什么master提示fail to launch worker呢??????

加载中
返回顶部
顶部