1
回答
Spark2.0 SparkML Caused by: java.lang.ClassNotFoundException: libsvm.DefaultSource
利用AWS快速构建适用于生产的无服务器应用程序,免费试用12个月>>>   
def main(args: Array[String]): Unit = {
	val spark = SparkSession
	  .builder()
	  .appName("KMeansExample")
	  .master("local[*]")
	  .config("spark.sql.warehouse.dir", "D:\\Data\\spark-warehouse")
	  .getOrCreate()

    // $example on$
    // Loads data.
    val dataset = spark.read.format("libsvm").load("D:\\Data\\sample_kmeans_data.txt")

    // Trains a k-means model.
    val kmeans = new KMeans().setK(2).setSeed(1L)
    val model = kmeans.fit(dataset)

    // Evaluate clustering by computing Within Set Sum of Squared Errors.
    val WSSSE = model.computeCost(dataset)
    println(s"Within Set Sum of Squared Errors = $WSSSE")

    // Shows the result.
    println("Cluster Centers: ")
    model.clusterCenters.foreach(println)
    // $example off$

    spark.stop()
  }
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.11</artifactId>
    <version>2.0.0</version>
    <scope>provided</scope>
</dependency>

这个是官方的案例。我spark版本是2.0

错误是这个。我google下,网上说是spark1.5之前引用会有这个问题。但是我的版本是2.0现在卡在这里了,哪位能指点下

Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: libsvm. Please find packages at http://spark-packages.org
    at org.apache.spark.sql.execution.datasources.DataSource.lookupDataSource(DataSource.scala:145)
    at org.apache.spark.sql.execution.datasources.DataSource.providingClass$lzycompute(DataSource.scala:78)
    at org.apache.spark.sql.execution.datasources.DataSource.providingClass(DataSource.scala:78)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:310)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:149)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:132)
    at com.study.spark.ml.KMeansExample$.main(KMeansExample.scala:20)
    at com.study.spark.ml.KMeansExample.main(KMeansExample.scala)
Caused by: java.lang.ClassNotFoundException: libsvm.DefaultSource
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5$$anonfun$apply$1.apply(DataSource.scala:130)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5$$anonfun$apply$1.apply(DataSource.scala:130)
    at scala.util.Try$.apply(Try.scala:192)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5.apply(DataSource.scala:130)
    at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$5.apply(DataSource.scala:130)
    at scala.util.Try.orElse(Try.scala:84)
    at org.apache.spark.sql.execution.datasources.DataSource.lookupDataSource(DataSource.scala:130)
    ... 7 more
 

<无标签>
举报
Galy_绿
发帖于1周前 1回/20阅
顶部