0
回答
在idea上建立一个sbt工程总是编译不过去,代码没有任何错误提示,求大神解答
【阿里云】新一代云服务器,9元体验半年,限时领取!>>>   

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
[info] Loading global plugins from /home/aurora/.sbt/0.13/plugins
[info] Updating {file:/home/aurora/.sbt/0.13/plugins/}global-plugins...
[info] Resolving org.jetbrains#sbt-structure-extractor-0-13;7.0.0-12-ga98ec5e ...
[info] Resolving com.googlecode.java-diff-utils#diffutils;1.2 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-sbt#sbt;0.13.15 ...
[info] Resolving org.scala-sbt#main;0.13.15 ...
[info] Resolving org.scala-sbt#actions;0.13.15 ...
[info] Resolving org.scala-sbt#classpath;0.13.15 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
[info] Resolving org.scala-sbt#interface;0.13.15 ...
[info] Resolving org.scala-sbt#io;0.13.15 ...
[info] Resolving org.scala-sbt#control;0.13.15 ...
[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...
[info] Resolving org.scala-sbt#completion;0.13.15 ...
[info] Resolving org.scala-sbt#collections;0.13.15 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Resolving org.scala-sbt#api;0.13.15 ...
[info] Resolving org.scala-sbt#classfile;0.13.15 ...
[info] Resolving org.scala-sbt#logging;0.13.15 ...
[info] Resolving org.scala-sbt#process;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...
[info] Resolving org.scala-sbt#relation;0.13.15 ...
[info] Resolving org.scala-sbt#compile;0.13.15 ...
[info] Resolving org.scala-sbt#persist;0.13.15 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...
[info] Resolving org.scala-sbt#ivy;0.13.15 ...
[info] Resolving org.scala-sbt#cross;0.13.15 ...
[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...
[info] Resolving com.jcraft#jsch;0.1.50 ...
[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...
[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...
[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#run;0.13.15 ...
[info] Resolving org.scala-sbt#task-system;0.13.15 ...
[info] Resolving org.scala-sbt#tasks;0.13.15 ...
[info] Resolving org.scala-sbt#tracking;0.13.15 ...
[info] Resolving org.scala-sbt#cache;0.13.15 ...
[info] Resolving org.scala-sbt#testing;0.13.15 ...
[info] Resolving org.scala-sbt#test-agent;0.13.15 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.15 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...
[info] Resolving org.scala-sbt#command;0.13.15 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-sbt#logic;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...
[info] Resolving org.scala-lang#jline;2.10.6 ...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Loading project definition from /home/aurora/scala_workspace/untitled/project
[info] Updating {file:/home/aurora/scala_workspace/untitled/project/}untitled-build...
[info] Resolving org.scala-sbt#global-plugins;0.0 ...
[info] Resolving org.jetbrains#sbt-structure-extractor-0-13;7.0.0-12-ga98ec5e ...
[info] Resolving com.googlecode.java-diff-utils#diffutils;1.2 ...
[info] Resolving org.scala-lang#scala-library;2.10.6 ...
[info] Resolving org.scala-sbt#sbt;0.13.15 ...
[info] Resolving org.scala-sbt#main;0.13.15 ...
[info] Resolving org.scala-sbt#actions;0.13.15 ...
[info] Resolving org.scala-sbt#classpath;0.13.15 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
[info] Resolving org.scala-sbt#interface;0.13.15 ...
[info] Resolving org.scala-sbt#io;0.13.15 ...
[info] Resolving org.scala-sbt#control;0.13.15 ...
[info] Resolving org.scala-sbt#launcher-interface;1.0.1 ...
[info] Resolving org.scala-sbt#completion;0.13.15 ...
[info] Resolving org.scala-sbt#collections;0.13.15 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Resolving org.scala-sbt#api;0.13.15 ...
[info] Resolving org.scala-sbt#classfile;0.13.15 ...
[info] Resolving org.scala-sbt#logging;0.13.15 ...
[info] Resolving org.scala-sbt#process;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.15 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.15 ...
[info] Resolving org.scala-sbt#relation;0.13.15 ...
[info] Resolving org.scala-sbt#compile;0.13.15 ...
[info] Resolving org.scala-sbt#persist;0.13.15 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.15 ...
[info] Resolving org.scala-sbt#ivy;0.13.15 ...
[info] Resolving org.scala-sbt#cross;0.13.15 ...
[info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-48dd0744422128446aee9ac31aa356ee203cc9f4 ...
[info] Resolving com.jcraft#jsch;0.1.50 ...
[info] Resolving org.scala-sbt#serialization_2.10;0.1.2 ...
[info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.1 ...
[info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
[info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
[info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
[info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
[info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
[info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
[info] Resolving org.scala-sbt#run;0.13.15 ...
[info] Resolving org.scala-sbt#task-system;0.13.15 ...
[info] Resolving org.scala-sbt#tasks;0.13.15 ...
[info] Resolving org.scala-sbt#tracking;0.13.15 ...
[info] Resolving org.scala-sbt#cache;0.13.15 ...
[info] Resolving org.scala-sbt#testing;0.13.15 ...
[info] Resolving org.scala-sbt#test-agent;0.13.15 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.15 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.15 ...
[info] Resolving org.scala-sbt#command;0.13.15 ...
[info] Resolving org.scala-sbt#template-resolver;0.1 ...
[info] Resolving org.scala-sbt#logic;0.13.15 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.15 ...
[info] Resolving org.scala-lang#jline;2.10.6 ...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Set current project to untitled (in build file:/home/aurora/scala_workspace/untitled/)
> ;reload; */*:dumpStructureTo /tmp/sbt-structure.xml download resolveClassifiers resolveSbtClassifiers
[info] Loading global plugins from /home/aurora/.sbt/0.13/plugins
[info] Loading project definition from /home/aurora/scala_workspace/untitled/project
[info] Set current project to untitled (in build file:/home/aurora/scala_workspace/untitled/)
[info] Updating {file:/home/aurora/scala_workspace/untitled/}untitled...
[info] Resolving org.scala-lang#scala-library;2.12.2 ...
[info] Resolving org.scala-lang#scala-compiler;2.12.2 ...
[info] Resolving org.scala-lang#scala-reflect;2.12.2 ...
[info] Resolving org.scala-lang.modules#scala-xml_2.12;1.0.6 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Done updating.
[info] Resolving org.scala-lang#scala-library;2.12.2 ...
[info] Resolving org.scala-lang#scala-reflect;2.12.2 ...
[info] Resolving org.scala-lang.modules#scala-xml_2.12;1.0.6 ...
[info] Resolving org.scala-lang#scala-compiler;2.12.2 ...
[info] Resolving jline#jline;2.14.3 ...
[info] Writing structure to /tmp/sbt-structure.xml...
[info] Done.
[success] Total time: 1 s, completed 2017-6-19 14:40:42
> ; {file:/home/aurora/scala_workspace/untitled/}untitled/products; {file:/home/aurora/scala_workspace/untitled/}untitled/test:products
[info] Compiling 6 Scala sources to /home/aurora/scala_workspace/untitled/target/scala-2.12/classes...
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:4: object apache is not a member of package org
[error] import org.apache.spark.SparkContext._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:5: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: not found: type SparkContext
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: com.sun.corba.se.impl.orbutil.graph.Graph does not take type parameters
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type SparkContext
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type Graph
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:3: object apache is not a member of package org
[error] import org.apache.spark.broadcast.Broadcast
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:26: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VD,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type SparkContext
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                 ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                                                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type SparkContext
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Graph
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type VertexRDD
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Broadcast
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                                                                              ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:163: not found: type EdgeTriplet
[error]   private def sendMsg(et:EdgeTriplet[VertexState,Long]) = {
[error]                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                                                                         ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:288: not found: value Graph
[error]     val compressedGraph = Graph(newVerts,edges)
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:289: not found: value PartitionStrategy
[error]       .partitionBy(PartitionStrategy.EdgePartition2D).groupEdges(_+_)
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:292: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VertexState,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:319: not found: type Graph
[error]    private def printlouvain(graph:Graph[VertexState,Long]) = {
[error]                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:327: not found: type Graph
[error]    private def printedgetriplets(graph:Graph[VertexState,Long]) = {
[error]                                        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:7: object apache is not a member of package org
[error] import org.apache.spark.Logging
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type SparkContext
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type Graph
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                                               ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type SparkContext
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type Graph
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:3: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:4: not found: object scopt
[error] import scopt.OptionParser._
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:32: not found: value scopt
[error]     val parser = new scopt.OptionParser[Config](this.getClass().toString()){
[error]                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:33: not found: value opt
[error]       opt[String]('i',"input") action {(x,c)=> c.copy(input=x)}  text("input file or path  Required.")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:34: not found: value opt
[error]       opt[String]('o',"output") action {(x,c)=> c.copy(output=x)} text("output path Required")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:35: not found: value opt
[error]       opt[String]('m',"master") action {(x,c)=> c.copy(master=x)} text("spark master, local[N] or spark://host:port default=local")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:36: not found: value opt
[error]       opt[String]('h',"sparkhome") action {(x,c)=> c.copy(sparkHome=x)} text("SPARK_HOME Required to run on cluster")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:37: not found: value opt
[error]       opt[String]('n',"jobname") action {(x,c)=> c.copy(appName=x)} text("job name")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:38: not found: value opt
[error]       opt[Int]('p',"parallelism") action {(x,c)=> c.copy(parallelism=x)} text("sets spark.default.parallelism and minSplits on the edge file. default=based on input partitions")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:39: not found: value opt
[error]       opt[Int]('x',"minprogress") action {(x,c)=> c.copy(minProgress=x)} text("Number of vertices that must change communites for the algorithm to consider progress. default=2000")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:40: not found: value opt
[error]        opt[Int]('y',"progresscounter") action {(x,c)=> c.copy(progressCounter=x)} text("Number of times the algorithm can fail to make progress before exiting. default=1")
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:41: not found: value opt
[error]       opt[String]('d',"edgedelimiter") action {(x,c)=> c.copy(edgedelimiter=x)} text("specify input file edge delimiter. default=\",\"")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:42: not found: value opt
[error]       opt[String]('j',"jars") action {(x,c)=> c.copy(jars=x)} text("comma seperated list of jars")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:43: not found: value opt
[error]       opt[Boolean]('z',"ipaddress") action {(x,c)=> c.copy(ipaddress=x)} text("Set to true to convert ipaddresses to Long ids. Defaults to false")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: not found: value arg
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value copy is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value properties is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:78: not found: type SparkContext
[error]     var sc: SparkContext = null
[error]             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:81: not found: type SparkContext
[error]       sc = new SparkContext(master, jobname)
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:85: not found: type SparkContext
[error]       sc = new SparkContext(master,jobname,sparkhome,jars.split(","))
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:104: not found: value Graph
[error]     val graph = Graph.fromEdges(edgeRDD, None)
[error]                 ^
[error] 59 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 202 s, completed 2017-6-19 16:32:35
> ; {file:/home/aurora/scala_workspace/untitled/}untitled/products; {file:/home/aurora/scala_workspace/untitled/}untitled/test:products
[info] Compiling 6 Scala sources to /home/aurora/scala_workspace/untitled/target/scala-2.12/classes...
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:4: object apache is not a member of package org
[error] import org.apache.spark.SparkContext._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:5: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: not found: type SparkContext
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: com.sun.corba.se.impl.orbutil.graph.Graph does not take type parameters
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type SparkContext
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type Graph
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:3: object apache is not a member of package org
[error] import org.apache.spark.broadcast.Broadcast
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:26: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VD,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type SparkContext
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                 ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                                                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type SparkContext
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Graph
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type VertexRDD
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Broadcast
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                                                                              ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:163: not found: type EdgeTriplet
[error]   private def sendMsg(et:EdgeTriplet[VertexState,Long]) = {
[error]                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                                                                         ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:288: not found: value Graph
[error]     val compressedGraph = Graph(newVerts,edges)
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:289: not found: value PartitionStrategy
[error]       .partitionBy(PartitionStrategy.EdgePartition2D).groupEdges(_+_)
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:292: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VertexState,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:319: not found: type Graph
[error]    private def printlouvain(graph:Graph[VertexState,Long]) = {
[error]                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:327: not found: type Graph
[error]    private def printedgetriplets(graph:Graph[VertexState,Long]) = {
[error]                                        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:7: object apache is not a member of package org
[error] import org.apache.spark.Logging
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type SparkContext
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type Graph
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                                               ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type SparkContext
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type Graph
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:3: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:4: not found: object scopt
[error] import scopt.OptionParser._
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:32: not found: value scopt
[error]     val parser = new scopt.OptionParser[Config](this.getClass().toString()){
[error]                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:33: not found: value opt
[error]       opt[String]('i',"input") action {(x,c)=> c.copy(input=x)}  text("input file or path  Required.")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:34: not found: value opt
[error]       opt[String]('o',"output") action {(x,c)=> c.copy(output=x)} text("output path Required")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:35: not found: value opt
[error]       opt[String]('m',"master") action {(x,c)=> c.copy(master=x)} text("spark master, local[N] or spark://host:port default=local")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:36: not found: value opt
[error]       opt[String]('h',"sparkhome") action {(x,c)=> c.copy(sparkHome=x)} text("SPARK_HOME Required to run on cluster")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:37: not found: value opt
[error]       opt[String]('n',"jobname") action {(x,c)=> c.copy(appName=x)} text("job name")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:38: not found: value opt
[error]       opt[Int]('p',"parallelism") action {(x,c)=> c.copy(parallelism=x)} text("sets spark.default.parallelism and minSplits on the edge file. default=based on input partitions")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:39: not found: value opt
[error]       opt[Int]('x',"minprogress") action {(x,c)=> c.copy(minProgress=x)} text("Number of vertices that must change communites for the algorithm to consider progress. default=2000")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:40: not found: value opt
[error]        opt[Int]('y',"progresscounter") action {(x,c)=> c.copy(progressCounter=x)} text("Number of times the algorithm can fail to make progress before exiting. default=1")
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:41: not found: value opt
[error]       opt[String]('d',"edgedelimiter") action {(x,c)=> c.copy(edgedelimiter=x)} text("specify input file edge delimiter. default=\",\"")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:42: not found: value opt
[error]       opt[String]('j',"jars") action {(x,c)=> c.copy(jars=x)} text("comma seperated list of jars")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:43: not found: value opt
[error]       opt[Boolean]('z',"ipaddress") action {(x,c)=> c.copy(ipaddress=x)} text("Set to true to convert ipaddresses to Long ids. Defaults to false")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: not found: value arg
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value copy is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value properties is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:78: not found: type SparkContext
[error]     var sc: SparkContext = null
[error]             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:81: not found: type SparkContext
[error]       sc = new SparkContext(master, jobname)
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:85: not found: type SparkContext
[error]       sc = new SparkContext(master,jobname,sparkhome,jars.split(","))
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:104: not found: value Graph
[error]     val graph = Graph.fromEdges(edgeRDD, None)
[error]                 ^
[error] 59 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 514 s, completed 2017-6-19 20:31:51
> ; {file:/home/aurora/scala_workspace/untitled/}untitled/products; {file:/home/aurora/scala_workspace/untitled/}untitled/test:products
[info] Compiling 6 Scala sources to /home/aurora/scala_workspace/untitled/target/scala-2.12/classes...
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:4: object apache is not a member of package org
[error] import org.apache.spark.SparkContext._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:5: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: not found: type SparkContext
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: com.sun.corba.se.impl.orbutil.graph.Graph does not take type parameters
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type SparkContext
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type Graph
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:3: object apache is not a member of package org
[error] import org.apache.spark.broadcast.Broadcast
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:26: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VD,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type SparkContext
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                 ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                                                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type SparkContext
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Graph
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type VertexRDD
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Broadcast
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                                                                              ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:163: not found: type EdgeTriplet
[error]   private def sendMsg(et:EdgeTriplet[VertexState,Long]) = {
[error]                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                                                                         ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:288: not found: value Graph
[error]     val compressedGraph = Graph(newVerts,edges)
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:289: not found: value PartitionStrategy
[error]       .partitionBy(PartitionStrategy.EdgePartition2D).groupEdges(_+_)
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:292: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VertexState,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:319: not found: type Graph
[error]    private def printlouvain(graph:Graph[VertexState,Long]) = {
[error]                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:327: not found: type Graph
[error]    private def printedgetriplets(graph:Graph[VertexState,Long]) = {
[error]                                        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:7: object apache is not a member of package org
[error] import org.apache.spark.Logging
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type SparkContext
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type Graph
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                                               ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type SparkContext
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type Graph
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:3: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:4: not found: object scopt
[error] import scopt.OptionParser._
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:32: not found: value scopt
[error]     val parser = new scopt.OptionParser[Config](this.getClass().toString()){
[error]                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:33: not found: value opt
[error]       opt[String]('i',"input") action {(x,c)=> c.copy(input=x)}  text("input file or path  Required.")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:34: not found: value opt
[error]       opt[String]('o',"output") action {(x,c)=> c.copy(output=x)} text("output path Required")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:35: not found: value opt
[error]       opt[String]('m',"master") action {(x,c)=> c.copy(master=x)} text("spark master, local[N] or spark://host:port default=local")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:36: not found: value opt
[error]       opt[String]('h',"sparkhome") action {(x,c)=> c.copy(sparkHome=x)} text("SPARK_HOME Required to run on cluster")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:37: not found: value opt
[error]       opt[String]('n',"jobname") action {(x,c)=> c.copy(appName=x)} text("job name")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:38: not found: value opt
[error]       opt[Int]('p',"parallelism") action {(x,c)=> c.copy(parallelism=x)} text("sets spark.default.parallelism and minSplits on the edge file. default=based on input partitions")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:39: not found: value opt
[error]       opt[Int]('x',"minprogress") action {(x,c)=> c.copy(minProgress=x)} text("Number of vertices that must change communites for the algorithm to consider progress. default=2000")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:40: not found: value opt
[error]        opt[Int]('y',"progresscounter") action {(x,c)=> c.copy(progressCounter=x)} text("Number of times the algorithm can fail to make progress before exiting. default=1")
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:41: not found: value opt
[error]       opt[String]('d',"edgedelimiter") action {(x,c)=> c.copy(edgedelimiter=x)} text("specify input file edge delimiter. default=\",\"")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:42: not found: value opt
[error]       opt[String]('j',"jars") action {(x,c)=> c.copy(jars=x)} text("comma seperated list of jars")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:43: not found: value opt
[error]       opt[Boolean]('z',"ipaddress") action {(x,c)=> c.copy(ipaddress=x)} text("Set to true to convert ipaddresses to Long ids. Defaults to false")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: not found: value arg
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value copy is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value properties is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:78: not found: type SparkContext
[error]     var sc: SparkContext = null
[error]             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:81: not found: type SparkContext
[error]       sc = new SparkContext(master, jobname)
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:85: not found: type SparkContext
[error]       sc = new SparkContext(master,jobname,sparkhome,jars.split(","))
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:104: not found: value Graph
[error]     val graph = Graph.fromEdges(edgeRDD, None)
[error]                 ^
[error] 59 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 609 s, completed 2017-6-19 20:49:41
> ; {file:/home/aurora/scala_workspace/untitled/}untitled/products; {file:/home/aurora/scala_workspace/untitled/}untitled/test:products
[info] Compiling 6 Scala sources to /home/aurora/scala_workspace/untitled/target/scala-2.12/classes...
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:4: object apache is not a member of package org
[error] import org.apache.spark.SparkContext._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:5: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: not found: type SparkContext
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/HDFSLouvainRunner.scala:18: com.sun.corba.se.impl.orbutil.graph.Graph does not take type parameters
[error]   override def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:3: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type SparkContext
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:79: not found: type Graph
[error]   def saveLevel(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:3: object apache is not a member of package org
[error] import org.apache.spark.broadcast.Broadcast
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:4: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:24: not found: type Graph
[error]    def createLouvainGraph[VD: ClassTag](graph: Graph[VD,Long]) : Graph[VertexState,Long]= {
[error]                                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:26: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VD,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type SparkContext
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                 ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:51: not found: type Graph
[error]   def louvainFromStandardGraph[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int) = {
[error]                                                                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                                                                                                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type SparkContext
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                  ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:62: not found: type Graph
[error]   def louvain(sc:SparkContext, graph:Graph[VertexState,Long], minProgress:Int=1,progressCounter:Int=1) : (Double,Graph[VertexState,Long],Int)= {
[error]                                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Graph
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type VertexRDD
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:193: not found: type Broadcast
[error]   private def louvainVertJoin(louvainGraph:Graph[VertexState,Long], msgRDD:VertexRDD[Map[(Long,Long),Long]], totalEdgeWeight:Broadcast[Long], even:Boolean) = {
[error]                                                                                                                              ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:163: not found: type EdgeTriplet
[error]   private def sendMsg(et:EdgeTriplet[VertexState,Long]) = {
[error]                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                                                                         ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:249: not found: type Graph
[error]   def compressGraph(graph:Graph[VertexState,Long],debug:Boolean=true) : Graph[VertexState,Long] = {
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:288: not found: value Graph
[error]     val compressedGraph = Graph(newVerts,edges)
[error]                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:289: not found: value PartitionStrategy
[error]       .partitionBy(PartitionStrategy.EdgePartition2D).groupEdges(_+_)
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:292: not found: type EdgeTriplet
[error]     val nodeWeightMapFunc = (e:EdgeTriplet[VertexState,Long]) => Iterator((e.srcId,e.attr), (e.dstId,e.attr))
[error]                                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:319: not found: type Graph
[error]    private def printlouvain(graph:Graph[VertexState,Long]) = {
[error]                                   ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainCore.scala:327: not found: type Graph
[error]    private def printedgetriplets(graph:Graph[VertexState,Long]) = {
[error]                                        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:7: object apache is not a member of package org
[error] import org.apache.spark.Logging
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type SparkContext
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:40: not found: type Graph
[error]   def run[VD: ClassTag](sc:SparkContext,graph:Graph[VD,Long]) = {
[error]                                               ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type SparkContext
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                    ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/LouvainHarness.scala:88: not found: type Graph
[error]   def finalSave(sc:SparkContext,level:Int,q:Double,graph:Graph[VertexState,Long]) = {
[error]                                                          ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:2: object apache is not a member of package org
[error] import org.apache.spark.SparkContext
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:3: object apache is not a member of package org
[error] import org.apache.spark.graphx._
[error]            ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:4: not found: object scopt
[error] import scopt.OptionParser._
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:32: not found: value scopt
[error]     val parser = new scopt.OptionParser[Config](this.getClass().toString()){
[error]                      ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:33: not found: value opt
[error]       opt[String]('i',"input") action {(x,c)=> c.copy(input=x)}  text("input file or path  Required.")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:34: not found: value opt
[error]       opt[String]('o',"output") action {(x,c)=> c.copy(output=x)} text("output path Required")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:35: not found: value opt
[error]       opt[String]('m',"master") action {(x,c)=> c.copy(master=x)} text("spark master, local[N] or spark://host:port default=local")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:36: not found: value opt
[error]       opt[String]('h',"sparkhome") action {(x,c)=> c.copy(sparkHome=x)} text("SPARK_HOME Required to run on cluster")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:37: not found: value opt
[error]       opt[String]('n',"jobname") action {(x,c)=> c.copy(appName=x)} text("job name")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:38: not found: value opt
[error]       opt[Int]('p',"parallelism") action {(x,c)=> c.copy(parallelism=x)} text("sets spark.default.parallelism and minSplits on the edge file. default=based on input partitions")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:39: not found: value opt
[error]       opt[Int]('x',"minprogress") action {(x,c)=> c.copy(minProgress=x)} text("Number of vertices that must change communites for the algorithm to consider progress. default=2000")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:40: not found: value opt
[error]        opt[Int]('y',"progresscounter") action {(x,c)=> c.copy(progressCounter=x)} text("Number of times the algorithm can fail to make progress before exiting. default=1")
[error]        ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:41: not found: value opt
[error]       opt[String]('d',"edgedelimiter") action {(x,c)=> c.copy(edgedelimiter=x)} text("specify input file edge delimiter. default=\",\"")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:42: not found: value opt
[error]       opt[String]('j',"jars") action {(x,c)=> c.copy(jars=x)} text("comma seperated list of jars")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:43: not found: value opt
[error]       opt[Boolean]('z',"ipaddress") action {(x,c)=> c.copy(ipaddress=x)} text("Set to true to convert ipaddresses to Long ids. Defaults to false")
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: not found: value arg
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value copy is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                       ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:44: value properties is not a member of Any
[error]       arg[(String,String)]("<property>=<value>....") unbounded() optional() action {case((k,v),c)=> c.copy(properties = c.properties :+ (k,v)) }
[error]                                                                                                                           ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:78: not found: type SparkContext
[error]     var sc: SparkContext = null
[error]             ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:81: not found: type SparkContext
[error]       sc = new SparkContext(master, jobname)
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:85: not found: type SparkContext
[error]       sc = new SparkContext(master,jobname,sparkhome,jars.split(","))
[error]                ^
[error] /home/aurora/scala_workspace/untitled/src/main/scala/Main.scala:104: not found: value Graph
[error]     val graph = Graph.fromEdges(edgeRDD, None)
[error]                 ^
[error] 59 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 351 s, completed 2017-6-19 21:42:04

In
举报
AuroraTan
发帖于7天前 0回/29阅
顶部