[Scala Study]K-Means Clustering using Scala

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. package org.apache.spark.examples.ml
   
2. import org.apache.spark.{SparkContext, SparkConf}
   
3. import org.apache.spark.mllib.linalg.{VectorUDT, Vectors}
   
4. import org.apache.spark.ml.clustering.KMeans
   
5. import org.apache.spark.sql.{Row, SQLContext}
   
6. import org.apache.spark.sql.types.{StructField, StructType}
   
7. 
   
8. 
   
9. /**
   
10. * An example demonstrating a k-means clustering.
    
11. * Run with
    
12. * {{{
    
13. * bin/run-example ml.KMeansExample <file> <k>
    
14. * }}}
    
15. */
    
16. object KMeansExample {
    
17. 
    
18.   final val FEATURES_COL = "features"
    
19. 
    
20.   def main(args: Array[String]): Unit = {
    
21.     if (args.length != 2) {
    
22.       // scalastyle:off println
    
23.       System.err.println("Usage: ml.KMeansExample <file> <k>")
    
24.       // scalastyle:on println
    
25.       System.exit(1)
    
26.     }
    
27.     val input = args(0)
    
28.     val k = args(1).toInt
    
29. 
    
30.     // Creates a Spark context and a SQL context
    
31.     val conf = new SparkConf().setAppName(s"${this.getClass.getSimpleName}")
    
32.     val sc = new SparkContext(conf)
    
33.     val sqlContext = new SQLContext(sc)
    
34. 
    
35.     // Loads data
    
36.     val rowRDD = sc.textFile(input).filter(_.nonEmpty)
    
37.       .map(_.split(" ").map(_.toDouble)).map(Vectors.dense).map(Row(_))
    
38.     val schema = StructType(Array(StructField(FEATURES_COL, new VectorUDT, false)))
    
39.     val dataset = sqlContext.createDataFrame(rowRDD, schema)
    
40. 
    
41.     // Trains a k-means model
    
42.     val kmeans = new KMeans()
    
43.       .setK(k)
    
44.       .setFeaturesCol(FEATURES_COL)
    
45.     val model = kmeans.fit(dataset)
    
46. 
    
47.     // Shows the result
    
48.     // scalastyle:off println
    
49.     println("Final Centers: ")
    
50.     model.clusterCenters.foreach(println)
    
51.     // scalastyle:on println
    
52. 
    
53.     sc.stop()
    
54.   }
    
55. }

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Clustering Cluster k-means SCALA study features example package import