[Case Study]K-Means Clustering using Python

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. from __future__ import print_function
   
2. 
   
3. import sys
   
4. import re
   
5. 
   
6. import numpy as np
   
7. from pyspark import SparkContext
   
8. from pyspark.ml.clustering import KMeans, KMeansModel
   
9. from pyspark.mllib.linalg import VectorUDT, _convert_to_vector
   
10. from pyspark.sql import SQLContext
    
11. from pyspark.sql.types import Row, StructField, StructType
    
12. 
    
13. """
    
14. A simple example demonstrating a k-means clustering.
    
15. Run with:
    
16.   bin/spark-submit examples/src/main/python/ml/kmeans_example.py <input> <k>
    
17. This example requires NumPy (http://www.numpy.org/).
    
18. """
    
19. 
    
20. 
    
21. def parseVector(line):
    
22.     array = np.array([float(x) for x in line.split(' ')])
    
23.     return _convert_to_vector(array)
    
24. 
    
25. 
    
26. if __name__ == "__main__":
    
27. 
    
28.     FEATURES_COL = "features"
    
29. 
    
30.     if len(sys.argv) != 3:
    
31.         print("Usage: kmeans_example.py <file> <k>", file=sys.stderr)
    
32.         exit(-1)
    
33.     path = sys.argv[1]
    
34.     k = sys.argv[2]
    
35. 
    
36.     sc = SparkContext(appName="PythonKMeansExample")
    
37.     sqlContext = SQLContext(sc)
    
38. 
    
39.     lines = sc.textFile(path)
    
40.     data = lines.map(parseVector)
    
41.     row_rdd = data.map(lambda x: Row(x))
    
42.     schema = StructType([StructField(FEATURES_COL, VectorUDT(), False)])
    
43.     df = sqlContext.createDataFrame(row_rdd, schema)
    
44. 
    
45.     kmeans = KMeans().setK(2).setSeed(1).setFeaturesCol(FEATURES_COL)
    
46.     model = kmeans.fit(df)
    
47.     centers = model.clusterCenters()
    
48. 
    
49.     print("Cluster Centers: ")
    
50.     for center in centers:
    
51.         print(center)
    
52. 
    
53.     sc.stop()

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Clustering Case study Cluster k-means python examples python simple future import