[Case Study]Decision Tree Classification using Python

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. from __future__ import print_function
   
2. 
   
3. import sys
   
4. 
   
5. # $example on$
   
6. from pyspark import SparkContext, SQLContext
   
7. from pyspark.ml import Pipeline
   
8. from pyspark.ml.classification import DecisionTreeClassifier
   
9. from pyspark.ml.feature import StringIndexer, VectorIndexer
   
10. from pyspark.ml.evaluation import MulticlassClassificationEvaluator
    
11. # $example off$
    
12. 
    
13. if __name__ == "__main__":
    
14.     sc = SparkContext(appName="decision_tree_classification_example")
    
15.     sqlContext = SQLContext(sc)
    
16. 
    
17.     # $example on$
    
18.     # Load the data stored in LIBSVM format as a DataFrame.
    
19.     data = sqlContext.read.format("libsvm").load("data/mllib/sample_libsvm_data.txt")
    
20. 
    
21.     # Index labels, adding metadata to the label column.
    
22.     # Fit on whole dataset to include all labels in index.
    
23.     labelIndexer = StringIndexer(inputCol="label", outputCol="indexedLabel").fit(data)
    
24.     # Automatically identify categorical features, and index them.
    
25.     # We specify maxCategories so features with > 4 distinct values are treated as continuous.
    
26.     featureIndexer =\
    
27.         VectorIndexer(inputCol="features", outputCol="indexedFeatures", maxCategories=4).fit(data)
    
28. 
    
29.     # Split the data into training and test sets (30% held out for testing)
    
30.     (trainingData, testData) = data.randomSplit([0.7, 0.3])
    
31. 
    
32.     # Train a DecisionTree model.
    
33.     dt = DecisionTreeClassifier(labelCol="indexedLabel", featuresCol="indexedFeatures")
    
34. 
    
35.     # Chain indexers and tree in a Pipeline
    
36.     pipeline = Pipeline(stages=[labelIndexer, featureIndexer, dt])
    
37. 
    
38.     # Train model.  This also runs the indexers.
    
39.     model = pipeline.fit(trainingData)
    
40. 
    
41.     # Make predictions.
    
42.     predictions = model.transform(testData)
    
43. 
    
44.     # Select example rows to display.
    
45.     predictions.select("prediction", "indexedLabel", "features").show(5)
    
46. 
    
47.     # Select (prediction, true label) and compute test error
    
48.     evaluator = MulticlassClassificationEvaluator(
    
49.         labelCol="indexedLabel", predictionCol="prediction", metricName="precision")
    
50.     accuracy = evaluator.evaluate(predictions)
    
51.     print("Test Error = %g " % (1.0 - accuracy))
    
52. 
    
53.     treeModel = model.stages[2]
    
54.     # summary only
    
55.     print(treeModel)
    
56.     # $example off$

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Case study Decision cation python Using example future import