[Case Study]Gradient Boosted Tree Classification using Python

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. from __future__ import print_function
   
2. 
   
3. import sys
   
4. 
   
5. from pyspark import SparkContext
   
6. from pyspark.ml.classification import GBTClassifier
   
7. from pyspark.ml.feature import StringIndexer
   
8. from pyspark.ml.regression import GBTRegressor
   
9. from pyspark.mllib.evaluation import BinaryClassificationMetrics, RegressionMetrics
   
10. from pyspark.sql import Row, SQLContext
    
11. 
    
12. """
    
13. A simple example demonstrating a Gradient Boosted Trees Classification/Regression Pipeline.
    
14. Note: GBTClassifier only supports binary classification currently
    
15. Run with:
    
16.   bin/spark-submit examples/src/main/python/ml/gradient_boosted_trees.py
    
17. """
    
18. 
    
19. 
    
20. def testClassification(train, test):
    
21.     # Train a GradientBoostedTrees model.
    
22. 
    
23.     rf = GBTClassifier(maxIter=30, maxDepth=4, labelCol="indexedLabel")
    
24. 
    
25.     model = rf.fit(train)
    
26.     predictionAndLabels = model.transform(test).select("prediction", "indexedLabel") \
    
27.         .map(lambda x: (x.prediction, x.indexedLabel))
    
28. 
    
29.     metrics = BinaryClassificationMetrics(predictionAndLabels)
    
30.     print("AUC %.3f" % metrics.areaUnderROC)
    
31. 
    
32. 
    
33. def testRegression(train, test):
    
34.     # Train a GradientBoostedTrees model.
    
35. 
    
36.     rf = GBTRegressor(maxIter=30, maxDepth=4, labelCol="indexedLabel")
    
37. 
    
38.     model = rf.fit(train)
    
39.     predictionAndLabels = model.transform(test).select("prediction", "indexedLabel") \
    
40.         .map(lambda x: (x.prediction, x.indexedLabel))
    
41. 
    
42.     metrics = RegressionMetrics(predictionAndLabels)
    
43.     print("rmse %.3f" % metrics.rootMeanSquaredError)
    
44.     print("r2 %.3f" % metrics.r2)
    
45.     print("mae %.3f" % metrics.meanAbsoluteError)
    
46. 
    
47. 
    
48. if __name__ == "__main__":
    
49.     if len(sys.argv) > 1:
    
50.         print("Usage: gradient_boosted_trees", file=sys.stderr)
    
51.         exit(1)
    
52.     sc = SparkContext(appName="PythonGBTExample")
    
53.     sqlContext = SQLContext(sc)
    
54. 
    
55.     # Load the data stored in LIBSVM format as a DataFrame.
    
56.     df = sqlContext.read.format("libsvm").load("data/mllib/sample_libsvm_data.txt")
    
57. 
    
58.     # Map labels into an indexed column of labels in [0, numLabels)
    
59.     stringIndexer = StringIndexer(inputCol="label", outputCol="indexedLabel")
    
60.     si_model = stringIndexer.fit(df)
    
61.     td = si_model.transform(df)
    
62.     [train, test] = td.randomSplit([0.7, 0.3])
    
63.     testClassification(train, test)
    
64.     testRegression(train, test)
    
65.     sc.stop()

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Case study gradient Boosted python cation example simple future import