[博客精选]Building a decision tree from scratch - a beginner tutorial

相似文件 换一批

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

Building a decision tree from scratch - a beginner tutorialby Patrick L. Lê

Suppose you have a population. You want to divide this population into relevant subgroups based on specific features characterizing each subgroup, so that you can accurately predict outcomes associated with each subgroup. For instance, you could :

• Use the list of the people on the Titanic, and by dividing them into subgroups depending on specific criteria (e.g. female vs male, passengers in 1st class vs 2nd and 3rd class, age class....) determines if they were (probably) going to survive or not.
• Look at the people who bought product on your e-commerce website, divide this population into segments depending on specific features (e.g. returning visitors vs new visitors, localization, ...) and determines for future visitors if they are (probably) going to buy your product or not.
  

In sum, you want to create a model that predicts the value of a target variable (e.g. survive/die; buy/not buy) based on simple decision rules inferred from the data features (e.g. female vs male, age, etc.).

The result is a decision tree that offers the great advantage to be easily vizualized and simple to understand. For instance, the picture below, fromwikipedia, shows the probability of passengers of the Titanic to survive depending on their sex, age and number of spouses or siblings aboard. Note how each branching is based on answering a question (the decision rule) and how the graph looks like an inverted tree.

本帖隐藏的内容

9.pdf (4.4 MB)

2016-8-19 01:55:10 上传

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Decision Tutorial Building beginner scratch scratch

1. def buildtree(rows,scoref=entropy): #rows is the set, either whole dataset or part of it in the recursive call,
   
2.                                     #scoref is the method to measure heterogeneity. By default it's entropy.
   
3.   if len(rows)==0: return decisionnode() #len(rows) is the number of units in a set
   
4.   current_score=scoref(rows)
   
5. 
   
6.   # Set up some variables to track the best criteria
   
7.   best_gain=0.0
   
8.   best_criteria=None
   
9.   best_sets=None
   
10.   
    
11.   column_count=len(rows[0])-1   #count the # of attributes/columns.
    
12.                                 #It's -1 because the last one is the target attribute and it does not count.
    
13.   for col in range(0,column_count):
    
14.     # Generate the list of all possible different values in the considered column
    
15.     global column_values        #Added for debugging
    
16.     column_values={}            
    
17.     for row in rows:
    
18.        column_values[row[col]]=1   
    
19.     # Now try dividing the rows up for each value in this column
    
20.     for value in column_values.keys(): #the 'values' here are the keys of the dictionnary
    
21.       (set1,set2)=divideset(rows,col,value) #define set1 and set2 as the 2 children set of a division
    
22.       
    
23.       # Information gain
    
24.       p=float(len(set1))/len(rows) #p is the size of a child set relative to its parent
    
25.       gain=current_score-p*scoref(set1)-(1-p)*scoref(set2) #cf. formula information gain
    
26.       if gain>best_gain and len(set1)>0 and len(set2)>0: #set must not be empty
    
27.         best_gain=gain
    
28.         best_criteria=(col,value)
    
29.         best_sets=(set1,set2)
    
30.         
    
31.   # Create the sub branches   
    
32.   if best_gain>0:
    
33.     trueBranch=buildtree(best_sets[0])
    
34.     falseBranch=buildtree(best_sets[1])
    
35.     return decisionnode(col=best_criteria[0],value=best_criteria[1],
    
36.                         tb=trueBranch,fb=falseBranch)
    
37.   else:
    
38.     return decisionnode(results=uniquecounts(rows))

复制代码

kankan

THS