[Case Study]Linear Discriminant Analysis using Julia

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

Linear Discriminant Analysis

A linear discriminant classifier can be built using the lda function and a dataframe. Here I am using the iris data set that I have divided into a training set (to build the classifier) and a testing set to validate against:

1. julia> lda_mod = lda(fm, iris[train,:])
   
2. Formula: Species ~ :(+(Sepal_Length,Sepal_Width,Petal_Length,Petal_Width))

复制代码

By default, rank-reduced linear discriminant analysis is performed. This (probably) will perform a dimensionality reduction if there is more than two groups (similar to principle components analysis).

The scaling matrix is used to "sphere" or "whiten" the input data so that its sample covariance matrix is the identity matrix (this decreases the complexity of the classification computation). In other words, the whitened data has a sample covariance that corresponds to the unit n-sphere.

Note: rank reduction was successful so the scaling matrix is of rank two rather than three.

1. julia> scaling(lda_mod)

复制代码

4x2 Array{Float64,2}:  0.660804   0.849652  1.59634    1.81788  -1.87905   -0.978034 -2.85134    2.14334

Prediction is as simple as plugging a dataframe and the model into the predict function. The model will extract the appropriate columns from the dataframe assuming they are named correctly:

1. julia> lda_pred = predict(lda_mod,iris[test,:])
   
2. 38x1 PooledDataArray{UTF8String,Uint32,2}:
   
3. "setosa"   
   
4. ⋮         
   
5. "virginica"
   
6. 
   
7. julia> 100*sum(lda_pred .== y[test])/length(y[test])
   
8. 100.0

复制代码

Regularized linear discriminant analysis has an additional parameter gamma. This regularization is analogous to ridge regression and can be used to 'nudge' a singular matrix into a non singular matrix (or help penalize the biased estimates of the eigenvalues - see paper below). This is important when the sample size is small and the sample covariance matrix may not be invertible.

The gamma values supplied should be between 0 and 1 inclusive. The value represents the percentage of shrinkage along the diagonals of the sample covariance matrix towards its average eigenvalue.

1. julia> lda_mod = lda(fm, iris[train,:], gamma=0.2)
   
2. 
   
3. julia> scaling(lda_mod)

复制代码

4x2 Array{Float64,2}: -0.122872   0.39509   0.554429   1.50014  -0.938699  -0.282481 -1.70349    0.797025

Rank-reduction can be disabled setting the parameter rrlda to false. Default is true. When it is disabled, we can see the scaling matrix is square:

1. julia> lda_mod = lda(fm, iris[train,:], rrlda=false)
   
2. 
   
3. julia> scaling(lda_mod)

复制代码

4x4 Array{Float64,2}: -0.708728   0.919018  -0.970648   2.99623 -0.85916   -2.03842   -2.20533   -2.15067 -0.76332    1.29499    0.677356  -3.26619 -1.38388   -2.33625    4.27793    2.95553

Lastly, a tolerance parameter can be set and is used in determining the rank of all covariance matrices. It is relative to the largest eigenvalue of the sample covariance matrix and should be between 0 and 1.

1. julia> lda_mod = lda(fm, iris[train,:], tol=0.1)
   
2. ERROR: Rank deficiency detected with tolerance=0.1.
   
3. in error at error.jl:21

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：discriminant Discriminan Case study Analysis Analysi function training against testing