楼主: lion003
9413 50

Scala for Machine Learning [推广有奖]

21楼
Lisrelchen 发表于 2016-4-20 09:07:21 |只看作者 |坛友微信交流群
  1. Initializing clusters

  3. The initialization of the cluster centroids is important to ensure fast convergence of K-means. Solutions range from the simple random generation of centroids to the application of genetic algorithms to evaluate the fitness of centroid candidates. We selected an efficient and fast initialization algorithm developed by M. Agha and W. Ashour [4:4].

  5. The steps of the initialization are as follows:

  7. Compute the standard deviation of the set of observations.
  8. Select the dimension k {xk,0, xk,1 … xk,n} with maximum standard deviation.
  9. Rank the observations by their increasing value of standard deviation for the dimension k.
  10. Divide the ranked observations set equally into K sets {Sm}.
  11. Find the median values, size (Sm)/2.
  12. Use the corresponding observations as centroids.
  13. The initialization algorithm is implemented by the private initialize method:

  15.   def initialize(xt:XTSeries[Array[T]]): List[Cluster[T]]={
  16.    val stats = statistics(xt) //1
  17.    val maxSDevDim = Range(0,stats.size).maxBy (stats( _ ).stdDev)//2
  18.    val rankedObs = xt.zipWithIndex
  19.                      .map(x=> (x._1(maxSDevDim), x._2)) //2
  20.                      .sortWith( _._1  < _._1) //3
  21.    val halfSegSize = ((rankedObs.size>>1)/K).floor.toInt //4
  22.    val centroids = rankedObs.filter(isContained( _, halfSegSize, rankedObs.size) ).map(n => xt(n._2)) //6
  23.    Range(0, K).foldLeft(List[Cluster[T]]())((xs, i) => Cluster[T](centroids(i)) :: xs) //7
  24. }
复制代码

使用道具

22楼
Lisrelchen 发表于 2016-4-20 09:09:11 |只看作者 |坛友微信交流群
  1. Step 2 – cluster assignment
  2. The second step in the K-means algorithm is the assignment of the observations to the clusters for which the centroids have been initialized in step 1. This feat is accomplished by the private assignToClusters method:

  4. def assignToClusters(xt: XTSeries[Array[T]], clusters: List[Cluster[T]], membership: Array[Int]): Int = {
  5.   xt.toArray
  6.     .zipWithIndex
  7.     .filter(x => { //1
  8.        val nearestCluster = getNearestCluster(clusters, x._1)//2
  9.        val reassigned = nearestCluster != membership(x._2)
  10.        clusters(nearestCluster) += x._2 //3
  11.        membership(x._2) = nearestCluster //4
  12.        reassigned
  13.      }).size
  14. }
  15. The core of the assignment of observations to each cluster is the filter on the time series (line 1). The filter computes the index of the closest cluster and checks whether the observation is to be reassigned (line 2). The observation at the index x._2 is added to the nearest cluster, clusters(nearestCluster) (line 3). The current membership of the observations is then updated (line 4).

  17. The cluster closest to an observation data is computed by the getNearestCluster method as follows:

  19. def getNearestCluster(clusters: List[Cluster[T]], x:Array[T]): Int={
  20.   clusters.zipWithIndex..foldLeft((Double.MaxValue,0))((p,c) => {
  21.       val measure = distance(c._1.center, x)
  22.       if(measure < p._1) (measure, c._2) else p
  23.    })._2
复制代码

使用道具

23楼
Lisrelchen 发表于 2016-4-20 09:10:17 |只看作者 |坛友微信交流群
  1. Step 3 – iterative reconstruction
  2. The final step is to implement the iterative computation of the reconstruction error. In this implementation, the iteration terminates when no more observations are reassigned to different clusters. As with other data processing units, the extraction of K-means clusters is encapsulated by the pipe operator |>, so that clustering can be integrated into a workflow using dependency injection described in the Dependency Injection section in Chapter 2, Hello World!.

  4. The generation of the K clusters is executed by the data transformation |>:

  6. def |> :PartialFunction[XTSeries[Array[T]], List[Cluster[T]]] = {
  7.   case xt: XTSeries[Array[T]] if(xt.size>2 && xt(0).size>0) => {
  8.     val clusters = initialize(xt)  //1

  10.     if( clusters.isEmpty) List.empty
  11.     else  {
  12.       val membership = Array.fill(xt.size)(0)
  13.       val reassigned = assignToClusters(xt,clusters,membership)//2
  14.       var newClusters: List[Cluster[T]] = List.empty
  15.       Range(0, maxIters).find( _ => {
  16.         newClusters = clusters.map( c => {
  17.           if( c.size > 0) c.moveCenter(xt, dimension(xt))
  18.           else clusters.filter( _.size > 0)
  19.                        .maxBy( _.stdDev(xt, distance))
  20.         }) //3
  21.         assignToClusters(xt, newClusters, membership) == 0
  22.       }) match {
  23.         case Some(index) => newClusters
  24.         case None => { … }
  25.     } //4
  26.   }
  27. }
复制代码

使用道具

24楼
Lisrelchen 发表于 2016-4-20 09:17:49 |只看作者 |坛友微信交流群
  1. /**
  2. * Copyright (c) 2013-2015  Patrick Nicolas - Scala for Machine Learning - All rights reserved
  3. *
  4. * The source code in this file is provided by the author for the sole purpose of illustrating the
  5. * concepts and algorithms presented in "Scala for Machine Learning". It should not be used to
  6. * build commercial applications.
  7. * ISBN: 978-1-783355-874-2 Packt Publishing.
  8. * Unless required by applicable law or agreed to in writing, software is distributed on an "AS IS" BASIS,
  9. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  10. *
  11. * Version 0.98
  12. */
  13. package org.scalaml.supervised.bayes

  15. // Scala classes
  16. import scala.collection.mutable.ArraySeq
  17. import scala.annotation.implicitNotFound
  18. import scala.util.{Try, Success, Failure}

  20. import org.apache.log4j.Logger

  22. import org.scalaml.core.XTSeries
  23. import org.scalaml.core.Types.ScalaMl._
  24. import org.scalaml.stats.{Validation, ClassValidation, Stats}
  25. import org.scalaml.core.Design.PipeOperator
  26. import org.scalaml.supervised.Supervised
  27. import org.scalaml.util.DisplayUtils
  28. import NaiveBayesModel._, XTSeries._, Stats._

  30.    
  31.                 /**
  32.                  * <p>Generic Binomial Naive Bayes classification class. The class is used for both training
  33.                  * and run-time classification. The training of the model is executed during the instantiation
  34.                  * of the class to avoid having an uninitialized model. A conversion from a parameterized
  35.                  * array, Array[T] to an array of double, DblVector, has to be implicitly defined for
  36.                  * training the model.<br>
  37.                  * As a classifier, the method implement the generic data transformation PipeOperator and the
  38.                  * Supervised interface.<br>
  39.                  * <pre><span style="font-size:9pt;color: #351c75;font-family: &quot;Helvetica Neue&quot;
  40.                  * ,Arial,Helvetica,sans-serif;">
  41.                  *  Naive Bayes formula: p(C}x) = p(x|C).p(C)/p(x) => p(C|x) = p(x1|C).p(x2|C). .. p(xn|C).p(C)</span></pre></p>
  42.                  * @constructor Instantiate a parameterized NaiveBayes model
  43.                  * @param smoothing Laplace or Lidstone smoothing factor
  44.                  * @param xt  Input labeled time series used for training
  45.                  * @param density Density function used to compute the discriminant
  46.                  *
  47.                  * @throws IllegalArgumentException if one of the class parameters is undefined
  48.                  * @author Patrick Nicolas
  49.                  * @since February 13, 2014
  50.                  * @note Scala for Machine learning Chapter 5 Naive Bayes Models / Naive Bayes classifiers
  51.                  */
  52. final class NaiveBayes[T <% Double](
  53.                 smoothing: Double,
  54.                 xt: XTSeries[(Array[T], Int)],
  55.                 density: Density)        extends PipeOperator[XTSeries[Array[T]], Array[Int]] with Supervised[T] {

  57.         import NaiveBayes._
  58.         check(smoothing, xt)
  59.        
  60.         private val logger = Logger.getLogger("NaiveBayes")
  61.        
  62.                 // The model is instantiated during training for both
  63.                 // classes if the training is successful. It is None otherwise
  64.         private[this] val model: Option[BinNaiveBayesModel[T]] =
  65.                         Try(BinNaiveBayesModel[T](train(1), train(0), density)) match {
  66.                 case Success(nb) => Some(nb)
  67.                 case Failure(e) => DisplayUtils.none("NaiveBayes.model", logger, e)
  68.         }
  69.                
  70.                 /**
  71.                  * <p>Run-time classification of a time series using the Naive Bayes model. The method invoke
  72.                  * the actual classification method in one of the NaiveBayes models.</p>
  73.                  * @throws MatchError if the input time series is undefined or have no elements or the
  74.                  * model was not properly trained
  75.                  * @return PartialFunction of time series of elements of type T as input to the Naive Bayes
  76.                  * and array of class indices as output
  77.                  */
  78.         override def |> : PartialFunction[XTSeries[Array[T]], Array[Int]] = {
  79.                 case xt: XTSeries[Array[T]] if( !xt.isEmpty && model != None) =>
  80.                         xt.toArray.map( model.get.classify( _))
  81.         }
  82.        
  83.                 /**
  84.                  * <p>Compute the F1 statistics for the Naive Bayes.</p>
  85.                  * @param xt Time series of features of type Array[T], and class indices as labels
  86.                  * @param index of the class, the time series or observation should belong to
  87.                  * @return F1 measure if the model has been properly trained (!= None), None otherwise
  88.                  */
  89.         override def validate(xt: XTSeries[(Array[T], Int)], index: Int): Option[Double] = model match {
  90.                 case Some(m) => Some(ClassValidation(xt.map(x =>(m.classify(x._1), x._2)) , index).f1)
  91.                 case None => DisplayUtils.none("NaiveBayes Model undefined", logger)
  92.         }


  95.                 /**
  96.                  * Textual representation of the Naive Bayes classifier with labels for features.
  97.                  * It returns "No Naive Bayes model" if no model exists
  98.                  * @return Stringized features with their label if model exists.
  99.                  */
  100.         def toString(labels: Array[String]): String =
  101.                 model.map(m => if( labels.isEmpty ) m.toString else m.toString(labels))
  102.                                 .getOrElse("No Naive Bayes model")

  104.                 /**
  105.                  * Default textual representation of the Naive Bayes classifier with labels for features.
  106.                  * It returns "No Naive Bayes model" if no model exists
  107.                  * @return Stringized features with their label if model exists.
  108.                  */
  109.         override def toString: String = toString(Array.empty)
  110.    
  111.                 /**
  112.                  * Train the Naive Bayes model on one of the two classes (positive = 1) or negative (=0)
  113.                  */
  114.         @implicitNotFound("NaiveBayes; Conversion from array[T] to DblVector is undefined")
  115.         private def train(label: Int)(implicit f: Array[T] => DblVector): Likelihood[T] = {
  116.                 val xi = xt.toArray
  117.                                 // Extract then filter each observation to be associated to a specific label.
  118.                                 // The implicit conversion from Array of type T to Array of type Double is invoked
  119.                 val values = xi.filter( _._2 == label).map(x => f(x._1))
  120.                 assert( !values.isEmpty, "NaiveBayes.train Filtered value is undefined")
  121.                
  122.                         // Gets the dimension of a feature
  123.                 val dim = xi(0)._1.size
  124.                 val vSeries = XTSeries[DblVector](values)
  125.        
  126.                         // Create a likelihood instance for this class 'label'. The
  127.                         // tuple (mean, standard deviation) (2nd argument) is computed
  128.                         // by invoking XTSeries.statistics then the Lidstone mean adjustment.
  129.                         // The last argument, class likelihood p(C) is computed as the ratio of the
  130.                         // number of observations associated to this class/label over total number of observations.
  131.                 Likelihood(label,
  132.                                 statistics(vSeries).map(stat => (stat.lidstoneMean(smoothing, dim), stat.stdDev) ),
  133.                                 values.size.toDouble/xi.size)
  134.         }
  135. }

  137.                 /**
  138.                  * Singleton that define the constructors for the NaiveBayes classifier and
  139.                  * validate its parameters
  140.                  * @author Patrick Nicolas
  141.                  * @since February 13, 2014
  142.                  * @note Scala for Machine learning Chapter 5 Naive Bayes Model
  143.                  */
  144. object NaiveBayes {       
  145.                 /**
  146.                  * Default constructor for the NaiveBayes class
  147.                  * @param smoothing Laplace or Lidstone smoothing factor
  148.                  * @param xt Input labeled time series used for training
  149.                  * @param density Density function used to compute the discriminant
  150.                  */
  151.         def apply[T <% Double](
  152.                         smoothing: Double,
  153.                         xt: XTSeries[(Array[T], Int)],
  154.                         density: Density): NaiveBayes[T] = new NaiveBayes[T](smoothing, xt, density)
  155.                
  156.                 /**
  157.                  * Constructor for the NaiveBayes class with a Laplace smoothing function and
  158.                  * a Gaussian density function.
  159.                  * @param xt  Input labeled time series used for training
  160.                  */
  161.         def apply[T <% Double](xt: XTSeries[(Array[T], Int)]): NaiveBayes[T] =
  162.                         new NaiveBayes[T](1.0, xt, gauss)
  163.                
  164.         /*
  165.         def |>[T <% Double](): PartialFunction[XTSeries[Array[T]], Array[Int]] = {
  166.                 case xt: XTSeries[Array[T]] if(!xt.isEmpty && {
  167.                         model =
  168.                   model != None }) =>
  169.                         xt.toArray.map( model.get.classify( _))
  170.         }
  171.         *
  172.         */
  173.         def |>[T <% Double](model: Option[BinNaiveBayesModel[T]]): PartialFunction[XTSeries[Array[T]], Array[Int]] = {
  174.                 case xt: XTSeries[Array[T]] if( !xt.isEmpty && model != None) =>
  175.                         xt.toArray.map( model.get.classify( _))
  176.         }
  177.        
  178.        
  179.        
  180.         private def check[T <% Double](smoothing: Double, xt: XTSeries[(Array[T], Int)]): Unit = {
  181.                 require(smoothing > 0.0 && smoothing <= 1.0,
  182.                           s"NaiveBayes: Laplace or Lidstone smoothing factor $smoothing is out of range")
  183.                 require( !xt.isEmpty,
  184.                                 "NaiveBayes: Time series input for training Naive Bayes is undefined")
  185.         }
  186. }


  189. // ------------------------------  EOF --------------------------------------------
复制代码

使用道具

25楼
Lisrelchen 发表于 2016-4-20 09:18:49 |只看作者 |坛友微信交流群
  1. /**
  2. * Copyright (c) 2013-2015  Patrick Nicolas - Scala for Machine Learning - All rights reserved
  3. *
  4. * The source code in this file is provided by the author for the sole purpose of illustrating the
  5. * concepts and algorithms presented in "Scala for Machine Learning". It should not be used to
  6. * build commercial applications.
  7. * ISBN: 978-1-783355-874-2 Packt Publishing.
  8. * Unless required by applicable law or agreed to in writing, software is distributed on an "AS IS" BASIS,
  9. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  10. *
  11. * Version 0.98
  12. */
  13. package org.scalaml.supervised.bayes

  15. import org.scalaml.stats.Stats
  16. import org.scalaml.util.{FormatUtils, DisplayUtils}
  17. import org.scalaml.core.Types.ScalaMl.XYTSeries
  18. import NaiveBayesModel._

  20.                 /**
  21.                  * <p>Class that represents a likelihood for each feature for Naive Bayes classifier.<br>
  22.                  * The prior consists of a label (index), the mean of the prior of each dimension of the model,
  23.                  * the standard deviation of the prior of each dimension of the model and the class likeliHood.<br>
  24.                  * The Naive Bayes assume that the dimension of the model are independent, making the log of
  25.                  * the prior additive.</p>
  26.                  * @constructor Create a likelihood for a specific class.
  27.                  * @throws IllegalArgumentException if the array of mean and standard deviation of the
  28.                  * likelihood is undefined or if the class likelihood (prior) is out of range ]0,1]
  29.                  * @param label  Name or label of the class or prior for which the likelihood is computed.
  30.                  * @param muSigma Array of tuples (mean, standard deviation) of the prior observations for the model
  31.                  * @param prior  Probability of occurrence for the class specified by the label.
  32.                  *
  33.                  * @author Patrick Nicolas
  34.                  * @since March 11, 2014
  35.                  * @note Scala for Machine Learning Chapter 5 Naive Bayes Models
  36.                  */
  37. protected class Likelihood[T <% Double](val label: Int, val muSigma: XYTSeries, val prior: Double) {
  38.         import Stats._, Likelihood._
  39.   
  40.         check(muSigma, prior)
  41.   
  42.                 /**
  43.                  * <p>Compute the log p(C|x of log of the conditional probability of the class given an
  44.                  * observation, obs and a probability density distribution.<br>
  45.                  * The default density probability function is Normal(0, 1)</p>
  46.                  * @param obs parameterized observation
  47.                  * @param density probability density function (default Gauss)
  48.                  * @throws IllegalArgumentException if the observations are undefined
  49.                  * @return log of the conditional probability p(C|x)
  50.                  */
  51.         final def score(obs: Array[T], density: Density): Double = {
  52.                 require( !obs.isEmpty, "Likelihood.score Undefined observations")
  53.                
  54.                         // Compute the Log of sum of the likelihood and the class prior probability
  55.                         // The log likelihood is computed by adding the log of the density for each dimension.
  56.                         // Sum {log p(xi|C) }
  57.                 (obs, muSigma).zipped.foldLeft(0.0)((post, xms) => {
  58.                         val mean = xms._2._1                // mean
  59.                         val stdDev = xms._2._2        // standard deviation
  60.                         val _obs = xms._1
  61.                         val logLikelihood = density(mean, stdDev, _obs)
  62.                                
  63.                                 // Avoid large value by setting a minimum value for the density probability
  64.                         post + Math.log(if(logLikelihood <  MINLOGARG) MINLOGVALUE else logLikelihood)
  65.                 }) + Math.log(prior) // Add the class likelihood p(C)
  66.         }
  67.        
  68.                 /**
  69.                  * <p>DisplayUtils the content of this Likelihood class with associated labels.</p>
  70.                  * @param labels Label of variables used to display content
  71.                  */
  72.         def toString(labels: Array[String]): String = {
  73.                 import org.scalaml.core.Types.ScalaMl
  74.                
  75.                 val muSigmaStr = muSigma.map(musig => (musig._1, if(musig._2 > 0.0) musig._2 else -1.0))
  76.                         // Format the tuple muSigma= (mean, standard deviation) and the Prior
  77.                 FormatUtils.format(muSigma, "Label\tMeans", "Standard Deviation", FormatUtils.MediumFormat, labels) +
  78.                 FormatUtils.format(prior, "Class likelihood", FormatUtils.MediumFormat)
  79.         }
  80.        
  81.         override def toString: String = toString(Array.empty)
  82. }


  85.                 /**
  86.                  * <p>Companion object for the Naive Bayes Likelihood class. The singleton
  87.                  * is used to define the constructor apply for the class.</p>
  88.                  * @author Patrick Nicolas
  89.                  * @since March 11, 2014
  90.                  * @note Scala for Machine Learning Chapter 5 Naive Bayes Models
  91.                  */
  92. object Likelihood {
  93.         private val MINLOGARG = 1e-32
  94.         private val MINLOGVALUE = -MINLOGARG

  96.                 /**
  97.                  * Default constructor for he class Likelihood.
  98.                  * @param label  Name or label of the class or prior for which the likelihood is computed.
  99.                  * @param muSigma Array of tuples (mean, standard deviation) of the prior observations
  100.                  * for the model
  101.                  * @param prior  Probability of occurrence for the class specified by the label.
  102.                  */
  103.         def apply[T <% Double](label: Int, muSigma: XYTSeries, prior: Double): Likelihood[T] =
  104.                 new Likelihood[T](label, muSigma, prior)
  105.    
  106.         private def check(muSigma: XYTSeries, prior: Double): Unit =  {
  107.                 require( !muSigma.isEmpty,
  108.                                 "Likelihood.check Historical mean and standard deviation is undefined")
  109.                 require(prior > 0.0  && prior <= 1.0,
  110.                                 s"Likelihood.check Prior for the NB prior $prior is out of range")
  111.         }
  112. }


  115. // --------------------------------  EOF --------------------------------------------------------------
复制代码

使用道具

26楼
Lisrelchen 发表于 2016-4-20 09:27:04 |只看作者 |坛友微信交流群
  1. package org.scalaml.supervised.bayes

  3. import org.scalaml.stats.Stats
  4. import org.scalaml.core.Design.Model
  5. import NaiveBayesModel._

  7. abstract class NaiveBayesModel[T <% Double](val density: Density) extends Model {

  9.         def classify(x: Array[T]): Int
  10.         def toString(labels: Array[String]): String
  11. }

  13. object NaiveBayesModel {

  15.         type Density= (Double*) => Double
  16. }


  19. protected class BinNaiveBayesModel[T <% Double](
  20.                 positives: Likelihood[T],
  21.                 negatives: Likelihood[T],
  22.                 density: Density) extends NaiveBayesModel[T](density) {

  24.         override def classify(x: Array[T]): Int = {
  25.                 require( !x.isEmpty,
  26.                                 "BinNaiveBayesModel.classify Undefined observations")
  27.                
  28.                 // Simply select one of the two classes with the highest log posterior probability
  29.                 if (positives.score(x, density) > negatives.score(x, density)) 1 else 0
  30.         }

  32.                        
  33.                        
  34.        
  35.         override def toString(labels: Array[String]): String = {
  36.                 require( !labels.isEmpty, "BinNaiveBayesModel.toString Undefined labels")
  37.                 s"\nPositive class\n${positives.toString(labels)}\nNegative class\n${negatives.toString(labels)}"
  38.         }
  39.        
  40.         override def toString: String =
  41.                         s"\nPositive\n${positives.toString}\nNegative\n${negatives.toString}"
  42. }


  45. object BinNaiveBayesModel {

  47.         def apply[T <% Double](
  48.                         positives: Likelihood[T],
  49.                         negatives: Likelihood[T],
  50.                         density: Density): BinNaiveBayesModel[T] =
  51.                                         new BinNaiveBayesModel(positives, negatives, density)
  52. }

  54. protected class MultiNaiveBayesModel[T <% Double](
  55.                 likelihoodSet: List[Likelihood[T]],
  56.                 density: Density) extends NaiveBayesModel[T](density) {
  57.   
  58.         require(!likelihoodSet.isEmpty,
  59.                         "MultiNaiveBayesModel Cannot classify using Multi-NB with undefined classes")
  60.   
  61.         override def classify(x: Array[T]): Int = {
  62.                 require( !x.isEmpty, "MultiNaiveBayesModel.classify Vector input is undefined")
  63.                
  64.                         // The classification is performed by ordering the class according to the
  65.                         // log of their posterior probability and selecting the top one (highest
  66.                         // posterior probability)
  67.                 likelihoodSet.sortWith((p1, p2) => p1.score(x, density) > p2.score(x, density)).head.label
  68.         }

  70.        
  71.        
  72.         override def toString(labels: Array[String]): String = {
  73.                 require( !labels.isEmpty, "MultiNaiveBayesModel.toString Vector input is undefined")
  74.                          
  75.                 val buf = new StringBuilder
  76.                 likelihoodSet.zipWithIndex.foreach(l => {
  77.                         buf.append(s"\nclass ${l._2}: ${l._1.toString(labels)}")
  78.                 })
  79.                 buf.toString
  80.         }
  81. }


  84. object MultiNaiveBayesModel {

  86.         def apply[T <% Double](
  87.                         likelihoodSet: List[Likelihood[T]],
  88.                         density: Density): MultiNaiveBayesModel[T] =
  89.                                         new MultiNaiveBayesModel[T](likelihoodSet, density)
  90. }


  93. // --------------------------------  EOF --------------------------------------------------------------
复制代码

使用道具

27楼
Lisrelchen 发表于 2016-4-20 09:31:56 |只看作者 |坛友微信交流群
  1. package org.scalaml.supervised.crf
  2.        
  3.         // Scala standard library
  4. import scala.util.{Try, Success, Failure}
  5.         // 3rd party frameworks or libraries
  6. import iitb.CRF.{CRF, CrfParams, DataSequence, DataIter, FeatureGenerator}
  7. import iitb.Model.{FeatureGenImpl, CompleteModel}
  8. import org.apache.log4j.Logger
  9.         // ScalaMl classes
  10. import org.scalaml.core.XTSeries
  11. import org.scalaml.core.Types.ScalaMl._
  12. import org.scalaml.core.Design.{PipeOperator, Model}
  13. import org.scalaml.supervised.Supervised
  14. import org.scalaml.util.DisplayUtils
  15. import org.scalaml.workflow.data.DataSource
  16. import CrfConfig._


  19. final protected class CrfModel(val weights: DblVector) extends Model {
  20.         require(!weights.isEmpty, "CrfModel Cannot create a model with undefined weights")

  22.         def this(className: String) =
  23.                         this({ Model.read(className).map( _.split(",").map(_.toDouble)).getOrElse(Array.empty) })
  24.        

  26.         override def >> : Boolean = write(weights.mkString(","))
  27. }


  30. final class Crf(
  31.                 nLabels: Int,
  32.                 config: CrfConfig,
  33.                 delims: CrfSeqDelimiter,
  34.                 taggedObs: String) extends PipeOperator[String, Double] {
  35.         import Crf._
  36.         check(nLabels)
  37.   
  38.         private val logger = Logger.getLogger("Crf")

  40.         class TaggingGenerator(nLabels: Int)        
  41.                         extends FeatureGenImpl(new CompleteModel(nLabels), nLabels, true)


  44.         private[this] val features = new TaggingGenerator(nLabels)
  45.         private[this] val crf = new CRF(nLabels, features, config.params)


  48.         private val model: Option[CrfModel] = train match {
  49.                 case Success(model) => Some(model)
  50.                 case Failure(e) => DisplayUtils.none("Crf.model could not be created", logger, e)
  51.         }
  52.   

  54.         override def |> : PartialFunction[String, Double] = {
  55.                 case obs: String if( !obs.isEmpty && model != None) => {
  56.                         val dataSeq =  new CrfTrainingSet(nLabels, obs, delims.obsDelim)
  57.                         crf.apply(dataSeq)
  58.                 }
  59.         }
  60.        

  62.         final def weights: Option[DblVector] = model.map( _.weights)


  65.         private def train: Try[CrfModel] = {
  66.                 val seqIter = CrfSeqIter(nLabels, taggedObs, delims)
  67.                 Try {
  68.                         features.train(seqIter)
  69.                         new CrfModel(crf.train(seqIter))
  70.                 }
  71.         }
  72. }


  75. object Crf {
  76.         final val NUM_LABELS_LIMITS = (1, 512)
  77.        

  79.         def apply(nLabels: Int, state: CrfConfig, delims: CrfSeqDelimiter, taggedObs: String): Crf =
  80.                 new Crf(nLabels, state, delims, taggedObs)
  81.   
  82.   
  83.         private def check(nLabels: Int): Unit = {
  84.                 require(nLabels > NUM_LABELS_LIMITS._1 && nLabels < NUM_LABELS_LIMITS._2,
  85.                                 s"Number of labels for generating tags for CRF $nLabels is out of range")
  86.         }
  87. }


  90. // ---------------------------- EOF ------------------------------------------------------
复制代码

使用道具

28楼
Lisrelchen 发表于 2016-4-20 09:34:39 |只看作者 |坛友微信交流群
  1. package org.scalaml.supervised.crf

  3. import iitb.CRF.{CRF, CrfParams, DataSequence, DataIter, FeatureGenerator}
  4. import iitb.Model.{FeatureGenImpl, CompleteModel}
  5. import org.scalaml.core.XTSeries
  6. import org.scalaml.workflow.data.DataSource
  7. import org.scalaml.core.Design.{PipeOperator, Config}
  8. import java.io.IOException
  9. import org.scalaml.core.Types.ScalaMl._


  12. protected class CrfConfig(w0: Double, maxIters: Int, lambda: Double, eps: Double) extends Config {
  13.         import CrfConfig._
  14.         check(w0, maxIters, lambda, eps)
  15.                

  17.         val params = s"initValue ${String.valueOf(w0)} maxIters ${String.valueOf(maxIters)} " +
  18.                         s"lambda ${String.valueOf(lambda)} scale true eps $eps"
  19. }


  22. object CrfConfig {
  23.         private val INIT_WEIGHTS_LIMITS = (0.1, 2.5)
  24.         private val MAX_ITERS_LIMITS = (10, 250)
  25.         private val LAMBDA_LIMITS = (1e-15, 1.5)
  26.         private val EPS_LIMITS = (1e-5, 0.2)
  27.        

  29.         def apply(w0: Double, maxIters: Int, lambda: Double, eps:Double): CrfConfig =
  30.                 new CrfConfig(w0, maxIters, lambda, eps)
  31.        
  32.        
  33.         private def check(w0: Double, maxIters: Int, lambda: Double,  eps: Double): Unit = {
  34.                 require(w0 >= INIT_WEIGHTS_LIMITS._1 && w0 <= INIT_WEIGHTS_LIMITS._2,
  35.                                 s"Initialization of the CRF weights $w0 is out of range")
  36.                 require( maxIters >= MAX_ITERS_LIMITS._1 && maxIters <= MAX_ITERS_LIMITS._2,
  37.                                 s"Maximum number of iterations for CRF training $maxIters is out of range")
  38.                 require( lambda >= LAMBDA_LIMITS._1 && lambda <= LAMBDA_LIMITS._2,
  39.                                 s"The factor for the L2 penalty for CRF $lambda is out of range")
  40.                 require( eps > EPS_LIMITS._1 && eps<= EPS_LIMITS._2,
  41.                                  s"The convergence criteria for the CRF training $eps is out of range")
  42.     }
  43. }



  47. // ---------------------------- EOF ------------------------------------------------------
复制代码

使用道具

29楼
Lisrelchen 发表于 2016-4-20 09:38:51 |只看作者 |坛友微信交流群
  1. package org.scalaml.supervised.crf

  3.         // IITB library classes
  4. import iitb.CRF.{CRF, CrfParams, DataSequence, DataIter}
  5. import iitb.Model.FeatureImpl
  6. import iitb.Segment.{DataCruncher, LabelMap}
  7.         // ScalaMl classes
  8. import org.scalaml.core.Types

  10. class CrfSeqDelimiter(val obsDelim: String, val labelsDelim: String, val trainingDelim: String) {
  11.         require(obsDelim != Types.nullString,
  12.                         "Delimiter for observations in CRF training sequence is undefined")
  13.         require(labelsDelim != Types.nullString,
  14.                         "Delimiter for labels in CRF training sequence is undefined")
  15.         require(trainingDelim != Types.nullString,
  16.                         "Delimiter for training sequences in CRF training sequence is undefined")
  17. }

  19. class CrfSeqIter(val nLabels: Int, val input: String, val delim: CrfSeqDelimiter) extends DataIter {
  20.         import CrfSeqIter._
  21.         check(nLabels, input, delim)
  22.        

  24.         lazy val trainData = DataCruncher.readTagged(nLabels, input, input, delim.obsDelim,
  25.                         delim.labelsDelim, delim.trainingDelim, new LabelMap)
  26.    

  28.         override def hasNext: Boolean = trainData.hasNext
  29.      

  31.         override def next: DataSequence = trainData.next
  32.    

  34.         override def startScan: Unit = trainData.startScan
  35. }


  38. object CrfSeqIter {
  39.         private val MAX_NUM_LABELS = 1000
  40.         private val DEFAULT_SEQ_DELIMITER = new CrfSeqDelimiter(",\t/ -():.;'?#`&_", "//", "\n")


  43.         def apply(
  44.                         nLabels: Int,
  45.                         input: String,
  46.                         delim: CrfSeqDelimiter): CrfSeqIter = new CrfSeqIter(nLabels, input, delim)
  47.        
  48.         def apply(nLabels: Int, input: String): CrfSeqIter =        
  49.                         new CrfSeqIter(nLabels, input, DEFAULT_SEQ_DELIMITER)
  50.    
  51.         private def check(nLabels: Int, input: String, delim: CrfSeqDelimiter): Unit = {
  52.                 require(nLabels > 0 && nLabels < MAX_NUM_LABELS,
  53.                                 s"CrfSeqIter.check Number of labels for the CRF model $nLabels is out of range")
  54.                 require(input != Types.nullString,
  55.                                 "CrfSeqIter.check  input for the CRF training files is undefined")
  56.         }
  57. }

  59. // ---------------------------- EOF ------------------------------------------------------
复制代码

使用道具

30楼
Lisrelchen 发表于 2016-4-20 09:40:36 |只看作者 |坛友微信交流群
  1. package org.scalaml.supervised.crf

  3. import iitb.CRF.{CRF, CrfParams, DataSequence, DataIter}
  4. import java.util.Properties
  5. import iitb.Model.FeatureImpl
  6. import iitb.Segment.{DataCruncher, LabelMap}


  9. class CrfTrainingSet(val nLabels: Int, val entry: String, val delim: String) extends DataSequence {
  10.         import CrfTrainingSet._
  11.        
  12.         check(nLabels, entry, delim)
  13.        
  14.         private[this] val words: Array[String] = entry.split(delim)
  15.         private[this] val map: Array[Int] = new Array[Int](nLabels)
  16.    
  17.         override def set_y(k: Int, label: Int): Unit = map(k) = label
  18.         override def y(k: Int): Int = map(k)
  19.         override def length: Int = words.size
  20.         override def x(k: Int): Object = words(k)
  21. }


  24. object CrfTrainingSet {
  25.         import Crf._
  26.        
  27.         private def check(nLabels: Int, entry: String, delim: String): Unit = {
  28.                 require(nLabels >= NUM_LABELS_LIMITS._1 && nLabels < NUM_LABELS_LIMITS._2)
  29.         }
  30. }


  33. // ---------------------------- EOF ------------------------------------------------------
复制代码

使用道具

返回列表
123456下一页
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加JingGuanBbs
拉您进交流群
  • 手机版  |
  • 意见反馈  |
  • 帮助  |
  • 新手入门  |
  • 用户手册  |
  • 友情链接  |

    • 如有投资本站、合作意向或投放广告，请联系：13661292478（刘老师）

    联系客服

    邮箱：service@pinggu.org 投诉或不良信息处理：（010-68466864）

    010-68466864

    京ICP备16021002-2号 京B2-20170662号 京公网安备 11010802022788号 论坛法律顾问：王进律师 知识产权保护声明   免责及隐私声明

    GMT+8, 2024-4-24 06:02