楼主: NewOccidental
1097 4

Ramp:Python Library for Prototyping of Machine Learning Solutions [推广有奖]

  • 0关注
  • 6粉丝

已卖：1554份资源

副教授

31%

还不是VIP/贵宾

-

TA的文库  其他...

Complex Data Analysis

东西方金融数据分析

eBook with Data and Code

0%
威望
0 级
论坛币
11734 个
通用积分
2.2450
学术水平
119 点
热心指数
115 点
信用等级
114 点
经验
8940 点
帖子
173
精华
10
在线时间
30 小时
注册时间
2006-9-19
最后登录
2022-11-3
楼主
NewOccidental 发表于 2016-4-25 03:12:28 |AI写论文

+2 论坛币
k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群
赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

Ramp

Ramp is a python library for rapid prototyping of machine learning solutions. It's a light-weight pandas-based machine learning framework pluggable with existing python machine learning and statistics tools (scikit-learn, rpy2, etc.). Ramp provides a simple, declarative syntax for exploring features, algorithms and transformations quickly and efficiently.

ramp-master.zip (70.55 KB)


二维码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0
关键词：solutions Solution Learning machine Library framework learning existing features provides

相关帖子

沙发
NewOccidental 发表于 2016-4-25 03:13:42
  1. import urllib2

  3. import pandas as pd
  4. import sklearn
  5. from sklearn import decomposition

  7. import ramp
  8. from ramp.features import *
  9. from ramp.metrics import PositiveRate, Recall

  11. import logging
  12. logger = logging.getLogger()
  13. logger.setLevel(logging.DEBUG)

  15. # fetch and clean iris data from UCI
  16. data = pd.read_csv(urllib2.urlopen(
  17.     "http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data"))
  18. data = data.drop([149]) # bad line
  19. columns = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'class']
  20. data.columns = columns

  22. # all features
  23. features = [FillMissing(f, 0) for f in columns[:-1]]

  25. # features, log transformed features, and interaction terms
  26. expanded_features = (
  27.     features +
  28.     [Log(F(f) + 1) for f in features] +
  29.     [
  30.         F('sepal_width') ** 2,
  31.         combo.Interactions(features),
  32.     ]
  33. )

  35. reporters = [
  36.     ramp.reporters.MetricReporter.factory(Recall(.4)),
  37.     ramp.reporters.DualThresholdMetricReporter.factory(Recall(), PositiveRate())
  38. ]


  41. # Define several models and feature sets to explore,
  42. # run 5 fold cross-validation on each and print the results.
  43. # We define 2 models and 4 feature sets, so this will be
  44. # 4 * 2 = 8 models tested.
  45. outcomes = ramp.shortcuts.cv_factory(
  46.     data=data,
  47.     folds=10,

  49.     target=[AsFactor('class')],

  51.     reporter_factories=reporters,

  53.     # Try out two algorithms
  54.     estimator=[
  55.         sklearn.ensemble.RandomForestClassifier(
  56.             n_estimators=20),
  57.         sklearn.linear_model.LogisticRegression(),
  58.         ],

  60.     # and 4 feature sets
  61.     features=[
  62.         expanded_features,

  64.         # Feature selection
  65.         # [trained.FeatureSelector(
  66.         #     expanded_features,
  67.         #     # use random forest's importance to trim
  68.         #     ramp.selectors.BinaryFeatureSelector(),
  69.         #     target=AsFactor('class'), # target to use
  70.         #     data=data,
  71.         #     n_keep=5, # keep top 5 features
  72.         #     )],

  74.         # Reduce feature dimension (pointless on this dataset)
  75.         [combo.DimensionReduction(expanded_features,
  76.                             decomposer=decomposition.PCA(n_components=4))],

  78.         # Normalized features
  79.         [Normalize(f) for f in expanded_features],
  80.     ]
  81. )

  83. print outcomes.values()[0]['reporters'][0]
复制代码
藤椅
NewOccidental 发表于 2016-4-25 03:14:16
  1. import pandas
  2. from ramp import *
  3. from ramp.estimators.sk import BinaryProbabilities
  4. import sklearn
  5. from sklearn import naive_bayes
  6. import gensim
  7. import tempfile

  9. try:
  10.     training_data = pandas.read_csv('train.csv')
  11. except IOError:
  12.     raise IOError("You need to download the 'Detecting Insults' dataset \
  13.                   from Kaggle to run this example. \
  14.                   http://www.kaggle.com/c/detecting-insults-in-social-commentary")


  17. tmpdir = tempfile.mkdtemp()
  18. context = DataContext(
  19.               store=tmpdir,
  20.               data=training_data)


  23. base_config = Configuration(
  24.     target='Insult',
  25.     metrics=[metrics.AUC()],
  26.     )

  28. base_features = [
  29.     Length('Comment'),
  30.     Log(Length('Comment') + 1)
  31. ]

  33. factory = ConfigFactory(
  34.     base_config,
  35.     features=[
  36.         # first feature set is basic attributes
  37.         base_features,

  39.         # second feature set adds word features
  40.         base_features + [
  41.             text.NgramCounts(
  42.                 text.Tokenizer('Comment'),
  43.                 mindocs=5,
  44.                 bool_=True)],

  46.         # third feature set creates character 5-grams
  47.         # and then selects the top 1000 most informative
  48.         base_features + [
  49.             trained.FeatureSelector(
  50.                 [text.NgramCounts(
  51.                     text.CharGrams('Comment', chars=5),
  52.                     bool_=True,
  53.                     mindocs=30)
  54.                 ],
  55.                 selector=selectors.BinaryFeatureSelector(),
  56.                 n_keep=1000,
  57.                 target=F('Insult')),
  58.             ],

  60.         # the fourth feature set creates 100 latent vectors
  61.         # from the character 5-grams
  62.         base_features + [
  63.             text.LSI(
  64.                 text.CharGrams('Comment', chars=5),
  65.                 mindocs=30,
  66.                 num_topics=100),
  67.             ]
  68.     ],

  70.     # we'll try two estimators (and wrap them so
  71.     # we get class probabilities as output):
  72.     model=[
  73.         BinaryProbabilities(
  74.             sklearn.linear_model.LogisticRegression()),
  75.         BinaryProbabilities(
  76.             naive_bayes.GaussianNB())
  77.     ]
  78. )


  81. for config in factory:
  82.     models.cv(config, context, folds=5, repeat=2,
  83.               print_results=True)


  86. def probability_of_insult(config, ctx, txt):
  87.     # create a unique index for this text
  88.     idx = int(md5(txt).hexdigest()[:10], 16)

  90.     # add the new comment to our DataFrame
  91.     d = DataFrame(
  92.             {'Comment':[txt]},
  93.             index=pandas.Index([idx]))
  94.     ctx.data = ctx.data.append(d)

  96.     # Specify which instances to predict with predict_index
  97.     # and make the prediction
  98.     pred, predict_x, predict_y = models.predict(
  99.             config,
  100.             ctx,
  101.             predict_index=pandas.Index([idx]))

  103.     return pred[idx]
复制代码
板凳
NewOccidental 发表于 2016-4-25 03:14:58
  1. # class Filter(Storable):

  3. #     def __init__(self, exclude_func=None, include_func=None):
  4. #         self.exclude_func = exclude_func
  5. #         self.include_func = include_func

  7. #     def filter(self, df):
  8. #         if self.include_func is not None:
  9. #             df =

  11. def filter_incomplete(df):
  12.     df = df.dropna()
  13.     return df
复制代码
报纸
NewOccidental 发表于 2016-4-25 03:16:36
  1. import types

  3. import numpy as np


  6. __all__ = ['Wrapper', 'Estimator', 'ConstantClassifier', 'ConstantRegressor',
  7.            'Probabilities', 'BinaryProbabilities', 'wrap_sklearn_like_estimator']


  10. class Wrapper(object):
  11.     def __init__(self, obj):
  12.         self._obj = obj

  14.     def __getattr__(self, attr):

  16.         if hasattr(self._obj, attr):
  17.             attr_value = getattr(self._obj,attr)

  19.             if isinstance(attr_value, types.MethodType):
  20.                 def callable(*args, **kwargs):
  21.                     return attr_value(*args, **kwargs)
  22.                 return callable
  23.             else:
  24.                 return attr_value

  26.         else:
  27.             raise AttributeError

  29.     def __getstate__(self): return self.__dict__
  30.     def __setstate__(self, d): self.__dict__.update(d)


  33. class Estimator(Wrapper):
  34.     def __init__(self, estimator):
  35.         self.base_estimator_ = estimator
  36.         super(Estimator, self).__init__(estimator)

  38.     def __repr__(self):
  39.         return repr(self.base_estimator_)

  41.     def fit(self, x, y, **kwargs):
  42.         return self.base_estimator_.fit(x.values, y.values, **kwargs)

  44.     def predict_maxprob(self, x, **kwargs):
  45.         """
  46.         Most likely value. Generally equivalent to predict.
  47.         """
  48.         return self.base_estimator_.predict(x.values, **kwargs)

  50.     def predict(self, x, **kwargs):
  51.         """
  52.         Model output. Not always the same as scikit_learn predict. E.g., in the
  53.         case of logistic regression, returns the probability of each outome.
  54.         """
  55.         return self.base_estimator_.predict(x.values, **kwargs)


  58. class Probabilities(Estimator):
  59.     """
  60.     Wraps a scikit-learn-like estimator to return probabilities (if
  61.     it supports it)
  62.     """
  63.     def __init__(self, estimator, binary=False):
  64.         """
  65.         binary: If True, predict returns only the probability
  66.             for the positive class. If False, returns probabilities for
  67.             all classes.
  68.         """
  69.         self.binary = binary
  70.         super(Probabilities, self).__init__(estimator)

  72.     def __str__(self):
  73.         return u"Probabilites for %s" % self.base_estimator_

  75.     def predict(self, x):
  76.         probs = self.base_estimator_.predict_proba(x)
  77.         if probs.shape[1] == 2 or self.binary:
  78.             return probs[:,1]
  79.         return probs


  82. class BinaryProbabilities(Probabilities):
  83.     def __init__(self, estimator):
  84.         super(BinaryProbabilities, self).__init__(estimator, binary=True)


  87. class ConstantClassifier(object):

  89.     def __init__(self, func):
  90.         self.func = func
  91.         self.constant = None

  93.     def fit(self, x, y, **kwargs):
  94.         self.constant = self.func(y)

  96.     def predict(self, x, **kwargs):
  97.         return np.full((x.shape[0],), int(self.constant > .5))

  99.     def predict_proba(self, x, **kwargs):
  100.         p = np.zeros((x.shape[0], 2))
  101.         p[:,0] = 1 - self.constant
  102.         p[:,1] = self.constant
  103.         return p


  106. class ConstantRegressor(object):

  108.     def __init__(self, func):
  109.         self.func = func
  110.         self.constant = None

  112.     def fit(self, x, y, **kwargs):
  113.         self.constant = self.func(y)

  115.     def predict(self, x, **kwargs):
  116.         return np.full((x.shape[0],), self.constant)


  119. def wrap_sklearn_like_estimator(estimator):
  120.     if isinstance(estimator, Estimator):
  121.         return estimator
  122.     elif estimator is None:
  123.         return None
  124.     elif not (hasattr(estimator, "fit") and (hasattr(estimator, "predict")
  125.                                           or hasattr(estimator, "predict_proba"))):
  126.         raise ValueError, "Invalid estimator: %s" % estimator
  127.     elif hasattr(estimator, "predict_proba"):
  128.         return Probabilities(estimator)
  129.     else:
  130.         return Estimator(estimator)
复制代码
返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加好友,备注jltj
拉您入交流群

京ICP备16021002号-2 京B2-20170662号 京公网安备 11010802022788号 论坛法律顾问：王进律师 知识产权保护声明   免责及隐私声明

GMT+8, 2025-12-31 20:53