运行pyspark的logistic回归报错

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. data = RandomRDDs.normalVectorRDD(sc, 100000, 10, seed=2)
   
2. 
   
3. 
   
4. def tologisticregressiondata(x):
   
5.     return LabeledPoint(rd.randint(0,1), x)
   
6. 
   
7. dataforlogisticregression = data.map(tologisticregressiondata)
   
8. 
   
9. cdata = dataforlogisticregression.randomSplit([0.8, 0.2])
   
10. ctrain = cdata[0]
    
11. ctest = cdata[1]
    
12. 
    
13. cmodel = LogisticRegressionWithLBFGS.train(ctrain)

复制代码

-------------------------------------------------------------------------------------------------------------------------------------------
18/12/28 13:57:29 WARN HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
18/12/28 13:57:29 WARN metastore: Failed to connect to the MetaStore Server...
18/12/28 13:57:30 WARN metastore: Failed to connect to the MetaStore Server...
18/12/28 13:57:31 WARN metastore: Failed to connect to the MetaStore Server...
18/12/28 13:57:32 WARN Hive: Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate o                      rg.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
请问为什么会这样呢，其他方法，比如线性回归，SVM，随机森林等等都没有问题，只有logistic回归会报这个错误

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：spark pyspark logistic hive mllib

尝试用随机梯度就没有问题

>>> multi_class_data = [
...     LabeledPoint(0.0, [0.0, 1.0, 0.0]),
...     LabeledPoint(1.0, [1.0, 0.0, 0.0]),
...     LabeledPoint(2.0, [0.0, 0.0, 1.0])
... ]
>>> data = sc.parallelize(multi_class_data)
>>> mcm = LogisticRegressionWithLBFGS.train(data, iterations=10, numClasses=3)

=================================

这里会报这个错
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/spark-current/python/pyspark/mllib/classification.py", line 398, in train
    return _regression_train_wrapper(train, LogisticRegressionModel, data, initialWeights)
  File "/usr/lib/spark-current/python/pyspark/mllib/regression.py", line 216, in _regression_train_wrapper
    return modelClass(weights, intercept, numFeatures, numClasses)
  File "/usr/lib/spark-current/python/pyspark/mllib/classification.py", line 176, in __init__
    self._dataWithBiasSize)
TypeError: 'float' object cannot be interpreted as an integer

您好，如果您的求助没有解决，请到项目交易发布需求，会有更快更专业的用户帮助您 https://bbs.pinggu.org/prj/