[Case Study]Random RDD using Python

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. """
   
2. Randomly generated RDDs.
   
3. """
   
4. from __future__ import print_function
   
5. 
   
6. import sys
   
7. 
   
8. from pyspark import SparkContext
   
9. from pyspark.mllib.random import RandomRDDs
   
10. 
    
11. 
    
12. if __name__ == "__main__":
    
13.     if len(sys.argv) not in [1, 2]:
    
14.         print("Usage: random_rdd_generation", file=sys.stderr)
    
15.         exit(-1)
    
16. 
    
17.     sc = SparkContext(appName="PythonRandomRDDGeneration")
    
18. 
    
19.     numExamples = 10000  # number of examples to generate
    
20.     fraction = 0.1  # fraction of data to sample
    
21. 
    
22.     # Example: RandomRDDs.normalRDD
    
23.     normalRDD = RandomRDDs.normalRDD(sc, numExamples)
    
24.     print('Generated RDD of %d examples sampled from the standard normal distribution'
    
25.           % normalRDD.count())
    
26.     print('  First 5 samples:')
    
27.     for sample in normalRDD.take(5):
    
28.         print('    ' + str(sample))
    
29.     print()
    
30. 
    
31.     # Example: RandomRDDs.normalVectorRDD
    
32.     normalVectorRDD = RandomRDDs.normalVectorRDD(sc, numRows=numExamples, numCols=2)
    
33.     print('Generated RDD of %d examples of length-2 vectors.' % normalVectorRDD.count())
    
34.     print('  First 5 samples:')
    
35.     for sample in normalVectorRDD.take(5):
    
36.         print('    ' + str(sample))
    
37.     print()
    
38. 
    
39.     sc.stop()
    
40. Status API Training Shop Blog About Pricing
    
41. © 2015 GitHub, Inc. Terms Privacy Security Contact Help

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Case study python random Using study generated examples Random number future