[Case Study]Text Search using Python, Scala and Java

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. text_file = spark.textFile("hdfs://...")
   
2. errors = text_file.filter(lambda line: "ERROR" in line)
   
3. # Count all the errors
   
4. errors.count()
   
5. # Count errors mentioning MySQL
   
6. errors.filter(lambda line: "MySQL" in line).count()
   
7. # Fetch the MySQL errors as an array of strings
   
8. errors.filter(lambda line: "MySQL" in line).collect()

复制代码

1. Val textFile = spark.textFile("hdfs://...")
   
2. val errors = textFile.filter(line => line.contains("ERROR"))
   
3. // Count all the errors
   
4. errors.count()
   
5. // Count errors mentioning MySQL
   
6. errors.filter(line => line.contains("MySQL")).count()
   
7. // Fetch the MySQL errors as an array of strings
   
8. errors.filter(line => line.contains("MySQL")).collect()

复制代码

1. JavaRDD<String> textFile = spark.textFile("hdfs://...");
   
2. JavaRDD<String> errors = textFile.filter(new Function<String, Boolean>() {
   
3.   public Boolean call(String s) { return s.contains("ERROR"); }
   
4. });
   
5. // Count all the errors
   
6. errors.count();
   
7. // Count errors mentioning MySQL
   
8. errors.filter(new Function<String, Boolean>() {
   
9.   public Boolean call(String s) { return s.contains("MySQL"); }
   
10. }).count();
    
11. // Fetch the MySQL errors as an array of strings
    
12. errors.filter(new Function<String, Boolean>() {
    
13.   public Boolean call(String s) { return s.contains("MySQL"); }
    
14. }).collect();

复制代码

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Case study python search Using SCALA Java