Scala Text Mining Library

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

lib-text

本帖隐藏的内容

lib-text-master.zip (1.23 MB)

2017-7-7 00:31:24 上传

A little text processing library for Scala.

Overview

This is a little text processing library which supports language identification, tokenization, stopword filtering and provides some useful helper functions. The tokenization has been tuned to work well with text conventions commonly used in social media such as Twitter, and supports URLs, emoji, hashtags, emails and @-mentions cleanly. Stopword filtering is currently supported for

• German
• English
• Spanish
• French
• Indonesian
• Japanese
• Malay
• Dutch
• Portuguese
• Swedish
• Turkish
• Arabic
  

More to come.

Usage

Add to your project dependencies:

resolvers += "peoplepattern" at "https://dl.bintray.com/peoplepattern/maven/"libraryDependencies += "com.peoplepattern" %% "lib-text" % "0.3"

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

1. Example
   
2. 
   
3. import com.peoplepattern.text.Implicits._
   
4. 
   
5. val txt = "Did you get your personalised print with your copy of #MadeintheAM on Black Friday? If not, there's still time! http://www.myplaydirect.com/one-direction"
   
6. 
   
7. txt.lang
   
8. // Some(en)
   
9. 
   
10. txt.tokens
    
11. // Vector(Did, you, get, your, personalised, print, with, your, copy, of, #MadeintheAM, on, Black, Friday, ?, If, not, ,, there's, still, time, !, http://www.myplaydirect.com/one-direction)
    
12. 
    
13. txt.terms
    
14. // Set(print, personalised, black, copy, friday, time)
    
15. 
    
16. txt.termsPlus
    
17. // Set(print, personalised, black, #madeintheam, copy, friday, time)
    
18. 
    
19. txt.termBigrams
    
20. // Set(black friday, personalised print)

复制代码