Python 3 Text Processing with NLTK 3 Cookbook

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. Author:Jacob Perkins
   
2. Isbn:978-1-78216-785-3
   
3. Year:2014
   
4. Pages:304
   
5. Language:English
   
6. File size:1.8 MB
   
7. File format:PDF

复制代码

本帖隐藏的内容

Python 3 Text Processing with NLTK 3 Cookbook.rar (1.4 MB) 本附件包括：

• Python 3 Text Processing with NLTK 3 Cookbook.pdf

2016-12-29 09:34:09 上传

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏2 回帖

关键词：Processing Cookbook processI Process python

1. Stemming words
   
2. How to do it...
   
3. NLTK comes with an implementation of the Porter stemming algorithm, which is very easy to
   
4. use. Simply instantiate the PorterStemmer class and call the stem() method with the word
   
5. you want to stem:
   
6. >>> from nltk.stem import PorterStemmer
   
7. >>> stemmer = PorterStemmer()
   
8. >>> stemmer.stem('cooking')
   
9. 'cook'
   
10. >>> stemmer.stem('cookery')
    
11. 'cookeri'

复制代码

1. Lemmatizing words with WordNet
   
2. How to do it...
   
3. We will use the WordNetLemmatizer class to find lemmas:
   
4. >>> from nltk.stem import WordNetLemmatizer
   
5. >>> lemmatizer = WordNetLemmatizer()
   
6. >>> lemmatizer.lemmatize('cooking')
   
7. 'cooking'
   
8. >>> lemmatizer.lemmatize('cooking', pos='v')
   
9. 'cook'
   
10. >>> lemmatizer.lemmatize('cookbooks')
    
11. 'cookbook'

复制代码

thanks ~~~~~~~~~~~~~

1. Removing repeating characters
   
2. import re
   
3. class RepeatReplacer(object):
   
4. def __init__(self):
   
5. self.repeat_regexp = re.compile(r'(\w*)(\w)\2(\w*)')
   
6. self.repl = r'\1\2\3'
   
7. def replace(self, word):
   
8. repl_word = self.repeat_regexp.sub(self.repl, word)
   
9. if repl_word != word:
   
10. return self.replace(repl_word)
    
11. else:
    
12. return repl_word
    
13. And now some example use cases:
    
14. >>> from replacers import RepeatReplacer
    
15. >>> replacer = RepeatReplacer()
    
16. >>> replacer.replace('looooove')
    
17. 'love'
    
18. >>> replacer.replace('oooooh')
    
19. 'oh'
    
20. >>> replacer.replace('goose')
    
21. 'gose'

复制代码

1. Spelling correction with Enchant
   
2. How to do it...
   
3. We will create a new class called SpellingReplacer in replacers.py, and this
   
4. time, the replace() method will check Enchant to see whether the word is valid.
   
5. If not, we will look up the suggested alternatives and return the best match using
   
6. nltk.metrics.edit_distance():
   
7. import enchant
   
8. from nltk.metrics import edit_distance
   
9. class SpellingReplacer(object):
   
10. def __init__(self, dict_name='en', max_dist=2):
    
11. self.spell_dict = enchant.Dict(dict_name)
    
12. self.max_dist = max_dist
    
13. def replace(self, word):
    
14. if self.spell_dict.check(word):
    
15. return word
    
16. suggestions = self.spell_dict.suggest(word)
    
17. if suggestions and edit_distance(word, suggestions[0]) <=
    
18. self.max_dist:
    
19. return suggestions[0]
    
20. else:
    
21. return word
    
22. The preceding class can be used to correct English spellings, as follows:
    
23. >>> from replacers import SpellingReplacer
    
24. >>> replacer = SpellingReplacer()
    
25. >>> replacer.replace('cookbok')
    
26. 'cookbook'

复制代码

1. Replacing synonyms
   
2. How to do it...
   
3. We'll first create a WordReplacer class in replacers.py that takes a word
   
4. replacement mapping:
   
5. class WordReplacer(object):
   
6. def __init__(self, word_map):
   
7. self.word_map = word_map
   
8. def replace(self, word):
   
9. return self.word_map.get(word, word)
   
10. Then, we can demonstrate its usage for simple word replacement:
    
11. >>> from replacers import WordReplacer
    
12. >>> replacer = WordReplacer({'bday': 'birthday'})
    
13. >>> replacer.replace('bday')
    
14. 'birthday'
    
15. >>> replacer.replace('happy')
    
16. 'happy'

复制代码

感谢分享