TheanoLM - An Extensible Toolkit for Neural Network Language Modeling

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. Abstract
   
2. We present a new tool for training neural network language models (NNLMs), scoring sentences, and generating text. The tool
   
3. has been written using Python library Theano, which allows researcher to easily extend it and tune any aspect of the training
   
4. process. Regardless of the flexibility, Theano is able to generate extremely fast native code that can utilize a GPU or multiple
   
5. CPU cores in order to parallelize the heavy numerical computations.
   
6. 
   
7. The tool has been evaluated in difficult Finnish and English conversational speech recognition tasks, and significant
   
8. improvement was obtained over our best back-off n-gram models. The results that we obtained in the Finnish task were compared
   
9. to those from existing RNNLM and RWTHLM toolkits, and found to be as good or better, while training times were an order of magnitude shorter.

复制代码

本帖隐藏的内容

TheanoLM — An Extensible Toolkit for Neural Network Language Modeling.pdf (96.17 KB)

2017-9-11 03:06:14 上传

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Modeling Language network Toolkit Theano

1. from __future__ import print_function
   
2. import theano
   
3. import theano.tensor as T
   
4. from theano.tensor.shared_randomstreams import RandomStreams
   
5. import sys
   
6. import time
   
7. import numpy as np
   
8. import pprint
   
9. 
   
10. sys.path.append("../..")
    
11. import DeepLearningStack
    
12. from DeepLearningStack import RecurrentNet
    
13. 
    
14. 
    
15. 
    
16. #create a deep network defined in AlexNet.xml
    
17. if __name__=="__main__":
    
18. 
    
19.     #create deep net
    
20.     num_timesteps     = 2
    
21.     print ("Creating Recurrent Net for %d time steps"%num_timesteps)
    
22.     config            = "RNNArch.xml"
    
23.     #random number generator used to initialize the weights
    
24.     rng               = RandomStreams(seed=int(time.time()))
    
25.     #for each time step, create a non-recurrent input
    
26.     nonrcrnt_ins      = []
    
27.     #size of the input layer is 136x128,
    
28.     for t in range(num_timesteps):
    
29.         in_matrix     = T.matrix("data-step-%d"%t,dtype=theano.config.floatX)#the input is concatenation of action history and beliefs
    
30.         nonrcrnt_ins.append( {"input":[in_matrix,(136,128)]} )#for each input, provide a symbolic variable and its size
    
31.     #create the recurrent inputs for time step 0
    
32.     #size of the recurrent inputs according to RNNArch.xml (batch size is 128, each size is dim x batch_size)
    
33.     # rct1: 128x128
    
34.     # rct2: 128x128
    
35.     # fc3 : 10x128
    
36.     #fc1_step0         = T.matrix("fc1-step-0",dtype=theano.config.floatX)#the input is concatenation of action history and beliefs
    
37.     fc2_step0         = T.matrix("fc2-step-0",dtype=theano.config.floatX)#the input is concatenation of action history and beliefs
    
38.     #rcrnt_ins         = {"fc1":[fc1_step0,(128,128)],"fc2":[fc2_step0,(128,128)]}
    
39.     rcrnt_ins         = {"fc2":[fc2_step0,(128,128)]}
    
40. 
    
41.     #create the graph structure
    
42.     rnn               = RecurrentNet.RecurrentNet( rng, nonrcrnt_ins, rcrnt_ins, config, unrolled_len=num_timesteps)
    
43.     print("RNN params:")
    
44.     pprint.pprint(rnn.params)
    
45. 
    
46.     #create a function for RNN
    
47.    
    
48.     #inputs to the graph consists of the recurrent inputs for time step 0 and non-recurrent inputs for all time steps
    
49.     inputs            = [nonrcrnt_ins[k]["input"][0] for k in range(len(nonrcrnt_ins))]
    
50.     for k in rcrnt_ins.keys():
    
51.         inputs.append(rcrnt_ins[k][0])
    
52.     #outputs of the network include the outputs for each time step
    
53.     outputs           = [rnn.name2layer[i]["fc2"].output for i in rnn.name2layer.keys()]
    
54.     print("compiling the function")
    
55.     f                 = theano.function(inputs=inputs,outputs=outputs)
    
56.     #draw the RNN
    
57.     graph_img_name    = "RNN.png"
    
58.     print("creating graph picture in:",graph_img_name)
    
59.     theano.printing.pydotprint(f, outfile=graph_img_name, var_with_name_simple=True)  

复制代码

kankan

谢谢楼主分享！