【Tutorial】Deep MNIST for Experts

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

1. Deep MNIST for Experts
   
2. 
   
3. TensorFlow is a powerful library for doing large-scale numerical computation. One of the tasks at which it excels is implementing and training deep neural networks. In this tutorial we will learn the basic building blocks of a TensorFlow model while constructing a deep convolutional MNIST classifier.
   
4. 
   
5. This introduction assumes familiarity with neural networks and the MNIST dataset. If you don't have a background with them, check out the introduction for beginners. Be sure to install TensorFlow before starting.
   
6. 
   
7. About this tutorial
   
8. 
   
9. The first part of this tutorial explains what is happening in the mnist_softmax.py code, which is a basic implementation of a Tensorflow model. The second part shows some ways to improve the accuracy.
   
10. 
    
11. You can copy and paste each code snippet from this tutorial into a Python environment, or you can choose to just read through the code.
    
12. 
    
13. What we will accomplish in this tutorial:
    
14. 
    
15. Create a softmax regression function that is a model for recognizing MNIST digits, based on looking at every pixel in the image
    
16. Use Tensorflow to train the model to recognize digits by having it "look" at thousands of examples (and run our first Tensorflow session to do so)
    
17. Check the model's accuracy with our test data
    
18. Build, train, and test a multilayer convolutional neural network to improve the results

复制代码

https://www.tensorflow.org/get_started/mnist/pros

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Tutorial experts Expert tutor Tori background building training library learn

1. Load MNIST Data
   
2. 
   
3. If you are copying and pasting in the code from this tutorial, start here with these two lines of code which will download and read in the data automatically:
   
4. 
   
5. from tensorflow.examples.tutorials.mnist import input_data
   
6. mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

复制代码

1. Start TensorFlow InteractiveSession
   
2. 
   
3. TensorFlow relies on a highly efficient C++ backend to do its computation. The connection to this backend is called a session. The common usage for TensorFlow programs is to first create a graph and then launch it in a session.
   
4. 
   
5. Here we instead use the convenient InteractiveSession class, which makes TensorFlow more flexible about how you structure your code. It allows you to interleave operations which build a computation graph with ones that run the graph. This is particularly convenient when working in interactive contexts like IPython. If you are not using an InteractiveSession, then you should build the entire computation graph before starting a session and launching the graph.
   
6. 
   
7. import tensorflow as tf
   
8. sess = tf.InteractiveSession()

复制代码

1. Placeholders
   
2. 
   
3. We start building the computation graph by creating nodes for the input images and target output classes.
   
4. 
   
5. x = tf.placeholder(tf.float32, shape=[None, 784])
   
6. y_ = tf.placeholder(tf.float32, shape=[None, 10])
   
7. Here x and y_ aren't specific values. Rather, they are each a placeholder -- a value that we'll input when we ask TensorFlow to run a computation.
   
8. 
   
9. The input images x will consist of a 2d tensor of floating point numbers. Here we assign it a shape of [None, 784], where 784 is the dimensionality of a single flattened 28 by 28 pixel MNIST image, and None indicates that the first dimension, corresponding to the batch size, can be of any size. The target output classes y_ will also consist of a 2d tensor, where each row is a one-hot 10-dimensional vector indicating which digit class (zero through nine) the corresponding MNIST image belongs to.
   
10. 
    
11. The shape argument to placeholder is optional, but it allows TensorFlow to automatically catch bugs stemming from inconsistent tensor shapes.

复制代码

1. Variables
   
2. 
   
3. We now define the weights W and biases b for our model. We could imagine treating these like additional inputs, but TensorFlow has an even better way to handle them: Variable. A Variable is a value that lives in TensorFlow's computation graph. It can be used and even modified by the computation. In machine learning applications, one generally has the model parameters be Variables.
   
4. 
   
5. W = tf.Variable(tf.zeros([784,10]))
   
6. b = tf.Variable(tf.zeros([10]))
   
7. We pass the initial value for each parameter in the call to tf.Variable. In this case, we initialize both W and b as tensors full of zeros. W is a 784x10 matrix (because we have 784 input features and 10 outputs) and b is a 10-dimensional vector (because we have 10 classes).
   
8. 
   
9. Before Variables can be used within a session, they must be initialized using that session. This step takes the initial values (in this case tensors full of zeros) that have already been specified, and assigns them to each Variable. This can be done for all Variables at once:
   
10. 
    
11. sess.run(tf.global_variables_initializer())

复制代码

1. Predicted Class and Loss Function
   
2. 
   
3. We can now implement our regression model. It only takes one line! We multiply the vectorized input images x by the weight matrix W, add the bias b.
   
4. 
   
5. y = tf.matmul(x,W) + b
   
6. We can specify a loss function just as easily. Loss indicates how bad the model's prediction was on a single example; we try to minimize that while training across all the examples. Here, our loss function is the cross-entropy between the target and the softmax activation function applied to the model's prediction. As in the beginners tutorial, we use the stable formulation:
   
7. 
   
8. cross_entropy = tf.reduce_mean(
   
9.     tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
   
10. Note that tf.nn.softmax_cross_entropy_with_logits internally applies the softmax on the model's unnormalized model prediction and sums across all classes, and tf.reduce_mean takes the average over these sums.

复制代码

1. Train the Model
   
2. 
   
3. Now that we have defined our model and training loss function, it is straightforward to train using TensorFlow. Because TensorFlow knows the entire computation graph, it can use automatic differentiation to find the gradients of the loss with respect to each of the variables. TensorFlow has a variety of built-in optimization algorithms. For this example, we will use steepest gradient descent, with a step length of 0.5, to descend the cross entropy.
   
4. 
   
5. train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
   
6. What TensorFlow actually did in that single line was to add new operations to the computation graph. These operations included ones to compute gradients, compute parameter update steps, and apply update steps to the parameters.
   
7. 
   
8. The returned operation train_step, when run, will apply the gradient descent updates to the parameters. Training the model can therefore be accomplished by repeatedly running train_step.
   
9. 
   
10. for _ in range(1000):
    
11.   batch = mnist.train.next_batch(100)
    
12.   train_step.run(feed_dict={x: batch[0], y_: batch[1]})
    
13. We load 100 training examples in each training iteration. We then run the train_step operation, using feed_dict to replace the placeholder tensors x and y_ with the training examples. Note that you can replace any tensor in your computation graph using feed_dict -- it's not restricted to just placeholders.

复制代码

1. Evaluate the Model
   
2. 
   
3. How well did our model do?
   
4. 
   
5. First we'll figure out where we predicted the correct label. tf.argmax is an extremely useful function which gives you the index of the highest entry in a tensor along some axis. For example, tf.argmax(y,1) is the label our model thinks is most likely for each input, while tf.argmax(y_,1) is the true label. We can use tf.equal to check if our prediction matches the truth.
   
6. 
   
7. correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
   
8. That gives us a list of booleans. To determine what fraction are correct, we cast to floating point numbers and then take the mean. For example, [True, False, True, True] would become [1,0,1,1] which would become 0.75.
   
9. 
   
10. accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
    
11. Finally, we can evaluate our accuracy on the test data. This should be about 92% correct.
    
12. 
    
13. print(accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

复制代码

1. Weight Initialization
   
2. 
   
3. To create this model, we're going to need to create a lot of weights and biases. One should generally initialize weights with a small amount of noise for symmetry breaking, and to prevent 0 gradients. Since we're using ReLU neurons, it is also good practice to initialize them with a slightly positive initial bias to avoid "dead neurons". Instead of doing this repeatedly while we build the model, let's create two handy functions to do it for us.
   
4. 
   
5. def weight_variable(shape):
   
6.   initial = tf.truncated_normal(shape, stddev=0.1)
   
7.   return tf.Variable(initial)
   
8. 
   
9. def bias_variable(shape):
   
10.   initial = tf.constant(0.1, shape=shape)
    
11.   return tf.Variable(initial)

复制代码

1. Convolution and Pooling
   
2. 
   
3. TensorFlow also gives us a lot of flexibility in convolution and pooling operations. How do we handle the boundaries? What is our stride size? In this example, we're always going to choose the vanilla version. Our convolutions uses a stride of one and are zero padded so that the output is the same size as the input. Our pooling is plain old max pooling over 2x2 blocks. To keep our code cleaner, let's also abstract those operations into functions.
   
4. 
   
5. def conv2d(x, W):
   
6.   return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
   
7. 
   
8. def max_pool_2x2(x):
   
9.   return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
   
10.                         strides=[1, 2, 2, 1], padding='SAME')

复制代码