在用CNN训练MNIST数据时，发现预测结果的输出是196*10，而我设置的batch是64，实际的输出应该是64*10才对。
InvalidArgumentError (see above for traceback): logits and labels must be same size: logits_size=[196,10] labels_size=[64,10]
花了两天时间，总算是把这个问题搞清楚了。原因在于padding 参数中 VALID和SAME的选择。
先上code
import tensorflow as tf
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '1'
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
# input data
from tensorflow.examples.tutorials.mnist import input_data
# mnist = input_data.read_data_sets('/tmp/data/', one_hot=True)
mnist = input_data.read_data_sets('./MNIST_data', one_hot=True) # runing on server
learning_rate = 0.001
training_iters = 200000
batch_size = 64
display_step = 10
n_input = 784
n_classes = 10
dropout = 0.75
x = tf.placeholder(tf.float32, [None, n_input])
y = tf.placeholder(tf.float32, [None, n_classes])
keep_prob = tf.placeholder(tf.float32) # dropout
def conv2d(name, x, W, b, s=1):
return tf.nn.relu(tf.nn.conv2d(x, W, strides=[1, s, s, 1], padding='SAME'))
def maxpool2d(name, x, k=2, s=2):
return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, s, s, 1],
padding='VALID', name=name)
def norm(name, l_input, lsize=4):
return tf.nn.lrn(l_input, lsize, bias=1.0, alpha=0.001 / 9.0,
beta=0.75, name=name)
def alex_net(x, weights, biases, dropout):
# Reshape input picture
x = tf.reshape(x, shape=[-1, 28, 28, 1])
conv1 = conv2d('conv1', x, weights['wc1'], biases['bc1'], s=1)
print ('cov1.shape: ', conv1.get_shape().as_list())
pool1 = maxpool2d('pool1', conv1, k=2, s=2)
print ('pool1.shape: ', pool1.get_shape().as_list())
norm1 = norm('norm1', pool1)
conv2 = conv2d('conv2', norm1, weights['wc2'], biases['bc2'], s=1)
pool2 = maxpool2d('pool2', conv2, k=2, s=2)
print ('pool2.shape: ', pool2.get_shape().as_list())
norm2 = norm('pool2', pool2)
fc1 = tf.reshape(norm2, [-1, weights['wd1'].get_shape().as_list()[0]])
fc1 = tf.add(tf.matmul(fc1, weights['wd1']), biases['bd1'])
fc1 = tf.nn.relu(fc1)
fc2 = tf.add(tf.matmul(fc1, weights['wd2']), biases['bd2'])
fc2 = tf.nn.relu(fc2)
out = tf.matmul(fc2, weights['out']) + biases['out']
return out
weights = {
'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64])),
'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128])),
'wd1': tf.Variable(tf.random_normal([4*4*128, 1024])),
'wd2': tf.Variable(tf.random_normal([1024, 1024])),
'out': tf.Variable(tf.random_normal([1024, n_classes]))
}
biases = {
'bc1': tf.Variable(tf.random_normal([64])),
'bc2': tf.Variable(tf.random_normal([128])),
'bd1': tf.Variable(tf.random_normal([1024])),
'bd2': tf.Variable(tf.random_normal([1024])),
'out': tf.Variable(tf.random_normal([n_classes]))
}
pred = alex_net(x, weights, biases, keep_prob)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
init = tf.global_variables_initializer()
with tf.Session(config=config) as sess:
sess.run(init)
step = 1
while step * batch_size < training_iters:
batch_x, batch_y = mnist.train.next_batch(batch_size)
sess.run(optimizer, feed_dict={x: batch_x, y: batch_y,
keep_prob: dropout})
if step % display_step == 0:
loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x,
y: batch_y,
keep_prob: 1.})
print("Iter " + str(step*batch_size) + ", Minibatch Loss= " + \
"{:.6f}".format(loss) + ", Training Accuracy= " + \
"{:.5f}".format(acc))
step += 1
print("Optimization Finished!")
print("Testing Accuracy:", \
sess.run(accuracy, feed_dict={x: mnist.test.images[:256],
y: mnist.test.labels[:256],
keep_prob: 1.}))
batch_szie 是 64，图片大小是28*28*1.
所以输入的时候是 64*28*28*1。
我的网络结构：
conv1 ( stride=1, padding=‘SAME’, 64)
pool1 ( kernel size = 2, stride = 2, padding=‘VALID’)
conv2 ( stride=1, padding=‘SAME’, 128)
pool2 ( kernel size = 2, stride = 2, padding=‘VALID’)
fc1 ( 4*4*128, 1024), 4*4*128 是一个image的size，要和pool2输出的size一样
fc1 ( 2014, 1024)
fc1 ( 1024, 10)
我之所以fc1以为输出的是4*4，是因为我对SAME和VALID对应的计算方法没有理解。这里先介绍不同padding下怎么计算。
在conv层，如果设置 padding=‘SAME’，那么output_size = input_szie / stride. 通常设置stride为1，即让size不变，至于抽取信息，我们留给pooling层来处理。
对应的如果pooling层也设置padding=‘SAME’的话，output size 只和 stride有关： 28 / 2 = 14。 但如果padding=‘VALID’，output size 和kernel size， stride有关。那么( 28 - 2)/2 + 1 = 14
所以，按照我上面conv用SAME，pooling用VALID的方法，每一层的size变化应该是
conv1 => 28 / 1 = 28
pool1 => （28 - 2) / 2 + 1 = 14
conv2 => 14 / 1 = 14
pool2 => （14 - 2) / 2 + 1 = 7
所以得到的size是 7*7*128。但是我fc1 中设置的参数是4*4*128。
fc1 = tf.reshape(norm2, [-1, weights['wd1'].get_shape().as_list()[0]])
所以上面的这句代码实际上把64* 7*7*128 => 196*7*7*128.
所以最后输出的batch_szie是196，而不是64.
所以说，要把
'wd1': tf.Variable(tf.random_normal([4*4*128, 1024])),
改为
'wd1': tf.Variable(tf.random_normal([7*7*128, 1024])),
这两篇文章做参考。
logits and labels must be same size, batch size is different
Tensorflow - padding = VALID/SAME
