tf3_6.py 完全解析神经网络搭建学习
-
测验过的代码
#coding:utf-8
#版本信息：ubuntu18.04 python3.6.8 tensorflow1.14.0
#作者：九除以三还是三哦 如有错误，欢迎评论指正！！
#0导入模块，生成模拟数据集。
import tensorflow as tf
import numpy as np
BATCH_SIZE=8
seed=23455
rng=np.random.RandomState(seed)
X=rng.rand(32,2)
Y=[[int(x0+x1<1)]for (x0,x1)in X]
print("X:\n",X)
print("Y:\n",Y)
x=tf.compat.v1.placeholder(tf.float32,shape=(None,2))
y_=tf.compat.v1.placeholder(tf.float32,shape=(None,1))
w1=tf.Variable(tf.random.normal([2,3],stddev=1,seed=1))
w2=tf.Variable(tf.random.normal([3,1],stddev=1,seed=1))
a=tf.matmul(x,w1)
y=tf.matmul(a,w2)
#2定义损失函数及反向传播方法。
loss=tf.reduce_mean(tf.square(y-y_))
train_step=tf.compat.v1.train.GradientDescentOptimizer(0.001).minimize(loss)
#train_step = tf.compat.v1.train.MomentumOptimizer(0.001,0.9).minimize(loss)
#train_step = tf.compat.v1.train.AdamOptimizer(0.001).minimize(loss)
#3生成会话，训练STEPS轮
with tf.compat.v1.Session() as sess:
init_op=tf.compat.v1.global_variables_initializer()
sess.run(init_op)
print("w1:\n",sess.run(w1))
print("w2:\n",sess.run(w2))
print("\n")
#训练模型
STEPS=3000
for i in range(STEPS):
start=(i*BATCH_SIZE)%32
end=start+BATCH_SIZE
sess.run(train_step,feed_dict={x:X[start:end],y_:Y[start:end]})
if i%500==0:
total_loss=sess.run(loss,feed_dict={x:X,y_:Y})
print("After %d training step(s), loss_mse on all data is %g" % (i, total_loss))
print("\n")
print("w1:\n",sess.run(w1))
print("w2:\n",sess.run(w2))
-
由于版本不同，会和视频中的出现一些差异，po来评论区的一张对比图：
与视频中的差异对比.png
-
运行结果
还有warning没有处理，应该是numpy版本过高的问题
程序还可以运行，先不搞这个了
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/hahaha/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
X:
[[0.83494319 0.11482951]
[0.66899751 0.46594987]
[0.60181666 0.58838408]
[0.31836656 0.20502072]
[0.87043944 0.02679395]
[0.41539811 0.43938369]
[0.68635684 0.24833404]
[0.97315228 0.68541849]
[0.03081617 0.89479913]
[0.24665715 0.28584862]
[0.31375667 0.47718349]
[0.56689254 0.77079148]
[0.7321604 0.35828963]
[0.15724842 0.94294584]
[0.34933722 0.84634483]
[0.50304053 0.81299619]
[0.23869886 0.9895604 ]
[0.4636501 0.32531094]
[0.36510487 0.97365522]
[0.73350238 0.83833013]
[0.61810158 0.12580353]
[0.59274817 0.18779828]
[0.87150299 0.34679501]
[0.25883219 0.50002932]
[0.75690948 0.83429824]
[0.29316649 0.05646578]
[0.10409134 0.88235166]
[0.06727785 0.57784761]
[0.38492705 0.48384792]
[0.69234428 0.19687348]
[0.42783492 0.73416985]
[0.09696069 0.04883936]]
Y:
[[1], [0], [0], [1], [1], [1], [1], [0], [1], [1], [1], [0], [0], [0], [0], [0], [0], [1], [0], [0], [1], [1], [0], [1], [0], [1], [1], [1], [1], [1], [0], [1]]
w1:
[[-0.8113182 1.4845988 0.06532937]
[-2.4427042 0.0992484 0.5912243 ]]
w2:
[[-0.8113182 ]
[ 1.4845988 ]
[ 0.06532937]]
After 0 training step(s), loss_mse on all data is 5.13118
After 500 training step(s), loss_mse on all data is 0.429111
After 1000 training step(s), loss_mse on all data is 0.409789
After 1500 training step(s), loss_mse on all data is 0.399923
After 2000 training step(s), loss_mse on all data is 0.394146
After 2500 training step(s), loss_mse on all data is 0.390597
w1:
[[-0.7000663 0.9136318 0.08953571]
[-2.3402493 -0.14641273 0.58823055]]
w2:
[[-0.06024271]
[ 0.9195618 ]
[-0.06820712]]
-
拓展1
改变优化器，观察loss值的减小，AdamOptimizer好像更好一点
AdamOptimizer.png
GradientDescentOptimizer.png
MomentumOptimizer.png
-
拓展2
改变batch_size的值，观察对于loss值的影响
batch_size表示一次喂入神经网络的数据数量，前面的几个值的loss相差不大，但batch_size过大时，神经网络吃不消，loss值偏大
batch_size=4.png
batch_size=8.png
batch_size=16.png
batch_size=64.png
batch_size=200.png
