线性回归简述
在这里，我们仅仅讨论单变量的线型回归模型。不对回归算法进行过多的展开。重点放在Tensorflow的学习上。
下图展示的分别是：单变量线性回归模型的公式；学习的参数；损失函数（采用的均方误差）；目标函数的优化求解（SGD）。
image
Tensorflow 线性回归
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
rng = np.random
参数设置
# 参数设置
learning_rate = 0.01
training_epochs = 1000
display_step = 50
生成训练数据
# 生成训练数据
train_X = np.asarray([3.3,4.4,5.5,6.71,6.93,4.168,9.779,6.182,7.59,2.167,
7.042,10.791,5.313,7.997,5.654,9.27,3.1])
train_Y = np.asarray([1.7,2.76,2.09,3.19,1.694,1.573,3.366,2.596,2.53,1.221,
2.827,3.465,1.65,2.904,2.42,2.94,1.3])
n_samples = train_X.shape[0]
构造线型回归模型
# tf 图的输入
X = tf.placeholder("float")
Y = tf.placeholder("float")
# 设置模型的权重与偏置
W = tf.Variable(rng.randn(), name="weight")
b = tf.Variable(rng.randn(), name="bias")
# 构造一个线性模型
pred = tf.add(tf.multiply(X, W), b)
定义损失函数
# 损失函数设置为均方误差
cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples)
定义优化方法
# 梯度下降
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
# 初始化变量(i.e. assign their default value)
init = tf.global_variables_initializer()
训练
# 开始训练
with tf.Session() as sess:
sess.run(init)
# Fit all training data
for epoch in range(training_epochs):
for (x, y) in zip(train_X, train_Y):
sess.run(optimizer, feed_dict={X: x, Y: y})
#现实每50轮迭代的结果
if (epoch+1) % display_step == 0:
c = sess.run(cost, feed_dict={X: train_X, Y:train_Y})
print ("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c), \
"W=", sess.run(W), "b=", sess.run(b))
print ("Optimization Finished!")
training_cost = sess.run(cost, feed_dict={X: train_X, Y: train_Y})
print ("Training cost=", training_cost, "W=", sess.run(W), "b=", sess.run(b), '\n')
#绘图
plt.plot(train_X, train_Y, 'ro', label='Original data')
plt.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line')
plt.legend()
plt.show()
Epoch: 0050 cost= 0.160369754 W= 0.41108337 b= -0.36027926
Epoch: 0100 cost= 0.150733337 W= 0.40147883 b= -0.2911848
Epoch: 0150 cost= 0.142209828 W= 0.39244553 b= -0.22619964
Epoch: 0200 cost= 0.134670869 W= 0.38394934 b= -0.16507955
Epoch: 0250 cost= 0.128002644 W= 0.37595856 b= -0.10759445
Epoch: 0300 cost= 0.122104712 W= 0.36844307 b= -0.05352829
Epoch: 0350 cost= 0.116888084 W= 0.3613746 b= -0.0026777028
Epoch: 0400 cost= 0.112274118 W= 0.35472643 b= 0.04514854
Epoch: 0450 cost= 0.108193211 W= 0.34847358 b= 0.09013041
Epoch: 0500 cost= 0.104583815 W= 0.34259278 b= 0.13243689
Epoch: 0550 cost= 0.101391472 W= 0.33706158 b= 0.17222734
Epoch: 0600 cost= 0.098568030 W= 0.33185956 b= 0.20965117
Epoch: 0650 cost= 0.096070863 W= 0.32696673 b= 0.2448493
Epoch: 0700 cost= 0.093862340 W= 0.32236505 b= 0.2779539
Epoch: 0750 cost= 0.091909051 W= 0.3180369 b= 0.3090903
Epoch: 0800 cost= 0.090181611 W= 0.31396636 b= 0.33837357
Epoch: 0850 cost= 0.088653855 W= 0.31013775 b= 0.365916
Epoch: 0900 cost= 0.087302707 W= 0.3065368 b= 0.39182106
Epoch: 0950 cost= 0.086107843 W= 0.30315018 b= 0.41618422
Epoch: 1000 cost= 0.085051164 W= 0.29996493 b= 0.43909845
Optimization Finished!
Training cost= 0.085051164 W= 0.29996493 b= 0.43909845
拟合曲线
Tensorflow 线性回归（Eager API）
from __future__ import absolute_import, division, print_function
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
import tensorflow.contrib.eager as tfe
设置Eager API
# Set Eager API
tfe.enable_eager_execution()
生成训练数据
# Training Data
train_X = [3.3, 4.4, 5.5, 6.71, 6.93, 4.168, 9.779, 6.182, 7.59, 2.167,
7.042, 10.791, 5.313, 7.997, 5.654, 9.27, 3.1]
train_Y = [1.7, 2.76, 2.09, 3.19, 1.694, 1.573, 3.366, 2.596, 2.53, 1.221,
2.827, 3.465, 1.65, 2.904, 2.42, 2.94, 1.3]
n_samples = len(train_X)
设置参数
# Parameters
learning_rate = 0.01
display_step = 100
num_steps = 1000
初始化参数
# Weight and Bias
W = tfe.Variable(np.random.randn())
b = tfe.Variable(np.random.randn())
构建线性回归模型
# Linear regression (Wx + b)
def linear_regression(inputs):
return inputs * W + b
定义损失函数（均方误差）
# Mean square error
def mean_square_fn(model_fn, inputs, labels):
return tf.reduce_sum(tf.pow(model_fn(inputs) - labels, 2)) / (2 * n_samples)
调用优化器（SGD）
# SGD Optimizer
optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
# Compute gradients
grad = tfe.implicit_gradients(mean_square_fn)
训练
# Initial cost, before optimizing
print("Initial cost= {:.9f}".format(
mean_square_fn(linear_regression, train_X, train_Y)),
"W=", W.numpy(), "b=", b.numpy())
# Training
for step in range(num_steps):
optimizer.apply_gradients(grad(linear_regression, train_X, train_Y))
if (step + 1) % display_step == 0 or step == 0:
print("Epoch:", '%04d' % (step + 1), "cost=",
"{:.9f}".format(mean_square_fn(linear_regression, train_X, train_Y)),
"W=", W.numpy(), "b=", b.numpy())
# Graphic display
plt.plot(train_X, train_Y, 'ro', label='Original data')
plt.plot(train_X, np.array(W * train_X + b), label='Fitted line')
plt.legend()
plt.show()
Initial cost= 34.570991516 W= -0.89701766 b= 0.09529222
Epoch: 0001 cost= 10.462613106 W= -0.3445475 b= 0.17387688
Epoch: 0100 cost= 0.090614140 W= 0.31795835 b= 0.32859808
Epoch: 0200 cost= 0.087662779 W= 0.31037292 b= 0.38237545
Epoch: 0300 cost= 0.085347883 W= 0.30365503 b= 0.4300023
Epoch: 0400 cost= 0.083532237 W= 0.29770544 b= 0.472182
Epoch: 0500 cost= 0.082108125 W= 0.29243633 b= 0.5095375
Epoch: 0600 cost= 0.080991186 W= 0.28776988 b= 0.54262066
Epoch: 0700 cost= 0.080115080 W= 0.28363708 b= 0.5719204
Epoch: 0800 cost= 0.079427943 W= 0.27997696 b= 0.5978689
Epoch: 0900 cost= 0.078888975 W= 0.27673545 b= 0.6208498
Epoch: 1000 cost= 0.078466244 W= 0.27386472 b= 0.64120203
拟合曲线
Logistic模型
Logistic模型图解
交叉熵
softmax
