一、线性回归
ex1 - 一元线性回归
1.训练数据集：x,y
6.1101,17.592
5.5277,9.1302
8.5186,13.662
7.0032,11.854
5.8598,6.8233
8.3829,11.886
7.4764,4.3483
8.5781,12
6.4862,6.5987
5.0546,3.8166
...
2.训练数据可视化：
3.假设函数
H(theta) = theta0*x0 + theta1*x1 %x0=1
4.代价函数
predictionVector = X*theta;
E = predictionVector-y;
%每个误差算平方，然后再加起来
errorQuadraticSum = E' * E;
%代价要在误差的平方和上除以2m
J = errorQuadraticSum/(2*m);
5.假设模型中，参数theta是未知的，利用梯度下降-最小化代价函数，来求得最优参数theta：
function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
m = length(y); %训练数据的条数
J_history = zeros(num_iters, 1); %记下每次迭代后，的代价值
for iter = 1:num_iters
Predictions = X*theta;
%批量梯度下降，总共有(n+1)个theta，J关于各个theta的偏导数组成一个(n+1)维向量：(X'*(Predictions-y))
theta = theta - alpha * (X'*(Predictions-y))/m; %alpha是学习率
J_history(iter) = computeCost(X, y, theta); %记录每次迭代后的代价值
end
end
6.最优参数theta的假设模型 曲线：
7.画出： 代价函数的曲面，代价函数的等高线
theta0_vals = linspace(-10, 10, 100);
theta1_vals = linspace(-1, 4, 100);
% initialize J_vals to a matrix of 0's
J_vals = zeros(length(theta0_vals), length(theta1_vals));
% Fill out J_vals
for i = 1:length(theta0_vals)
for j = 1:length(theta1_vals)
t = [theta0_vals(i); theta1_vals(j)];
J_vals(i,j) = computeCost(X, y, t);
end
end
% Because of the way meshgrids work in the surf command, we need to
% transpose J_vals before calling surf, or else the axes will be flipped
J_vals = J_vals';
% Surface plot
figure;
[X1,X2] = meshgrid(theta0_vals, theta1_vals);
surf(X1,X2,J_vals);
xlabel('\theta_0'); ylabel('\theta_1');
% Contour plot
figure;
% Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100
% 10^(-2) ~ 10^3 共20个点
contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 3, 20))
xlabel('\theta_0'); ylabel('\theta_1');
hold on;
plot(theta(1), theta(2), 'rx', 'MarkerSize', 10, 'LineWidth', 2);
ex1_multi - 多元线性回归
1.训练数据集：x1,x2, y
2104,3,399900
1600,3,329900
2400,3,369000
1416,2,232000
3000,4,539900
1985,4,299900
1534,3,314900
1427,3,198999
...
2.特征之间 数值大小 差别较大，需要标准化特征值：
function [X_norm, mu, sigma] = featureNormalize(X)
X_norm = X; %标准化后的X
mu = zeros(1, size(X, 2)); %均值，1行n列
sigma = zeros(1, size(X, 2)); %标准差，1行n列
for i = 1 : size(X, 2) %遍历X的列
mu(1,i) = mean(X(:,i)); %求出每一列的均值
sigma(1,i) = std(X(:,i)); %求出每一列的标准差
end
for j = 1 : size(X, 2)
X_norm(:,j) = (X(:,j)-mu(1,j)) / sigma(1,j); % x_norm = (x-mu)/sigma
end
end
3.代价函数：
function J = computeCostMulti(X, y, theta)
m = length(y); %训练数据条数
J = 0;
predictionVector = X*theta;
E = predictionVector-y;
%每个误差算平方，然后再加起来
errorQuadraticSum = E' * E;
%代价要在误差的平方和上除以2m
J = errorQuadraticSum/(2*m);
end
4.梯度下降
function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)
m = length(y);
J_history = zeros(num_iters, 1);
for iter = 1:num_iters
Predictions = X*theta;
E = Predictions - y;
%批量梯度下降，总共有(n+1)个theta，J关于各个theta的偏导数组成一个(n+1)维向量：(X'*E)
theta = theta - alpha * (X'*E)/m; %alpha是学习率
% Save the cost J in every iteration
J_history(iter) = computeCostMulti(X, y, theta);
end
end
5.画出 代价值 随梯度下降迭代次数 的变化曲线：
alpha = 0.1; %学习率
num_iters = 100; %总迭代次数
theta = zeros(3, 1); %初识参数值
[theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters); %梯度下降
%画出 代价值曲线
figure;
%numel(J_history) 是 J_history的元素个数
plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2);
xlabel('Number of iterations');
ylabel('Cost J');
正规方程法求参数theta
X * theta = y 这个方程不一定有解。
只有当y在X的列空间上才有解。
但是y不一定在X的列空间，需要将y投影到X的列空间上。
求出X * theta = y_tou的解，也就是X * theta = y的最优解。
function [theta] = normalEqn(X, y)
theta = zeros(size(X, 2), 1);
%正规方程法无需标准化特征值。
theta = pinv(X'*X)*X'*y;
end
