# Linear regression with multiple variables

Linear-Regression - Shuai-Xie -Github

ex1data2.txt（房屋尺寸，卧室数量，房屋价格）

``````2104,3,399900
1600,3,329900
2400,3,369000
1416,2,232000
3000,4,539900
1985,4,299900
……
``````

### Part 1: Feature Normalization

``````%% ================ Part 1: Feature Normalization ================

X = data(:, 1:2); % 房屋特征2个 (m*2)
y = data(:, 3); % 价格 (m*1)

m = length(y); % 样本数量

% 输出10个样本点
fprintf('First 10 examples from the dataset: \n');

% X 2个特征，y 1个价格
% 因为fprintf按列输出矩阵，所以要将矩阵先转置，使得每一列对应一个实例
fprintf(' x = [%.0f %.0f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
% fprintf(' x = [%.0f %.0f], y = %.0f \n', data(1:10, :)'); % 也可以

fprintf('Program paused. Press enter to continue.\n');
pause;

% 特征缩放
fprintf('Normalizing Features ...\n');
% X是特征缩放后的特征矩阵，mu是平均值矩阵，sigma是方差矩阵
[X, mu, sigma] = featureNormalize(X);
X = [ones(m, 1), X]; % X 补充 theta0

% 格式化输出特征缩放后的矩阵
fprintf(' x = [%.6f %.6f %.6f], y = %.0f \n', [X(1:10,:) y(1:10,:)]');
fprintf('Program paused. Press enter to continue.\n');
pause;
``````
``````Loading data ...
First 10 examples from the dataset:
x = [2104 3], y = 399900
x = [1600 3], y = 329900
x = [2400 3], y = 369000
x = [1416 2], y = 232000
x = [3000 4], y = 539900
x = [1985 4], y = 299900
x = [1534 3], y = 314900
x = [1427 3], y = 198999
x = [1380 3], y = 212000
x = [1494 3], y = 242500
Program paused. Press enter to continue.
Normalizing Features ...
x = [1.000000 0.130010 -0.223675], y = 399900
x = [1.000000 -0.504190 -0.223675], y = 329900
x = [1.000000 0.502476 -0.223675], y = 369000
x = [1.000000 -0.735723 -1.537767], y = 232000
x = [1.000000 1.257476 1.090417], y = 539900
x = [1.000000 -0.019732 1.090417], y = 299900
x = [1.000000 -0.587240 -0.223675], y = 314900
x = [1.000000 -0.721881 -0.223675], y = 198999
x = [1.000000 -0.781023 -0.223675], y = 212000
x = [1.000000 -0.637573 -0.223675], y = 242500
Program paused. Press enter to continue.
``````

featureNormalize 函数

``````function [X_norm, mu, sigma] = featureNormalize(X)
% 标准化 X 的特征到[0,1]之间

% 初始化
X_norm = X; % X(47*2)

% 利用matlab自带函数mean，std求平均值和标准差
mu = mean(X); % mu(1*2) a row vector containing the mean value of each column.
sigma = std(X); % sigma(1*2)

% size(X, 1) 获得 X 矩阵的行数，即样本数 = 47
% size(X, 2) 获得 X 矩阵的列数，即特征数 = 2
m = size(X, 1);

% 缩放 X 的 2 个特征
for i = 1:m
X_norm(i, 1) = ( X_norm(i, 1) - mu(1) ) / sigma(1);
X_norm(i, 2) = ( X_norm(i, 2) - mu(2) ) / sigma(2);
end

end
``````

``````%% ================ Part 2: Gradient Descent ================

% Choose some alpha value 3个学习率
alpha = 0.01; % 学习率
alpha1 = 0.03;
alpha2 = 0.1;
num_iters = 400; % 迭代次数

% Init Theta and Run Gradient Descent
theta = zeros(3, 1); % 3个特征
theta1 = zeros(3, 1);
theta2 = zeros(3, 1);
[theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters); % 多变量梯度下降
[theta1, J_history1] = gradientDescentMulti(X, y, theta1, alpha1, num_iters);
[theta2, J_history2] = gradientDescentMulti(X, y, theta2, alpha2, num_iters);

% Plot the convergence graph
figure;
plot(1:numel(J_history), J_history, '-b', 'LineWidth', 2);
hold on; % 继续画图
plot(1:numel(J_history1), J_history1, '-r', 'LineWidth', 2);
plot(1:numel(J_history2), J_history2, '-k', 'LineWidth', 2);
% numel: number of elements 返回元素个数，作为x轴迭代次数
xlabel('Number of iterations');
ylabel('Cost J');
legend('alpha = 0.01', 'alpha = 0.03', 'alpha = 0.1'); % 图例

% 显示theta值
fprintf('Theta computed from gradient descent: \n');
fprintf(' %f \n', theta); % 3*1

% 估计房屋价格 1650 sq-ft, 3 br house
% x1不需要特征缩放为1，将后面的x2，x3特征缩放
% price = 0; % You should change this
price = [1, (1650 - mu(1,1)) / sigma(1,1), (3 - mu(1,2)) / sigma(1,2)] * theta;

fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...

fprintf('Program paused. Press enter to continue.\n');
pause;
``````
``````Running gradient descent ...
334302.063993
100087.116006
3673.548451
Predicted price of a 1650 sq-ft, 3 br house (using gradient descent): \$289314.620338
Program paused. Press enter to continue.
``````

``````function [theta, J_history] = gradientDescentMulti(X, y, theta, alpha, num_iters)

% Initialize some useful values
J_history = zeros(num_iters, 1);
m = length(y);

% 迭代
for iter = 1:num_iters

tmp = (X' * ( X * theta - y )); % 向量运算 求和项 (3*m)*(m*1)=(3*1)
theta = theta - alpha / m * tmp;

% Save the cost J in every iteration
J_history(iter) = computeCostMulti(X, y, theta); % 根据theta求J

end
``````

computeCostMulti 函数

``````function J = computeCostMulti(X, y, theta)

% Initialize some useful values
m = length(y); % number of training examples

E = X * theta - y; % 偏差矩阵(m*1)
J = (E' * E) / (2 * m); % (E'*E) 方差和

end
``````

### Part 3: Normal Equations

``````%% ================ Part 3: Normal Equations ================
fprintf('Solving with normal equations...\n');

X = data(:, 1:2);
y = data(:, 3);
m = length(y);

% Add intercept term to X
X = [ones(m, 1) X];

% Calculate the parameters from the normal equation
theta = normalEqn(X, y);

% Display normal equation result
fprintf('Theta computed from the normal equations: \n');
fprintf(' %f \n', theta);

% Estimate the price of a 1650 sq-ft, 3 br house
% 不需要特征缩放
price = [1, 1650, 3] * theta;

fprintf(['Predicted price of a 1650 sq-ft, 3 br house ' ...
'(using normal equations): \$%f\n'], price);
``````

``````Solving with normal equations...
Theta computed from the normal equations:
89597.909544
139.210674
-8738.019113
Predicted price of a 1650 sq-ft, 3 br house (using normal equations): \$293081.464335
``````

normalEqn 函数

``````function [theta] = normalEqn(X, y)

theta = pinv(X' * X) * X' * y; % 调用方程

end
``````