video1
linear regression is one kind of supervised learning
definition of supervised learning：当给定n个数据/sample/example，e.g.：
%2C(x_2%2Cy_2)%2C(x_3%2Cy_3)......%5C%7D)
remark：其中
是input space，
是output space or lable
The question is to find a function
，such that %3Dy')
.When you input 
. you will get a prediction of 
（的目的是逼近其真实值）
That means the function is a good predictor of y for a future input x（to predict the data，instead of fitting data）
Statistical Learning Definition
hypothesis：
the product space
x
The training set %2C(x_2%2Cy_2)%2C(x_3%2Cy_3)...(x_n%2Cy_n)%5C%7D)
,which is in 
,and the 
samples drawn i.i.d. from 
There is an unknown probability distribution on the product space,written)
Assuming that 
is a compact domain in Euclidean space and a bounded subset of 
.
is the hypothesis space, a space of functions
algorithm:
A learning algorithm is a map 
that looks at 
and selects from a function
such that%5Capprox%20y)
in a predictive way
To measure the degree of approximation of function , a loss function 
x
,（一个是正确的值y，一个是预测的y，即f(x)）and then,we define the expected or true error of (在loss function 的基础上求均值)is
This is the expected loss on a new example drawn at random from $\mu $
但由于未知，以上方程解不出来，故由大数定理，根据sample的数据，造一个去逼近:
The empirical error of fs
但为了确保小等价于小，需要满足
video2 linear regression：
problem settings：
Elements：
Assumptions：
So，The empirical error of is:
Matrix Form:
Then the empirical error of can be written as matrix form:
Conclusion:
Assuming that has full column rank, minimization of the empirical error leads to the estimator of the function : 
,where %5E%7B-1%7DX%5ETy)
