EM算法看一次忘一次，干脆整理一下好了。

一. 最大似然算法

基本思想：从已经观察到有限个samples推测出该群体最合理的distribution.

$max \ Lik\left(D,z | \theta \right)$ in order to find the best $\theta$ . theta is distribution parameters( means or var)

二. EM算法

基本思想：从已经观察到的有限个samples和一些未知的samples推测出该群体最合理的distribution.

$max \ Lik\left(D,z | \theta \right)$ in order to find the best $\theta$ . $z$ are hidden variables.

E step : fix $\theta_k$ . max Likelihood function with respect to hidden variable $z_k$ .

M step : fix $z_k$ obtained from previous step, max Likelihood function with respect to the parameter $\theta_{k+1}$ .

坐标上升法

http://www.cmlab.csie.ntu.edu.tw/~cyy/learning/tutorials/EM.pdf