高斯混合模型(Gaussian Mixture Model)

高斯混合模型是指如下形式的概率分布

S06P03-高斯混合模型 - 图1%3D%5Csum%7Bk%3D1%7D%5EKp_k%5Cphi(X%7C%5Ctheta_k)%0A#card=math&code=P%28X%7C%5Ctheta%29%3D%5Csum%7Bk%3D1%7D%5EKp_k%5Cphi%28X%7C%5Ctheta_k%29%0A)

其中S06P03-高斯混合模型 - 图2为权重且S06P03-高斯混合模型 - 图3；S06P03-高斯混合模型 - 图4#card=math&code=%5Cphi%28X%7C%5Ctheta_k%29)为高斯分布，S06P03-高斯混合模型 - 图5#card=math&code=%5Ctheta_k%3D%28%5Cmu_k%2C%5CSigma_k%29)，称为第S06P03-高斯混合模型 - 图6个分模型
高斯混合模型其实就是多个高斯分布叠加组成的概率分布，包含了权重以及各个分模型的参数；一般混合模型可用任意模型替换高斯分布

案例

考虑一个学生身高的样本，假设身高是符合高斯分布的，于是可以直接用极大似然估计来求解。但是现在给出一个条件，男生和女生是符合不同的高斯分布的，此时如果知道每一个样本是什么性别，那么对男女同学分别进行极大似然估计即可；但是样本并没有给出性别，只有身高数据，于是这个问题中含有一个隐变量，即学生的性别。这就构成一个含有隐变量的高斯混合模型，可以考虑运用EM算法来求解

EM算法参数估计

给定样本量为N的观测S06P03-高斯混合模型 - 图7#card=math&code=X%3D%28X_1%2CX_2%2C%5Ccdots%2CX_N%29)，假设其符合高斯混合模型，那么有隐变量S06P03-高斯混合模型 - 图8#card=math&code=Z%3D%281%2C2%2C%5Ccdots%2CK%29)表示样本属于第S06P03-高斯混合模型 - 图9个分模型，显然可以看做S06P03-高斯混合模型 - 图10符合如下分布

S06P03-高斯混合模型 - 图11

高斯混合模型是一个生成模型，参数S06P03-高斯混合模型 - 图12#card=math&code=%5Ctheta%3D%28p_1%2Cp_2%2C%5Ccdots%2Cp_k%2C%5Cmu_1%2C%5Cmu_2%2C%5Ccdots%2C%5Cmu_k%2C%5CSigma_1%2C%5CSigma_2%2C%5Ccdots%2C%5CSigma_k%29)。在概率图中一般用阴影表示已观测，可以画出其概率有向图

S06P03-高斯混合模型 - 图13 S06P03-高斯混合模型 - 图14%3D%5CsumZP(X_i%2CZ%7C%5Ctheta)%3D%5Csum%7Bk%3D1%7D%5EKP(Xi%2CZ%3Dk%7C%5Ctheta)%0A#card=math&code=P%28X_i%7C%5Ctheta%29%3D%5Csum_ZP%28X_i%2CZ%7C%5Ctheta%29%3D%5Csum%7Bk%3D1%7D%5EKP%28X_i%2CZ%3Dk%7C%5Ctheta%29%0A)

根据概率图，可以写出联合概率分布

S06P03-高斯混合模型 - 图15%3DP(Z%3Dk%7C%5Ctheta)P(X_i%7CZ%3Dk%2C%5Ctheta)%3Dp_kN(X_i%7C%5Cmu_k%2C%5CSigma_k)%0A%5Cend%7Baligned%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0AP%28X_i%2CZ%3Dk%7C%5Ctheta%29%3DP%28Z%3Dk%7C%5Ctheta%29P%28X_i%7CZ%3Dk%2C%5Ctheta%29%3Dp_kN%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%0A%5Cend%7Baligned%7D%0A)

于是对于EM算法的第S06P03-高斯混合模型 - 图16次迭代：

  1. E-Step
    计算出后验概率S06P03-高斯混合模型 - 图17%7D)%3D%5Cfrac%7Bpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k%5E%7B(t)%7D)%7D%7B%5Csum%7Bk%3D1%7D%5EKpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k)%5E%7B(t)%7D%7D%0A#card=math&code=P%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%3D%5Cfrac%7Bp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%5E%7B%28t%29%7D%29%7D%7B%5Csum%7Bk%3D1%7D%5EKp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%29%5E%7B%28t%29%7D%7D%0A)

写出S06P03-高斯混合模型 - 图18%7D)#card=math&code=Q%28%5Ctheta%2C%5Ctheta%5E%7B%28t%29%7D%29)S06P03-高斯混合模型 - 图19%7D)%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%5Clog%20P(X_i%2CZ%3Dk%7C%5Ctheta)%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EK%5Cfrac%7Bp_k%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k%5E%7B(t)%7D)%7D%7B%5Csum%7Bk%3D1%7D%5EKpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k%5E%7B(t)%7D)%7D%5Clog%20p_kN(X_i%7C%5Cmu_k%2C%5CSigma_k)%0A%5Cend%7Baligned%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0AQ%28%5Ctheta%2C%5Ctheta%5E%7B%28t%29%7D%29%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Clog%20P%28X_i%2CZ%3Dk%7C%5Ctheta%29%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EK%5Cfrac%7Bp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%5E%7B%28t%29%7D%29%7D%7B%5Csum%7Bk%3D1%7D%5EKp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%5E%7B%28t%29%7D%29%7D%5Clog%20p_kN%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%0A%5Cend%7Baligned%7D%0A)

  1. M-Step
    优化问题为S06P03-高斯混合模型 - 图20%7D%26%3D%5Carg%5Cmax%5Ctheta%20Q(%5Ctheta%2C%5Ctheta%5E%7B(t)%7D)%5C%5C%0A%26%3D%5Carg%5Cmax%5Ctheta%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%5Clog%20p_kN(X_i%7C%5Cmu_k%2C%5CSigma_k)%0A%5Cend%7Baligned%7D%5C%5C%0As.t.%5Cquad%5Csum%7Bk%3D1%7D%5EKpk%3D1%0A%5Cend%7Bgathered%7D%0A#card=math&code=%5Cbegin%7Bgathered%7D%0A%5Cbegin%7Baligned%7D%0A%5Ctheta%5E%7B%28t%2B1%29%7D%26%3D%5Carg%5Cmax%5Ctheta%20Q%28%5Ctheta%2C%5Ctheta%5E%7B%28t%29%7D%29%5C%5C%0A%26%3D%5Carg%5Cmax%5Ctheta%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Clog%20p_kN%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%0A%5Cend%7Baligned%7D%5C%5C%0As.t.%5Cquad%5Csum%7Bk%3D1%7D%5EKpk%3D1%0A%5Cend%7Bgathered%7D%0A)
    S06P03-高斯混合模型 - 图21%26%3D%5Csum    %7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5Clog%20p_kN(X_i%7C%5Cmu_k%2C%5CSigma_k)%2B%5Clambda%5Cleft(1-%5Csum%7Bk%3D1%7D%5EKpk%5Cright)%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5B%5Clog%20p_k%2B%5Clog%20N(X_i%7C%5Cmu_k%2C%5CSigma_k)%5D%2B%5Clambda%5Cleft(1-%5Csum%7Bk%3D1%7D%5EKpk%5Cright)%0A%5Cend%7Baligned%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0AL%28%5Ctheta%29%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Clog%20p_kN%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%2B%5Clambda%5Cleft%281-%5Csum%7Bk%3D1%7D%5EKpk%5Cright%29%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5B%5Clog%20p_k%2B%5Clog%20N%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%5D%2B%5Clambda%5Cleft%281-%5Csum%7Bk%3D1%7D%5EKp_k%5Cright%29%0A%5Cend%7Baligned%7D%0A)

求S06P03-高斯混合模型 - 图22%7D#card=math&code=pk%5E%7B%28t%2B1%29%7D)![](https://g.yuque.com/gr/latex?%5Cbegin%7Bgathered%7D%0A%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20p_k%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%7Bp_k%7D-%5Clambda%3D0%5C%5C%0A%5CRightarrow%20p_k%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%7B%5Clambda%7D%0A%5Cend%7Bgathered%7D%0A#card=math&code=%5Cbegin%7Bgathered%7D%0A%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20p_k%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%7Bp_k%7D-%5Clambda%3D0%5C%5C%0A%5CRightarrow%20p_k%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%7B%5Clambda%7D%0A%5Cend%7Bgathered%7D%0A)

代入S06P03-高斯混合模型 - 图23得S06P03-高斯混合模型 - 图24%7D)%7D%7B%5Clambda%7D%3D1%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5CRightarrow%5Clambda%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN1%5C%5C%0A%26%3DN%0A%5Cend%7Baligned%7D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5CRightarrow%20pk%5E%7B(t%2B1)%7D%26%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%7BN%7D%5C%5C%0A%26%3D%7B1%5Cover%20N%7D%5Csum%7Bi%3D1%7D%5EN%5Cfrac%7Bpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k%5E%7B(t)%7D)%7D%7B%5Csum%7Bk%3D1%7D%5EKpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k)%5E%7B(t)%7D%7D%0A%5Cend%7Baligned%7D%0A%5Cend%7Bgathered%7D%0A#card=math&code=%5Cbegin%7Bgathered%7D%0A%5Csum%7Bk%3D1%7D%5EK%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%7B%5Clambda%7D%3D1%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5CRightarrow%5Clambda%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN%5Csum%7Bk%3D1%7D%5EKP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5C%5C%0A%26%3D%5Csum%7Bi%3D1%7D%5EN1%5C%5C%0A%26%3DN%0A%5Cend%7Baligned%7D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5CRightarrow%20pk%5E%7B%28t%2B1%29%7D%26%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%7BN%7D%5C%5C%0A%26%3D%7B1%5Cover%20N%7D%5Csum%7Bi%3D1%7D%5EN%5Cfrac%7Bpk%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%5E%7B%28t%29%7D%29%7D%7B%5Csum%7Bk%3D1%7D%5EKp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%29%5E%7B%28t%29%7D%7D%0A%5Cend%7Baligned%7D%0A%5Cend%7Bgathered%7D%0A)

求S06P03-高斯混合模型 - 图25%7D%2C%5CSigmak%5E%7B(t%2B1)%7D#card=math&code=%5Cmu_k%5E%7B%28t%2B1%29%7D%2C%5CSigma_k%5E%7B%28t%2B1%29%7D)，只用把S06P03-高斯混合模型 - 图26#card=math&code=L%28%5Ctheta%29)中跟S06P03-高斯混合模型 - 图27相关的项提出来计算就可以了S06P03-高斯混合模型 - 图28%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5Clog%20N(X_i%7C%5Cmu_k%2C%5CSigma_k)%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5Clog%5Cleft%5B%7B1%5Cover%7B(2%5Cpi%7D)%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D%5Cexp%5Cleft(-%7B1%5Cover2%7D(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)%5Cright)%5Cright%5D%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%5Cleft%5B%5Clog%7B1%5Cover%7B(2%5Cpi%7D)%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D-%7B1%5Cover2%7D(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)%5Cright%5D%0A%5Cend%7Baligned%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0AL%28%5Cmu%29%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Clog%20N%28X_i%7C%5Cmu_k%2C%5CSigma_k%29%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Clog%5Cleft%5B%7B1%5Cover%7B%282%5Cpi%7D%29%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D%5Cexp%5Cleft%28-%7B1%5Cover2%7D%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%5Cright%29%5Cright%5D%5C%5C%0A%26%3D%5Csum%7Bk%3D1%7D%5EK%5Csum_%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5Cleft%5B%5Clog%7B1%5Cover%7B%282%5Cpi%7D%29%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D-%7B1%5Cover2%7D%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%5Cright%5D%0A%5Cend%7Baligned%7D%0A)

对S06P03-高斯混合模型 - 图29求导有S06P03-高斯混合模型 - 图30%7D)%5CSigmak%5E%7B-1%7D(X_i-%5Cmu_k)%3D0%0A%5Cend%7Baligned%7D%5Ctag%7B1%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0A%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%5Cmu_k%7D%3D%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%3D0%0A%5Cend%7Baligned%7D%5Ctag%7B1%7D%0A)

对S06P03-高斯混合模型 - 图31求导有S06P03-高斯混合模型 - 图32%7D)%5Cleft%5B%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigmak%7D%5Clog%7B1%5Cover%7B(2%5Cpi%7D)%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D-%7B1%5Cover2%7D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)%5Cright%5D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%5Clog%7B1%5Cover%7B(2%5Cpi%7D)%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D%26%3D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%5Cleft%5B-%5Clog(2%5Cpi)%5E%7Bp%2F2%7D-%7B1%5Cover2%7D%5Clog%7C%5CSigma_k%7C%5Cright%5D%5C%5C%0A%26%3D-%7B1%5Cover2%7D%7B1%5Cover%7C%5CSigma_k%7C%7D%7C%5CSigma_k%7C%5CSigma_k%5E%7B-1%7D%5C%5C%0A%26%3D-%7B%5CSigma_k%5E%7B-1%7D%5Cover2%7D%0A%5Cend%7Baligned%7D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)%26%3D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7Dtr%5Cleft%5B(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)%5Cright%5D%5C%5C%0A%26%3D%5Cfrac%7B%5Cpartial%20tr%5Cleft%5B%5CSigma_k%5E%7B-1%7D(X_i-%5Cmu_k)(X_i-%5Cmu_k)%5ET%5Cright%5D%7D%7B%5Cpartial%5CSigma_k%5E%7B-1%7D%7D%5Cfrac%7B%5Cpartial%5CSigma_k%5E%7B-1%7D%7D%7B%5Cpartial%5CSigma_k%7D%5C%5C%0A%26%3D-(X_i-%5Cmu_k)(X_i-%5Cmu_k)%5ET%5CSigma_k%5E%7B-2%7D%0A%5Cend%7Baligned%7D%0A%5Cend%7Bgathered%7D%0A#card=math&code=%5Cbegin%7Bgathered%7D%0A%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%5CSigma_k%7D%3D%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%5Cleft%5B%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%5Clog%7B1%5Cover%7B%282%5Cpi%7D%29%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D-%7B1%5Cover2%7D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%5Cright%5D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%5Clog%7B1%5Cover%7B%282%5Cpi%7D%29%5E%7Bp%2F2%7D%7C%5CSigma_k%7C%5E%7B1%2F2%7D%7D%26%3D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%5Cleft%5B-%5Clog%282%5Cpi%29%5E%7Bp%2F2%7D-%7B1%5Cover2%7D%5Clog%7C%5CSigma_k%7C%5Cright%5D%5C%5C%0A%26%3D-%7B1%5Cover2%7D%7B1%5Cover%7C%5CSigma_k%7C%7D%7C%5CSigma_k%7C%5CSigma_k%5E%7B-1%7D%5C%5C%0A%26%3D-%7B%5CSigma_k%5E%7B-1%7D%5Cover2%7D%0A%5Cend%7Baligned%7D%5C%5C%0A%5C%5C%0A%5Cbegin%7Baligned%7D%0A%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7D%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%26%3D%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%5CSigma_k%7Dtr%5Cleft%5B%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%5Cright%5D%5C%5C%0A%26%3D%5Cfrac%7B%5Cpartial%20tr%5Cleft%5B%5CSigma_k%5E%7B-1%7D%28X_i-%5Cmu_k%29%28X_i-%5Cmu_k%29%5ET%5Cright%5D%7D%7B%5Cpartial%5CSigma_k%5E%7B-1%7D%7D%5Cfrac%7B%5Cpartial%5CSigma_k%5E%7B-1%7D%7D%7B%5Cpartial%5CSigma_k%7D%5C%5C%0A%26%3D-%28X_i-%5Cmu_k%29%28X_i-%5Cmu_k%29%5ET%5CSigma_k%5E%7B-2%7D%0A%5Cend%7Baligned%7D%0A%5Cend%7Bgathered%7D%0A)
![](https://g.yuque.com/gr/latex?%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%5CSigma_k%7D%3D%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7B1%5Cover2%7D%5Cleft%5B(X_i-%5Cmu_k)(X_i-%5Cmu_k)%5ET-%5CSigma_k%5Cright%5D%5CSigma_k%5E%7B-2%7D%3D0%5Ctag%7B2%7D%0A#card=math&code=%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%5CSigma_k%7D%3D%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7B1%5Cover2%7D%5Cleft%5B%28X_i-%5Cmu_k%29%28X_i-%5Cmu_k%29%5ET-%5CSigma_k%5Cright%5D%5CSigma_k%5E%7B-2%7D%3D0%5Ctag%7B2%7D%0A)
S06P03-高斯混合模型 - 图33%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)X_i%7D%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%0A#card=math&code=%5Cmu_k%5E%7B%28t%2B1%29%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29X_i%7D%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%0A)
S06P03-高斯混合模型 - 图34%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)(X_i-%5Cmu_k)(X_i-%5Cmu_k)%5ET%7D%7B%5Csum%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%0A#card=math&code=%5CSigma_k%5E%7B%28t%2B1%29%7D%3D%5Cfrac%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%28X_i-%5Cmu_k%29%28X_i-%5Cmu_k%29%5ET%7D%7B%5Csum%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%0A)

GMM算法步骤

(1)初始化参数S06P03-高斯混合模型 - 图35%7D%3D(p_1%2Cp_2%2C%5Ccdots%2Cp_k%2C%5Cmu_1%2C%5Cmu_2%2C%5Ccdots%2C%5Cmu_k%2C%5CSigma_1%2C%5CSigma_2%2C%5Ccdots%2C%5CSigma_k)#card=math&code=%5Ctheta%5E%7B%280%29%7D%3D%28p_1%2Cp_2%2C%5Ccdots%2Cp_k%2C%5Cmu_1%2C%5Cmu_2%2C%5Ccdots%2C%5Cmu_k%2C%5CSigma_1%2C%5CSigma_2%2C%5Ccdots%2C%5CSigma_k%29)
(2)E-Step：依据当前模型，计算后验概率

S06P03-高斯混合模型 - 图36%7D)%3D%5Cfrac%7Bpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k%5E%7B(t)%7D)%7D%7B%5Csum%7Bk%3D1%7D%5EKpk%5E%7B(t)%7DN(X_i%7C%5Cmu_k%5E%7B(t)%7D%2C%5CSigma_k)%5E%7B(t)%7D%7D%0A#card=math&code=P%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%3D%5Cfrac%7Bp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%5E%7B%28t%29%7D%29%7D%7B%5Csum%7Bk%3D1%7D%5EKp_k%5E%7B%28t%29%7DN%28X_i%7C%5Cmu_k%5E%7B%28t%29%7D%2C%5CSigma_k%29%5E%7B%28t%29%7D%7D%0A)

(3)M-Step：计算下一轮迭代的参数

S06P03-高斯混合模型 - 图37%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP(Z%3Dk%7CX_i%2C%5Ctheta%5E%7B(t)%7D)%7D%7BN%7D%5C%5C%0A%5Cmu_k%5E%7B(t%2B1)%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)X_i%7D%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%5C%5C%0A%5CSigma_k%5E%7B(t%2B1)%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)(X_i-%5Cmu_k)(X_i-%5Cmu_k)%5ET%7D%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP(Z%3Dk%7CXi%2C%5Ctheta%5E%7B(t)%7D)%7D%0A%5Cend%7Bgathered%7D%0A#card=math&code=%5Cbegin%7Bgathered%7D%0Ap_k%5E%7B%28t%2B1%29%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%7BN%7D%5C%5C%0A%5Cmu_k%5E%7B%28t%2B1%29%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29X_i%7D%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%5C%5C%0A%5CSigma_k%5E%7B%28t%2B1%29%7D%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP%28Z%3Dk%7CXi%2C%5Ctheta%5E%7B%28t%29%7D%29%28X_i-%5Cmu_k%29%28X_i-%5Cmu_k%29%5ET%7D%7B%5Csum%5Climits%7Bi%3D1%7D%5ENP%28Z%3Dk%7CX_i%2C%5Ctheta%5E%7B%28t%29%7D%29%7D%0A%5Cend%7Bgathered%7D%0A)

(4)重复步骤(2)、(3)，直至算法收敛