Math

  1. MLE
    23.Cheatsheet - 图1%5Cmathop%7B%3D%7D%5Climits%20%7Biid%7D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7BN%7D%5Clog%20p(x%7Bi%7D%7C%5Ctheta)%0A#card=math&code=%5Ctheta%7BMLE%7D%3D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7D%5Clog%20p%28X%7C%5Ctheta%29%5Cmathop%7B%3D%7D%5Climits%20%7Biid%7D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7BN%7D%5Clog%20p%28x%7Bi%7D%7C%5Ctheta%29%0A&height=47&width=334#crop=0&crop=0&crop=1&crop=1&id=ASEQu&originHeight=66&originWidth=469&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. MAP
    23.Cheatsheet - 图2%3D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7Dp(X%7C%5Ctheta)%5Ccdot%20p(%5Ctheta)%0A#card=math&code=%5Ctheta%7BMAP%7D%3D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7Dp%28%5Ctheta%7CX%29%3D%5Cmathop%7Bargmax%7D%5Climits%20%7B%5Ctheta%7Dp%28X%7C%5Ctheta%29%5Ccdot%20p%28%5Ctheta%29%0A&height=28&width=306#crop=0&crop=0&crop=1&crop=1&id=avY06&originHeight=41&originWidth=429&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. Gaussian Distribution
    23.Cheatsheet - 图3%3D%5Cfrac%7B1%7D%7B(2%5Cpi)%5E%7Bp%2F2%7D%7C%5CSigma%7C%5E%7B1%2F2%7D%7De%5E%7B-%5Cfrac%7B1%7D%7B2%7D(x-%5Cmu)%5E%7BT%7D%5CSigma%5E%7B-1%7D(x-%5Cmu)%7D%5C%5C%0A%26%5CDelta%3D(x-%5Cmu)%5E%7BT%7D%5CSigma%5E%7B-1%7D(x-%5Cmu)%3D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7Bp%7D(x-%5Cmu)%5E%7BT%7Du%7Bi%7D%5Cfrac%7B1%7D%7B%5Clambda%7Bi%7D%7Du%7Bi%7D%5E%7BT%7D(x-%5Cmu)%3D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7Bp%7D%5Cfrac%7By%7Bi%7D%5E%7B2%7D%7D%7B%5Clambda%7Bi%7D%7D%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%26p%28x%7C%5Cmu%2C%5CSigma%29%3D%5Cfrac%7B1%7D%7B%282%5Cpi%29%5E%7Bp%2F2%7D%7C%5CSigma%7C%5E%7B1%2F2%7D%7De%5E%7B-%5Cfrac%7B1%7D%7B2%7D%28x-%5Cmu%29%5E%7BT%7D%5CSigma%5E%7B-1%7D%28x-%5Cmu%29%7D%5C%5C%0A%26%5CDelta%3D%28x-%5Cmu%29%5E%7BT%7D%5CSigma%5E%7B-1%7D%28x-%5Cmu%29%3D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7Bp%7D%28x-%5Cmu%29%5E%7BT%7Du%7Bi%7D%5Cfrac%7B1%7D%7B%5Clambda%7Bi%7D%7Du%7Bi%7D%5E%7BT%7D%28x-%5Cmu%29%3D%5Csum%5Climits%20%7Bi%3D1%7D%5E%7Bp%7D%5Cfrac%7By%7Bi%7D%5E%7B2%7D%7D%7B%5Clambda%7Bi%7D%7D%0A%5Cend%7Balign%7D%0A&height=89&width=426#crop=0&crop=0&crop=1&crop=1&id=K01kC&originHeight=125&originWidth=597&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  4. 已知 23.Cheatsheet - 图4%2C%20y%5Csim%20Ax%2Bb#card=math&code=x%5Csim%5Cmathcal%7BN%7D%28%5Cmu%2C%5CSigma%29%2C%20y%5Csim%20Ax%2Bb&height=19&width=160#crop=0&crop=0&crop=1&crop=1&id=t4QBJ&originHeight=27&originWidth=223&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，有：
    23.Cheatsheet - 图5%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7Dy%5Csim%5Cmathcal%7BN%7D%28A%5Cmu%2Bb%2C%20A%5CSigma%20A%5ET%29%0A%5Cend%7Balign%7D%0A&height=20&width=152#crop=0&crop=0&crop=1&crop=1&id=ImE0H&originHeight=29&originWidth=213&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  5. 记 23.Cheatsheet - 图6%5ET%3D(x%7Ba%2Cm%5Ctimes%201%7D%2C%20x%7Bb%2Cn%5Ctimes1%7D)%5ET%2C%5Cmu%3D(%5Cmu%7Ba%2Cm%5Ctimes1%7D%2C%20%5Cmu%7Bb%2Cn%5Ctimes1%7D)%2C%5CSigma%3D%5Cbegin%7Bpmatrix%7D%5CSigma%7Baa%7D%26%5CSigma%7Bab%7D%5C%5C%5CSigma%7Bba%7D%26%5CSigma%7Bbb%7D%5Cend%7Bpmatrix%7D#card=math&code=x%3D%28x1%2C%20x_2%2C%5Ccdots%2Cx_p%29%5ET%3D%28x%7Ba%2Cm%5Ctimes%201%7D%2C%20x%7Bb%2Cn%5Ctimes1%7D%29%5ET%2C%5Cmu%3D%28%5Cmu%7Ba%2Cm%5Ctimes1%7D%2C%20%5Cmu%7Bb%2Cn%5Ctimes1%7D%29%2C%5CSigma%3D%5Cbegin%7Bpmatrix%7D%5CSigma%7Baa%7D%26%5CSigma%7Bab%7D%5C%5C%5CSigma%7Bba%7D%26%5CSigma%7Bbb%7D%5Cend%7Bpmatrix%7D&height=39&width=519#crop=0&crop=0&crop=1&crop=1&id=TadFg&originHeight=56&originWidth=727&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，已知 23.Cheatsheet - 图7#card=math&code=x%5Csim%5Cmathcal%7BN%7D%28%5Cmu%2C%5CSigma%29&height=19&width=81#crop=0&crop=0&crop=1&crop=1&id=gFxBy&originHeight=27&originWidth=114&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，则：
    ![](https://g.yuque.com/gr/latex?%5Cbegin%7Balign%7D%26x_a%5Csim%5Cmathcal%7BN%7D(%5Cmu_a%2C%5CSigma    %7Baa%7D)%5C%5C%0A%26xb%7Cx_a%5Csim%5Cmathcal%7BN%7D(%5Cmu%7Bb%7Ca%7D%2C%5CSigma%7Bb%7Ca%7D)%5C%5C%0A%26%5Cmu%7Bb%7Ca%7D%3D%5CSigma%7Bba%7D%5CSigma%7Baa%7D%5E%7B-1%7D(xa-%5Cmu_a)%2B%5Cmu_b%5C%5C%0A%26%5CSigma%7Bb%7Ca%7D%3D%5CSigma%7Bbb%7D-%5CSigma%7Bba%7D%5CSigma%7Baa%7D%5E%7B-1%7D%5CSigma%7Bab%7D%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%26xa%5Csim%5Cmathcal%7BN%7D%28%5Cmu_a%2C%5CSigma%7Baa%7D%29%5C%5C%0A%26xb%7Cx_a%5Csim%5Cmathcal%7BN%7D%28%5Cmu%7Bb%7Ca%7D%2C%5CSigma%7Bb%7Ca%7D%29%5C%5C%0A%26%5Cmu%7Bb%7Ca%7D%3D%5CSigma%7Bba%7D%5CSigma%7Baa%7D%5E%7B-1%7D%28xa-%5Cmu_a%29%2B%5Cmu_b%5C%5C%0A%26%5CSigma%7Bb%7Ca%7D%3D%5CSigma%7Bbb%7D-%5CSigma%7Bba%7D%5CSigma%7Baa%7D%5E%7B-1%7D%5CSigma%7Bab%7D%0A%5Cend%7Balign%7D%0A&height=86&width=191#crop=0&crop=0&crop=1&crop=1&id=QwLSb&originHeight=122&originWidth=268&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Linear Regression

Model

  1. Dataset:
    23.Cheatsheet - 图8%2C(x_2%2C%20y_2)%2C%5Ccdots%2C(x_N%2C%20y_N)%5C%7D%0A#card=math&code=%5Cmathcal%7BD%7D%3D%5C%7B%28x_1%2C%20y_1%29%2C%28x_2%2C%20y_2%29%2C%5Ccdots%2C%28x_N%2C%20y_N%29%5C%7D%0A&height=18&width=238#crop=0&crop=0&crop=1&crop=1&id=lwWPp&originHeight=26&originWidth=333&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. Notation:
    23.Cheatsheet - 图9%5ET%2CY%3D(y_1%2Cy_2%2C%5Ccdots%2Cy_N)%5ET%0A#card=math&code=X%3D%28x_1%2Cx_2%2C%5Ccdots%2Cx_N%29%5ET%2CY%3D%28y_1%2Cy_2%2C%5Ccdots%2Cy_N%29%5ET%0A&height=20&width=288#crop=0&crop=0&crop=1&crop=1&id=qyk4q&originHeight=29&originWidth=403&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. Model:
    23.Cheatsheet - 图10%3Dw%5ETx%0A#card=math&code=f%28w%29%3Dw%5ETx%0A&height=20&width=78#crop=0&crop=0&crop=1&crop=1&id=O63qz&originHeight=29&originWidth=110&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Loss Function

  1. 最小二乘误差/高斯噪声的MLE23.Cheatsheet - 图11%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%7C%7Cw%5ETx_i-y_i%7C%7C%5E2_2%0A#card=math&code=L%28w%29%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%7C%7Cw%5ETx_i-y_i%7C%7C%5E2_2%0A&height=47&width=164#crop=0&crop=0&crop=1&crop=1&id=lfWCE&originHeight=66&originWidth=230&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

闭式解

23.Cheatsheet - 图12%5E%7B-1%7DX%5ETY%3DX%5E%2BY%5C%5C%0AX%3DU%5CSigma%20V%5ET%5C%5C%0AX%5E%2B%3DV%5CSigma%5E%7B-1%7DU%5ET%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%5Chat%7Bw%7D%3D%28X%5ETX%29%5E%7B-1%7DX%5ETY%3DX%5E%2BY%5C%5C%0AX%3DU%5CSigma%20V%5ET%5C%5C%0AX%5E%2B%3DV%5CSigma%5E%7B-1%7DU%5ET%0A%5Cend%7Balign%7D%0A&height=61&width=183#crop=0&crop=0&crop=1&crop=1&id=e3gkN&originHeight=86&originWidth=257&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

正则化

23.Cheatsheet - 图13%2B%5Clambda%7C%7Cw%7C%7C_1%2C%5Clambda%5Cgt0%5C%5C%0AL2-Laplasian%5C%20priori-Sparsity%26%3A%5Cmathop%7Bargmin%7D%5Climits_wL(w)%2B%5Clambda%7C%7Cw%7C%7C%5E2_2%2C%5Clambda%20%5Cgt%200%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%0AL1-Gaussian%20%5C%20priori%26%3A%5Cmathop%7Bargmin%7D%5Climits_wL%28w%29%2B%5Clambda%7C%7Cw%7C%7C_1%2C%5Clambda%5Cgt0%5C%5C%0AL2-Laplasian%5C%20priori-Sparsity%26%3A%5Cmathop%7Bargmin%7D%5Climits_wL%28w%29%2B%5Clambda%7C%7Cw%7C%7C%5E2_2%2C%5Clambda%20%5Cgt%200%0A%5Cend%7Balign%7D%0A&height=61&width=425#crop=0&crop=0&crop=1&crop=1&id=FIcyt&originHeight=86&originWidth=596&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Linear Classification

Hard

PCA

  1. Idea: 在线性模型上加入激活函数
  2. Loss Function:

23.Cheatsheet - 图14%3D%5Csum%5Climits%7Bx_i%5Cin%5Cmathcal%7BD%7D%7Bwrong%7D%7D-yiw%5ETx_i%0A#card=math&code=L%28w%29%3D%5Csum%5Climits%7Bxi%5Cin%5Cmathcal%7BD%7D%7Bwrong%7D%7D-y_iw%5ETx_i%0A&height=38&width=162#crop=0&crop=0&crop=1&crop=1&id=HCakC&originHeight=54&originWidth=227&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. Parameters:

23.Cheatsheet - 图15

Fisher

  1. Idea: 投影，类内小，类间大。
  2. Loss Function:


23.Cheatsheet - 图16%3D%5Cfrac%7Bw%5ETSbw%7D%7Bw%5ETS_ww%7D%5C%5C%0A%26S_b%3D(%5Coverline%7Bx%7Bc1%7D%7D-%5Coverline%7Bx%7Bc2%7D%7D)(%5Coverline%7Bx%7Bc1%7D%7D-%5Coverline%7Bx%7Bc2%7D%7D)%5ET%5C%5C%0A%26S_w%3DS_1%2BS_2%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%26J%28w%29%3D%5Cfrac%7Bw%5ETS_bw%7D%7Bw%5ETS_ww%7D%5C%5C%0A%26S_b%3D%28%5Coverline%7Bx%7Bc1%7D%7D-%5Coverline%7Bx%7Bc2%7D%7D%29%28%5Coverline%7Bx%7Bc1%7D%7D-%5Coverline%7Bx_%7Bc2%7D%7D%29%5ET%5C%5C%0A%26S_w%3DS_1%2BS_2%0A%5Cend%7Balign%7D%0A&height=80&width=191#crop=0&crop=0&crop=1&crop=1&id=oWSby&originHeight=113&originWidth=268&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. 闭式解，投影方向:


23.Cheatsheet - 图17%0A#card=math&code=Sw%5E%7B-1%7D%28%5Coverline%7Bx%7Bc1%7D%7D-%5Coverline%7Bx_%7Bc2%7D%7D%29%0A&height=20&width=95#crop=0&crop=0&crop=1&crop=1&id=vrByT&originHeight=29&originWidth=134&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Soft

判别模型

Logistic Regression
  1. Idea，激活函数:


23.Cheatsheet - 图18%26%3D%5Cfrac%7B1%7D%7B1%2B%5Cexp(-a)%7D%5C%5C%0Aa%26%3Dw%5ETx%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7Dp%28C_1%7Cx%29%26%3D%5Cfrac%7B1%7D%7B1%2B%5Cexp%28-a%29%7D%5C%5C%0Aa%26%3Dw%5ETx%0A%5Cend%7Balign%7D%0A&height=58&width=159#crop=0&crop=0&crop=1&crop=1&id=qOrgo&originHeight=83&originWidth=223&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. Loss Function(交叉熵):


23.Cheatsheet - 图19%3D%5Cmathop%7Bargmax%7Dw%5Csum%5Climits%7Bi%3D1%7D%5EN(yi%5Clog%20p_1%2B(1-y_i)%5Clog%20p_0)%0A#card=math&code=%5Chat%7Bw%7D%3D%5Cmathop%7Bargmax%7D_wJ%28w%29%3D%5Cmathop%7Bargmax%7D_w%5Csum%5Climits%7Bi%3D1%7D%5EN%28y_i%5Clog%20p_1%2B%281-y_i%29%5Clog%20p_0%29%0A&height=47&width=381#crop=0&crop=0&crop=1&crop=1&id=grgFY&originHeight=66&originWidth=533&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. 解法，SGD

  2. 23.Cheatsheet - 图20%3D%5Csum%5Climits%7Bi%3D1%7D%5EN(y_i-p_1)x_i%0A#card=math&code=J%27%28w%29%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%28y_i-p_1%29x_i%0A&height=47&width=148#crop=0&crop=0&crop=1&crop=1&id=eQRuW&originHeight=66&originWidth=208&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

生成模型

GDA
  1. Model
    1. 23.Cheatsheet - 图21#card=math&code=y%5Csim%20Bernoulli%28%5Cphi%29&height=18&width=111#crop=0&crop=0&crop=1&crop=1&id=OECLl&originHeight=26&originWidth=156&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. 23.Cheatsheet - 图22#card=math&code=x%7Cy%3D1%5Csim%5Cmathcal%7BN%7D%28%5Cmu_1%2C%5CSigma%29&height=19&width=127#crop=0&crop=0&crop=1&crop=1&id=kqbs8&originHeight=27&originWidth=178&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    3. 23.Cheatsheet - 图23#card=math&code=x%7Cy%3D0%5Csim%5Cmathcal%7BN%7D%28%5Cmu_0%2C%5CSigma%29&height=19&width=127#crop=0&crop=0&crop=1&crop=1&id=ExxuC&originHeight=27&originWidth=178&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. MAP
    23.Cheatsheet - 图24p(Y)%5Cnonumber%5C%5C%0A%26%3D%5Cmathop%7Bargmax%7D%7B%5Cphi%2C%5Cmu_0%2C%5Cmu_1%2C%5CSigma%7D%5Csum%5Climits%7Bi%3D1%7D%5EN((1-yi)%5Clog%5Cmathcal%7BN%7D(%5Cmu_0%2C%5CSigma)%2By_i%5Clog%20%5Cmathcal%7BN%7D(%5Cmu_1%2C%5CSigma)%2By_i%5Clog%5Cphi%2B(1-y_i)%5Clog(1-%5Cphi))%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%0A%26%5Cmathop%7Bargmax%7D%7B%5Cphi%2C%5Cmu0%2C%5Cmu_1%2C%5CSigma%7D%5Clog%20p%28X%7CY%29p%28Y%29%5Cnonumber%5C%5C%0A%26%3D%5Cmathop%7Bargmax%7D%7B%5Cphi%2C%5Cmu0%2C%5Cmu_1%2C%5CSigma%7D%5Csum%5Climits%7Bi%3D1%7D%5EN%28%281-y_i%29%5Clog%5Cmathcal%7BN%7D%28%5Cmu_0%2C%5CSigma%29%2By_i%5Clog%20%5Cmathcal%7BN%7D%28%5Cmu_1%2C%5CSigma%29%2By_i%5Clog%5Cphi%2B%281-y_i%29%5Clog%281-%5Cphi%29%29%0A%5Cend%7Balign%7D%0A&height=80&width=560#crop=0&crop=0&crop=1&crop=1&id=TplXV&originHeight=113&originWidth=783&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 解
    23.Cheatsheet - 图25xi%7D%7BN_0%7D%5C%5C%0A%5CSigma%26%3D%5Cfrac%7BN_1S_1%2BN_2S_2%7D%7BN%7D%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%5Cphi%26%3D%5Cfrac%7BN_1%7D%7BN%7D%5C%5C%0A%5Cmu_1%26%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5ENyix_i%7D%7BN_1%7D%5C%5C%0A%5Cmu_0%26%3D%5Cfrac%7B%5Csum%5Climits%7Bi%3D1%7D%5EN%281-y_i%29x_i%7D%7BN_0%7D%5C%5C%0A%5CSigma%26%3D%5Cfrac%7BN_1S_1%2BN_2S_2%7D%7BN%7D%0A%5Cend%7Balign%7D%0A&height=196&width=133#crop=0&crop=0&crop=1&crop=1&id=MCL7S&originHeight=275&originWidth=187&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Naive Bayesian
  1. Model, 对单个数据点的各个维度作出限制
    23.Cheatsheet - 图26
    1. 23.Cheatsheet - 图27 为连续变量：23.Cheatsheet - 图28%3D%5Cmathcal%7BN%7D(%5Cmu_i%2C%5Csigma_i%5E2)#card=math&code=p%28x_i%7Cy%29%3D%5Cmathcal%7BN%7D%28%5Cmu_i%2C%5Csigma_i%5E2%29&height=20&width=127#crop=0&crop=0&crop=1&crop=1&id=cKI0p&originHeight=29&originWidth=178&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. 23.Cheatsheet - 图29 为离散变量：类别分布（Categorical）：23.Cheatsheet - 图30%3D%5Cthetai%2C%5Csum%5Climits%7Bi%3D1%7D%5EK%5Cthetai%3D1#card=math&code=p%28x_i%3Di%7Cy%29%3D%5Ctheta_i%2C%5Csum%5Climits%7Bi%3D1%7D%5EK%5Ctheta_i%3D1&height=47&width=172#crop=0&crop=0&crop=1&crop=1&id=aT0PR&originHeight=66&originWidth=241&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    3. 23.Cheatsheet - 图31%3D%5Cphi%5Ey(1-%5Cphi)%5E%7B1-y%7D#card=math&code=p%28y%29%3D%5Cphi%5Ey%281-%5Cphi%29%5E%7B1-y%7D&height=20&width=129#crop=0&crop=0&crop=1&crop=1&id=E94dZ&originHeight=29&originWidth=181&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 解：和GDA相同

Dimension Reduction

中心化：

23.Cheatsheet - 图32(EN-%5Cfrac%7B1%7D%7BN%7D%5Cmathbb%7BI%7D%7BN1%7D%5Cmathbb%7BI%7D%7B1N%7D)%5ETX%5Cnonumber%5C%5C%0A%26%3D%5Cfrac%7B1%7D%7BN%7DX%5ETH%5E2X%3D%5Cfrac%7B1%7D%7BN%7DX%5ETHX%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7DS%0A%26%3D%5Cfrac%7B1%7D%7BN%7DX%5ET%28E_N-%5Cfrac%7B1%7D%7BN%7D%5Cmathbb%7BI%7D%7BN1%7D%5Cmathbb%7BI%7D%7B1N%7D%29%28E_N-%5Cfrac%7B1%7D%7BN%7D%5Cmathbb%7BI%7D%7BN1%7D%5Cmathbb%7BI%7D_%7B1N%7D%29%5ETX%5Cnonumber%5C%5C%0A%26%3D%5Cfrac%7B1%7D%7BN%7DX%5ETH%5E2X%3D%5Cfrac%7B1%7D%7BN%7DX%5ETHX%0A%5Cend%7Balign%7D%0A&height=70&width=325#crop=0&crop=0&crop=1&crop=1&id=CP4l2&originHeight=98&originWidth=455&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

PCA

  1. Idea: 坐标变换，寻找线性无关的新基矢，取信息损失最小的前几个维度
  2. Loss Function:
    23.Cheatsheet - 图33
  3. 解：
    1. 特征分解法
      23.Cheatsheet - 图34
    2. SVD for X/S
      23.Cheatsheet - 图35
    3. SVD for T
      23.Cheatsheet - 图36

p-PCA

  1. Model:
    23.Cheatsheet - 图37%5C%5C%0Ax%26%3DWz%2B%5Cmu%2B%5Cvarepsilon%5C%5C%0A%5Cvarepsilon%26%5Csim%5Cmathcal%7BN%7D(0%2C%5Csigma%5E2%5Cmathbb%7BI%7D%7Bpp%7D)%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%0Az%26%5Csim%5Cmathcal%7BN%7D%28%5Cmathbb%7BO%7D%7Bq1%7D%2C%5Cmathbb%7BI%7D%7Bqq%7D%29%5C%5C%0Ax%26%3DWz%2B%5Cmu%2B%5Cvarepsilon%5C%5C%0A%5Cvarepsilon%26%5Csim%5Cmathcal%7BN%7D%280%2C%5Csigma%5E2%5Cmathbb%7BI%7D%7Bpp%7D%29%0A%5Cend%7Balign%7D%0A&height=61&width=108#crop=0&crop=0&crop=1&crop=1&id=SNR5O&originHeight=86&originWidth=152&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. Learning: E-M
  3. Inference:
    23.Cheatsheet - 图38%3D%5Cmathcal%7BN%7D(W%5ET(WW%5ET%2B%5Csigma%5E2%5Cmathbb%7BI%7D)%5E%7B-1%7D(x-%5Cmu)%2C%5Cmathbb%7BI%7D-W%5ET(WW%5ET%2B%5Csigma%5E2%5Cmathbb%7BI%7D)%5E%7B-1%7DW)%0A#card=math&code=p%28z%7Cx%29%3D%5Cmathcal%7BN%7D%28W%5ET%28WW%5ET%2B%5Csigma%5E2%5Cmathbb%7BI%7D%29%5E%7B-1%7D%28x-%5Cmu%29%2C%5Cmathbb%7BI%7D-W%5ET%28WW%5ET%2B%5Csigma%5E2%5Cmathbb%7BI%7D%29%5E%7B-1%7DW%29%0A&height=20&width=443#crop=0&crop=0&crop=1&crop=1&id=LQtSM&originHeight=29&originWidth=620&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

SVM

  1. 强对偶关系：凸优化+（松弛）Slater 条件->强对偶。
  2. 参数求解：KKT条件
    1. 可行域
    2. 互补松弛+梯度为0

Hard-margin

  1. Idea: 最大化间隔
  2. Model:
    23.Cheatsheet - 图39%5Cge1%2Ci%3D1%2C2%2C%5Ccdots%2CN%0A#card=math&code=%5Cmathop%7Bargmin%7D_%7Bw%2Cb%7D%5Cfrac%7B1%7D%7B2%7Dw%5ETw%5C%20s.t.%5C%20y_i%28w%5ETx_i%2Bb%29%5Cge1%2Ci%3D1%2C2%2C%5Ccdots%2CN%0A&height=39&width=337#crop=0&crop=0&crop=1&crop=1&id=KPX1u&originHeight=56&originWidth=472&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 对偶问题
    23.Cheatsheet - 图40
  4. 模型参数

Soft-margin

  1. Idea:允许少量错误
  2. Model:
    23.Cheatsheet - 图41%5C%7D%5C%5C%0A%5Cmathop%7Bargmin%7D%7Bw%2Cb%7D%5Cfrac%7B1%7D%7B2%7Dw%5ETw%2BC%5Csum%5Climits%7Bi%3D1%7D%5EN%5Cxii%5C%20s.t.%5C%20y_i(w%5ETx_i%2Bb)%5Cge1-%5Cxi_i%2C%5Cxi_i%5Cge0%2Ci%3D1%2C2%2C%5Ccdots%2CN%0A#card=math&code=error%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Cmax%5C%7B0%2C1-yi%28w%5ETx_i%2Bb%29%5C%7D%5C%5C%0A%5Cmathop%7Bargmin%7D%7Bw%2Cb%7D%5Cfrac%7B1%7D%7B2%7Dw%5ETw%2BC%5Csum%5Climits_%7Bi%3D1%7D%5EN%5Cxi_i%5C%20s.t.%5C%20y_i%28w%5ETx_i%2Bb%29%5Cge1-%5Cxi_i%2C%5Cxi_i%5Cge0%2Ci%3D1%2C2%2C%5Ccdots%2CN%0A&height=97&width=643#crop=0&crop=0&crop=1&crop=1&id=VafnL&originHeight=137&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Kernel

对称的正定函数都可以作为正定核。

Exp Family

  1. 表达式
    23.Cheatsheet - 图42%3Dh(x)%5Cexp(%5Ceta%5ET%5Cphi(x)-A(%5Ceta))%3D%5Cfrac%7B1%7D%7B%5Cexp(A(%5Ceta))%7Dh(x)%5Cexp(%5Ceta%5ET%5Cphi(x))%0A#card=math&code=p%28x%7C%5Ceta%29%3Dh%28x%29%5Cexp%28%5Ceta%5ET%5Cphi%28x%29-A%28%5Ceta%29%29%3D%5Cfrac%7B1%7D%7B%5Cexp%28A%28%5Ceta%29%29%7Dh%28x%29%5Cexp%28%5Ceta%5ET%5Cphi%28x%29%29%0A&height=38&width=421#crop=0&crop=0&crop=1&crop=1&id=IruJb&originHeight=54&originWidth=590&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 对数配分函数
    23.Cheatsheet - 图43%3D%5Cmathbb%7BE%7D%7Bp(x%7C%5Ceta)%7D%5B%5Cphi(x)%5D%5C%5C%0AA’’(%5Ceta)%3DVar%7Bp(x%7C%5Ceta)%7D%5B%5Cphi(x)%5D%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%20%0AA%27%28%5Ceta%29%3D%5Cmathbb%7BE%7D%7Bp%28x%7C%5Ceta%29%7D%5B%5Cphi%28x%29%5D%5C%5C%0AA%27%27%28%5Ceta%29%3DVar%7Bp%28x%7C%5Ceta%29%7D%5B%5Cphi%28x%29%5D%0A%5Cend%7Balign%7D%0A&height=41&width=155#crop=0&crop=0&crop=1&crop=1&id=hupWi&originHeight=59&originWidth=217&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 指数族分布满足最大熵定理

PGM

Representation

  1. 有向图
    23.Cheatsheet - 图44%3D%5Cprod%5Climits%7Bi%3D1%7D%5Epp(x_i%7Cx%7Bparent(i)%7D)%0A#card=math&code=p%28x1%2Cx_2%2C%5Ccdots%2Cx_p%29%3D%5Cprod%5Climits%7Bi%3D1%7D%5Epp%28xi%7Cx%7Bparent%28i%29%7D%29%0A&height=45&width=237#crop=0&crop=0&crop=1&crop=1&id=tg1x6&originHeight=63&originWidth=332&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)


D-separation
23.Cheatsheet - 图45%3D%5Cfrac%7Bp(x)%7D%7B%5Cint%20p(x)dx%7Bi%7D%7D%3D%5Cfrac%7B%5Cprod%5Climits%7Bj%3D1%7D%5Epp(xj%7Cx%7Bparents(j)%7D)%7D%7B%5Cint%5Cprod%5Climits%7Bj%3D1%7D%5Epp(x_j%7Cx%7Bparents(j)%7D)dxi%7D%3D%5Cfrac%7Bp(x_i%7Cx%7Bparents(i)%7D)p(x%7Bchild(i)%7D%7Cx_i)%7D%7B%5Cint%20p(x_i%7Cx%7Bparents(i)%7D)p(x%7Bchild(i)%7D%7Cx_i)dx_i%7D%0A#card=math&code=p%28x_i%7Cx%7B-i%7D%29%3D%5Cfrac%7Bp%28x%29%7D%7B%5Cint%20p%28x%29dx%7Bi%7D%7D%3D%5Cfrac%7B%5Cprod%5Climits%7Bj%3D1%7D%5Epp%28xj%7Cx%7Bparents%28j%29%7D%29%7D%7B%5Cint%5Cprod%5Climits%7Bj%3D1%7D%5Epp%28x_j%7Cx%7Bparents%28j%29%7D%29dxi%7D%3D%5Cfrac%7Bp%28x_i%7Cx%7Bparents%28i%29%7D%29p%28x%7Bchild%28i%29%7D%7Cx_i%29%7D%7B%5Cint%20p%28x_i%7Cx%7Bparents%28i%29%7D%29p%28x_%7Bchild%28i%29%7D%7Cx_i%29dx_i%7D%0A&height=86&width=555#crop=0&crop=0&crop=1&crop=1&id=ZCfQc&originHeight=122&originWidth=777&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. 无向图
    23.Cheatsheet - 图46%3D%5Cfrac%7B1%7D%7BZ%7D%5Cprod%5Climits%7Bi%3D1%7D%5E%7BK%7D%5Cphi(x%7Bci%7D)%5C%5C%0AZ%3D%5Csum%5Climits%7Bx%5Cin%5Cmathcal%7BX%7D%7D%5Cprod%5Climits%7Bi%3D1%7D%5E%7BK%7D%5Cphi(x%7Bci%7D)%5C%5C%0A%5Cphi(x%7Bci%7D)%3D%5Cexp(-E(x%7Bci%7D))%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7Dp%28x%29%3D%5Cfrac%7B1%7D%7BZ%7D%5Cprod%5Climits%7Bi%3D1%7D%5E%7BK%7D%5Cphi%28x%7Bci%7D%29%5C%5C%0AZ%3D%5Csum%5Climits%7Bx%5Cin%5Cmathcal%7BX%7D%7D%5Cprod%5Climits%7Bi%3D1%7D%5E%7BK%7D%5Cphi%28x%7Bci%7D%29%5C%5C%0A%5Cphi%28x%7Bci%7D%29%3D%5Cexp%28-E%28x%7Bci%7D%29%29%0A%5Cend%7Balign%7D%0A&height=116&width=151#crop=0&crop=0&crop=1&crop=1&id=dVcFH&originHeight=164&originWidth=211&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 有向转无向
    1. 将每个节点的父节点两两相连
    2. 将有向边替换为无向边

Learning

参数学习-EM

  1. 目的：解决具有隐变量的混合模型的参数估计（极大似然估计）
  2. 参数：
    23.Cheatsheet - 图47%0A#card=math&code=%5Ctheta%7BMLE%7D%3D%5Cmathop%7Bargmax%7D%5Climits%5Ctheta%5Clog%20p%28x%7C%5Ctheta%29%0A&height=28&width=168#crop=0&crop=0&crop=1&crop=1&id=K5nqo&originHeight=41&originWidth=237&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 迭代求解：
    23.Cheatsheet - 图48%5Dp(z%7Cx%2C%5Ctheta%5Et)dz%3D%5Cmathbb%7BE%7D%7Bz%7Cx%2C%5Ctheta%5Et%7D%5B%5Clog%20p(x%2Cz%7C%5Ctheta)%5D%0A#card=math&code=%5Ctheta%5E%7Bt%2B1%7D%3D%5Cmathop%7Bargmax%7D%5Climits%7B%5Ctheta%7D%5Cintz%5Clog%20%5Bp%28x%2Cz%7C%5Ctheta%29%5Dp%28z%7Cx%2C%5Ctheta%5Et%29dz%3D%5Cmathbb%7BE%7D%7Bz%7Cx%2C%5Ctheta%5Et%7D%5B%5Clog%20p%28x%2Cz%7C%5Ctheta%29%5D%0A&height=37&width=409#crop=0&crop=0&crop=1&crop=1&id=BnZY6&originHeight=53&originWidth=573&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  4. 原理
    23.Cheatsheet - 图49%5Cle%5Clog%20p(x%7C%5Ctheta%5E%7Bt%2B1%7D)%0A#card=math&code=%5Clog%20p%28x%7C%5Ctheta%5Et%29%5Cle%5Clog%20p%28x%7C%5Ctheta%5E%7Bt%2B1%7D%29%0A&height=20&width=164#crop=0&crop=0&crop=1&crop=1&id=zsyuU&originHeight=29&originWidth=231&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  5. 广义EM
    1. E step：
      23.Cheatsheet - 图50%3D%5Cmathop%7Bargmax%7D_q%5Cint_zq%5Et(z)%5Clog%5Cfrac%7Bp(x%2Cz%7C%5Ctheta)%7D%7Bq%5Et(z)%7Ddz%2Cfixed%5C%20%5Ctheta%0A#card=math&code=%5Chat%7Bq%7D%5E%7Bt%2B1%7D%28z%29%3D%5Cmathop%7Bargmax%7D_q%5Cint_zq%5Et%28z%29%5Clog%5Cfrac%7Bp%28x%2Cz%7C%5Ctheta%29%7D%7Bq%5Et%28z%29%7Ddz%2Cfixed%5C%20%5Ctheta%0A&height=41&width=319#crop=0&crop=0&crop=1&crop=1&id=sFPss&originHeight=59&originWidth=447&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. M step：
      23.Cheatsheet - 图51%5Clog%5Cfrac%7Bp(x%2Cz%7C%5Ctheta)%7D%7Bq%5E%7Bt%2B1%7D(z)%7Ddz%2Cfixed%5C%20%5Chat%7Bq%7D%0A#card=math&code=%5Chat%7B%5Ctheta%7D%3D%5Cmathop%7Bargmax%7D_%5Ctheta%20%5Cint_zq%5E%7Bt%2B1%7D%28z%29%5Clog%5Cfrac%7Bp%28x%2Cz%7C%5Ctheta%29%7D%7Bq%5E%7Bt%2B1%7D%28z%29%7Ddz%2Cfixed%5C%20%5Chat%7Bq%7D%0A&height=41&width=296#crop=0&crop=0&crop=1&crop=1&id=KOKFZ&originHeight=59&originWidth=416&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

Inference

  1. 精确推断
    1. VE
    2. BP
      23.Cheatsheet - 图52%3D%5Csum%5Climitsj%5Cphi_j(j)%5Cphi%7Bij%7D(ij)%5Cprod%5Climits%7Bk%5Cin%20Neighbour(j)-i%7Dm%7Bk%5Cto%20j%7D(j)%0A#card=math&code=m%7Bj%5Cto%20i%7D%28i%29%3D%5Csum%5Climits_j%5Cphi_j%28j%29%5Cphi%7Bij%7D%28ij%29%5Cprod%5Climits%7Bk%5Cin%20Neighbour%28j%29-i%7Dm%7Bk%5Cto%20j%7D%28j%29%0A&height=38&width=312#crop=0&crop=0&crop=1&crop=1&id=Hfbvw&originHeight=54&originWidth=437&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    3. MP
      23.Cheatsheet - 图53-i%7Dm%7Bk%5Cto%20j%7D%0A#card=math&code=m%7Bj%5Cto%20i%7D%3D%5Cmax%5Climits%7Bj%7D%5Cphi_j%5Cphi%7Bij%7D%5Cprod%5Climits%7Bk%5Cin%20Neighbour%28j%29-i%7Dm%7Bk%5Cto%20j%7D%0A&height=38&width=243#crop=0&crop=0&crop=1&crop=1&id=Kczx0&originHeight=54&originWidth=340&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 近似推断
    1. 确定性近似，VI
      1. 变分表达式
        23.Cheatsheet - 图54%3D%5Cmathop%7Bargmax%7D%7Bq(Z)%7DL(q)%0A#card=math&code=%5Chat%7Bq%7D%28Z%29%3D%5Cmathop%7Bargmax%7D%7Bq%28Z%29%7DL%28q%29%0A&height=32&width=133#crop=0&crop=0&crop=1&crop=1&id=ig92w&originHeight=45&originWidth=187&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
      2. 平均场近似下的 VI-坐标上升
        23.Cheatsheet - 图55%7D%5B%5Clog%20p(X%2CZ)%5D%3D%5Clog%20%5Chat%7Bp%7D(X%2CZj)%5C%5C%0Aq_j(Z_j)%3D%5Chat%7Bp%7D(X%2CZ_j)%0A#card=math&code=%5Cmathbb%7BE%7D%7B%5Cprod%5Climits_%7Bi%5Cne%20j%7Dq_i%28Z_i%29%7D%5B%5Clog%20p%28X%2CZ%29%5D%3D%5Clog%20%5Chat%7Bp%7D%28X%2CZ_j%29%5C%5C%0Aq_j%28Z_j%29%3D%5Chat%7Bp%7D%28X%2CZ_j%29%0A&height=52&width=643#crop=0&crop=0&crop=1&crop=1&id=Coxrb&originHeight=74&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
      3. SGVI-变成优化问题，重参数法
        23.Cheatsheet - 图56%7DL(q)%3D%5Cmathop%7Bargmax%7D%7B%5Cphi%7DL(%5Cphi)%5C%5C%0A%5Cnabla%5Cphi%20L(%5Cphi)%3D%5Cmathbb%7BE%7D%7Bq%5Cphi%7D%5B(%5Cnabla%5Cphi%5Clog%20q%5Cphi)(%5Clog%20p%5Ctheta(x%5Ei%2Cz)-%5Clog%20q%5Cphi(z))%5D%5C%5C%0A%3D%5Cmathbb%7BE%7D%7Bp(%5Cvarepsilon)%7D%5B%5Cnabla_z%5B%5Clog%20p%5Ctheta(x%5Ei%2Cz)-%5Clog%20q%5Cphi(z)%5D%5Cnabla%5Cphi%20g%5Cphi(%5Cvarepsilon%2Cx%5Ei)%5D%5C%5C%0Az%3Dg%5Cphi(%5Cvarepsilon%2Cx%5Ei)%2C%5Cvarepsilon%5Csim%20p(%5Cvarepsilon)%0A#card=math&code=%5Cmathop%7Bargmax%7D%7Bq%28Z%29%7DL%28q%29%3D%5Cmathop%7Bargmax%7D%7B%5Cphi%7DL%28%5Cphi%29%5C%5C%0A%5Cnabla%5Cphi%20L%28%5Cphi%29%3D%5Cmathbb%7BE%7D%7Bq%5Cphi%7D%5B%28%5Cnabla%5Cphi%5Clog%20q%5Cphi%29%28%5Clog%20p%5Ctheta%28x%5Ei%2Cz%29-%5Clog%20q%5Cphi%28z%29%29%5D%5C%5C%0A%3D%5Cmathbb%7BE%7D%7Bp%28%5Cvarepsilon%29%7D%5B%5Cnablaz%5B%5Clog%20p%5Ctheta%28x%5Ei%2Cz%29-%5Clog%20q%5Cphi%28z%29%5D%5Cnabla%5Cphi%20g%5Cphi%28%5Cvarepsilon%2Cx%5Ei%29%5D%5C%5C%0Az%3Dg%5Cphi%28%5Cvarepsilon%2Cx%5Ei%29%2C%5Cvarepsilon%5Csim%20p%28%5Cvarepsilon%29%0A&height=107&width=643#crop=0&crop=0&crop=1&crop=1&id=vReNB&originHeight=150&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. 随机性近似
      1. 蒙特卡洛方法采样
        1. CDF 采样
        2. 拒绝采样， 23.Cheatsheet - 图57#card=math&code=q%28z%29&height=18&width=25#crop=0&crop=0&crop=1&crop=1&id=E2vnu&originHeight=26&originWidth=36&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，使得 23.Cheatsheet - 图58%5Cge%20p(z_i)#card=math&code=%5Cforall%20z_i%2CMq%28z_i%29%5Cge%20p%28z_i%29&height=18&width=124#crop=0&crop=0&crop=1&crop=1&id=Fg1M0&originHeight=26&originWidth=174&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，拒绝因子：23.Cheatsheet - 图59%7D%7BMq(z%5Ei)%7D%5Cle1#card=math&code=%5Calpha%3D%5Cfrac%7Bp%28z%5Ei%29%7D%7BMq%28z%5Ei%29%7D%5Cle1&height=42&width=108#crop=0&crop=0&crop=1&crop=1&id=CuZsH&originHeight=60&originWidth=152&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
        3. 重要性采样
          23.Cheatsheet - 图60%7D%5Bf(z)%5D%3D%5Cint%20p(z)f(z)dz%3D%5Cint%20%5Cfrac%7Bp(z)%7D%7Bq(z)%7Df(z)q(z)dz%5Csimeq%5Cfrac%7B1%7D%7BN%7D%5Csum%5Climits%7Bi%3D1%7D%5ENf(z_i)%5Cfrac%7Bp(z_i)%7D%7Bq(z_i)%7D%0A#card=math&code=%5Cmathbb%7BE%7D%7Bp%28z%29%7D%5Bf%28z%29%5D%3D%5Cint%20p%28z%29f%28z%29dz%3D%5Cint%20%5Cfrac%7Bp%28z%29%7D%7Bq%28z%29%7Df%28z%29q%28z%29dz%5Csimeq%5Cfrac%7B1%7D%7BN%7D%5Csum%5Climits_%7Bi%3D1%7D%5ENf%28z_i%29%5Cfrac%7Bp%28z_i%29%7D%7Bq%28z_i%29%7D%0A&height=47&width=438#crop=0&crop=0&crop=1&crop=1&id=Zi8ho&originHeight=66&originWidth=614&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
        4. 重要性重采样：重要性采样+重采样
      2. MCMC：构建马尔可夫链概率序列，使其收敛到平稳分布 23.Cheatsheet - 图61#card=math&code=p%28z%29&height=18&width=26#crop=0&crop=0&crop=1&crop=1&id=tEoMb&originHeight=26&originWidth=37&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)。
        1. 转移矩阵（提议分布）
          23.Cheatsheet - 图62%5Ccdot%20Q%7Bz%5Cto%20z%5E%7D%5Calpha(z%2Cz%5E)%3Dp(z%5E*)%5Ccdot%20Q%7Bz%5E%5Cto%20z%7D%5Calpha(z%5E%2Cz)%5C%5C%0A%5Calpha(z%2Cz%5E)%3D%5Cmin%5C%7B1%2C%5Cfrac%7Bp(z%5E)Q%7Bz%5E*%5Cto%20z%7D%7D%7Bp(z)Q%7Bz%5Cto%20z%5E*%7D%7D%5C%7D%0A#card=math&code=p%28z%29%5Ccdot%20Q%7Bz%5Cto%20z%5E%2A%7D%5Calpha%28z%2Cz%5E%2A%29%3Dp%28z%5E%2A%29%5Ccdot%20Q%7Bz%5E%2A%5Cto%20z%7D%5Calpha%28z%5E%2A%2Cz%29%5C%5C%0A%5Calpha%28z%2Cz%5E%2A%29%3D%5Cmin%5C%7B1%2C%5Cfrac%7Bp%28z%5E%2A%29Q%7Bz%5E%2A%5Cto%20z%7D%7D%7Bp%28z%29Q%7Bz%5Cto%20z%5E%2A%7D%7D%5C%7D%0A&height=61&width=643#crop=0&crop=0&crop=1&crop=1&id=MKWMO&originHeight=87&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
        2. 算法（MH）：
          1. 通过在0，1之间均匀分布取点 23.Cheatsheet - 图63
          2. 生成 23.Cheatsheet - 图64#card=math&code=z%5E%2A%5Csim%20Q%28z%5E%2A%7Cz%5E%7Bi-1%7D%29&height=20&width=100#crop=0&crop=0&crop=1&crop=1&id=Jt1N9&originHeight=29&originWidth=141&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
          3. 计算 23.Cheatsheet - 图65 值
          4. 如果 23.Cheatsheet - 图66，则 23.Cheatsheet - 图67，否则 23.Cheatsheet - 图68
      3. Gibbs 采样：给定初始值 23.Cheatsheet - 图69在 23.Cheatsheet - 图70 时刻，采样 23.Cheatsheet - 图71#card=math&code=zi%5E%7Bt%2B1%7D%5Csim%20p%28z_i%7Cz%7B-i%7D%29&height=21&width=101#crop=0&crop=0&crop=1&crop=1&id=QjriJ&originHeight=30&originWidth=142&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，从第一个维度一个个采样。

GMM

  1. Model
    23.Cheatsheet - 图72%3D%5Csum%5Climits%7Bk%3D1%7D%5EKp_k%5Cmathcal%7BN%7D(x%7C%5Cmu_k%2C%5CSigma_k)%0A#card=math&code=p%28x%29%3D%5Csum%5Climits%7Bk%3D1%7D%5EKp_k%5Cmathcal%7BN%7D%28x%7C%5Cmu_k%2C%5CSigma_k%29%0A&height=47&width=166#crop=0&crop=0&crop=1&crop=1&id=z6ZYd&originHeight=66&originWidth=233&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 求解-EM
    23.Cheatsheet - 图73%26%3D%5Csum%5Climitsz%5B%5Clog%5Cprod%5Climits%7Bi%3D1%7D%5ENp(xi%2Cz_i%7C%5Ctheta)%5D%5Cprod%20%5Climits%7Bi%3D1%7D%5ENp(zi%7Cx_i%2C%5Ctheta%5Et)%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits_z%5B%5Csum%5Climits%7Bi%3D1%7D%5EN%5Clog%20p(xi%2Cz_i%7C%5Ctheta)%5D%5Cprod%20%5Climits%7Bi%3D1%7D%5ENp(zi%7Cx_i%2C%5Ctheta%5Et)%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Csum%5Climits%7Bz_i%7D%5Clog%20p(x_i%2Cz_i%7C%5Ctheta)p(z_i%7Cx_i%2C%5Ctheta%5Et)%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Csum%5Climits%7Bz_i%7D%5Clog%20p%7Bzi%7D%5Cmathcal%7BN(x_i%7C%5Cmu%7Bzi%7D%2C%5CSigma%7Bzi%7D)%7D%5Cfrac%7Bp%7Bzi%7D%5Et%5Cmathcal%7BN%7D(x_i%7C%5Cmu%7Bzi%7D%5Et%2C%5CSigma%7Bzi%7D%5Et)%7D%7B%5Csum%5Climits_kp_k%5Et%5Cmathcal%7BN%7D(x_i%7C%5Cmu_k%5Et%2C%5CSigma_k%5Et)%7D%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7DQ%28%5Ctheta%2C%5Ctheta%5Et%29%26%3D%5Csum%5Climits_z%5B%5Clog%5Cprod%5Climits%7Bi%3D1%7D%5ENp%28xi%2Cz_i%7C%5Ctheta%29%5D%5Cprod%20%5Climits%7Bi%3D1%7D%5ENp%28zi%7Cx_i%2C%5Ctheta%5Et%29%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits_z%5B%5Csum%5Climits%7Bi%3D1%7D%5EN%5Clog%20p%28xi%2Cz_i%7C%5Ctheta%29%5D%5Cprod%20%5Climits%7Bi%3D1%7D%5ENp%28zi%7Cx_i%2C%5Ctheta%5Et%29%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Csum%5Climits%7Bz_i%7D%5Clog%20p%28x_i%2Cz_i%7C%5Ctheta%29p%28z_i%7Cx_i%2C%5Ctheta%5Et%29%5Cnonumber%5C%5C%0A%26%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Csum%5Climits%7Bz_i%7D%5Clog%20p%7Bzi%7D%5Cmathcal%7BN%28x_i%7C%5Cmu%7Bzi%7D%2C%5CSigma%7Bzi%7D%29%7D%5Cfrac%7Bp%7Bzi%7D%5Et%5Cmathcal%7BN%7D%28x_i%7C%5Cmu%7Bzi%7D%5Et%2C%5CSigma%7Bzi%7D%5Et%29%7D%7B%5Csum%5Climits_kp_k%5Et%5Cmathcal%7BN%7D%28x_i%7C%5Cmu_k%5Et%2C%5CSigma_k%5Et%29%7D%0A%5Cend%7Balign%7D%0A&height=202&width=376#crop=0&crop=0&crop=1&crop=1&id=iy6EB&originHeight=284&originWidth=526&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    ![](https://g.yuque.com/gr/latex?p_k%5E%7Bt%2B1%7D%3D%5Cfrac%7B1%7D%7BN%7D%5Csum%5Climits    %7Bi%3D1%7D%5ENp(zi%3Dk%7Cx_i%2C%5Ctheta%5Et)%0A#card=math&code=p_k%5E%7Bt%2B1%7D%3D%5Cfrac%7B1%7D%7BN%7D%5Csum%5Climits%7Bi%3D1%7D%5ENp%28z_i%3Dk%7Cx_i%2C%5Ctheta%5Et%29%0A&height=47&width=188#crop=0&crop=0&crop=1&crop=1&id=RuH2a&originHeight=66&originWidth=263&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

序列模型-HMM，LDS，Particle

  1. 假设：
    1. 齐次 Markov 假设（未来只依赖于当前）：
      23.Cheatsheet - 图74%3Dp(i%7Bt%2B1%7D%7Ci_t)%0A#card=math&code=p%28i%7Bt%2B1%7D%7Cit%2Ci%7Bt-1%7D%2C%5Ccdots%2Ci1%2Co_t%2Co%7Bt-1%7D%2C%5Ccdots%2Co1%29%3Dp%28i%7Bt%2B1%7D%7Ci_t%29%0A&height=18&width=311#crop=0&crop=0&crop=1&crop=1&id=ST6n8&originHeight=26&originWidth=435&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. 观测独立假设：
      23.Cheatsheet - 图75%3Dp(ot%7Ci_t)%0A#card=math&code=p%28o_t%7Ci_t%2Ci%7Bt-1%7D%2C%5Ccdots%2Ci1%2Co%7Bt-1%7D%2C%5Ccdots%2Co_1%29%3Dp%28o_t%7Ci_t%29%0A&height=18&width=269#crop=0&crop=0&crop=1&crop=1&id=gXxvU&originHeight=26&originWidth=377&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 参数
    23.Cheatsheet - 图76%0A#card=math&code=%5Clambda%3D%28%5Cpi%2CA%2CB%29%0A&height=18&width=84#crop=0&crop=0&crop=1&crop=1&id=J3qFx&originHeight=26&originWidth=119&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

离散线性隐变量-HMM

  1. Evaluation：23.Cheatsheet - 图77#card=math&code=p%28O%7C%5Clambda%29&height=18&width=44#crop=0&crop=0&crop=1&crop=1&id=E2uMf&originHeight=26&originWidth=62&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，Forward-Backward 算法
    23.Cheatsheet - 图78%3D%5Csum%5Climits%7Bi%3D1%7D%5ENp(O%2Ci_T%3Dq_i%7C%5Clambda)%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5CalphaT(i)%3D%5Csum%5Climits%7Bi%3D1%7D%5ENbi(o_1)%5Cpi_i%5Cbeta_1(i)%5C%5C%0A%5Calpha%7Bt%2B1%7D(j)%3D%5Csum%5Climits%7Bi%3D1%7D%5ENb%7Bj%7D(ot)a%7Bij%7D%5Calphat(i)%5C%5C%0A%5Cbeta_t(i)%3D%5Csum%5Climits%7Bj%3D1%7D%5ENbj(o%7Bt%2B1%7D)a%7Bij%7D%5Cbeta%7Bt%2B1%7D(j)%0A#card=math&code=p%28O%7C%5Clambda%29%3D%5Csum%5Climits%7Bi%3D1%7D%5ENp%28O%2Ci_T%3Dq_i%7C%5Clambda%29%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5CalphaT%28i%29%3D%5Csum%5Climits%7Bi%3D1%7D%5ENbi%28o_1%29%5Cpi_i%5Cbeta_1%28i%29%5C%5C%0A%5Calpha%7Bt%2B1%7D%28j%29%3D%5Csum%5Climits%7Bi%3D1%7D%5ENb%7Bj%7D%28ot%29a%7Bij%7D%5Calphat%28i%29%5C%5C%0A%5Cbeta_t%28i%29%3D%5Csum%5Climits%7Bj%3D1%7D%5ENbj%28o%7Bt%2B1%7D%29a%7Bij%7D%5Cbeta%7Bt%2B1%7D%28j%29%0A&height=150&width=643#crop=0&crop=0&crop=1&crop=1&id=jPFKl&originHeight=210&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. Learning：23.Cheatsheet - 图79#card=math&code=%5Clambda%3D%5Cmathop%7Bargmax%7D%5Climits%7B%5Clambda%7Dp%28O%7C%5Clambda%29&height=28&width=126#crop=0&crop=0&crop=1&crop=1&id=nKMIi&originHeight=41&originWidth=176&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，EM 算法（Baum-Welch）
    ![](https://g.yuque.com/gr/latex?%5Clambda%5E%7Bt%2B1%7D%3D%5Cmathop%7Bargmax%7D    %5Clambda%5Csum%5ClimitsI%5Clog%20p(O%2CI%7C%5Clambda)p(O%2CI%7C%5Clambda%5Et)%5C%5C%3D%5Csum%5Climits_I%5B%5Clog%20%5Cpi%7Bi1%7D%2B%5Csum%5Climits%7Bt%3D2%7D%5ET%5Clog%20a%7Bi%7Bt-1%7D%2Cit%7D%2B%5Csum%5Climits%7Bt%3D1%7D%5ET%5Clog%20b%7Bi_t%7D(o_t)%5Dp(O%2CI%7C%5Clambda%5Et)%0A#card=math&code=%5Clambda%5E%7Bt%2B1%7D%3D%5Cmathop%7Bargmax%7D%5Clambda%5Csum%5ClimitsI%5Clog%20p%28O%2CI%7C%5Clambda%29p%28O%2CI%7C%5Clambda%5Et%29%5C%5C%3D%5Csum%5Climits_I%5B%5Clog%20%5Cpi%7Bi1%7D%2B%5Csum%5Climits%7Bt%3D2%7D%5ET%5Clog%20a%7Bi%7Bt-1%7D%2Cit%7D%2B%5Csum%5Climits%7Bt%3D1%7D%5ET%5Clog%20b_%7Bi_t%7D%28o_t%29%5Dp%28O%2CI%7C%5Clambda%5Et%29%0A&height=86&width=643#crop=0&crop=0&crop=1&crop=1&id=oTony&originHeight=120&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. Decoding：23.Cheatsheet - 图80#card=math&code=I%3D%5Cmathop%7Bargmax%7D%5Climits%7BI%7Dp%28I%7CO%2C%5Clambda%29&height=28&width=139#crop=0&crop=0&crop=1&crop=1&id=xc332&originHeight=41&originWidth=194&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)，Viterbi 算法-动态规划
    ![](https://g.yuque.com/gr/latex?%5Cdelta    %7Bt%7D(j)%3D%5Cmax%5Climits%7Bi_1%2C%5Ccdots%2Ci%7Bt-1%7D%7Dp(o1%2C%5Ccdots%2Co_t%2Ci_1%2C%5Ccdots%2Ci%7Bt-1%7D%2Cit%3Dq_i)%5C%5C%5Cdelta%7Bt%2B1%7D(j)%3D%5Cmax%5Climits%7B1%5Cle%20i%5Cle%20N%7D%5Cdelta_t(i)a%7Bij%7Dbj(o%7Bt%2B1%7D)%5C%5C%5Cpsi%7Bt%2B1%7D(j)%3D%5Cmathop%7Bargmax%7D%5Climits%7B1%5Cle%20i%5Cle%20N%7D%5Cdeltat(i)a%7Bij%7D%0A#card=math&code=%5Cdelta%7Bt%7D%28j%29%3D%5Cmax%5Climits%7Bi1%2C%5Ccdots%2Ci%7Bt-1%7D%7Dp%28o1%2C%5Ccdots%2Co_t%2Ci_1%2C%5Ccdots%2Ci%7Bt-1%7D%2Cit%3Dq_i%29%5C%5C%5Cdelta%7Bt%2B1%7D%28j%29%3D%5Cmax%5Climits%7B1%5Cle%20i%5Cle%20N%7D%5Cdelta_t%28i%29a%7Bij%7Dbj%28o%7Bt%2B1%7D%29%5C%5C%5Cpsi%7Bt%2B1%7D%28j%29%3D%5Cmathop%7Bargmax%7D%5Climits%7B1%5Cle%20i%5Cle%20N%7D%5Cdeltat%28i%29a%7Bij%7D%0A&height=92&width=643#crop=0&crop=0&crop=1&crop=1&id=APWkR&originHeight=129&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

连续线性隐变量-LDS

  1. Model
    23.Cheatsheet - 图81%26%5Csim%5Cmathcal%7BN%7D(A%5Ccdot%20z%7Bt-1%7D%2BB%2CQ)%5C%5C%0Ap(x_t%7Cz_t)%26%5Csim%5Cmathcal%7BN%7D(C%5Ccdot%20z_t%2BD%2CR)%5C%5C%0Az_1%26%5Csim%5Cmathcal%7BN%7D(%5Cmu_1%2C%5CSigma_1)%0A%5Cend%7Balign%7D%0A#card=math&code=%5Cbegin%7Balign%7D%0Ap%28z_t%7Cz%7Bt-1%7D%29%26%5Csim%5Cmathcal%7BN%7D%28A%5Ccdot%20z_%7Bt-1%7D%2BB%2CQ%29%5C%5C%0Ap%28x_t%7Cz_t%29%26%5Csim%5Cmathcal%7BN%7D%28C%5Ccdot%20z_t%2BD%2CR%29%5C%5C%0Az_1%26%5Csim%5Cmathcal%7BN%7D%28%5Cmu_1%2C%5CSigma_1%29%0A%5Cend%7Balign%7D%0A&height=58&width=208#crop=0&crop=0&crop=1&crop=1&id=bfDZw&originHeight=83&originWidth=291&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 滤波
    23.Cheatsheet - 图82%3Dp(x%7B1%3At%7D%2Cz_t)%2Fp(x%7B1%3At%7D)%5Cpropto%20p(x%7B1%3At%7D%2Cz_t)%5C%5C%3Dp(x_t%7Cz_t)p(z_t%7Cx%7B1%3At-1%7D)p(x%7B1%3At-1%7D)%5Cpropto%20p(x_t%7Cz_t)p(z_t%7Cx%7B1%3At-1%7D)%0A#card=math&code=p%28zt%7Cx%7B1%3At%7D%29%3Dp%28x%7B1%3At%7D%2Cz_t%29%2Fp%28x%7B1%3At%7D%29%5Cpropto%20p%28x%7B1%3At%7D%2Cz_t%29%5C%5C%3Dp%28x_t%7Cz_t%29p%28z_t%7Cx%7B1%3At-1%7D%29p%28x%7B1%3At-1%7D%29%5Cpropto%20p%28x_t%7Cz_t%29p%28z_t%7Cx%7B1%3At-1%7D%29%0A&height=39&width=643#crop=0&crop=0&crop=1&crop=1&id=FDHsj&originHeight=56&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 递推求解-线性高斯模型
    1. Prediction
      23.Cheatsheet - 图83%3D%5Cint%7Bz%7Bt-1%7D%7Dp(zt%7Cz%7Bt-1%7D)p(z%7Bt-1%7D%7Cx%7B1%3At-1%7D)dz%7Bt-1%7D%3D%5Cint%7Bz%7Bt-1%7D%7D%5Cmathcal%7BN%7D(Az%7Bt-1%7D%2BB%2CQ)%5Cmathcal%7BN%7D(%5Cmu%7Bt-1%7D%2C%5CSigma%7Bt-1%7D)dz%7Bt-1%7D%0A#card=math&code=p%28z_t%7Cx%7B1%3At-1%7D%29%3D%5Cint%7Bz%7Bt-1%7D%7Dp%28zt%7Cz%7Bt-1%7D%29p%28z%7Bt-1%7D%7Cx%7B1%3At-1%7D%29dz%7Bt-1%7D%3D%5Cint%7Bz%7Bt-1%7D%7D%5Cmathcal%7BN%7D%28Az%7Bt-1%7D%2BB%2CQ%29%5Cmathcal%7BN%7D%28%5Cmu%7Bt-1%7D%2C%5CSigma%7Bt-1%7D%29dz_%7Bt-1%7D%0A&height=39&width=591#crop=0&crop=0&crop=1&crop=1&id=xQDo9&originHeight=56&originWidth=827&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    2. Update:
      image.png

连续非线性隐变量-粒子滤波

通过采样(SIR)解决：

23.Cheatsheet - 图85%5D%3D%5Cintzf(z)p(z)dz%3D%5Cint_zf(z)%5Cfrac%7Bp(z)%7D%7Bq(z)%7Dq(z)dz%3D%5Csum%5Climits%7Bi%3D1%7D%5ENf(zi)%5Cfrac%7Bp(z_i)%7D%7Bq(z_i)%7D%0A#card=math&code=%5Cmathbb%7BE%7D%5Bf%28z%29%5D%3D%5Cint_zf%28z%29p%28z%29dz%3D%5Cint_zf%28z%29%5Cfrac%7Bp%28z%29%7D%7Bq%28z%29%7Dq%28z%29dz%3D%5Csum%5Climits%7Bi%3D1%7D%5ENf%28z_i%29%5Cfrac%7Bp%28z_i%29%7D%7Bq%28z_i%29%7D%0A&height=47&width=399#crop=0&crop=0&crop=1&crop=1&id=moLFm&originHeight=66&originWidth=558&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)

  1. 采样
    23.Cheatsheet - 图86p(zt%7Cz%7Bt-1%7D)%7D%7Bq(zt%7Cz%7B1%3At-1%7D%2Cx%7B1%3At%7D)%7Dw%7Bt-1%7D%5Ei%5C%5C%0Aq(zt%7Cz%7B1%3At-1%7D%2Cx%7B1%3At%7D)%3Dp(z_t%7Cz%7Bt-1%7D)%0A#card=math&code=wt%5Ei%5Cpropto%5Cfrac%7Bp%28x_t%7Cz_t%29p%28z_t%7Cz%7Bt-1%7D%29%7D%7Bq%28zt%7Cz%7B1%3At-1%7D%2Cx%7B1%3At%7D%29%7Dw%7Bt-1%7D%5Ei%5C%5C%0Aq%28zt%7Cz%7B1%3At-1%7D%2Cx%7B1%3At%7D%29%3Dp%28z_t%7Cz%7Bt-1%7D%29%0A&height=63&width=643#crop=0&crop=0&crop=1&crop=1&id=ddQCk&originHeight=89&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 重采样

CRF

  1. PDF
    23.Cheatsheet - 图87%3D%5Cfrac%7B1%7D%7BZ(x%2C%5Ctheta)%7D%5Cexp%5B%5Ctheta%5ETH(yt%2Cy%7Bt-1%7D%2Cx)%5D%0A#card=math&code=p%28Y%3Dy%7CX%3Dx%29%3D%5Cfrac%7B1%7D%7BZ%28x%2C%5Ctheta%29%7D%5Cexp%5B%5Ctheta%5ETH%28yt%2Cy%7Bt-1%7D%2Cx%29%5D%0A&height=38&width=306#crop=0&crop=0&crop=1&crop=1&id=NPD78&originHeight=54&originWidth=429&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  2. 边缘概率
    23.Cheatsheet - 图88%3D%5Csum%5Climits%7By%7B1%3At-1%7D%7D%5Csum%5Climits%7By%7Bt%2B1%3AT%7D%7D%5Cfrac%7B1%7D%7BZ%7D%5Cprod%5Climits%7Bt’%3D1%7D%5ET%5Cphi%7Bt’%7D(y%7Bt’-1%7D%2Cy%7Bt’%7D%2Cx)%5C%5C%0Ap(yt%3Di%7Cx)%3D%5Cfrac%7B1%7D%7BZ%7D%5CDelta_l%5CDelta_r%5C%5C%0A%5CDelta_l%3D%5Csum%5Climits%7By%7B1%3At-1%7D%7D%5Cphi%7B1%7D(y0%2Cy_1%2Cx)%5Cphi_2(y_1%2Cy_2%2Cx)%5Ccdots%5Cphi%7Bt-1%7D(y%7Bt-2%7D%2Cy%7Bt-1%7D%2Cx)%5Cphit(y%7Bt-1%7D%2Cyt%3Di%2Cx)%5C%5C%0A%5CDelta_r%3D%5Csum%5Climits%7By%7Bt%2B1%3AT%7D%7D%5Cphi%7Bt%2B1%7D(yt%3Di%2Cy%7Bt%2B1%7D%2Cx)%5Cphi%7Bt%2B2%7D(y%7Bt%2B1%7D%2Cy%7Bt%2B2%7D%2Cx)%5Ccdots%5Cphi_T(y%7BT-1%7D%2CyT%2Cx)%0A#card=math&code=p%28y_t%3Di%7Cx%29%3D%5Csum%5Climits%7By%7B1%3At-1%7D%7D%5Csum%5Climits%7By%7Bt%2B1%3AT%7D%7D%5Cfrac%7B1%7D%7BZ%7D%5Cprod%5Climits%7Bt%27%3D1%7D%5ET%5Cphi%7Bt%27%7D%28y%7Bt%27-1%7D%2Cy%7Bt%27%7D%2Cx%29%5C%5C%0Ap%28y_t%3Di%7Cx%29%3D%5Cfrac%7B1%7D%7BZ%7D%5CDelta_l%5CDelta_r%5C%5C%0A%5CDelta_l%3D%5Csum%5Climits%7By%7B1%3At-1%7D%7D%5Cphi%7B1%7D%28y0%2Cy_1%2Cx%29%5Cphi_2%28y_1%2Cy_2%2Cx%29%5Ccdots%5Cphi%7Bt-1%7D%28y%7Bt-2%7D%2Cy%7Bt-1%7D%2Cx%29%5Cphit%28y%7Bt-1%7D%2Cyt%3Di%2Cx%29%5C%5C%0A%5CDelta_r%3D%5Csum%5Climits%7By%7Bt%2B1%3AT%7D%7D%5Cphi%7Bt%2B1%7D%28yt%3Di%2Cy%7Bt%2B1%7D%2Cx%29%5Cphi%7Bt%2B2%7D%28y%7Bt%2B1%7D%2Cy%7Bt%2B2%7D%2Cx%29%5Ccdots%5Cphi_T%28y%7BT-1%7D%2CyT%2Cx%29%0A&height=169&width=643#crop=0&crop=0&crop=1&crop=1&id=xB1r8&originHeight=237&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
    23.Cheatsheet - 图89%3D%5CDelta_l%3D%5Csum%5Climits    %7Bj%5Cin%20S%7D%5Cphit(y%7Bt-1%7D%3Dj%2Cyt%3Di%2Cx)%5Calpha%7Bt-1%7D(j)%5C%5C%0A%5CDeltar%3D%5Cbeta_t(i)%3D%5Csum%5Climits%7Bj%5Cin%20S%7D%5Cphi%7Bt%2B1%7D(y_t%3Di%2Cy%7Bt%2B1%7D%3Dj%2Cx)%5Cbeta%7Bt%2B1%7D(j)%0A#card=math&code=%5Calpha_t%28i%29%3D%5CDelta_l%3D%5Csum%5Climits%7Bj%5Cin%20S%7D%5Cphit%28y%7Bt-1%7D%3Dj%2Cyt%3Di%2Cx%29%5Calpha%7Bt-1%7D%28j%29%5C%5C%0A%5CDeltar%3D%5Cbeta_t%28i%29%3D%5Csum%5Climits%7Bj%5Cin%20S%7D%5Cphi%7Bt%2B1%7D%28y_t%3Di%2Cy%7Bt%2B1%7D%3Dj%2Cx%29%5Cbeta_%7Bt%2B1%7D%28j%29%0A&height=79&width=643#crop=0&crop=0&crop=1&crop=1&id=DUQOd&originHeight=111&originWidth=900&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)
  3. 学习
    23.Cheatsheet - 图90-%5Csum%5Climits%7By%7Bt-1%7D%7D%5Csum%5Climits%7By_t%7Dp(y%7Bt-1%7D%2Cyt%7Cx%5Ei)f(y%7Bt-1%7D%2Cyt%2Cx%5Ei)%5D%0A#card=math&code=%5Cnabla%5Clambda%20L%3D%5Csum%5Climits%7Bi%3D1%7D%5EN%5Csum%5Climits%7Bt%3D1%7D%5ET%5Bf%28y%7Bt-1%7D%2Cy_t%2Cx%5Ei%29-%5Csum%5Climits%7By%7Bt-1%7D%7D%5Csum%5Climits%7Byt%7Dp%28y%7Bt-1%7D%2Cyt%7Cx%5Ei%29f%28y%7Bt-1%7D%2Cy_t%2Cx%5Ei%29%5D%0A&height=50&width=424#crop=0&crop=0&crop=1&crop=1&id=QG1Wz&originHeight=71&originWidth=594&originalType=binary&ratio=1&rotation=0&showTitle=false&status=done&style=none&title=)