• 先验分布：关于参数 贝叶斯推断 - 图1 的先验估计

贝叶斯推断 - 图2%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cright%29%0A)

  • 后验分布：在给出数据 贝叶斯推断 - 图3 后，关于参数 贝叶斯推断 - 图4 的估计

贝叶斯推断 - 图5%20%3D%5Cfrac%7Bp%5Cleft(%20%5Ctheta%20%2Cx%20%5Cright)%7D%7Bp%5Cleft(%20x%20%5Cright)%7D%3D%5Cfrac%7Bp%5Cleft(%20x%5Cmid%20%5Ctheta%20%5Cright)%20%5Ccdot%20p%5Cleft(%20%5Ctheta%20%5Cright)%7D%7B%5Cint%7B%5Ctheta%7D%7Bp%5Cleft(%20x%5Cmid%20%5Ctheta%20%5Cright)%20%5Ccdot%20p%5Cleft(%20%5Ctheta%20%5Cright)%20%5Cmathrm%7Bd%7D%5Ctheta%7D%7D%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cmid%20x%20%5Cright%29%20%3D%5Cfrac%7Bp%5Cleft%28%20%5Ctheta%20%2Cx%20%5Cright%29%7D%7Bp%5Cleft%28%20x%20%5Cright%29%7D%3D%5Cfrac%7Bp%5Cleft%28%20x%5Cmid%20%5Ctheta%20%5Cright%29%20%5Ccdot%20p%5Cleft%28%20%5Ctheta%20%5Cright%29%7D%7B%5Cint%7B%5Ctheta%7D%7Bp%5Cleft%28%20x%5Cmid%20%5Ctheta%20%5Cright%29%20%5Ccdot%20p%5Cleft%28%20%5Ctheta%20%5Cright%29%20%5Cmathrm%7Bd%7D%5Ctheta%7D%7D%0A)

例子

已知盒内4球，有红白两色，取一次后放回，连取4次，有3红1白

我们设盒内红球有 贝叶斯推断 - 图6 个，白球有 贝叶斯推断 - 图7 个，其中 贝叶斯推断 - 图8.

设取出红球个数为 贝叶斯推断 - 图9 个，那么有：

贝叶斯推断 - 图10%20%3DC%7B4%7D%5E%7Bx%7D%5Ccdot%20%5Cleft(%20%5Cfrac%7B%5Ctheta%7D%7B4%7D%20%5Cright)%20%5Ex%5Cleft(%20%5Cfrac%7B4-%5Ctheta%7D%7B4%7D%20%5Cright)%20%5E%7B4-x%7D%0A#card=math&code=p%5Cleft%28%20x%5Cmid%20%5Ctheta%20%5Cright%29%20%3DC%7B4%7D%5E%7Bx%7D%5Ccdot%20%5Cleft%28%20%5Cfrac%7B%5Ctheta%7D%7B4%7D%20%5Cright%29%20%5Ex%5Cleft%28%20%5Cfrac%7B4-%5Ctheta%7D%7B4%7D%20%5Cright%29%20%5E%7B4-x%7D%0A)

如果 贝叶斯推断 - 图11 ，那么：

贝叶斯推断 - 图12%20%3D%5Cfrac%7B1%7D%7B64%7D%5Ctheta%20%5E3%5Cleft(%204-%5Ctheta%20%5Cright)%0A#card=math&code=p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%5Cright%29%20%3D%5Cfrac%7B1%7D%7B64%7D%5Ctheta%20%5E3%5Cleft%28%204-%5Ctheta%20%5Cright%29%0A)

对于不同的 贝叶斯推断 - 图13 ，有：

贝叶斯推断 - 图14%20%26%3D0%5C%5C%0A%09p%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%3D1%20%5Cright)%20%26%3D%5Cfrac%7B3%7D%7B64%7D%5C%5C%0A%09p%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%3D2%20%5Cright)%20%26%3D%5Cfrac%7B16%7D%7B64%7D%5C%5C%0A%09p%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%3D3%20%5Cright)%20%26%3D%5Cfrac%7B27%7D%7B64%7D%5C%5C%0A%09p%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%3D4%20%5Cright)%20%26%3D0%5C%5C%0A%5Cend%7Baligned%7D%0A#card=math&code=%5Cbegin%7Baligned%7D%0A%09p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%3D0%20%5Cright%29%20%26%3D0%5C%5C%0A%09p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%3D1%20%5Cright%29%20%26%3D%5Cfrac%7B3%7D%7B64%7D%5C%5C%0A%09p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%3D2%20%5Cright%29%20%26%3D%5Cfrac%7B16%7D%7B64%7D%5C%5C%0A%09p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%3D3%20%5Cright%29%20%26%3D%5Cfrac%7B27%7D%7B64%7D%5C%5C%0A%09p%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%3D4%20%5Cright%29%20%26%3D0%5C%5C%0A%5Cend%7Baligned%7D%0A)

由贝叶斯公式可知：

贝叶斯推断 - 图15%20%3D%5Cfrac%7Bp%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%5Cright)%20%5Ccdot%20p%5Cleft(%20%5Ctheta%20%5Cright)%7D%7B%5Csum%7B%5Ctheta%20%3D0%7D%5E4%7Bp%5Cleft(%20x%3D3%5Cmid%20%5Ctheta%20%5Cright)%20%5Ccdot%20p%5Cleft(%20%5Ctheta%20%5Cright)%7D%7D%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cmid%20x%3D3%20%5Cright%29%20%3D%5Cfrac%7Bp%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%5Cright%29%20%5Ccdot%20p%5Cleft%28%20%5Ctheta%20%5Cright%29%7D%7B%5Csum%7B%5Ctheta%20%3D0%7D%5E4%7Bp%5Cleft%28%20x%3D3%5Cmid%20%5Ctheta%20%5Cright%29%20%5Ccdot%20p%5Cleft%28%20%5Ctheta%20%5Cright%29%7D%7D%0A)

如果按正常情况，我们认为 $\theta $ 是服从均匀分布的，即 贝叶斯推断 - 图16%3D1%2F5%2C%5C%2C(i%3D0%2C%5Cldots%2C4)#card=math&code=p%28%5Ctheta%3Di%29%3D1%2F5%2C%5C%2C%28i%3D0%2C%5Cldots%2C4%29).

那么

贝叶斯推断 - 图17%20%3D%5Cbegin%7Bcases%7D%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%200%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B3%7D%7B46%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B16%7D%7B46%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B27%7D%7B46%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%200%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cright%29%20%3D%5Cbegin%7Bcases%7D%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%200%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B3%7D%7B46%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B16%7D%7B46%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%20%5Cfrac%7B27%7D%7B46%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%09%5Cfrac%7B1%7D%7B5%7D%5Crightarrow%200%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A)

贝叶斯推断 - 图18

但我们也可以认为：盒子里没有红球和白球的概率是极小的，大概率是红球白球一样多，于是我们可以认为 贝叶斯推断 - 图19#card=math&code=p%28%5Ctheta%20%29) 的先验分布为：

贝叶斯推断 - 图20%20%3D%5Cbegin%7Bcases%7D%0A%090%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B2%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%090%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cright%29%20%3D%5Cbegin%7Bcases%7D%0A%090%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B2%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%090%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A)

那么我们在此先验分布之上建立的后验分布为：

贝叶斯推断 - 图21%20%3D%5Cbegin%7Bcases%7D%0A%090%5Crightarrow%200%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%5Crightarrow%20%5Cfrac%7B3%7D%7B62%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B2%7D%5Crightarrow%20%5Cfrac%7B32%7D%7B62%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%5Crightarrow%20%5Cfrac%7B27%7D%7B62%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%090%5Crightarrow%200%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A#card=math&code=p%5Cleft%28%20%5Ctheta%20%5Cright%29%20%3D%5Cbegin%7Bcases%7D%0A%090%5Crightarrow%200%2C%5Cqquad%20i%3D0%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%5Crightarrow%20%5Cfrac%7B3%7D%7B62%7D%2C%5Cqquad%20i%3D1%5C%5C%0A%09%5Cfrac%7B1%7D%7B2%7D%5Crightarrow%20%5Cfrac%7B32%7D%7B62%7D%2C%5Cqquad%20i%3D2%5C%5C%0A%09%5Cfrac%7B1%7D%7B4%7D%5Crightarrow%20%5Cfrac%7B27%7D%7B62%7D%2C%5Cqquad%20i%3D3%5C%5C%0A%090%5Crightarrow%200%2C%5Cqquad%20i%3D4%5C%5C%0A%5Cend%7Bcases%7D%0A)

贝叶斯推断 - 图22

观察这张图其实很有意思，尽管摸出了3个红球，但后验分布还是认为有两个红球的概率最大，和我们日常经验更为趋近——即使2红2白也是有可能摸出3个红球的，一次的发生可能是偶然因素。

绘图的MATLAB代码为：

  1. clc,clear
  2. x=0:4;
  3. y=[1/5,1/5,1/5,1/5,1/5];
  4. y1=[0,3/46,16/46,27/46,0];
  5. y2=[0,1/4,1/2,1/4,0];
  6. y3=[0,3/62,32/62,27/62,0];
  8. figure
  9. hold on
  10. area(x,y,'FaceColor',[0.7 0.7 0.7],'EdgeColor','k','FaceAlpha',0.5)
  11. area(x,y1,'FaceColor',[0.5 0.9 0.6],'EdgeColor',[0 0.5 0.1],'FaceAlpha',0.5)
  12. legend('Prior ','Posterior')
  14. figure
  15. hold on
  16. area(x,y2,'FaceColor',[0.7 0.7 0.7],'EdgeColor','k','FaceAlpha',0.5)
  17. area(x,y3,'FaceColor',[0.5 0.9 0.6],'EdgeColor',[0 0.5 0.1],'FaceAlpha',0.5)
  18. legend('Prior ','Posterior')