在使用GF、WS变异算子进行变异的时候,不再随机选择层,而是固定对最后一层进行变异,观察生成的结果的分布。
生成了svhn_lenet5的变异模型共4000个,每个变异算子2000个,看如此能否填平分布图里的沟沟,改善svhn_lenet5的效果。
❌当利用WS、GF算子只对最后一层进行扰动的时候,2000个WS的变异模型去重之后只剩5个,2000个GF的变异模型去重之后也只剩5个
结合之前细化模型的表现,怀疑利用GF和WS变异算子生成的变异模型的精度都极其接近原模型
利用第三批生成的变异模型分别对WS、GF、NAI、NEB算子生成的模型进行统计
发现利用GF和WS算子扰动生成的模型相较于NAI、NEB算子扰动生成的模型与原模型的精度差异要小得多
扰动程度排序:NAI>NEB>WS>GF(下表是基于4000个模型统计的结果)
unique | sum | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | |||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
mnist_lenet1 | GF | 255 | 255 | 255 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 853 | 614 | 23 | 106 | 72 | 56 | 47 | 44 | 34 | 21 | 15 | 30 | 32 | 14 | 9 | 14 | 8 | 16 | 21 | 15 | 19 | 18 | ||
NAI | 450 | 106 | 1 | 2 | 4 | 8 | 9 | 11 | 5 | 7 | 7 | 7 | 1 | 5 | 5 | 7 | 7 | 4 | 4 | 2 | 1 | 9 | ||
NEB | 465 | 227 | 1 | 7 | 13 | 23 | 19 | 21 | 18 | 11 | 10 | 10 | 13 | 7 | 9 | 8 | 7 | 3 | 6 | 9 | 14 | 18 | ||
mnist_lenet4 | GF | 428 | 428 | 428 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 875 | 649 | 17 | 97 | 92 | 65 | 57 | 43 | 33 | 20 | 26 | 33 | 21 | 26 | 19 | 15 | 16 | 11 | 12 | 12 | 16 | 18 | ||
NAI | 636 | 362 | 1 | 29 | 48 | 38 | 27 | 27 | 18 | 17 | 22 | 20 | 14 | 15 | 13 | 14 | 9 | 12 | 13 | 8 | 6 | 11 | ||
NEB | 630 | 448 | 5 | 48 | 65 | 55 | 39 | 35 | 20 | 25 | 12 | 24 | 14 | 17 | 12 | 9 | 13 | 8 | 5 | 10 | 11 | 21 | ||
mnist_lenet5 | GF | 556 | 556 | 556 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 955 | 810 | 174 | 173 | 109 | 53 | 35 | 30 | 22 | 19 | 19 | 39 | 17 | 14 | 17 | 15 | 9 | 13 | 9 | 8 | 10 | 25 | ||
NAI | 753 | 433 | 44 | 89 | 40 | 24 | 25 | 29 | 22 | 18 | 20 | 14 | 18 | 10 | 15 | 14 | 6 | 10 | 15 | 5 | 7 | 8 | ||
NEB | 765 | 653 | 92 | 163 | 107 | 64 | 51 | 29 | 21 | 17 | 21 | 17 | 14 | 5 | 5 | 7 | 8 | 5 | 4 | 4 | 6 | 13 | ||
cifar10_vgg16 | GF | 567 | 567 | 304 | 129 | 66 | 26 | 31 | 9 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 917 | 453 | 0 | 0 | 2 | 5 | 13 | 49 | 26 | 26 | 39 | 41 | 36 | 23 | 49 | 21 | 24 | 22 | 23 | 24 | 19 | 20 | ||
NAI | 770 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
NEB | 997 | 731 | 3 | 34 | 33 | 41 | 53 | 73 | 60 | 36 | 48 | 41 | 28 | 41 | 49 | 40 | 27 | 30 | 33 | 30 | 20 | 10 | ||
cifar10_resnet20 | GF | 520 | 519 | 501 | 15 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 537 | 504 | 3 | 15 | 38 | 52 | 69 | 62 | 50 | 36 | 37 | 36 | 23 | 21 | 16 | 9 | 2 | 10 | 7 | 7 | 6 | 5 | ||
NAI | 980 | 282 | 2 | 2 | 4 | 13 | 14 | 24 | 15 | 16 | 24 | 22 | 10 | 21 | 24 | 12 | 19 | 13 | 11 | 15 | 13 | 8 | ||
NEB | 984 | 799 | 2 | 25 | 52 | 61 | 82 | 87 | 69 | 70 | 52 | 48 | 53 | 34 | 26 | 20 | 26 | 22 | 17 | 20 | 21 | 12 | ||
fashion_lenet5 | GF | 484 | 484 | 484 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 952 | 619 | 10 | 6 | 12 | 33 | 60 | 39 | 32 | 34 | 37 | 53 | 45 | 31 | 33 | 23 | 30 | 33 | 27 | 23 | 20 | 38 | ||
NAI | 1000 | 289 | 0 | 0 | 0 | 2 | 2 | 11 | 24 | 28 | 28 | 24 | 21 | 21 | 20 | 12 | 15 | 17 | 16 | 14 | 18 | 16 | ||
NEB | 1000 | 923 | 0 | 7 | 40 | 92 | 85 | 78 | 77 | 71 | 66 | 59 | 63 | 42 | 48 | 32 | 34 | 31 | 30 | 25 | 30 | 13 | ||
svhn_lenet5 | GF | 477 | 477 | 7 | 150 | 218 | 75 | 17 | 6 | 4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
WS | 928 | 120 | 13 | 4 | 0 | 0 | 0 | 0 | 0 | 3 | 0 | 0 | 0 | 3 | 10 | 7 | 17 | 6 | 20 | 15 | 10 | 12 | ||
NAI | 1000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ||
NEB | 1000 | 210 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 4 | 6 | 10 | 11 | 21 | 24 | 19 | 26 | 20 | 21 | 22 | 25 |
分析表格可知,一般情况下,之前我们所使用的选择变异模型的规则选出来的模型
1.那些精度与原模型很接近的模型几乎都是利用GF算子生成的
2.将ratio从固定的几个值调整为一个范围可能可以避免生成较多重复的模型
3.根据模型的大小和变异算子(NAI调低,GF调高)需要适应性地调整ratio,如果真的选择与原模型预测结果差异在200-2000之内的模型,例如mnist_lenet1和svhn_lenet5的变异ratio就要调低,就目前的数据来看,生成很多变异程度较大的无用模型