1.Summary

  This is a survey of transfer learning, which describes some methods of transfer learning. Transfer learning is to apply the knowledge or patterns learned in a certain field or task to different but related fields or problems. That is, knowledge or patterns are migrated from the source domain to the target domain. Transfer learning is usually used when the amount of data in the source domain is large and the amount of data in the target domain is small.<br />        This paper describes four types of transfer learning methods: (1) Instances-based deep transfer learning, (2) Mapping-based deep transfer learning, (3) Network-based deep transfer learning, (4) Adversarial-based deep transfer learning. Instances-based deep transfer learning is instance based. First remove the instances in the source domain that are completely inconsistent with the target domain, then weight the remaining instances in the source domain, and finally use the weighted instances in the source domain and the instances in the target domain to train the model. Mapping-based deep transfer learning is to map instances from the source domain and target domain to the new data space, so that they are as similar as possible in the new data space. Network-based deep transfer learning is to transform the trained part of the network in the source domain into a part of the deep neural network in the target domain, and continue to train the whole model with a small number of data in the target domain. Adversarial-based deep transfer learning uses Gan's idea to embedding the features of source domain and target domain with generative model, and then classifies these embeddings with discriminative model (the labeled is from source domain or target domain). The goal is to make the effect of discriminative model worse and worse, and the embeddings of the source domain and the target domain are more and more similar.<br />        In order to let me more understand this paper, I should study some methods, models or concepts mentioned in the paper, such as TCA and GAN. I may also should read other transfer learning review articles or some papers that use the four methods mentioned above.

2.你对于论文的思考

  这是一篇关于深度学习中的迁移学习的综述，文章中总结的迁移学习的方法比较完备，对于每一种方法，也都列举出了一些相关的文章，但是文章中对各方法的描述比较简单，许多内容没有展开说，并且文章的侧重点在于列举迁移学习的各种方法，并没有对比这几种方法的优劣。<br />        这篇文章还是比较适合新手入门阅读的，可以快速了解迁移学习大致是干什么的、有什么方法能使用，一千多的高引用量也表明这篇文章还是不错的。

3. 其他

迁移学习的学习过程

迁移学习就是要将某个领域或任务上学习到的知识或模式应用到不同但相关的领域或问题中，如图上所示，迁移学习把源域中的知识迁移到了目标域中。

深度迁移学习（ Deep Transfer Learning ）方法

1.Instances-based deep transfer learning

先使用TrAdaBoost过滤掉源域中与目标域不同的实例，再对源域剩下的实例重新加权以构成类似于目标域的分布，最后通过使用来自源域的重新加权实例和来自目标域的原始实例来训练模型。

2.Mapping-based deep transfer learning

使用了TCA的思想，将源域和目标域都映射到新的数据空间，虽然源域和目标域之间存在差异，但是映射完成后，它们会变得更相似。用MMD（最大均值差异）来度量源域和目标域之间的距离，最终的损失函数为：

3.Network-based deep transfer learning

这种方法是把源域中前面的几层网络（已经用源域的大量数据训练好的）迁移给了目标域对应位置的网络。可以把源域中除了输出层之外的前面所有网络都原封不动迁移给目标域，这些网络中的参数都不再更新，只需要在这些网络层后面加一个输出层就好了，拼接好后，用目标域为数不多的数据来训练输出层中的参数；也可以选取源域中前面的几层网络迁移给目标域，再在后面添加一些隐藏层和一个输出层，再用目标域为数不多的数据来训练整个网络中的参数。

4.Adversarial-based deep transfer learning

利用了GAN的思想，使用生成模型对源域和目标域的特征进行表征，再利用判别模型对这些表征进行分类（标签为来自源域或目标域），最终目标是让判别模型的效果越来越差，即使得经过生成模型表征后的源域和目标域的特征越来越相似。
上图中先是分别用源域和目标域的数据训练黑色部分的网络结构，标签都是这数据原本标签，这相当于在做特征提取，最后一层隐藏层就是提取完的特征；然后开始训练蓝色部分的网络结构，这一部分分为生成模型和判别模型，先是将之前提取好的特征作为生成模型的输入，即把源域和目标域的特征重新表征，然后把这些表征输入到判别模型中，让判别模型判断它们是来自源域还是目标域。如果最后生成的这些表征足够相似，那么判别模型的效果会非常差，那么目的就达到了，此时源域数据以后成功迁移了。

A Survey on Deep Transfer Learning