- 分享主题:Masked-field Pre-training
- 论文标题:Masked-field Pre-training for User Intent Prediction
- 论文链接:https://dl.acm.org/doi/pdf/10.1145/3340531.3412726
- 分享人:唐共勇

1. Summary

【必写】,推荐使用 grammarly 检查语法问题,尽量参考论文 introduction 的写作方式。需要写出

  1. 这篇文章解决了什么问题?
  2. 作者使用了什么方法(不用太细节)来解决了这个问题?
  3. 你觉得你需要继续去研究哪些概念才会加深你对这篇文章的理解?

This article deals with the problem of user intention prediction and proposes a masked field pre-training framework The author named the proposed framework fi trans. In pre-training, use a large amount of unlabeled data to learn feature interaction patterns. The framework masks some data and uses other data to predict the masked part as a pre-training task, which is similar to the mask mechanism in Bert. When the pre-trained model is used to predict user intention, the problem of sparsely labeled data is solved to a certain extent, and a very good effect is achieved.

2. 你对于论文的思考

需要写出你自己对于论文的思考,例如优缺点,你的takeaways

优点:

  1. 针对用户意图预测提出了一套预训练框架,能够解决用户意图预测中标签数据过少,特征矩阵稀疏的情况
  2. 可以作为多标签分类任务的一种新的思路,通过预训练解决数据不足问题

缺点:

  1. 预训练开销过大,需要较好的配置
  2. 本质是将bert里的mask机制引入,创新不足

    3. 其他

    【可选】

模型结构
image.png
框架结构
image.png
实验结果
image.png