ACL2020_PretrainLanguageModel

预训练语言模型及部分应用

QuASE: Question-Answer Driven Sentence Encoding

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.772.pdf
  - 代码链接：https://github.com/CogComp/QuASE

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.745.pdf
  - 代码链接：https://github.com/facebookresearch/TaBERT

Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.740.pdf
  - 代码链接：https://github.com/allenai/dont-stop-pretraining

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.703.pdf
  - 代码链接：https://github.com/pytorch/fairseq

Toward Better Storylines with Sentence-Level Language Models

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.666.pdf
  - 代码链接：https://github.com/google-research/google-research/tree/master/better_storylines

tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.630.pdf
  - 代码链接：https://github.com/wuningxi/tBERT

FastBERT: a Self-distilling BERT with Adaptive Inference Time

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.537.pdf
  - 代码链接：https://github.com/autoliuweijie/FastBERT

Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.439.pdf
  - 代码链接：https://github.com/google-research/language/tree/master/language/conpono

DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.411.pdf
  - 代码链接：https://github.com/StonyBrookNLP/deformer

Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.315.pdf
  - 代码链接：https://github.com/lsvih/MWA

Span Selection Pre-training for Question Answering

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.247.pdf
  - 代码链接：https://github.com/IBM/span-selection-pretraining

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.204.pdf
  - 代码链接：https://github.com/castorini/DeeBERT

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.195.pdf
  - 代码链接：https://github.com/google-research/google-research/tree/master/mobilebert

Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.76.pdf
  - 代码链接：https://github.com/joongbo/tta

Few-Shot NLG with Pre-Trained Language Model

• 论文链接：https://www.aclweb.org/anthology/2020.acl-main.18.pdf
  - 代码链接：https://github.com/czyssrs/Few-Shot-NLG