EMNLP 2020

Selections from EMNLP 2020

This contains a selection of papers from EMNLP 2020. Some papers are missing — mainly papers on the following topics: Emotion Recognition/Detection, Aspect-Based Sentiment Analysis, Task-oriented/goal-oriented dialogue, Morphology, Word Segmentation, Spelling Correction, Speech/Spoken Language Understanding.

Schedule: https://2020.emnlp.org/schedule

EMNLP papers: https://www.aclweb.org/anthology/events/emnlp-2020/

Findings of EMNLP 2020: https://www.aclweb.org/anthology/volumes/2020.findings-emnlp/

CoNLL 2020: https://www.aclweb.org/anthology/events/conll-2020/

• General Remarks
• Tutorials
• Workshops
• Keynotes
• System Demonstrations

Topics

• Methodological
• Datasets, Pretrained models and Other Resources
• Parsing
• Syntax
• Semantics
  • Lexical Semantics
  • Word Sense Disambiguation
  • Sentence-level Semantics
• Pragmatics
• Discourse
• Argumentation
• Figurative Language
• Artifacts and Spurious Correlations
• Biases
  • Gender Bias
• Explainability
• Adversarial ML
  • Adversarial attacks
  • Adversarial training
• Commonsense Reasoning and Question Answering
  • Visual Question Answering
  • Knowledge and reasoning
• Learning with Less Data
  • Active Learning
  • Data Augmentation
  • Domain Adaptation
  • Few-shot Learning
  • Zero-shot Learning
• Language Models and Transformers
• Embeddings
• Cognitive Models and Psycholinguistics
• Compositionality
• Language Grounding
• Multimodal
• Multilingual

NLG papers

• Dialogue
• Sequence to Sequence
• Text Generation
  • Open-ended text generation
  • Meaning-to-text generation
  • Data-to-text generation
• OCR post-correction
• Summarization
• Simplification
• Machine Translation
• Style Transfer
• Paraphrasing

NLU papers

• Text Classification
• Information Retrieval
• Sequence Labeling
• POS tagging
• Entity Recognition, Linking and Representation
  • NER
  • Entity Linking
  • Entity Representation
  • Entity Set Expansion
• Coreference
• Relation Extraction
• Events
• Open Information Extraction
• Semantic Role Labeling
• Semantic Parsing
  • Text to SQL
  • AMR Parsing
• Topic Models
• Fact Checking
• Knowledge Graphs
• Natural Language Inference
• Emergent Language in Multi-Agent Communication

Applications

• Cybersecurity
• Language and Code
• Pandemics and other crises
• Clinical and Biomedical
• Mental Health
• Scientific Text Mining
• Essay Scoring
• News Media
• Social Media
• Literature and Narrative
• Movies
• Dictionaries
• Grammatical Error Correction
• Finance
• Law
• Customer Service
• Hate speech and offensive language
• Other applications
• Other
• More Findings of EMNLP papers

General Remarks

• The use of GatherTown for poster sessions was a great improvement over ACL 2020, particularly for the conference to feel more like a real conference. However, it was extremely glitchy (disconnecting every few minutes), and so my experience with it was very frustrating.
• The “Birds of a Feather” events have too many people in them. Breakout rooms are a good idea, but were not always used.
• The Zoom QA sessions should automatically turn everyone’s microphone’s by default.

Best paper: ++ Digital Voicing of Silent Speech. David Gaddy, Dan Klein (paper) (code) (data) (predicted audio samples)

Includes an interesting application of CCA (which improved performance substantially).

Tutorials

• Interpreting Predictions of NLP Models (Eric Wallace, Matt Gardner, Sameer Singh) (website) (video) (slides)

I. Overview

Why interpretability? Validation metrics are not enough! What else might you want: find errors, bugs, undesirable behavior in models and errors and bugs in data (isn’t this just good science and engineering?) Do inherently interpretable models generalize better? (see Gahdanu et al., 2018, Bogin et al., 2020, on CLEVR dataset]. Also: uncover patterns that models find, for scientific discovery (e.g., happened for alpha Go… has not happened so much for language yet).

Interpretability methods:

1. Probing internal representations
2. Testing model behavior using challenge sets, diagnostic sets, etc. Does a model capture some phenomena (lexical semantics, syntax, predicate-argument structure, logic, knowledge, etc.)
3. Baking interpretability into the model. For example: generating explanations as well as making predictions (but this is not inherently interpretable).
4. Looking at input features (e.g., rationales, saliency maps).
5. Looking for global decision rules (e.g., Ribeiro et al., High-Precision Model-Agnostic Explanations, 2018 or Wallace et al., Universal Adversarial Triggers for Attacking and Analyzing NLP, 2019).
6. Looking at training examples (e.g., influence functions).

The ACL 2020 tutorial on interpretability covers 1. and 2. “Baking interpretability into the model — does not have comprehensive resources”. This tutorial: 4., 5., 6.

Desirable properties: model-agnostic, fast and easy to compute, and faithful to underlying model (lots of caveats here!)

II. What Parts of an input led to a prediction?

II.1. Saliency maps (gradient and perturbation based methods)
Goal: compute relative importance of each token in the input — if you change or remove the token, how much is the prediction affected?

II.1.1 Saliency maps via input gradients. Straightforward. For NLP, this is the “derivative of output of model wrt an input probability”. But many options: what to use as output? (loss, top prediction probability.. and what do you do with multiple outputs, in say generation or tagging problems). How to turn gradient into a scalar score? (Lp norm? Dot product with embedding itself?) People have mostly agreed on \grad_e(t) L_y \dot e(t). (see Han et al., 2020 “Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions” https://arxiv.org/abs/2005.06676 ). What is the intuition behind the dot product of the embedding? Takes sign and magnitude of embedding into account.

Some problems: too local, and thus sensitive to slight perturbations; saturated outputs lead to unintuitive gradients; discontinuous gradients are problematic. These are not NLP-specific problems. How to fix? Don’t rely on single gradient calculation: SmoothGrad, Integrated Gradients (other approaches not covered: LRP, DeepLIFT, GradCAM… lots of work in computer vision).

SmoothGrad: add Gaussian noise to input and average the gradient. Integrated Gradients: average gradients along path from zero to input.

Problems with gradient-based saliency methods: need gradient access of the model, not “customizable”, difficult to apply to non-classification tasks.

II.1.2. Saliency maps via input perturbations

Same goal as gradient-based methods, but these are black box (model-agnostic). They are more flexible/customizable (can define what perturbations/neighborhoods we care about, different units of explanation (words, phrases, sentences), etc.), can we enforce constraints like grammaticality?

A simple approach: leave-one-out: remove tokens, and look at drop in prediction confidence (Li et al., Understanding Neural Networks through Representation Erasure 2017 https://arxiv.org/abs/1612.08220). Problem: “The movie is mediocre, maybe even bad“. Removing both words at once has a bigger effect than removing each word individually.

Another approach: LIME. Look at model’s predictions for nearby inputs; closer points are more important than further points. Fit a linear classifier, and use its weights as feature importances. Problems with LIME and leave-one-out: how to define “distance” between sentences? Hard for users to understand different explanations with different perturbations. Generating interpretations is expensive (many calls to the underlying model).

Bigger problem with saliency maps: linear representations can be quite limited. “Saliency maps are often not very actionable (in terms of finding artifacts)”.

II. 2. Perturbations as explanations (input reduction and adversarial perturbations).

II.2.1. Input reduction

II.2.2. Adversarial perturbations

• Fact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era
  (Preslav Nakov, Giovanni Da San Martino) (website) (video)

x

• The Amazing World of Neural Language Generation (Yangfeng Ji, Antoine Bosselut, Thomas Wolf, Asli Celikyilmaz) (video) (slides)

x

• Machine Reasoning for Knowledgeable, Explainable and Inferable Models (Nan Duan, Duyu Tang, Ming Zhou) (video)

• High Performance Natural Language Processing (Gabriel Ilharco, Cesar Ilharco, Iulia Turc, Tim Dettmers, Felipe Ferreira, Kenton Lee) (video) (slides)

x

• Representation, Learning and Reasoning on Spatial Language for Down-stream NLP Tasks (Parisa Kordjamshidi, James Pustejovsky, Marie-Francine Moens
  )
• Simultaneous Translation (Liang Huang, Colin Cherry, Mingbo Ma, Naveen Arivazhagan, Zhongjun He) (video)

Workshops

Full list of workshops: https://2020.emnlp.org/workshops

• Workshop on Insights from Negative Results in NLP
• BlackboxNLP 2020: Analyzing and interpreting neural networks for NLP
• Evaluation and Comparison of NLP Systems (Eval4NLP)
• SustaiNLP: Workshop on Simple and Efficient Natural Language Processing (Promoting computationally efficient NLP and justified model complexity. Encouraging conceptual NLP novelty, discouraging ‘more compute’”)
• CoNLL 2020
• Deep Learning Inside Out (DeeLIO): The First Workshop on Knowledge Extraction and Integration for DL Architectures (focus on “low-data regimes”) See Eduard Hovy’s fantastic talk, “Knowledge in Deep Networks”
• 4th Workshop on Structured Prediction for NLP
• Interactive and Executable Semantic Parsing (Int-Ex)
• 6th Workshop on Noisy User-generated Text (W-NUT 2020)
• NLP and Computational Social Science (NLP+CSS)
• Second Workshop for NLP Open Source Software (NLP-OSS)
• PrivateNLP 2020: The First Workshop on Privacy in NLP

Keynotes

1. Claire Cardie, Information Extraction Through the Years: How Did We Get Here? (video)

See also: Ralph Grishman, 25 Years of Information Extraction and Ehud Reiter’s blog post on the topic.

1. Rich Caruana, Friends Don’t Let Friends Deploy Black-Box Models: The Importance of Intelligibility in Machine Learning (video)
2. Janet Pierrehumbert, Linguistic Behaviour and the Realistic Testing of NLP Systems (video)

System Demonstrations

Topics

Methodological

See also: the papers in Evaluation and Comparison of NLP Systems (Eval4NLP) and the Workshop on Insights from Negative Results in NLP. Here are some methodological papers from EMNLP 2019 and ACL 2020.

++ Utility is in the Eye of the User: A Critique of NLP Leaderboards. Kawin Ethayarajh and Dan Jurafsky. (paper)

See also Anna Rogers’ blog post How the transformers broke NLP leaderboards

A Matter of Framing: The Impact of Linguistic Formalism on Probing Results. Ilia Kuznetsov and Iryna Gurevych. (paper) (code)

Precise Task Formalization Matters in Winograd Schema Evaluations. Haokun Liu, William Huang, Dhara Mungra and Samuel R. Bowman. (paper) (code)

Is the Best Better? Bayesian Statistical Model Comparison for Natural Language Processing. Piotr Szymański and Kyle Gorman. (paper)

With Little Power Comes Great Responsibility. Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald and Dan Jurafsky. (paper) (code)

Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction!. Bruno Taillé, Vincent Guigue, Geoffrey Scoutheeten and patrick Gallinari. (paper) (code)

(paper) (code)

Datasets, pretrained models and other resources

MAVEN: A Massive General Domain Event Detection Dataset. Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin and Jie Zhou. (paper) (code)

Multi/cross-lingual:

XGLUE: A New Benchmark Datasetfor Cross-lingual Pre-training, Understanding and Generation. Yaobo Liang et al. (paper) (code)

In languages other than English:

See also: mT5 (101 languages) and mBART (25 languages) and OSCAR.

CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French. AmirAli Bagher Zadeh, Yansheng Cao, Simon Hessner, Paul Pu Liang, Soujanya Poria and Louis-Philippe Morency. (paper) (code)

An Empirical Study of Pre-trained Transformers for Arabic Information Extraction. Wuwei Lan, Yang Chen, Wei Xu and Alan Ritter. (paper) (code)

(Findings) IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages. Divyanshu Kakwani et al. (paper) (code)

(Findings) RobBERT: a Dutch RoBERTa-based Language Model. Pieter Delobelle, Thomas Winters and Bettina Berendt. (paper) (code)

(Findings) Revisiting Pre-Trained Models for Chinese Natural Language Processing. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang and Guoping Hu. (paper) (code)

(Findings) PhoBERT: Pre-trained language models for Vietnamese. Dat Quoc Nguyen and Anh Tuan Nguyen. (paper) (code)

RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark. Tatiana Shavrina et al. (paper)

Parsing

(paper) (code)

(TACL) Consistent Unsupervised Estimators for Anchored PCFGs. Alexander Clark, Nathanaël Fijalkow (paper) (code)

Some Languages Seem Easier to Parse Because Their Treebanks Leak. Anders Søgaard. (paper) (code)

(TACL) The Return of Lexical Dependencies: Neural Lexicalized PCFGs (paper) (code)

(CL) Efficient Outside Computation. Daniel Gildea (paper)

(Findings) Rethinking Self-Attention: Towards Interpretability in Neural Parsing. Khalil Mrini et al. (paper) (code)

Constituency parsing:

(Findings) Improving Constituency Parsing with Span Attention. Yuanhe Tian, Yan Song, Fei Xia and Tong Zhang. (preprint) (paper) (code)

Span-based discontinuous constituency parsing: a family of exact chart-based algorithms with time complexities from O(n^6) down to O(n^3). Caio Corro. (preprint) (paper) (code)

Discontinuous Constituent Parsing as Sequence Labeling. David Vilares and Carlos Gómez-Rodríguez. (paper) (code)

Unsupervised Constituency parsing:

++ Unsupervised Parsing via Constituency Tests. Steven Cao, Nikita Kitaev and Dan Klein. (paper) (code)

On the Role of Supervision in Unsupervised Constituency Parsing. Haoyue Shi, Karen Livescu and Kevin Gimpel. (paper) (code)

Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders. Andrew Drozdov, Subendhu Rongali, Yi-Pei Chen, Tim O’Gorman, Mohit Iyyer and Andrew McCallum. (paper) (code)

Dependency parsing:

(Findings+WS) Graph-to-Graph Transformer for Transition-based Dependency Parsing. Alireza Mohammadshahi and James Henderson. (paper) (code)

Please Mind the Root: Decoding Arborescences for Dependency Parsing. Ran Zmigrod, Tim Vieira and Ryan Cotterell. (paper) (code)

(Findings+WS) Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank. Ethan C. Chau, Lucy H. Lin and Noah A. Smith. (paper) (code)

Cross-lingual dependency parsing:

UDapter: Language Adaptation for Truly Universal Dependency Parsing. Ahmet Üstün, Arianna Bisazza, Gosse Bouma and Gertjan van Noord. (paper) (code)

(Findings) Cross-Lingual Dependency Parsing by POS-Guided Word Reordering. Lu Liu, Yi Zhou, Jianhan Xu, Xiaoqing Zheng, Kai-Wei Chang and Xuanjing Huang. (paper) (code)

(Findings) Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders. Zhao Li and Kewei Tu. (paper) (code)

Syntax

++ Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model. Jun Yen Leung, Guy Emerson and Ryan Cotterell. (paper) (code)

(TACL) Syntactic Structure Distillation Pretraining for Bidirectional Encoders. Adhiguna Kuncoro et al. (paper)

Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks. Yuanhe Tian, Yan Song and Fei Xia. (paper) (code)

(CL) Tractable Lexical-Functional Grammar. Jürgen Wedekind, Ronald M. Kaplan (paper)

Parsing Gapping Constructions Based on Grammatical and Semantic Roles. Yoshihide Kato and Shigeki Matsubara. (paper) (code)

Semantics

Lexical Semantics

See also: Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA. Ieva Staliūnaitė and Ignacio Iacobacci.

++ Do Illegal Aliens”? Differentiating Denotation and Connotation in Vector Spaces. Albert Webson, Zhizhong Chen, Carsten Eickhoff and Ellie Pavlick.**Undocumented Workers" ==** (paper) (code)

Philosophy of Language prelude: Frege, Sense and Reference (1894): Semantic meaning = Reference (denotation) + “mode of presentation”. Most semantics to follow largely focused on reference and truth conditions — in philosophy of language these are roughly the externalists and referentialists such as Putnam and Kripke. However, one can also look at semantics as grounded on psychological concepts (e.g., Gilbert Harman, Conceptual Role Semantics) and how they are used. Two-Factor Semantics (Ned Block, endorsed by Susan Carey) integrates both. This paper tries to operationalize the disentanglement of denotation and connotation with an adversarial neural network.

Corpus: congressional record (denotation: legislation under discussion; connotation: partisanship). Does this method work? “undocumented” and “illegal_alians” are much closer in denotation space than in the original space!

Not all that related, but interesting: Media Persuasion through Slanted Language: Evidence from the Coverage of Immigration

++ Speakers Fill Lexical Semantic Gaps with Context. Tiago Pimentel, Rowan Hall Maudslay, Damian Blasi and Ryan Cotterell. (paper) (code)

Exploring Semantic Capacity of Terms. Jie Huang, Zilong Wang, Kevin Chang, Wen-mei Hwu and JinJun Xiong. (paper) (code)

Within-Between Lexical Relation Classification. Oren Barkan, Avi Caciularu and Ido Dagan. (paper) (code)

Hypernymy:

When Hearst Is not Enough: Improving Hypernymy Detection from Corpus with Distributional Models. Changlong Yu, Jialong Han, Peifeng Wang, Yangqiu Song, Hongming Zhang, Wilfred Ng and Shuming Shi. (paper) (code)

Semantic change detection:

Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection. Adam Tsakalidis and Maria Liakata. (paper) (code)

Contextualized Representations:

Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors. Qianchu Liu, Diana McCarthy and Anna Korhonen. (paper) (code)

Goal: post-processing technique to improve contextualized embeddings with static embeddings. Question: Why does rotation work so well? I am curious if there were any patterns in the words that did not align closely after the transformation. BERT uses a wider context than SGNS, GloVe, etc., which could lead to qualitatively different representations.

Word class flexibility: A deep contextualized approach. Bai Li, Guillaume Thomas, Yang Xu and Frank Rudzicz. (paper) (code)

Probing Pretrained Language Models for Lexical Semantics. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš and Anna Korhonen. (paper) (code)

(Findings) Quantifying the Contextualization of Word Representations with Semantic Class Probing. Mengjie Zhao, Philipp Dufter, Yadollah Yaghoobzadeh and Hinrich Schütze. (paper) (code)

(Findings) SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings. Masoud Jalili Sabet et al. (paper) (code)

XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization. Alessandro Raganato et al. (paper) (code)

BERT Knows Punta Cana is not just beautiful, it’s gorgeous: Ranking Scalar Adjectives with Contextualised Representations. Aina Garí Soler and Marianna Apidianaki. (paper) (code)

Word Sense Disambiguation

Improving Word Sense Disambiguation with Translations. Yixing Luan, Bradley Hauer, Lili Mou and Grzegorz Kondrak. (paper) (code)

With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation. Bianca Scarlini, Tommaso Pasini and Roberto Navigli. (paper) (code)

A Synset Relation-enhanced Framework with a Try-again Mechanism for Word Sense Disambiguation. Ming Wang and Yinglin Wang. (paper) (code)

Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations. Gábor Berend. (paper) (code)

Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation. Daniel Loureiro and Jose Camacho-Collados. (paper) (code)

(Findings) Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences. Boon Peng Yap, Andrew Koh and Eng Siong Chng. (paper) (code)

Word Sense Induction:

(Findings) An Evaluation Method for Diachronic Word Sense Induction. Ashjan Alsulaimani, Erwan Moreau and Carl Vogel. (paper) (code)

Bilingual Lexicon Induction:

LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space. Tasnim Mohiuddin, M Saiful Bari and Shafiq Joty. (paper) (code)

Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction. Xu Zhao, Zihao Wang, Hao Wu and Yong Zhang. (paper) (code)

Improving Bilingual Lexicon Induction for Low Frequency Words. Jiaji Huang, Xingyu Cai and Kenneth Church. (paper) (code)

Other:

++ Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks. Denis Emelin, Ivan Titov and Rico Sennrich. (paper) (code)

Multimodal:

(Findings+WS) Visual Objects As Context: Exploiting Visual Objects for Lexical Entailment. Masayasu Muraoka, Tetsuya Nasukawa and Bishwaranjan Bhattacharjee. (paper) (code)

Sentence-level Semantics

For NLI, see Natural Language Inference.

Compositional Phrase Alignment and Beyond. Yuki Arase and Jun’ichi Tsujii. (paper) (code)

Word Rotator’s Distance. Sho Yokoi, Ryo Takahashi, Reina Akama, Jun Suzuki and Kentaro Inui. (paper) (code) (for sentence similarity)

Sentence embeddings:

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. Nils Reimers and Iryna Gurevych. (preprint) (paper) (code, models and data)

A Bilingual Generative Transformer for Semantic Sentence Embedding. John Wieting, Graham Neubig and Taylor Berg-Kirkpatrick. (paper) (code)

On the Sentence Embeddings from Pre-trained Language Models. Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang and Lei Li. (paper) (code)

An Unsupervised Sentence Embedding Method by Mutual Information Maximization. Yan Zhang, Ruidan He, ZUOZHU LIU, Kwan Hui Lim and Lidong Bing. (paper) (code)

Cross-Thought for Sentence Encoder Pre-training. Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jingjing Liu and Jing Jiang. (paper) (code)

Do sequence-to-sequence VAEs learn global features of sentences?. Tom Bosc and Pascal Vincent. (paper) (code)

Pragmatics

See also: Pragmatic Issue-Sensitive Image Captioning. Allen Nie, Reuben Cohn-Gordon and Christopher Potts..

“I’d rather just go to bed”: Understanding Indirect Answers. Annie Louis, Dan Roth and Filip Radlinski. (paper) (code)

Help! Need Advice on Identifying Advice. Venkata Subrahmanyan Govindarajan, Benjamin Chen, Rebecca Warholic, Katrin Erk and Junyi Jessy Li. (paper) (code)

Quantifying Intimacy in Language. Jiaxin Pei and David Jurgens. (paper) (code)

Reactive Supervision: A New Method for Collecting Sarcasm Data. Boaz Shmueli, Lun-Wei Ku and Soumya Ray. (paper) (code)

Discourse

See also the papers at the 1st Workshop on Computational Approaches to Discourse (CODI).

• Eduard Hovy’s fantastic invited talk “Discourse processing in the time of DNNs” Data is not enough. What we need: planning, better discourse-annotated corpora with communicative intents/goals. And knowledge. The KR community split off from NLP in the late 70s and never came back. Question: “Do you think that building an accurate document representation could be done by combining formal discourse representations and distributional approaches to semantic representation” Answer: There is no other way.

BERT-enhanced Relational Sentence Ordering Network. Baiyun Cui, Yingming Li and Zhongfei Zhang. (paper) (code)

(Findings) Weakly-Supervised Modeling of Contextualized Event Embedding for Discourse Relations. I-Ta Lee, Maria Leonor Pacheco and Dan Goldwasser. (paper) (code)

(Findings) Towards Domain-Independent Text Structuring Trainable on Large Discourse Treebanks. Grigorii Guz and Giuseppe Carenini. (paper) (code)

MedFilter: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge. Sopan Khosla, Shikhar Vashishth, Jill Fain Lehman and Carolyn Rose. (paper) (code)

Predicting Reference: What do Language Models Learn about Discourse Models?. Shiva Upadhye, Leon Bergen and Andrew Kehler. (paper) (code)

Affective Event Classification with Discourse-enhanced Self-training. Yuan Zhuang, Tianyu Jiang and Ellen Riloff. (paper) (code)

++ QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines. Valentina Pyatkin, Ayal Klein, Reut Tsarfaty and Ido Dagan. (paper) (code)

Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and Knowledge Grounding. Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni and Julian McAuley. (paper) (code)

++ Centering-based Neural Coherence Modeling with Hierarchical Discourse Segments. Sungho Jeon and Michael Strube. (paper) (code)

MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision. Patrick Huber and Giuseppe Carenini. (paper) (code)

SLM: Learning a Discourse Language Representation with Sentence Unshuffling. Haejun Lee, Drew A. Hudson, Kangwook Lee and Christopher D. Manning. (paper) (code)

Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading. Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C.H. Hoi, Caiming Xiong, Irwin King and Michael Lyu. (paper) (code)

Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays. Wei Song, Ziyao Song, Ruiji Fu, Lizhen Liu, Miaomiao Cheng and Ting Liu. (paper) (code)

Argumentation

(Findings) The workweek is the best time to start a family – A Study of GPT-2 Based Claim Generation. Shai Gretz, Yonatan Bilu, Edo Cohen-Karlik and Noam Slonim. (paper) (code)

++ Detecting Attackable Sentences in Arguments. Yohan Jo, Seojin Bang, Emaad Manzoor, Eduard Hovy and Chris Reed. (paper) (code)

Extracting Implicitly Asserted Propositions in Argumentation. Yohan Jo, Jacky Visser, Chris Reed and Eduard Hovy. (paper) (code)

APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning. Liying Cheng, Lidong Bing, Qian Yu, Wei Lu and Luo Si. (paper) (code)

(Findings) Multilingual Argument Mining: Datasets and Analysis. Orith Toledo-Ronen, Matan Orbach, Yonatan Bilu, Artem Spector and Noam Slonim. (paper) (code)

Exploring the Role of Argument Structure in Online Debate Persuasion. Jialu Li, Esin Durmus and Claire Cardie. (paper) (code)

Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis. Roy Bar-Haim, Yoav Kantor, Lilach Eden, Roni Friedman, Dan Lahav and Noam Slonim. (paper) (code)

Unsupervised Stance Detection for Arguments from Consequences. Jonathan Kobbe, Ioana Hulpuș and Heiner Stuckenschmidt. (paper) (code)

Figurative Language

Identifying Exaggerated Language. Li Kong, Chuanyi Li, Jidong Ge, Bin Luo and Vincent Ng. (paper) (code)

++ Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation. Tuhin Chakrabarty, Smaranda Muresan and Nanyun Peng. (paper) (code)

(Findings) The Pragmatics behind Politics: Modelling Metaphor, Framing and Emotion in Political Discourse (paper) (code)

(Findings) Contextual Modulation for Relation-Level Metaphor Identification. Omnia Zayed, John P. McCrae and Paul Buitelaar. (paper) (code)

Artifacts and Spurious Correlations

Unknown artifacts:

Towards Debiasing NLU Models from Unknown Biases. Prasetya Ajie Utama, Nafise Sadat Moosavi and Iryna Gurevych. (paper) (code)

(Findings) Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles. Christopher Clark, Mark Yatskar and Luke Zettlemoyer. (paper) (code)

Other:

See also: (Findings+WS) Identifying Spurious Correlations for Robust Text Classification. Zhao Wang and Aron Culotta..

Translation Artifacts in Cross-lingual Transfer Learning. Mikel Artetxe, Gorka Labaka and Eneko Agirre. (paper) (code)

Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training. Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel and Tim Rocktäschel. (preprint) (paper) (code)

(Findings+WS) Improving QA Generalization by Concurrent Modeling of Multiple Biases. Mingzhu Wu, Nafise Sadat Moosavi, Andreas Rücklé and Iryna Gurevych. (paper) (code)

Biases

CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. Nikita Nangia, Clara Vania, Rasika Bhalerao and Samuel R. Bowman. (paper) (code)

“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models. Vered Shwartz, Rachel Rudinger and Oyvind Tafjord. (paper) (code)

Debiasing knowledge graph embeddings. Joseph Fisher, Arpit Mittal, Dave Palfrey and Christos Christodoulopoulos. (paper) (code)

Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation. Robert Munro and Alex (Carmen) Morrison. (paper) (code)

LOGAN: Local Group Bias Detection by Clustering. Jieyu Zhao and Kai-Wei Chang. (paper) (code)

(Findings) Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. Po-Sen Huang et al. (paper) (code)

(Findings) Towards Controllable Biases in Language Generation. Emily Sheng, Kai-Wei Chang, Prem Natarajan and Nanyun Peng. (paper) (code) (to analyze and mitigate social biases).

(Findings) UNQOVERing Stereotyping Biases via Underspecified Questions. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal and Vivek Srikumar. (paper) (code)

Gender Bias

Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias. Ana Valeria González, Maria Barrett, Rasmus Hvingelby, Kellie Webster and Anders Søgaard. (preprint) (paper) (code and data)

Classification/discovery of gender bias:

Unsupervised Discovery of Implicit Gender Bias. Anjalie Field and Yulia Tsvetkov. (paper) (code)

Multi-Dimensional Gender Bias Classification. Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela and Adina Williams. (paper) (code)

(Findings+WS) Automatically Identifying Gender Issues in Machine Translation using Perturbations. Hila Gonen and Kellie Webster. (paper) (code)

Mitigation for gender bias in text generation:

++ PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction. Xinyao Ma, Maarten Sap, Hannah Rashkin and Yejin Choi. (paper) (code)

Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning. Haochen Liu, Wentao Wang, Yiqi Wang, Hui Liu, Zitao Liu and Jiliang Tang. (preprint) (paper) (code)

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation. Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela and Jason Weston. (paper) (code)

Mitigation of gender bias in word embeddings:

See also Fair Embedding Engine: A Library for Analyzing and Mitigating Gender
Bias in Word Embeddings from the NLP-OSS Workshop.

(Findings) Neutralizing Gender Bias in Word Embeddings with Latent Disentanglement and Counterfactual Generation. Seungjae Shin et al. (paper) (code)

Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings. Vaibhav Kumar et al. (paper) (code)

Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation. Francisco Vargas and Ryan Cotterell. (paper) (code)

Explainability

See also: the papers in BlackboxNLP 2020: Analyzing and interpreting neural networks for NLP.

++ (Honorable Mention) GLUCOSE: GeneraLized and COntextualized Story Explanations. Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran and Jennifer Chu-Carroll. (paper) (code)

(Findings) Teaching Machine Comprehension with Compositional Explanations. Qinyuan Ye, Xiao Huang, Elizabeth Boschee and Xiang Ren. (paper) (code)

(Findings) Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?. Peter Hase et al. (paper) (code)

++ A Diagnostic Study of Explainability Techniques for Text Classification. Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma and Isabelle Augenstein. (paper) (code)

Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers. Hanjie Chen and Yangfeng Ji. (paper) (code)

Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks. Chengyue Jiang, Yinggong Zhao, Shanbo Chu, Libin Shen and Kewei Tu. (paper) (code)

Interpretation of NLP models through input marginalization. Siwon Kim, Jihun Yi, Eunji Kim and Sungroh Yoon. (paper) (code)

How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking. Nicola De Cao et al. (preprint) (paper) (code)

Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification. Sen et al. (paper) (code)

(Findings) Unsupervised Expressive Rules Provide Explainability and Assist Human Experts Grasping New Domains. Eyal Shnarch et al. (paper) (code)

(Findings) What-if I ask you to explain: Explaining the effects of perturbations in procedural text. Dheeraj Rajagopal et al. (paper) (code)

Rationales:

See also the paper from ACL 2020, Learning to Faithfully Rationalize by Construction.

(Findings) Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervision. Max Glockner, Ivan Habernal and Iryna Gurevych. (paper) (code)

An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction. Bhargavi Paranjape, Mandar Joshi, John Thickstun, Hannaneh Hajishirzi and Luke Zettlemoyer. (paper) (code)

Evaluating and Characterizing Human Rationales. Samuel Carton, Anirudh Rathore and Chenhao Tan. (paper) (code)

(Findings) Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs. Ana Marasović et al. (paper) (code)

Applications:

(Findings) Generalizable and Explainable Dialogue Generation via Explicit Action Learning. Xinting Huang, Jianzhong Qi, Yu Sun and Rui Zhang. (paper) (code)

HENIN: Learning Heterogeneous Neural Interaction Networks for Explainable Cyberbullying Detection on Social Media. Hsin-Yu Chen and Cheng-Te Li. (paper) (code)

F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering. Hendrik Schuff, Heike Adel and Ngoc Thang Vu. (paper) (code)

Towards Interpreting BERT for Reading Comprehension Based QA. Sahana Ramnath, Preksha Nema, Deep Sahni and Mitesh M. Khapra. (paper) (code)

Adversarial ML

Adversarial attacks

See also: Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks. Denis Emelin, Ivan Titov and Rico Sennrich.

(Findings) Reevaluating Adversarial Examples in Natural Language. John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji and Yanjun Qi. (paper) (code)

(Findings) Gradient-based Analysis of NLP Models is Manipulable. Junlin Wang, Jens Tuyls, Eric Wallace and Sameer Singh. (paper) (code)

Imitation Attacks and Defenses for Black-box Machine Translation Systems. Eric Wallace, Mitchell Stern and Dawn Song. (preprint) (paper) (code)

Adversarial Attack and Defense of Structured Prediction Models. Wenjuan Han, Liwen Zhang, Yong Jiang and Kewei Tu. (paper) (code)

BERT-ATTACK: Adversarial Attack Against BERT Using BERT. Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue and Xipeng Qiu. (preprint) (paper) (code)

BAE: BERT-based Adversarial Examples for Text Classification. Siddhant Garg and Goutham Ramakrishnan. (paper) (code)

T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack. Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang and Bo Li. (paper) (code)

Adversarial Semantic Collisions. Congzheng Song, Alexander Rush and Vitaly Shmatikov. (paper) (code)

Adversarial training

NOTE: there are other papers related to this scattered in other sections (GEC, gender biases, etc.). For an interesting application, see “Differentiating Denotation and Connotation in Vector Spaces”.

Adversarial Semantic Decoupling for Recognizing Open-Vocabulary Slots. Yuanmeng Yan, Keqing He, Hong Xu, Sihong Liu, Fanyu Meng, Min Hu and Weiran XU. (paper) (code)

Adversarial Self-Supervised Data-Free Distillation for Text Classification. Xinyin Ma, Yongliang Shen, Gongfan Fang, Chen Chen, Chenghao Jia and Weiming Lu. (paper) (code)

Neural Topic Modeling with Cycle-Consistent Adversarial Training. Xuemeng Hu, Rui Wang, Deyu Zhou and Yuxuan Xiong. (paper)

For domain adaptation:

Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models. Thuy-Trang Vu, Dinh Phung and Gholamreza Haffari. (paper) (code)

(Findings+WS) Domain Adversarial Fine-Tuning as an Effective Regularizer. Giorgos Vernikos, Katerina Margatina, Alexandra Chronopoulou and Ion Androutsopoulos. (paper) (code)

Improving robustness:

CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation. Tianlu Wang et al. (paper) (code)

(Findings+WS) Enhance Robustness of Sequence Labelling with Masked Adversarial Training. Luoxin Chen, Xinyue Liu, Weitong Ruan and Jianhua Lu. (paper) (code)

Commonsense Reasoning and Question Answering

++ Unsupervised Commonsense Question Answering with Self-Talk. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula and Yejin Choi. (preprint) (paper) (code)

Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering. Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan and Xiang Ren. (paper) (code)

A Simple Yet Strong Pipeline for HotpotQA. Dirk Groeneveld, Tushar Khot, Mausam - and Ashish Sabharwal. (paper) (code)

Neural Conversational QA: Learning to Reason vs Exploiting Patterns. Nikhil Verma et al. (paper) (code)

More Bang for Your Buck: Natural Perturbation for Robust Question Answering. Daniel Khashabi, Tushar Khot and Ashish Sabharwal. (paper) (code)

ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning. Michael Boratko, Xiang Li, Tim O’Gorman, Rajarshi Das, Dan Le and Andrew McCallum. (paper) (code)

(Findings) Question Answering with Long Multiple-Span Answers. Ming Zhu, Aman Ahuja, Da-Cheng Juan, Wei Wei and Chandan K Reddy. (paper) (code)

(Findings) Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering. Peifeng Wang, Nanyun Peng, Filip Ilievski, Pedro Szekely and Xiang Ren. (paper) (code)

(Findings) Open Domain Question Answering based on Text Enhanced Knowledge Graph with Hyperedge Infusion. Jiale Han, Bo Cheng and Xu Wang. (paper) (code)

Hierarchical Graph Network for Multi-hop Question Answering. Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang and Jingjing Liu. (paper) (code)

Context-Aware Answer Extraction in Question Answering. Yeon Seonwoo, Ji-Hoon Kim, Jung-Woo Ha and Alice Oh. (paper) (code)

Unsupervised Question Decomposition for Question Answering. Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho and Douwe Kiela. (preprint) (paper) (code)

STL-CQA: Structure-based Transformers with Localization and Encoding for Chart Question Answering. Hrituraj Singh and Sumit Shekhar. (paper) (code)

Look at the First Sentence: Position Bias in Question Answering. Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim and Jaewoo Kang. (paper) (code)

Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering. Yuxiang Wu, Sebastian Riedel, Pasquale Minervini and Pontus Stenetorp. (paper) (code)

What do Models Learn from Question Answering Datasets?. Priyanka Sen and Amir Saffari. (paper) (code)

Learning a Cost-Effective Annotation Policy for Question Answering. Bernhard Kratzwald, Stefan Feuerriegel and Huan Sun. (paper) (code)

Training Question Answering Models From Synthetic Data. Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary and Bryan Catanzaro. (paper) (code)

Dense Passage Retrieval for Open-Domain Question Answering. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen and Wen-tau Yih. (paper) (code)

Unsupervised Adaptation of Question Answering Systems via Generative Self-training. Steven Rennie, Etienne Marcheret, Neil Mallinar, David Nahamoo and Vaibhava Goel. (paper) (code)

Is Graph Structure Necessary for Multi-hop Question Answering?. Nan Shao, Yiming Cui, Ting Liu, Shijin Wang and Guoping Hu. (paper) (code)

++ (Findings) UNIFIEDQA: Crossing Format Boundaries with a Single QA System. Daniel Khashabi et al. (paper) (code)

See also the comparison with GPT-3 in Measuring Massive Multitask Language Understanding on the zero-shot setting.

AmbigQA: Answering Ambiguous Open-domain Questions. Sewon Min, Julian Michael, Hannaneh Hajishirzi and Luke Zettlemoyer. (paper) (code)

Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot and Ashish Sabharwal. (paper) (code)

++ (Findings+WS) What do we expect from Multiple-choice QA Systems?. Krunal Shah, Nitish Gupta and Dan Roth. (paper) (code)

ISAAQ - Mastering Textbook Questions with Pre-trained Transformers and Bottom-Up and Top-Down Attention. Jose Manuel Gomez-Perez and Raúl Ortega. (paper) (code)

IIRC: A Dataset of Incomplete Information Reading Comprehension Questions. James Ferguson, Matt Gardner, Hannaneh Hajishirzi, Tushar Khot and Pradeep Dasigi. (paper) (code)

A Simple and Effective Model for Answering Multi-span Questions. Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson and Jonathan Berant. (paper) (code)

Datasets:

XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić and Anna Korhonen. (paper) (code)

TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions. Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner and Dan Roth. (paper) (code)

SubjQA: A Dataset for Subjectivity and Review Comprehension. Johannes Bjerva, Nikita Bhutani, Behzad Golshan, Wang-Chiew Tan and Isabelle Augenstein. (paper) (code)

(Findings) FQuAD: French Question Answering Dataset. Martin D’Hoffschmidt, Wacim Belblidia, Quentin Heinrich, Tom Brendlé and Maxime Vidal. (paper) (code)

Visual Question Answering

(Findings) Visuo-Linguistic Question Answering (VLQA) Challenge. Shailaja Keyur Sampat, Yezhou Yang and Chitta Baral. (paper) (code)

Learning to Contrast the Counterfactual Samples for Robust Visual Question Answering. Zujie Liang, Weitao Jiang, Haifeng Hu and Jiaying Zhu. (paper) (code)

MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering. Tejas Gokhale, Pratyay Banerjee, Chitta Baral and Yezhou Yang. (paper) (code)

(Findings) Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation. Yiming Xu, Lin Chen, Zhongwei Cheng, Lixin Duan and Jiebo Luo. (paper) (code)

(Findings) ConceptBert: Concept-Aware Representation for Visual Question Answering. François Gardères, Maryam Ziaeefard, baptiste abeloos and Freddy Lecue. (paper) (code)

(Findings) MMFT-BERT: Multimodal Fusion Transformer with BERT Encodings for Visual Question Answering. Aisha Urooj, Amir Mazaheri, Niels Da vitoria lobo and Mubarak Shah. (paper) (code)

(Findings) Can Pre-training help VQA with Lexical Variations?. Shailza Jolly and Shubham Kapoor. (paper) (code)

Knowledge and reasoning

See also:

• The tutorial on Machine Reasoning for Knowledgeable, Explainable and Inferable Models (Nan Duan, Duyu Tang, Ming Zhou) (video).
• Deep Learning Inside Out (DeeLIO): The First Workshop on Knowledge Extraction and Integration for DL Architectures (focus on “low-data regimes”)

Knowledge:

++ Causal Inference of Script Knowledge. Noah Weber, Rachel Rudinger and Benjamin Van Durme. (paper) (code)

Distilling Structured Knowledge for Text-Based Relational Reasoning. Jin Dong, Marc-Antoine Rondeau and William L. Hamilton. (paper) (code)

Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning. Tao Shen, Yi Mao, Pengcheng He, Guodong Long, Adam Trischler and Weizhu Chen. (paper) (code)

Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection. Ruize Wang, Duyu Tang, Nan Duan, Wanjun Zhong, Zhongyu Wei, Xuanjing Huang, Daxin Jiang and Ming Zhou. (paper) (code)

Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning. Ye Liu, Sheng Zhang, Rui Song, Suo Feng and Yanghua Xiao. (paper) (code)

DualTKB: A Dual Learning Bridge between Text and Knowledge Base. Pierre Dognin, Igor Melnyk, Inkit Padhi, Cicero Nogueira dos Santos and Payel Das. (paper) (code)

(Findings) Using the Past Knowledge to Improve Sentiment Classification. Qi Qin, Wenpeng Hu and Bing Liu. (paper) (code)

(Findings) Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models. Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu SUN and Bin He. (paper) (code)

Math word problems:

A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving. Qinzhuo Wu, Qi Zhang, Jinlan Fu and Xuanjing Huang. (paper) (code)

Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems. Jinghui Qin, Lihui Lin, Xiaodan Liang, Rumin Zhang and Liang Lin. (preprint) (paper) (data and code)

Point to the Expression: Solving Algebraic Word Problems using the Expression-Pointer Transformer Model. Bugeun Kim, Kyung Seo Ki, Donggeon Lee and Gahgene Gweon. (paper) (code)

(Findings) LGPSolver - Solving Logic Grid Puzzles Automatically. Elgun Jabrayilzade and Selma Tekir. (paper) (code)

Reasoning:

Question Directed Graph Attention Network for Numerical Reasoning over Text. Kunlong Chen, Weidi Xu, Xingyi Cheng, Zou Xiaochuan, Yuyu Zhang, Le Song, Taifeng Wang, Yuan Qi and Wei Chu. (paper) (code)

++ PRover: Proof Generation for Interpretable Reasoning over Rules. Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava and Mohit Bansal. (paper) (code)

Multi-Step Inference for Reasoning Over Paragraphs. Jiangming Liu, Matt Gardner, Shay B. Cohen and Mirella Lapata. (paper) (code)

Towards Interpretable Reasoning over Paragraph Effects in Situation. Mucheng Ren, Xiubo Geng, Tao QIN, Heyan Huang and Daxin Jiang. (paper) (code)

Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering. Harsh Jhamtani and Peter Clark. (paper) (code)

Reasoning about Goals, Steps, and Temporal Ordering with WikiHow. Li Zhang, Qing Lyu and Chris Callison-Burch. (paper) (code)

Commonsense Reasoning:

Social Chemistry 101: Learning to Reason about Social and Moral Norms. Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap and Yejin Choi. (paper) (code)

(Findings+WS) Social Commonsense Reasoning with Multi-Head Knowledge Attention. Debjit Paul and Anette Frank. (paper) (code)

++ Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning. Lianhui Qin et al. (paper) (code)

(Findings+WS) Beyond Language: Learning Commonsense from Images for Reasoning. Wanqing Cui, Yanyan Lan, Liang Pang, Jiafeng Guo and Xueqi Cheng. (paper) (code)

Learning with Less Data

Active Learning

Cold-start Active Learning through Self-supervised Language Modeling. Michelle Yuan, Hsuan-Tien Lin and Jordan Boyd-Graber. (preprint) (paper) (code)

ALICE: Active Learning with Contrastive Natural Language Explanations. Weixin Liang, James Zou and Zhou Yu. (preprint) (paper) (code)

Active Learning for BERT: An Empirical Study. Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz and Noam Slonim. (paper) (code)

Textual Data Augmentation for Efficient Active Learning on Tiny Datasets. Husam Quteineh, Spyridon Samothrakis and Richard Sutcliffe. (paper) (code)

(Findings) Active Learning Approaches to Enhancing Neural Machine Translation. Yuekai Zhao, Haoran Zhang, Shuchang Zhou and Zhihua Zhang. (paper) (code)

(Findings) Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space. Dongyu Ru et al. (paper) (code)

Data Augmentation

DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks. BOSHENG DING, Linlin Liu, Lidong Bing, Canasai Kruengkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si and Chunyan Miao. (paper) (code)

(Findings+WS) How Effective is Task-Agnostic Data Augmentation for Pretrained Transformers?. Shayne Longpre, Yu Wang and Chris DuBois. (paper) (code)

(Findings+WS) Generative Data Augmentation for Commonsense Reasoning. Yiben Yang et al. (paper) (code)

Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging. Semih Yavuz et al. (paper) (code)

Sequence-Level Mixed Sample Data Augmentation. Demi Guo, Yoon Kim and Alexander Rush. (paper) (code)

Local Additivity Based Data Augmentation for Semi-supervised NER. Jiaao Chen, Zhenghui Wang, Ran Tian, Zichao Yang and Diyi Yang. (paper) (code)

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space. Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan and Ming Zhou. (paper) (code)

SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness. Nathan Ng, Kyunghyun Cho and Marzyeh Ghassemi. (paper) (code)

Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection. Jingfeng Yang, Diyi Yang and Zhaoran Ma. (paper) (code)

SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup. Rongzhi Zhang, Yue Yu and Chao Zhang. (paper) (code)

Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation. Ruibo Liu, Guangxuan Xu, Chenyan Jia, Weicheng Ma, Lili Wang and Soroush Vosoughi. (paper) (code)

(Findings) Finding Friends and Flipping Frenemies: Automatic Paraphrase Dataset Augmentation Using Graph Theory. Hannah Chen, Yangfeng Ji and David Evans. (paper) (code)

Domain Adaptation

See also some papers under Adversarial training.

(Findings+WS) Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA. Nina Poerner et al. (paper) (code)

Multi-Stage Pre-training for Low-Resource Domain Adaptation. Rong Zhang et al. (paper) (code)

Unified Feature and Instance Based Domain Adaptation for Aspect-Based Sentiment Analysis. Chenggong Gong, Jianfei Yu and Rui Xia. (paper) (code)

Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training. Hai Ye, Qingyu Tan, Ruidan He, Juntao Li, Hwee Tou Ng and Lidong Bing. (paper) (code)

Transformer Based Multi-Source Domain Adaptation. Dustin Wright and Isabelle Augenstein. (paper) (code)

(Findings) Margin-aware Unsupervised Domain Adaptation for Cross-lingual Text Labeling. Dejiao Zhang et al. (paper) (code)

Few-shot Learning

NOTE: A few other interesting few-shot papers are scattered in other sections.

Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks. Trapit Bansal, Rishikesh Jha, Tsendsuren Munkhdalai and Andrew McCallum. (paper) (code)

Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models. Ethan Wilcox, Peng Qian, Richard Futrell, Ryosuke Kohita, Roger Levy and Miguel Ballesteros. (paper) (code)

(Findings) Few-Shot Multi-Hop Relation Reasoning over Knowledge Bases. Chuxu Zhang, Lu Yu, Mandana Saebi, Meng Jiang and Nitesh Chawla. (paper) (code)

Adaptive Attentional Network for Few-Shot Knowledge Graph Completion. Jiawei Sheng, Shu Guo, Zhenyu Chen, Juwei Yue, Lihong Wang, Tingwen Liu and Hongbo Xu. (preprint) (paper) (code)

NER:

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning. Yi Yang and Arzoo Katiyar. (paper) (code)

Intent Detection:

Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference. Jianguo Zhang et al. (paper) (code)

(Findings+WS) Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection. Hoang Nguyen, Chenwei Zhang, Congying Xia and Philip Yu. (paper) (code)

(Findings+WS) Composed Variational Natural Language Generation for Few-shot Intents. Congying Xia, Caiming Xiong, Philip Yu and Richard Socher. (paper) (code)

WSD:

(Findings) Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation. Nithin Holla, Pushkar Mishra, Helen Yannakoudakis and Ekaterina Shutova. (paper) (code)

QA:

Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning. Yuncheng Hua, Yuan-Fang Li, Gholamreza Haffari, Guilin Qi and Tongtong Wu. (paper) (code)

Zero-shot Learning

Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs. Jueqing Lu, Lan Du, Ming Liu and Joanna Dipnall. (paper) (code)

MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale. Andreas Rücklé, Jonas Pfeiffer and Iryna Gurevych. (paper) (code)

Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations. Emily Allaway and Kathleen McKeown. (paper) (code)

Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering. Pratyay Banerjee and Chitta Baral. (paper) (code)

(Findings) Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering. Po-Nien Kung, Tse-Hsuan Yang, Yi-Cheng Chen, Sheng-Siang Yin and Yun-Nung Chen. (paper) (code)

Language Models and Transformers

++ KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations. Fabio Massimo Zanzotto et al. (paper) (code)

++ Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually). Alex Warstadt, Yian Zhang, Xiaocheng Li, Haokun Liu and Samuel R. Bowman. (paper) (code)

See also: When Do You Need Billions of Words of Pretraining Data?, Information-theoretic Probing Explains Reliance on Spurious Features.

++ Which BERT? A Survey Organizing Contextualized Encoders. Patrick Xia, Shijie Wu and Benjamin Van Durme. (paper) (code)

Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU. Brielen Madureira and David Schlangen. (paper) (code)

(Findings) Integrating Task Specific Information into Pretrained Language Models for Low Resource Fine Tuning. Rui Wang et al. (paper) (code)

++ ETC: Encoding Long and Structured Inputs in Transformers. Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang and Li Yang. (paper) (code)

Understanding the Difficulty of Training Transformers. Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen and Jiawei Han. (paper) (code)

Analyzing Redundancy in Pretrained Transformer Models. Fahim Dalvi, Hassan Sajjad, Nadir Durrani and Yonatan Belinkov. (paper) (code)

Assessing Phrasal Representation and Composition in Transformers. Lang Yu and Allyson Ettinger. (paper) (code)

(Findings+WS) Guiding Attention for Self-Supervised Learning with Transformers. Ameet Deshpande and Karthik Narasimhan. (paper) (code)

(Findings) Scheduled DropHead: A Regularization Method for Transformer Models. Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou and Ke Xu. (paper) (code)

(Findings+WS) Improve Transformer Models with Better Relative Position Embeddings. zhiheng huang, Davis Liang, Peng Xu and Bing Xiang. (paper) (code)

(Findings) Inserting Information Bottlenecks for Attribution in Transformers. Zhiying Jiang, Raphael Tang, Ji Xin and Jimmy Lin. (paper) (code)

(Findings+WS) Investigating Transferability in Pretrained Language Models. Alex Tamkin, Trisha Singh, Davide Giovanardi and Noah Goodman. (paper) (code)

Calibration of Pre-trained Transformers. Shrey Desai and Greg Durrett. (paper) (code)

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining. Chengyu Wang, Minghui Qiu, jun huang and XIAOFENG HE. (preprint) (paper) (code)

Retrofitting Structure-aware Transformer Language Model for End Tasks. Hao Fei, Yafeng Ren and Donghong Ji. (preprint) (paper) (code)

++ Pre-Training Transformers as Energy-Based Cloze Models. Kevin Clark, Minh-Thang Luong, Quoc Le and Christopher D. Manning. (paper) (code)

(Findings) Byte Pair Encoding is Suboptimal for Language Model Pretraining. Kaj Bostrom and Greg Durrett. (paper) (code)

Masking as an Efficient Alternative to Finetuning for Pretrained Language Models. Mengjie Zhao, Tao Lin, Fei Mi, Martin Jaggi and Hinrich Schütze. (paper) (code)

An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training. Kristjan Arumae, Qing Sun and Parminder Bhatia. (paper) (code)

Scaling Hidden Markov Language Models. Justin Chiu and Alexander Rush. (paper) (code)

Consistency of a Recurrent Language Model With Respect to Incomplete Decoding. Sean Welleck, Ilia Kulikov, Jaedeok Kim, Richard Yuanzhe Pang and Kyunghyun Cho. (paper) (code)

Analysis — what does BERT learn?

See also: what LMs know about lexical semantics (Probing Pretrained Language Models for Lexical Semantics. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš and Anna Korhonen. ), discourse (Predicting Reference: What do Language Models Learn about Discourse Models?. Shiva Upadhye, Leon Bergen and Andrew Kehler.), and event arguments BERTering RAMS: What and How Much does BERT Already Know About Event Arguments? - A Study on the RAMS Dataset. Varun Gangal, Eduard Hovy). They also contain social bias (CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models. Nikita Nangia, Clara Vania, Rasika Bhalerao and Samuel R. Bowman.).

On the Ability and Limitations of Transformers to Recognize Formal Languages. Satwik Bhattamishra, Kabir Ahuja and Navin Goyal. (paper) (code)

++ Asking without Telling: Exploring Latent Ontologies in Contextual Representations. Julian Michael, Jan A. Botha and Ian Tenney. (paper) (code)

(Findings) How Can Self-Attention Networks Recognize Dyck-n Languages?. Javid Ebrahimi, Dhruv Gelda and Wei Zhang. (paper) (code)

(Findings) On the Branching Bias of Syntax Extracted from Pre-trained Language Models. Huayang Li, Lemao Liu, Guoping Huang and Shuming Shi. (paper) (code)

++ How Much Knowledge Can You Pack Into the Parameters of a Language Model?. Adam Roberts, Colin Raffel and Noam Shazeer. (paper) (code)

Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA. Ieva Staliūnaitė and Ignacio Iacobacci. (paper) (code)

(Findings) Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming. Kanishka Misra, Allyson Ettinger and Julia Rayz. (paper) (code)

Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models. Isabel Papadimitriou and Dan Jurafsky. (paper) (code)

(Findings) Universal Dependencies According to BERT: Both More Specific and More General. Tomasz Limisiewicz, David Mareček and Rudolf Rosa. (paper) (code)

Investigating representations of verb bias in neural language models. Robert Hawkins, Takateru Yamakoshi, Thomas Griffiths and Adele Goldberg. (paper) (code)

Word Frequency Does Not Predict Grammatical Knowledge in Language Models. Charles Yu, Ryan Sie, Nicolas Tedeschi and Leon Bergen. (paper) (code)

(Findings) On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers. Marius Mosbach et al. (paper) (code)

Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models. Bill Yuchen Lin et al. (paper) (code)

Information-Theoretic Probing with Minimum Description Length. Elena Voita and Ivan Titov. (paper) (code) (video - NLP with Friends)

An information theoretic view on selecting linguistic probes. Zining Zhu and Frank Rudzicz. (paper) (code)

Intrinsic Probing through Dimension Selection. Lucas Torroba Hennigen, Adina Williams and Ryan Cotterell. (paper) (code)

Pareto Probing: Trading Off Accuracy for Complexity. Tiago Pimentel, Naomi Saphra, Adina Williams and Ryan Cotterell. (paper) (code)

Analysis, other:

Analyzing Individual Neurons in Pre-trained Language Models. Nadir Durrani, Hassan Sajjad, Fahim Dalvi and Yonatan Belinkov. (paper) (code)

What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding. Yu-An Wang and Yun-Nung Chen. (paper) (code)

Attention is Not Only a Weight: Analyzing Transformers with Vector Norms. Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi and Kentaro Inui. (paper) (code)

When BERT Plays the Lottery, All Tickets Are Winning. Sai Prasanna, Anna Rogers and Anna Rumshisky. (paper) (code)

BERT variants:

++ PatchBERT: Just-in-Time, Out-of-Vocabulary Patching. Sangwhan Moon and Naoaki Okazaki. (paper) (code)

(Findings) exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources. Wen Tai et al. (paper) (code)

(Findings+WS) BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA. Nora Kassner and Hinrich Schütze. (paper) (code)

(Findings) BERT-MK: Integrating Graph Contextualized Knowledge into Pre-trained Language Models. Bin He et al. (paper) (code)

(Findings) LIMIT-BERT : Linguistics Informed Multi-Task BERT. Junru Zhou, Zhuosheng Zhang, Hai Zhao and Shuailiang Zhang. (paper) (code)

(Findings) BERT-QE: Contextualized Query Expansion for Document Re-ranking. Zhi Zheng, Kai Hui, Ben He, Xianpei Han, Le Sun and Andrew Yates. (paper) (code)

(Findings) TopicBERT for Energy Efficient Document Classification. Yatin Chaudhary, Pankaj Gupta, Khushbu Saxena, Vivek Kulkarni, Thomas Runkler and Hinrich Schütze. (paper) (code)

Compression, Distillation, Pruning:

BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover’s Distance. jianquan li, Xiaokang Liu, Honghong Zhao, Ruifeng Xu, Min Yang and yaohong jin. (paper) (code)

BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei and Ming Zhou. (paper) (code)

Contrastive Distillation on Intermediate Representations for Language Model Compression. Siqi Sun, Zhe Gan, Yuwei Fang, Yu Cheng, Shuohang Wang and Jingjing Liu. (paper) (code)

TernaryBERT: Distillation-aware Ultra-low Bit BERT. Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang and Qun Liu. (paper) (code)

Structured Pruning of Large Language Models. Ziheng Wang, Jeremy Wohlwend and Tao Lei. (paper) (code)

On the weak link between importance and prunability of attention heads. Aakriti Budhraja, Madhura Pande, Preksha Nema, Pratyush Kumar and Mitesh M. Khapra. (paper) (code)

(Findings+WS) Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior. Zi Lin, Jeremiah Liu, Zi Yang, Nan Hua and Dan Roth. (paper) (code)

(Findings) Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning. Bingbing Li et al. (paper) (code)

Losing Heads in the Lottery: Pruning Transformer Attention in Neural Machine Translation. Maximiliana Behnke and Kenneth Heafield. (paper) (code)

Knowledge Distillation:

Autoregressive Knowledge Distillation through Imitation Learning. Alexander Lin, Jeremy Wohlwend, Howard Chen and Tao Lei. (paper) (code)

Lifelong Language Knowledge Distillation. Yung-Sung Chuang, Shang-Yu Su and Yun-Nung Chen. (paper) (code)

Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers. Yimeng Wu, Peyman Passban, Mehdi Rezagholizadeh and Qun Liu. (paper) (code)

Other:

Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting. Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu and Xiangzhan Yu. (paper) (code)

On Losses for Modern Language Models. Stéphane Aroca-Ouellette and Frank Rudzicz. (paper) (code)

Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data. Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao and Chao Zhang. (paper) (code)

Pretrained Language Model Embryology: The Birth of ALBERT. Cheng-Han Chiang, Sung-Feng Huang and Hung-yi Lee. (paper) (code)

On the importance of pre-training data volume for compact language models. Vincent Micheli, Martin d’Hoffschmidt and François Fleuret. (paper) (code)

Train No Evil: Selective Masking for Task-Guided Pre-Training. Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang, Zhiyuan Liu and Maosong Sun. (paper) (code)

Improving Low Compute Language Modeling with In-Domain Embedding Initialisation. Charles Welch, Rada Mihalcea and Jonathan K. Kummerfeld. (paper) (code)

Embeddings

Also papers in Lexical Semantics. For debiasing gender bias from word embeddings, see Gender Bias.

(Findings) More Embeddings, Better Sequence Labelers?. Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang and Kewei Tu. (paper) (code)

Cross/multi-lingual embeddings:

Interactive Refinement of Cross-Lingual Word Embeddings. Michelle Yuan, Mozhi Zhang, Benjamin Van Durme, Leah Findlater and Jordan Boyd-Graber. (paper) (code)

Pre-tokenization of Multi-word Expressions in Cross-lingual Word Embeddings. Naoki Otani, Satoru Ozaki, Xingyuan Zhao, Yucen Li, Micaelah St Johns and Lori Levin. (paper) (code)

A Simple Approach to Learning Unsupervised Multilingual Embeddings. Pratik Jawanpuria, Mayank Meghwanshi and Bamdev Mishra. (paper) (code)

Multilingual Offensive Language Identification with Cross-lingual Embeddings. Tharindu Ranasinghe and Marcos Zampieri. (paper) (code)

Word embeddings:

++ Deconstructing word embedding algorithms. Kian Kenyon-Dean, Edward Newell and Jackie Chi Kit Cheung. (paper) (code)

Are All Good Word Vector Spaces Isomorphic?. Ivan Vulić, Sebastian Ruder and Anders Søgaard. (paper) (code)

DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion. Zhen Han, Peng Chen, Yunpu Ma and Volker Tresp. (paper) (code)

Embedding Words in Non-Vector Space with Unsupervised Graph Learning. Max Ryabinin, Sergei Popov, Liudmila Prokhorenkova and Elena Voita. (paper) (code)

Improving Out-of-Scope Detection in Intent Classification by Using Embeddings of the Word Graph Space of the Classes. Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel and Claudio Pinhanez. (paper) (code)

Compositional Demographic Word Embeddings. Charles Welch, Jonathan K. Kummerfeld, Verónica Pérez-Rosas and Rada Mihalcea. (paper) (code)

(Findings+WS) PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding. Zhao Jinman, Shawn Zhong, Xiaomin Zhang and Yingyu Liang. (paper) (code)

(Findings+WS) General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference. Jingfei Du, Myle Ott, Haoran Li, Xing Zhou and Veselin Stoyanov. (paper) (code)

(Findings) Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition. Vasileios Lioutas et al. (paper) (code)

(Findings) Robust Backed-off Estimation of Out-of-Vocabulary Embeddings. Nobukazu Fukuda, Naoki Yoshinaga and Masaru Kitsuregawa. (paper) (code)

(Findings) Learning Numeral Embedding. Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen and Kewei Tu. (paper) (code)

Methods for Numeracy-Preserving Word Embeddings. Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika and Lawrence Carin. (paper) (code)

Cognitive Models and Psycholinguistics

See also the Workshop on Cognitive Modeling and Computational Linguistics (CMCL) at this conference.

Compositionality

See also: ROLE paper (BlackboxNLP)

++ COGS: A Compositional Generalization Challenge Based on Semantic Interpretation. Najoung Kim and Tal Linzen. (paper) (code)

++ (Findings) Improving Compositional Generalization in Semantic Parsing. Inbar Oren, Jonathan Herzig, Nitish Gupta, Matt Gardner and Jonathan Berant. (paper) (code)

Visually Grounded Continual Learning of Compositional Phrases. Xisen Jin, Junyi Du, Arka Sadhu, Ram Nevatia and Xiang Ren. (paper) (code)

Language Grounding

See also the following workshops:

• Spatial Language Understanding (“Spatial Language Understanding, Learning/Reasoning on Spatial Semantics, (sub)Symbolic Representations and Grounding Language in Perception”)
• NLP Beyond Text

For fun: “Semantic Understanding of Proefessional Soccer Commentaries” (UAI 2012),Hannaneh Hajishirzi, Mohammad Rastegari, Ali Farhadi, and Jessica Hodgins. And
“Reasoning in Robocup Soccer Narratives” (UAI 11), Hannaneh Hajishirzi, Julia Hockenmaier, Erik T. Mueller, and Eyal Amir. For social aspects of language use: the Cards Corpus (http://cardscorpus.christopherpotts.net/).

++ Experience Grounds Language. Yonatan Bisk et al. (paper)

(Findings+WS) Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions. Peter Jansen. (paper) (code)

Vision:

++ (Honorable Mention) Visually Grounded Compound PCFGs. Yanpeng Zhao and Ivan Titov. (paper) (code)

++ Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. Hao Tan and Mohit Bansal. (paper) (code)

Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube. Jack Hessel, Zhenhai Zhu, Bo Pang and Radu Soricut. (paper) (code)

Domain-Specific Lexical Grounding in Noisy Visual-Textual Documents. Gregory Yauney, Jack Hessel and David Mimno. (paper) (code)

(Findings+WS) Robust and Interpretable Grounding of Spatial References with Relation Networks. Tsung-Yen Yang, Andrew Lan and Karthik Narasimhan. (paper) (code)

(Findings) Be Different to Be Better! A Benchmark to Leverage the Complementarity of Language and Vision. Sandro Pezzelle et al. (paper) (code)

Navigation and Game Playing:

Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding. Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie and Jason Baldridge. (paper) (code)

Sub-Instruction Aware Vision-and-Language Navigation. Yicong Hong, Cristian Rodriguez, Qi Wu and Stephen Gould. (paper) (code)

(Findings) ArraMon: A Joint Navigation-Assembly Instruction Interpretation Task in Dynamic Environments. Hyounghun Kim et al. (paper) (code)

Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games. Subhajit Chaudhury et al. (paper) (code)

Keep CALM and Explore: Language Models for Action Generation in Text-based Games. Shunyu Yao, Rohan Rao, Matthew Hausknecht and Karthik Narasimhan. (paper) (code)

Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning. Xiaoxiao Guo, Mo Yu, Yupeng Gao, Chuang Gan, Murray Campbell and Shiyu Chang. (paper) (code)

Multimodal

For VQA, see Visual Question Answering.

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers. Jaemin Cho, jiasen lu, Dustin Schwenk, Hannaneh Hajishirzi and Aniruddha Kembhavi. (preprint) (paper) (code and demo)

Learning to Represent Image and Text with Denotation Graph. Bowen Zhang, Hexiang Hu, Vihan Jain, Eugene Ie and Fei Sha. (paper) (code)

Reading Between the Lines: Exploring Infilling in Visual Narratives. Khyathi Raghavi Chandu, Ruo-Ping Dong and Alan W Black. (paper) (code)

SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning. Tsu-Jui Fu, Xin Wang, Scott Grafton, Miguel Eckstein and William Yang Wang. (paper) (code)

(Findings) Fine-Grained Grounding for Multimodal Speech Recognition. Tejas Srinivasan, Ramon Sanabria, Florian Metze and Desmond Elliott. (paper) (code)

(Findings) Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection. Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi and Weiping Wang. (paper) (code)

(Findings) DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding. Zilong Wang, Mingjie Zhan, xuebo liu and ding liang. (paper) (code)

(Findings) Dual Low-Rank Multimodal Fusion. Tao Jin, Siyu Huang, Yingming Li and Zhongfei Zhang. (paper) (code)

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding. Qinxin Wang, Hao Tan, Sheng Shen, Michael Mahoney and Zhewei Yao. (paper) (code)

++ Does my multimodal model learn cross-modal interactions? It’s harder to tell than you might think!. Jack Hessel and Lillian Lee. (paper) (code)

Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis. Yao-Hung Hubert Tsai, Martin Ma, Muqiao Yang, Ruslan Salakhutdinov and Louis-Philippe Morency. (preprint) (paper) (code)

Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning. Wanyun Cui, Guangyu Zheng and Wei Wang. (paper) (code)

Keyphrase extraction:

Incorporating Multimodal Information in Open-Domain Web Keyphrase Extraction. Yansen Wang, Zhen Fan and Carolyn Rose. (paper) (code)

Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings. Yue Wang, Jing Li, Michael Lyu and Irwin King. (paper) (code)

Fake news detection:

Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. Reuben Tan, Bryan Plummer and Kate Saenko. (paper) (code)

Video and language

What is More Likely to Happen Next? Video-and-Language Future Event Prediction. Jie Lei, Licheng Yu, Tamara Berg and Mohit Bansal. (paper) (code)

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training. Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu and Jingjing Liu. (paper) (code)

VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles. Mingzhe Li, Xiuying Chen, Shen Gao, Zhangming Chan, Dongyan Zhao and Rui Yan. (paper) (code)

Multistage Fusion with Forget Gate for Multimodal Summarization in Open-Domain Videos. Nayu Liu, Xian Sun, Hongfeng Yu, Wenkai Zhang and Guangluan Xu. (paper) (code)

Summarization:

(Findings) ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization. Tzuf Paz-Argaman, Reut Tsarfaty, Gal Chechik and Yuval Atzmon. (paper) (code)

Captioning:

Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements. Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li and Zhiwei Guan. (paper) (code)

(Findings+WS) Pragmatic Issue-Sensitive Image Captioning. Allen Nie, Reuben Cohn-Gordon and Christopher Potts. (paper) (code)

CapWAP: Image Captioning with a Purpose. Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan Clark and Regina Barzilay. (paper) (code)

(Findings) Semi-Supervised Learning for Video Captioning. Ke Lin, Zhuoxin Gan and Liwei WANG. (paper) (code)

Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning. Zhiyuan Fang, Tejas Gokhale, Pratyay Banerjee, Chitta Baral and Yezhou Yang. (paper) (code)

Multilingual

(Findings) On the Language Neutrality of Pre-trained Multilingual Representations. Jindřich Libovický, Rudolf Rosa and Alexander Fraser. (paper) (code)

Cross-lingual:

Zero-Shot Cross-Lingual Transfer with Meta Learning. Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva and Isabelle Augenstein. (preprint) (paper) (code)

Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages. Zheng Li, Mukul Kumar, William Headden, Bing Yin, Ying Wei, Yu Zhang and Qiang Yang. (paper) (code)

Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies. Maryam Aminian, Mohammad Sadegh Rasooli and Mona Diab. (paper) (code)

MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych and Sebastian Ruder. (paper) (code)

The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures. Haim Dubossarsky, Ivan Vulić, Roi Reichart and Anna Korhonen. (paper) (code)

Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank. Eleftheria Briakou and Marine Carpuat. (paper) (code)

End-to-End Slot Alignment and Recognition for Cross-Lingual NLU. Weijia Xu, Batool Haider and Saab Mansour. (paper) (code)

CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs. Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzmán and Philipp Koehn. (paper) (code)

Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings. Phillip Keung, Yichao Lu, Julian Salazar and Vikas Bhardwaj. (paper) (code)

(Findings+WS) Cross-lingual Alignment Methods for Multilingual BERT: A Comparative Study. Saurabh Kulshreshtha, Jose Luis Redondo Garcia and Ching-Yun Chang. (paper) (code)

(Findings+WS) Cross-Lingual Text Classification with Minimal Resources by Transferring a Sparse Teacher. Giannis Karamanolakis, Daniel Hsu and Luis Gravano. (paper) (code)

(Findings+WS) Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension. Adyasha Maharana and Mohit Bansal. (paper) (code)

(Findings) The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification. Abdullatif Köksal and Arzucan Özgür. (paper) (code)

Multilingual:

Do Explicit Alignments Robustly Improve Multilingual Encoders?. Shijie Wu and Mark Dredze. (paper) (code)

Improving Multilingual Models with Language-Clustered Vocabularies. Hyung Won Chung, Dan Garrette, Kiat Chuan Tan and Jason Riesa. (paper) (code)

On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment. Zirui Wang, Zachary C. Lipton and Yulia Tsvetkov. (paper) (code)

From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers. Anne Lauscher, Vinit Ravishankar, Ivan Vulić and Goran Glavaš. (paper) (code)

X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models. Zhengbao Jiang, Antonios Anastasopoulos, Jun Araki, Haibo Ding and Graham Neubig. (paper) (code)

Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages. Michael A. Hedderich, David Adelani, Dawei Zhu, Jesujoba Alabi, Udia Markus and Dietrich Klakow. (paper) (code)

BERT:

(Findings) Extending Multilingual BERT to Low-Resource Languages. Zihan Wang, Karthikeyan K, Stephen Mayhew and Dan Roth. (paper) (code)

Identifying Elements Essential for BERT’s Multilinguality. Philipp Dufter and Hinrich Schütze. (paper) (code)

A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT. Masaaki Nagata, Katsuki Chousa and Masaaki Nishino. (paper) (code)

Linguistic Typology:

Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions. Arya D. McCarthy, Adina Williams, Shijia Liu, David Yarowsky and Ryan Cotterell. (paper) (code)

Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations. Arturo Oncevay, Barry Haddow and Alexandra Birch. (paper) (code)

New dataset:

The Multilingual Amazon Reviews Corpus. Phillip Keung, Yichao Lu, György Szarvas and Noah A. Smith. (paper) (code)

NLG papers

Dialogue

Task-oriented

NOTE: several task/goal-oriented papers are missing.

++ How to Make Neural Natural Language Generation as Reliable as Templates in Task-Oriented Dialogue. Henry Elder, Alexander O’Connor and Jennifer Foster. (paper) (code)

++ (TACL) Task-Oriented Dialogue as Dataflow Synthesis. Jacob Andreas. (paper) (project) (code) (blog)

Dialogue generation:

Cross Copy Network for Dialogue Generation. Changzhen Ji, Xin Zhou, Yating Zhang, Xiaozhong Liu, Changlong Sun, Conghui Zhu and Tiejun Zhao. (paper) (code)

Counterfactual Off-Policy Training for Neural Dialogue Generation. Qingfu Zhu, Wei-Nan Zhang, Ting Liu and William Yang Wang. (paper) (code)

(Findings+WS) Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning. Yifan Gao, Piji Li, Wei Bi, Xiaojiang Liu, Michael Lyu and Irwin King. (paper) (code)

(Findings) Group-wise Contrastive Learning for Neural Dialogue Generation. Hengyi Cai, Hongshen Chen, Yonghao Song, zhuoye Ding, Yongjun Bao, Weipeng Yan and Xiaofang Zhao. (paper) (code)

(Findings) Pretrained Language Models for Dialogue Generation with Multiple Input Sources. Yu Cao, Wei Bi, Meng Fang and Dacheng Tao. (paper) (code)

(Findings) Plug-and-Play Conversational Models. Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri and Pascale Fung. (paper) (code)

(Findings) Learning to Plan and Realize Separately for Open-Ended Dialogue Systems. Sashank Santhanam et al. (paper) (code)

Regularizing Dialogue Generation by Imitating Implicit Scenarios. Shaoxiong Feng, Xuancheng Ren, Hongshen Chen, Bin Sun, Kan Li and Xu SUN. (paper) (code)

(Findings) SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training. Huda Khayrallah and João Sedoc. (paper) (code)

Generating Dialogue Responses from a Semantic Latent Space. Wei-Jen Ko, Avik Ray, Yilin Shen and Hongxia Jin. (paper) (code)

Knowledge-grounded dialogue:

(Findings) Difference-aware Knowledge Selection for Knowledge-grounded Conversation Generation. Chujie Zheng, Yunbo Cao, Daxin Jiang and Minlie Huang. (paper) (code)

AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue. Jaehun Jung, Bokyung Son and Sungwon Lyu. (paper) (code)

Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation. Xiuyi Chen, Fandong Meng, Peng Li, Feilong Chen, Shuang Xu, Bo Xu and Jie Zhou. (paper) (code)

(Findings) Improving Knowledge-Aware Dialogue Response Generation by Using Human-Written Prototype Dialogues. Sixing Wu, Ying Li, Dawei Zhang and Zhonghai Wu. (paper) (code)

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models. Xueliang Zhao, wei wu, Can Xu, Chongyang Tao, Dongyan Zhao and Rui Yan. (paper) (code)

(Findings) Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation. Wen Zheng, Natasa Milic-Frayling and Ke Zhou. (paper) (code)

Persona consistency:

(Findings) A Multi-Persona Chatbot for Hotline Counselor Training. Orianna Demasi, Yu Li and Zhou Yu. (paper) (code)

CHARM: Inferring Personal Attributes from Conversations. Anna Tigunova, Andrew Yates, Paramita Mirza and Gerhard Weikum. (paper) (code)

Profile Consistency Identification for Open-domain Dialogue Agents. Haoyu Song, Yan Wang, Wei-Nan Zhang, Zhengyu Zhao, Ting Liu and Xiaojiang Liu. (paper) (code)

(Findings+WS) Toward Stance-based Personas for Opinionated Dialogues. Thomas Scialom, Serra Sinem Tekiroğlu, Jacopo Staiano and Marco Guerini. (paper) (code)

Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness. Hyunwoo Kim, Byeongchang Kim and Gunhee Kim. (paper) (code)

Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions. Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick and Julian McAuley. (paper) (code)

Towards Persona-Based Empathetic Conversational Models. Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu and Chunyan Miao. (preprint) (paper) (data and code)

Conversation disentanglement:

Online Conversation Disentanglement with Pointer Networks. Tao Yu and Shafiq Joty. (paper) (code)

Dialogue state tracking:

Slot Attention with Value Normalization for Multi-Domain Dialogue State Tracking. Yexiang Wang, Yi Guo and Siqi Zhu. (paper) (code)

Conversational Semantic Parsing for Dialog State Tracking. Jianpeng Cheng et al. (paper) (code)

(Findings) Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking. Su Zhu, Jieyu Li, Lu Chen and Kai Yu. (paper) (code)

(Findings) GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking. Peng Wu, Bowei Zou, Ridong Jiang and AiTi Aw. (paper) (code)

(Findings) Neural Dialogue State Tracking with Temporally Expressive Networks. Junfan Chen, Richong Zhang, Yongyi Mao and Jie Xu. (paper) (code)

Evaluation:

GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems. Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin and Xiaodan Liang. (paper) (code)

++ (Honorable Mention) Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems. Jan Deriu et al. (paper) (code)

New dataset:

MovieChats: Chat like Humans in a Closed Domain. Hui Su, Xiaoyu Shen, Zhou Xiao, Zheng Zhang, Ernie Chang, Cheng Zhang, Cheng Niu and Jie Zhou. (paper) (code) (in Chinese)

Information Seeking in the Spirit of Learning: A Dataset for Conversational Curiosity. Pedro Rodriguez, Paul Crook, Seungwhan Moon and Zhiguang Wang. (paper) (code)

A Visually-grounded First-person Dialogue Dataset with Verbal and Non-verbal Responses. Hisashi Kamezawa, Noriki Nishida, Nobuyuki Shimizu, Takashi Miyazaki and Hideki Nakayama. (paper) (code)

MedDialog: Large-scale Medical Dialogue Datasets. Guangtao Zeng et al. (paper) (data)

Response selection:

Response Selection for Multi-Party Conversations with Dynamic Topic Tracking. Weishi Wang, Steven C.H. Hoi and Shafiq Joty. (paper) (code)

Multi-turn Response Selection using Dialogue Dependency Relations. Qi Jia, Yizhu Liu, Siyu Ren, Kenny Zhu and Haifeng Tang. (preprint) (paper) (code)

The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection. Zibo Lin, Deng Cai, Yan Wang, Xiaojiang Liu, Haitao Zheng and Shuming Shi. (preprint) (paper) (code)

Dialogue Response Ranking Training with Large-Scale Human Feedback Data. Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett and Bill Dolan. (paper) (code)

Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks. YUFAN ZHAO, Can Xu and Wei Wu. (paper) (code)

MIME: MIMicking Emotions for Empathetic Response Generation. Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea and Soujanya Poria. (paper) (code)

(Findings+WS) Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots. Jia-Chen Gu et al. (paper) (code)

(Findings) Focus-Constrained Attention Mechanism for CVAE-based Response Generation. Zhi Cui, Yanran Li, Jiayi ZHANG, Jianwei Cui, Chen Wei and Bin Wang. (paper) (code)

Other:

Where Are You? Localization from Embodied Dialog. Meera Hahn, Jacob Krantz, Dhruv Batra, Devi Parikh, James Rehg, Stefan Lee and Peter Anderson. (paper) (code)

Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data. Rongsheng Zhang, Yinhe Zheng, Jianzhi Shao, Xiaoxi Mao, Yadong Xi and Minlie Huang. (paper) (code)

(Findings) Understanding User Resistance Strategies in Persuasive Conversations. Youzhi Tian, Weiyan Shi, Chen Li and Zhou Yu. (paper) (code)

Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts. Ece Takmaz, Mario Giulianelli, Sandro Pezzelle, Arabella Sinclair and Raquel Fernández. (paper) (code)

Personal Information Leakage Detection in Conversations. Qiongkai Xu, Lizhen Qu, Zeyu Gao and Gholamreza Haffari. (paper) (code)

Structured Attention for Unsupervised Dialogue Structure Induction. Liang Qiu et al. (preprint) (paper) (code)

Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations. Lingzhi Wang, Jing Li, Xingshan Zeng, Haisong Zhang and Kam-Fai Wong. (paper) (code)

(Findings+WS) ConveRT: Efficient and Accurate Conversational Representations from Transformers. Matthew Henderson et al. (paper) (code)

(Findings+WS) RMM: A Recursive Mental Model for Dialogue Navigation. Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz and Jianfeng Gao. (paper) (code)

(Findings) A Compare Aggregate Transformer for Understanding Document-grounded Dialogue. Longxuan Ma, Wei-Nan Zhang, runxin sun and Ting Liu. (paper) (code)

(Findings) TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog. Erik Ekstedt and Gabriel Skantze. (paper) (code)

(Findings) Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles. Carel van Niekerk et al. (paper) (code)

BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues. Hung Le, Doyen Sahoo, Nancy Chen and Steven C.H. Hoi. (paper) (code)

VD-BERT: A Unified Vision and Dialog Transformer with BERT. Yue Wang, Shafiq Joty, Michael Lyu, Irwin King, Caiming Xiong and Steven C.H. Hoi. (preprint) (paper) (code)

Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation. Kang Min Yoo, Hanbit Lee, Franck Dernoncourt, Trung Bui, Walter Chang and Sang-goo Lee. (paper) (code)

Parallel Interactive Networks for Multi-Domain Dialogue State Generation. Junfan Chen, Richong Zhang, Yongyi Mao and Jie Xu. (paper) (code)

Human-centric dialog training via offline reinforcement learning. Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu and Rosalind Picard. (paper) (code)

INSPIRED: Toward Sociable Recommendation Dialog Systems. Shirley Anugrah Hayati, Dongyeop Kang, Qingxiaoyang Zhu, Weiyan Shi and Zhou Yu. (paper) (code)

(Findings) Helpful or Hierarchical? Predicting the Communicative Strategies of Chat Participants, and their Impact on Success. Farzana Rashid et al. (paper) (code)

++ Sound Natural: Content Rephrasing in Dialog Systems. Arash Einolghozati, Anchit Gupta, Keith Diedrick and Sonal Gupta. (paper) (code)

Sequence to Sequence

(Findings) ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training. Weizhen Qi et al. (paper) (code)

Seq2Edits: Sequence Transduction Using Span-level Edit Operations. Felix Stahlberg and Shankar Kumar. (paper) (code)

Semantic Label Smoothing for Sequence to Sequence Problems. Michal Lukasik et al. (paper) (code)

(Findings) Semantic Matching for Sequence-to-Sequence Learning. Ruiyi Zhang, Changyou Chen, Xinyuan Zhang, Ke Bai and Lawrence Carin. (paper) (code)

A Streaming Approach For Efficient Batched Beam Search. Kevin Yang, Violet Yao, John DeNero and Dan Klein. (paper) (code)

Text Generation

Open-ended text generation

Sparse Text Generation. Pedro Henrique Martins, Zita Marinho and André F. T. Martins. (paper) (code)

(Findings+WS) TSDG: Content-aware Neural Response Generation with Two-stage Decoding Process. Junsheng Kong, Zhicheng Zhong, Yi Cai, Xin Wu and Da Ren. (paper) (code)

++ Authorship Attribution for Neural Text Generation. Adaku Uchendu, Thai Le, Kai Shu and Dongwon Lee. (preprint) (paper) (code)

Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation. Minki Kang, Moonsu Han and Sung Ju Hwang. (preprint) (paper) (code)

UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation. Jian Guan and Minlie Huang. (preprint) (paper) (code)

F^2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax. Byung-Ju Choi, Jimin Hong, David Park and Sang Wan Lee. (preprint) (paper) (code)

Gradient-guided Unsupervised Lexically Constrained Text Generation. Lei Sha. (paper) (code)

++ BLEU might be Guilty but References are not Innocent. Markus Freitag, David Grangier and Isaac Caswell. (preprint) (paper) (code)

Meaning-to-text generation

Controllable Meaning Representation to Text Generation: Linearization and Data Augmentation Strategies. Chris Kedzie and Kathleen McKeown. (paper) (code)

(Findings) Logic2Text: High-Fidelity Natural Language Generation from Logical Forms. Zhiyu Chen et al. (paper) (code)

AMR-to-text:

Multilingual AMR-to-Text Generation. Angela Fan and Claire Gardent. (paper) (code)

Online Back-Parsing for AMR-to-Text Generation. Xuefeng Bai, Linfeng Song and Yue Zhang. (paper) (code)

Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation. Yan Zhang, Zhijiang Guo, Zhiyang Teng, Wei Lu, Shay B. Cohen, ZUOZHU LIU and Lidong Bing. (paper) (code)

Data-to-text generation

ToTTo: A Controlled Table-To-Text Generation Dataset. Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang and Dipanjan Das. (paper) (code)

(Findings) Enhancing Content Planning for Table-to-Text Generation with Data Understanding and Verification. Heng Gong et al. (paper) (code)

(Findings) Data-to-Text Generation with Style Imitation. Shuai Lin, Wentao Wang, Zichao Yang, Xiaodan Liang, Frank F. Xu, Eric Xing and Zhiting Hu. (paper) (code)

Partially-Aligned Data-to-Text Generation with Distant Supervision. Zihao Fu, Bei Shi, Wai Lam, Lidong Bing and Zhiyuan Liu. (paper) (code)

KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation. Wenhu Chen, Yu Su, Xifeng Yan and William Yang Wang. (paper) (code)

(Findings) Make Templates Smarter: A Template Based Data2Text System Powered by Text Stitch Model. Bingfeng Luo, Zuo Bai, Kunfeng Lai and Jianping Shen. (paper) (code)

(Findings) How Decoding Strategies Affect the Verifiability of Generated Text. Luca Massarelli et al. (paper) (code)

MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. Peng Xu et al. (paper) (code)

++ Content Planning for Neural Story Generation with Aristotelian Rescoring. Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel and Nanyun Peng. (paper) (code)

PathQG: Neural Question Generation from Facts. Siyuan Wang, Zhongyu Wei, Zhihao Fan, Zengfeng Huang, Weijian Sun, Qi ZHANG and Xuanjing Huang. (paper) (code)

Inquisitive Question Generation for High Level Text Comprehension. Wei-Jen Ko, TE-YUAN CHEN, Yiyan Huang, Greg Durrett and Junyi Jessy Li. (paper) (code)

PALM: Pre-training an Autoencoding & Autoregressive Language Model for Context-conditioned Generation. Bin Bi et al. (paper) (code)

Homophonic Pun Generation with Lexically Constrained Rewriting. Zhiwei Yu, Hongyu Zang and Xiaojun Wan. (paper) (code)

++ COD3S: Diverse Generation with Discrete Semantic Signatures. Nathaniel Weir, João Sedoc and Benjamin Van Durme. (paper) (code)

Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations. Wanrong Zhu et al. (paper) (code)

Investigating African-American Vernacular English in Transformer-Based Text Generation. Sophie Groenwold et al. (paper) (code)

(Findings+WS) GRUEN for Evaluating Linguistic Quality of Generated Text. Wanzheng Zhu and Suma Bhat. (paper) (code)

(Findings) Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation. Giuseppe Russo, Nora Hollenstein, Claudiu Cristian Musat and Ce Zhang. (paper) (code)

(Findings) Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo Approach. Maosen Zhang, Nan Jiang, Lei Li and Yexiang Xue. (paper) (code)

(Findings) StyleDGPT: Stylized Response Generation with Pre-trained Language Models. Ze Yang, wei wu, Can Xu, Xinnian Liang, jiaqi bai, Liran Wang, Wei Wang and Zhoujun Li. (paper) (code)

(Findings) A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning. Deepak Gupta, Asif Ekbal and Pushpak Bhattacharyya. (paper) (code)

(Findings) Recursive Top-Down Production for Sentence Generation with Latent Trees. Shawn Tan, Yikang Shen, Alessandro Sordoni, Aaron Courville and Timothy J. O’Donnell. (paper) (code)

(Findings) Target Conditioning for One-to-Many Generation. Marie-Anne Lachaux, Armand Joulin and Guillaume Lample. (paper) (code)

(Findings) Regularization of Distinct Strategies for Unsupervised Question Generation. Junmo Kang, Giwon Hong, Haritz Puerto San Roman and Sung-Hyon Myaeng. (paper) (code)

++ (Findings) Narrative Text Generation with a Latent Discrete Plan. Harsh Jhamtani and Taylor Berg-Kirkpatrick. (paper) (code)

(Findings) Controllable Text Generation with Focused Variation. Lei Shu et al. (paper) (code)

(Findings) Adversarial Text Generation via Sequence Contrast Discrimination. Ke Wang and Xiaojun Wan. (paper) (code)

(Findings) Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data. Katja Filippova. (paper) (code)

(Findings) Incorporating Stylistic Lexical Preferences in Generative Language Models. Hrituraj Singh, Gaurav Verma and Balaji Vasan Srinivasan. (paper) (code)

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace and Sameer Singh. (paper) (code)

++ An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing. Martin Schmitt, Sahand Sharifzadeh, Volker Tresp and Hinrich Schütze. (paper) (code)

(Findings) A Sentiment-Controllable Topic-to-Essay Generator with Topic Knowledge Graph. Lin Qiao, Jianhao Yan, Fandong Meng, Zhendong Yang and Jie Zhou. (paper) (code)

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph. Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu and Minlie Huang. (paper) (code)

(Findings) CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning. Bill Yuchen Lin et al. (paper) (code)

Acrostic Poem Generation. Rajat Agarwal and Katharina Kann. (paper) (code)

Plug and Play Autoencoders for Conditional Text Generation. Florian Mai, Nikolaos Pappas, Ivan Montero, Noah A. Smith and James Henderson. (paper) (code)

PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking. Hannah Rashkin, Asli Celikyilmaz, Yejin Choi and Jianfeng Gao. (paper) (code)

POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training. Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett and Bill Dolan. (paper) (code)

STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation. Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng and Mohit Iyyer. (paper) (code)

Improving Text Generation with Student-Forcing Optimal Transport. Jianqiao Li et al. (paper) (code)

PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation. Xinyu Hua and Lu Wang. (paper) (code)

Routing Enforced Generative Model for Recipe Generation. Zhiwei Yu, Hongyu Zang and Xiaojun Wan. (paper) (code)

OCR post-correction

++ OCR Post Correction for Endangered Language Texts. Shruti Rijhwani, Antonios Anastasopoulos and Graham Neubig. (paper) (code)

Summarization

What Have We Achieved on Text Summarization?. Dandan Huang, Leyang Cui, Sen Yang, Guangsheng Bao, Kun Wang, Jun Xie and Yue Zhang. (paper) (code)

Intrinsic Evaluation of Summarization Datasets. Rishi Bommasani and Claire Cardie. (paper) (code)

Modeling Content Importance for Summarization with Pre-trained Language Models. Liqiang Xiao, Lu Wang, Hao He and Yaohui Jin. (paper) (code)

Short-text generation:

A Preliminary Exploration of GANs for Keyphrase Generation. Avinash Swaminathan et al. (paper) (code)

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation. Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv and Nan Duan. (paper) (code)

Incorporating background knowledge:

(Findings) KLearn: Background Knowledge Inference from Summarization Data. Maxime Peyrard and Robert West. (paper) (code)

(Findings) TLDR: Extreme Summarization of Scientific Documents. Isabel Cachola, Kyle Lo, Arman Cohan and Daniel Weld. (paper) (code)

Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach. Bowen Tan, Lianhui Qin, Eric Xing and Zhiting Hu. (paper) (code)

Using Reinforcement Learning:

Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning. Yuning Mao, Yanru Qu, Yiqing Xie, Xiang Ren and Jiawei Han. (preprint) (paper) (code)

Q-learning with Language Model for Edit-based Unsupervised Summarization. Ryosuke Kohita, Akifumi Wachi, Yang Zhao and Ryuki Tachibana. (paper) (code)

Abstractive:

Understanding Neural Abstractive Summarization Models via Uncertainty. Jiacheng Xu, Shrey Desai and Greg Durrett. (paper) (code)

On Extractive and Abstractive Neural Document Summarization with Transformer Language Models. Jonathan Pilault, Raymond Li, Sandeep Subramanian and Chris Pal. (paper) (code)

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization. Jiaao Chen and Diyi Yang. (paper) (code)

Friendly Topic Assistant for Transformer Based Abstractive Summarization. Zhengjue Wang, Zhibin Duan, Hao Zhang, chaojie wang, long tian, Bo Chen and Mingyuan Zhou. (paper) (code)

Pre-training for Abstractive Document Summarization by Reinstating Source Text. Yanyan Zou, Xingxing Zhang, Wei Lu, Furu Wei and Ming Zhou. (paper) (code)

(Findings+WS) A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining. Chenguang Zhu, Ruochen Xu, Michael Zeng and Xuedong Huang. (paper) (code)

(Findings) Reducing Quantity Hallucinations in Abstractive Summarization. Zheng Zhao, Shay B. Cohen and Bonnie Webber. (paper) (code)

Extractive:

Compressive Summarization with Plausibility and Salience Modeling. Shrey Desai, Jiacheng Xu and Greg Durrett. (paper) (code)

Better Highlighting: Creating Sub-Sentence Summary Highlights. Sangwoo Cho, Kaiqiang Song, Chen Li, Dong Yu, Hassan Foroosh and Fei Liu. (paper) (code)

Stepwise Extractive Summarization and Planning with Structured Transformers. Shashi Narayan, Joshua Maynez, Jakub Adamek, Daniele Pighin, Blaz Bratanic and Ryan McDonald. (paper) (code)

(Findings+WS) SupMMD: A Sentence Importance Model for Extractive Summarization using Maximum Mean Discrepancy. Umanga Bista, Alexander Mathews, Aditya Menon and Lexing Xie. (paper) (code)

Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network. Ruipeng Jia, Yanan Cao, Hengzhu Tang, Fang Fang, Cong Cao and Shi Wang. (paper) (code)

(Findings) Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization. Zhengyuan Liu, Ke Shi and Nancy Chen. (paper) (code)

(Findings) Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers. Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei and Ming Zhou. (paper) (code)

Multi-document:

(Findings) Corpora Evaluation and System Bias Detection in Multi-document Summarization. Alvin Dey, Tanya Chowdhury, Yash Kumar and Tanmoy Chakraborty. (paper) (code)

Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles. Yao Lu, Yue Dong and Laurent Charlin. (paper) (code)

A Spectral Method for Unsupervised Multi-Document Summarization. Kexiang Wang, Baobao Chang and Zhifang Sui. (paper) (code)

Coarse-to-Fine Query Focused Multi-Document Summarization. Yumo Xu and Mirella Lapata. (paper) (code)

(Findings) Abstractive Multi-Document Summarization via Joint Learning with Single-Document Summarization. Hanqi Jin and Xiaojun Wan. (paper) (code)

Evaluation:

Re-evaluating Evaluation in Text Summarization. Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, Pengfei Liu and Graham Neubig. (paper) (code)

Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning. Hanlu Wu, Tengfei Ma, Lingfei Wu, Tariro Manyumwa and Shouling Ji. (paper) (code)

(Findings+WS) CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems. Yiran Chen et al. (paper) (code)

Factuality evaluation and correction:

++ (Findings) Evaluating Factuality in Generation with Dependency-level Entailment. Tanya Goyal and Greg Durrett. (paper) (code)

Evaluating the Factual Consistency of Abstractive Text Summarization. Wojciech Kryscinski, Bryan McCann, Caiming Xiong and Richard Socher. (paper) (code)

Factual Error Correction for Abstractive Summarization Models. Meng Cao, Yue Dong, Jiapeng Wu and Jackie Chi Kit Cheung. (paper) (code)

Multi-Fact Correction in Abstractive Text Summarization. Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung and Jingjing Liu. (paper) (code)

Other:

Multi-hop Inference for Question-driven Summarization. Yang Deng, Wenxuan Zhang and Wai Lam. (paper) (code)

Learning to Fuse Sentences with Transformers for Summarization. Logan Lebanoff, Franck Dernoncourt, Doo Soon Kim, Lidan Wang, Walter Chang and Fei Liu. (paper) (code)

(Findings) TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising. Ziyi Yang et al. (paper) (code)

New dataset:

(Findings) WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization. Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown. (paper) (code)

MLSUM: The Multilingual Summarization Corpus. Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski and Jacopo Staiano. (paper) (code)

Few-shot/zero-shot:

Few-Shot Learning for Opinion Summarization. Arthur Bražinskas, Mirella Lapata and Ivan Titov. (paper) (code)

(Findings) Towards Zero-Shot Conditional Summarization with Adaptive Multi-Task Fine-Tuning. Travis Goodwin, Max Savery and Dina Demner-Fushman. (paper) (code)

Simplification

Zero-Shot Crosslingual Sentence Simplification. Jonathan Mallinson, Rico Sennrich and Mirella Lapata. (paper) (code)

(Findings+WS) PharmMT: A Neural Machine Translation Approach to Simplify Prescription Directions. Jiazhao Li et al. (paper) (code)

Machine Translation

See also the Fifth Conference on Machine Translation (WMT20) at this conference.

Towards Enhancing Faithfulness for Neural Machine Translation. Rongxiang Weng, Heng Yu, Xiangpeng Wei and Weihua Luo. (paper) (code)

(Findings+WS) On Long-Tailed Phenomena in Neural Machine Translation. Vikas Raunak, Siddharth Dalmia, Vivek Gupta and Florian Metze. (paper) (code)

Evaluation:

(Findings+WS) KoBE: Knowledge-Based Machine Translation Evaluation. Zorik Gekhman, Roee Aharoni, Genady Beryozkin, Markus Freitag and Wolfgang Macherey. (paper) (code)

COMET: A Neural Framework for MT Evaluation. Ricardo Rei, Craig Stewart, Ana C Farinha and Alon Lavie. (paper) (code)

Translation Quality Estimation by Jointly Learning to Score and Rank. Jingyi Zhang and Josef van Genabith. (paper) (code)

Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing. Brian Thompson and Matt Post. (paper) (code)

(Findings) The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation. Jie He, Tao Wang, Deyi Xiong and Qun Liu. (paper) (code)

(Findings) It’s not a Non-Issue: Negation as a Source of Error in Machine Translation. Md Mosharaf Hossain, Antonios Anastasopoulos, Eduardo Blanco and Alexis Palmer. (paper) (code)

Statistical Power and Translationese in Machine Translation Evaluation. Yvette Graham, Barry Haddow and Philipp Koehn. (paper) (code)

Low-resource:

Language Model Prior for Low-Resource Neural Machine Translation. Christos Baziotis, Barry Haddow and Alexandra Birch. (paper) (code)

Simulated Multiple Reference Training Improves Low-Resource Machine Translation. Huda Khayrallah, Brian Thompson, Matt Post and Philipp Koehn. (paper) (code)

ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization. Shiyue Zhang, Benjamin Frey and Mohit Bansal. (paper) (code)

(Findings) Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages. Wilhelmina Nekoto et al. (paper) (code)

Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT. Alexandra Chronopoulou, Dario Stojanovski and Alexander Fraser. (paper) (code)

Multilingual:

See also Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations. Arturo Oncevay, Barry Haddow and Alexandra Birch..

(Findings) A Multilingual View of Unsupervised Machine Translation. Xavier Garcia, Pierre Foret, Thibault Sellam and Ankur Parikh. (paper) (code)

(Findings) Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation. Luyu Gao, Xinyi Wang and Graham Neubig. (paper) (code)

Multi-task Learning for Multilingual Neural Machine Translation. Yiren Wang, ChengXiang Zhai and Hany Hassan. (paper) (code)

Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information. Zehui Lin et al. (paper) (code)

Multi-domain/domain adaptation:

(Findings) Vocabulary Adaptation for Domain Adaptation in Neural Machine Translation. Shoetsu Sato, Jin Sakuma, Naoki Yoshinaga, Masashi Toyoda and Masaru Kitsuregawa. (paper) (code)

Distilling Multiple Domains for Neural Machine Translation. Anna Currey, Prashant Mathur and Georgiana Dinu. (paper) (code)

(Findings) Factorized Transformer for Multi-Domain Neural Machine Translation. Yongchao Deng, Hongfei Yu, Heng Yu, Xiangyu Duan and Weihua Luo. (paper) (code)

Zero-shot:

Monolingual Adapters for Zero-Shot Neural Machine Translation. Jerin Philip, Alexandre Berard, Matthias Gallé and Laurent Besacier. (paper) (code)

(Findings) Sparse and Decorrelated Representations for Stable Zero-shot NMT. Bokyung Son and Sungwon Lyu. (paper) (code)

Non-autoregressive:

Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation. Jason Lee, Raphael Shu and Kyunghyun Cho. (paper) (code)

Non-Autoregressive Machine Translation with Latent Alignments. Chitwan Saharia, William Chan, Saurabh Saxena and Mohammad Norouzi. (paper) (code)

Incorporating a Local Translation Mechanism into Non-autoregressive Translation. Xiang Kong, Zhisong Zhang and Eduard Hovy. (paper) (code)

Document-level:

Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation. Pei Zhang, Boxing Chen, Niyu Ge and Kai Fan. (paper) (code)

Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning. Xiaomian Kang, Yang Zhao, Jiajun Zhang and Chengqing Zong. (paper) (code)

Post-editing:

(Findings+WS) Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing. Ke Wang, Jiayi Wang, Niyu Ge, Yangbin Shi, Yu Zhao and Kai Fan. (paper) (code)

Can Automatic Post-Editing Improve NMT?. Shamil Chollampatt, Raymond Hendy Susanto, Liling Tan and Ewa Szymanska. (paper) (code)

Other:

Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses. Prathyusha Jwalapuram, Shafiq Joty and Youlin Shen. (paper) (code)

Towards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems. Jindřich Libovický and Alexander Fraser. (paper) (code)

Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation. Dana Ruiter, Josef van Genabith and Cristina España-Bonet. (paper) (code)

Uncertainty-Aware Semantic Augmentation for Neural Machine Translation. Xiangpeng Wei, Heng Yu, Yue Hu, Rongxiang Weng, Luxi Xing and Weihua Luo. (paper) (code)

CSP: Code-Switching Pre-training for Neural Machine Translation. Zhen Yang, Bojie Hu, ambyera han, shen huang and Qi Ju. (preprint) (paper) (code)

Shallow-to-Deep Training for Neural Machine Translation. Bei Li, Ziyang Wang, Hui Liu, Yufan Jiang, Quan Du, Tong Xiao, Huizhen Wang and Jingbo Zhu. (preprint) (paper) (code)

Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation. Mehrad Moradshahi, Giovanni Campagna, Sina Semnani, Silei Xu and Monica Lam. (paper) (code)

Accurate Word Alignment Induction from Neural Machine Translation. Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang and Qun Liu. (paper) (code)

Token-level Adaptive Training for Neural Machine Translation. Shuhao Gu, Jinchao Zhang, Fandong Meng, Yang Feng, Wanying Xie, Jie Zhou and Dong Yu. (paper) (code)

Multi-Unit Transformers for Neural Machine Translation. Jianhao Yan, Fandong Meng and Jie Zhou. (paper) (code)

++ An Empirical Study of Generation Order for Machine Translation. William Chan, Mitchell Stern, Jamie Kiros and Jakob Uszkoreit. (paper) (code)

Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation. Wenxiang Jiao, Xing Wang, Shilin He, Irwin King, Michael Lyu and Zhaopeng Tu. (paper) (code)

Self-Paced Learning for Neural Machine Translation. Yu Wan, Baosong Yang, Derek F. Wong, Yikai Zhou, Lidia S. Chao, Haibo Zhang and Boxing Chen. (paper) (code)

Inference Strategies for Machine Translation with Conditional Masking. Julia Kreutzer, George Foster and Colin Cherry. (paper) (code)

On the Sparsity of Neural Machine Translation Models. Yong Wang, Longyue Wang, Victor Li and Zhaopeng Tu. (paper) (code)

(Findings) Fully Quantized Transformer for Machine Translation. Gabriele Prato, Ella Charlaix and Mehdi Rezagholizadeh. (preprint) (paper) (code)

(Findings+WS) Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation. Alessandro Raganato, Yves Scherrer and Jörg Tiedemann. (paper) (code)

(Findings) Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation. Baohao Liao, Yingbo Gao and Hermann Ney. (paper) (code)

(Findings) Finding the Optimal Vocabulary Size for Neural Machine Translation. Thamme Gowda and Jonathan May. (paper) (code)

(Findings) Adversarial Subword Regularization for Robust Neural Machine Translation. Jungsoo Park, Mujeen Sung, Jinhyuk Lee and Jaewoo Kang. (paper) (code)

(Findings) Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation. Weijia Xu, Xing Niu and Marine Carpuat. (paper) (code)

(Findings) Reference Language based Unsupervised Neural Machine Translation. Zuchao Li, Hai Zhao, Rui Wang, Masao Utiyama and Eiichiro Sumita. (paper) (code)

(Findings) Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation. Qiang Wang, Tong Xiao and Jingbo Zhu. (paper) (code)

Generating Diverse Translation from Model Distribution with Dropout. Xuanfu Wu, Yang Feng and Chenze Shao. (paper) (code)

Back-translation:

Iterative Domain-Repaired Back-Translation. Hao-Ran Wei, Zhirui Zhang, Boxing Chen and Weihua Luo. (paper) (code)

Dynamic Data Selection and Weighting for Iterative Back-Translation. Zi-Yi Dou, Antonios Anastasopoulos and Graham Neubig. (paper) (code)

Simulatenous Translation:

Learning Adaptive Segmentation Policy for Simultaneous Translation. Ruiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua Wu and Haifeng Wang. (paper) (code)

(Demo paper) SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation. Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, Juan Pino (paper) (code)

(Findings) Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training. Renjie Zheng et al. (paper) (code)

Simultaneous Translation, Multimodal:

Simultaneous Machine Translation with Visual Context. Ozan Caglayan, Julia Ive, Veneta Haralampieva, Pranava Madhyastha, Loïc Barrault and Lucia Specia. (paper) (code)

Towards Multimodal Simultaneous Neural Machine Translation. Aizhan Imankulova, Masahiro Kaneko, Tosho Hirasawa, Mamoru Komachi (paper) (code)

Style Transfer

++ Reformulating Unsupervised Style Transfer as Paraphrase Generation. Kalpesh Krishna, John Wieting and Mohit Iyyer. (paper) (project) (code)

Blank Language Models. Tianxiao Shen, Victor Quach, Regina Barzilay and Tommi Jaakkola. (paper) (code)

DGST: a Dual-Generator Network for Text Style Transfer. Xiao Li, Guanyi Chen, Chenghua Lin and Ruizhe Li. (paper) (code)

Unsupervised Text Style Transfer with Padded Masked Language Models. Eric Malmi, Aliaksei Severyn and Sascha Rothe. (paper) (code)

(Findings) Contextual Text Style Transfer. Yu Cheng, Zhe Gan, Yizhe Zhang, Oussama Elachqar, Dianqi Li and Jingjing Liu. (paper) (code)

(Findings) Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization. Kunal Chawla and Diyi Yang. (paper) (code)

Facilitating the Communication of Politeness through Fine-Grained Paraphrasing. Liye Fu, Susan Fussell and Cristian Danescu-Niculescu-Mizil. (preprint) (paper) (code) (video)

(Findings) From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer. Paulo Cavalin et al. (paper) (code)

Paraphrasing

(Findings) Robustness to Modification with Shared Words in Paraphrase Identification. Zhouxing Shi and Minlie Huang. (paper) (code)

(Findings+WS) A Study in Improving BLEU Reference Coverage with Diverse Automatic Paraphrasing. Rachel Bawden, Biao Zhang, Lisa Yankovskaya, Andre Tättar and Matt Post. (paper) (code)

(Findings) DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network. Yue Cao and Xiaojun Wan. (paper) (code)

NLU papers

Text Classification

META: Metadata-Empowered Weak Supervision for Text Classification. Dheeraj Mekala, Xinyang Zhang and Jingbo Shang. (paper) (code)

Hyperbolic models:

(Findings+WS) A Fully Hyperbolic Neural Model for Hierarchical Multi-Class Classification. Federico López and Michael Strube. (paper) (code)

(Findings) HyperText: Endowing FastText with Hyperbolic Geometry. Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen and Qun Liu. (paper) (code)

Robust text classification:

(Findings+WS) Identifying Spurious Correlations for Robust Text Classification. Zhao Wang and Aron Culotta. (paper) (code)

(Findings) Assessing Robustness of Text Classification through Maximal Safe Radius Computation. Emanuele La Malfa et al. (paper) (code)

Large-scale text classification:

(Findings) Multi-pretraining for Large-scale Text Classification. Kang-Min Kim, Bumsu Hyeon, Yeachan Kim, Jun-Hyung Park and SangKeun Lee. (paper) (code)

An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels. Ilias Chalkidis et al. (paper) (code)

Other:

Text Classification Using Label Names Only: A Language Model Self-Training Approach. Yu Meng, Yunyi Zhang, Jiaxin Huang, Chenyan Xiong, Heng Ji, Chao Zhang and Jiawei Han. (paper) (code)

FIND: Human-in-the-Loop Debugging Deep Text Classifiers. Piyawat Lertvittayakumjorn, Lucia Specia and Francesca Toni. (paper) (code)

Text Graph Transformer for Document Classification. Haopeng Zhang and Jiawei Zhang. (paper) (code)

(Findings) Cascaded Semantic and Positional Self-Attention Network for Document Classification. Juyong Jiang, Jie Zhang and Kai Zhang. (paper) (code)

(Findings) Balancing via Generation for Multi-Class Text Classification Improvement. Naama Tepper et al. (paper) (code)

HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification. Wenshuo Yang et al. (paper) (code)

Task-oriented Domain-specific Meta-Embedding for Text Classification. Xin Wu, Yi Cai, Yang Kai, Tao Wang and Qing Li. (paper) (code)

Less is More: Attention Supervision with Counterfactuals for Text Classification. Seungtaek Choi, Haeju Park, Jinyoung Yeo and Seung-won Hwang. (paper) (code)

Towards More Accurate Uncertainty Estimation In Text Classification. Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao and ChangTien Lu. (paper) (code)

Be More with Less: Hypergraph Attention Networks for Inductive Text Classification. Kaize Ding, Jianling Wang, Jundong Li, Dingcheng Li and Huan Liu. (paper) (code)

Information Retrieval

Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2. Yosi Mass and Haggai Roitman. (paper) (code)

Querying Across Genres for Medical Claims in News. Chaoyuan Zuo, Narayan Acharya and Ritwik Banerjee. (paper) (code)

(Findings) Document Ranking with a Pretrained Sequence-to-Sequence Model. Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep and Jimmy Lin. (paper) (code)

(Findings) Long Document Ranking with Query-Directed Sparse Transformer. Jyun-Yu Jiang, Chenyan Xiong, Chia-Jung Lee and Wei Wang. (paper) (code)

Cross/multi-lingual:

CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval. Shuo Sun and Kevin Duh. (paper) (code)

(Findings) Cross-Lingual Training of Neural Models for Document Ranking. Peng Shi, He Bai and Jimmy Lin. (paper) (code)

LAReQA: Language-Agnostic Answer Retrieval from a Multilingual Pool. Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips and Yinfei Yang. (paper) (code)

Sequence Labeling

To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging. Kasturi Bhattacharjee et al. (paper) (code)

AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network. Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang and Kewei Tu. (paper) (code)

Augmented Natural Language for Generative Sequence Labeling. Ben Athiwaratkun, Cicero Nogueira dos Santos, Jason Krone and Bing Xiang. (preprint) (paper) (code)

Uncertainty-Aware Label Refinement for Sequence Labeling. Tao Gui, Jiacheng Ye, Qi Zhang, Zhengyan Li, Zichu Fei, Yeyun Gong and Xuanjing Huang. (paper) (code)

An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks. Lifu Tu, Tianyu Liu and Kevin Gimpel. (paper) (code)

(Findings) Attending to Long-Distance Document Context for Sequence Labeling. Matthew Jörke, Jon Gillick, Matthew Sims and David Bamman. (paper) (code)

POS tagging

Unsupervised Cross-Lingual Part-of-Speech Tagging for Truly Low-Resource Scenarios. Ramy Eskander, Smaranda Muresan and Michael Collins. (paper) (code)

(Findings) Persian Ezafe Recognition Using Transformers and Its Role in Part-Of-Speech Tagging. Ehsan Doostmohammadi, Minoo Nassajian and Adel Rahimi. (paper) (code)

Entity Recognition, Linking and Representation

NER

Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition. Xiangji Zeng, Yunliang Li, Yuchen Zhai and Yin Zhang. (paper) (code)

Named Entity Recognition Only from Word Embeddings. Ying Luo, Hai Zhao and Junlang Zhan. (paper) (code)

Coarse-to-Fine Pre-training for Named Entity Recognition. Xue Mengge, Bowen Yu, Zhenyu Zhang, Tingwen Liu, Yue Zhang and Bin Wang. (paper) (code)

A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land?. Hongyu Lin, Yaojie Lu, Jialong Tang, Xianpei Han, Le Sun, Zhicheng Wei and Nicholas Jing Yuan. (paper) (code)

(Findings) Tri-Train: Automatic Pre-Fine Tuning between Pre-Training and Fine-Tuning for SciNER. Qingkai Zeng et al. (paper) (code)

Named Entity Recognition for Social Media Texts with Semantic Augmentation. Yuyang Nie, Yuanhe Tian, Xiang Wan, Yan Song and Bo Dai. (paper) (code)

(Findings+WS) A Dual-Attention Network for Joint Named Entity Recognition and Sentence Classification of Adverse Drug Events. Susmitha Wunnava et al. (paper) (code)

To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints. Barun Patra et al. (paper) (code)

Evaluation:

Interpretable Multi-dataset Evaluation for Named Entity Recognition. Jinlan Fu, Pengfei Liu and Graham Neubig. (paper) (code)

Nested NER:

See also the papers from ACL 2020 on nested NER, including Named Entity Recognition as Dependency Parsing.

HIT: Nested Named Entity Recognition via Head-Tail Pair and Token Interaction. Yu Wang, Yun Li, Hanghang Tong and Ziye Zhu. (paper) (code)

(Findings) Hierarchical Region Learning for Nested Named Entity Recognition. Xinwei Long, Shuzi Niu and Yucheng Li. (paper) (code)

Large-scale typing:

(Findings) Interpretable Entity Representations through Large-Scale Typing. Yasumasa Onoe and Greg Durrett. (paper) (code)

(Findings) Toward Recognizing More Entity Types in NER: An Efficient Implementation using Only Entity Lexicons (paper) (code)

Entity Linking

Zero-shot/few-shot:

Scalable Zero-shot Entity Linking with Dense Entity Retrieval. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel and Luke Zettlemoyer. (paper) (code)

(Findings) Zero-shot Entity Linking with Efficient Long Range Sequence Modeling. Zonghai Yao, Liangliang Cao and Huapu Pan. (paper) (code)

Muli/cross-lingual:

Design Challenges in Low-resource Cross-lingual Entity Linking. Xingyu Fu, Weijia Shi, Xiaodong Yu, Zian Zhao and Dan Roth. (paper) (code)

Entity Linking in 100 Languages. Jan A. Botha, Zifei Shan and Daniel Gillick. (paper) (code)

Dataset:

COMETA: A Corpus for Medical Entity Linking in the Social Media. Marco Basaldella, Fangyu Liu, Ehsan Shareghi and Nigel Collier. (paper) (code)

Other:

Efficient One-Pass End-to-End Entity Linking for Questions. Belinda Z. Li, Sewon Min, Srinivasan Iyer, Yashar Mehdad and Wen-tau Yih. (paper) (code)

(Findings) Joint Intent Detection and Entity Linking on Spatial Domain Queries. lei zhang, Runze Wang, Jingbo Zhou, Jingsong Yu, Zhenhua Ling and Hui Xiong. (paper) (code)

Entity Representation

Entities as Experts: Sparse Memory Access with Entity Supervision. Thibault Févry, Livio Baldini Soares, Nicholas FitzGerald, Eunsol Choi and Tom Kwiatkowski. (paper) (code)

(Findings+WS) E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT. Nina Poerner, Ulli Waltinger and Hinrich Schütze. (paper) (code)

LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention. Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda and Yuji Matsumoto. (paper) (code)

Learning Structured Representations of Entity Names using Active Learning and Weak Supervision. Kun Qian, Poornima Chozhiyath Raman, Yunyao Li and Lucian Popa. (paper) (code)

Entity tracking:

Understanding Procedural Text using Interactive Entity Networks. Jizhi Tang, Yansong Feng and Dongyan Zhao. (paper) (code)

A Dataset for Tracking Entities in Open Domain Procedural Text. Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson and Eduard Hovy. (paper) (code)

Entity Set Expansion

(Findings) Global Bootstrapping Neural Network for Entity Set Expansion. Lingyong Yan, Xianpei Han, Ben He and Le Sun. (paper) (code)

SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery. Jiaming Shen, Wenda Qiu, Jingbo Shang, Michelle Vanni, Xiang Ren and Jiawei Han. (paper) (code)

OTHER

TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization. Clément Jumel, Annie Louis and Jackie Chi Kit Cheung. (paper) (code)

Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment. Zhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li, Zhiyuan Liu and Tat-Seng Chua. (paper) (code)

ENT-DESC: Entity Description Generation by Exploring Knowledge Graph. Liying Cheng, Dekun Wu, Lidong Bing, Yan Zhang, Zhanming Jie, Wei Lu and Luo Si. (paper) (code)

Coreference

(Findings+WS) Paraphrasing vs Coreferring: Two Sides of the Same Coin. Yehudit Meged, Avi Caciularu, Vered Shwartz and Ido Dagan. (paper) (code)

Revealing the Myth of Higher-Order Inference in Coreference Resolution. Liyan Xu and Jinho D. Choi. (paper) (code)

Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks. Shubham Toshniwal, Sam Wiseman, Allyson Ettinger, Karen Livescu and Kevin Gimpel. (paper) (code)

Incremental Neural Coreference Resolution in Constant Memory. Patrick Xia, João Sedoc and Benjamin Van Durme. (paper) (code)

Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art. Jing Lu and Vincent Ng. (paper) (code)

Coreferential Reasoning Learning for Language Representation. Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun and Zhiyuan Liu. (preprint) (paper) (code)

Pre-training Mention Representations in Coreference Models. Yuval Varkel and Amir Globerson. (paper) (code)

(Findings+WS) Adapting Coreference Resolution to Twitter Conversations. Berfin Aktaş, Veronika Solopova, Annalena Kohnert and Manfred Stede. (paper) (code)

(Findings) A Knowledge-Driven Approach to Classifying Object and Attribute Coreferences in Opinion Mining. Jiahua Chen, Shuai Wang, Sahisnu Mazumder and Bing Liu. (paper) (code)

(Findings) Fast End-to-end Coreference Resolution for Korean. Cheoneum Park, Jamin Shin, Sungjoon Park, Joonho Lim and Changki Lee. (paper) (code)

Relation Extraction

++ Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data. . (paper) (code)

Pre-training Entity Relation Encoder with Intra-span and Inter-span Information. Yijun Wang, Changzhi Sun, Yuanbin Wu, Junchi Yan, Peng Gao and Guotong Xie. (paper) (code)

Document-level:

(Findings) The Dots Have Their Values: Exploiting the Node-Edge Connections in Graph-based Neural Models for Document-level Relation Extraction. (paper) (code)

Double Graph Based Reasoning for Document-level Relation Extraction. Shuang Zeng, Runxin Xu, Baobao Chang and Lei Li. (preprint) (paper) (code)

Global-to-Local Neural Networks for Document-Level Relation Extraction. Difeng Wang, Wei Hu, Ermei Cao and Weijian Sun. (preprint) (paper) (code and data)

Analysis:

Learning from Context or Names? An Empirical Study on Neural Relation Extraction. Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun and Jie Zhou. (paper) (code)

Unsupervised/self-supervised:

SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction. Xuming Hu, Lijie Wen, Yusong Xu, Chenwei Zhang and Philip Yu. (paper) (code)

(Findings) Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion. Ankur Goswami, Akshata Bhat, Hadar Ohana and Theodoros Rekatsinas. (paper) (code)

Other:

FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction. Dianbo Sui, Yubo Chen, Jun Zhao, Yantao Jia, Yuantao Xie and Weijian Sun. (paper) (code)

Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction. Rujun Han, Yichao Zhou and Nanyun Peng. (paper) (code)

Joint Learning (Entities and Relations):

Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders. Jue WANG and Wei Lu. (paper) (code)

Recurrent Interaction Network for Jointly Extracting Entities and Classifying Relations. Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao and xudong Liu. (paper) (code)

Distant supervision:

Denoising Relation Extraction from Document-level Distant Supervision. Chaojun Xiao, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Maosong Sun, Fen Lin and Leyu Lin. (paper) (code)

(Findings) Active Testing: An Unbiased Evaluation Method for Distantly Supervised Relation Extraction. Pengshuai Li, Xinsong Zhang, Weijia Jia and Wei Zhao. (paper) (code)

Events

Event-event relations:

Weakly Supervised Subevent Knowledge Acquisition. Wenlin Yao, Zeyu Dai, Maitreyi Ramaswamy, Bonan Min and Ruihong Huang. (paper) (code)

Analogous Process Structure Induction for Sub-event Sequence Prediction. Hongming Zhang, Muhao Chen, Haoyu Wang, Yangqiu Song and Dan Roth. (paper) (code)

Joint Constrained Learning for Event-Event Relation Extraction. Haoyu Wang, Muhao Chen, Hongming Zhang and Dan Roth. (paper) (code)

Connecting the Dots: Event Graph Schema Induction with Path Language Modeling. Manling Li, Qi Zeng, Ying Lin, Kyunghyun Cho, Heng Ji, Jonathan May, Nathanael Chambers and Clare Voss. (preprint) (paper) (code)

Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events. Miguel Ballesteros et al. (paper) (code)

Event extraction:

Incremental Event Detection via Knowledge Consolidation Networks. Pengfei Cao, Yubo Chen, Jun Zhao and Taifeng Wang. (paper) (code)

Biomedical Event Extraction as Sequence Labeling. Alan Ramponi, Rob van der Goot, Rosario Lombardo and Barbara Plank. (paper) (code)

(Findings) Biomedical Event Extraction with Hierarchical Knowledge Graphs. Kung-Hsiang Huang, Mu Yang and Nanyun Peng. (paper) (code)

Semi-supervised New Event Type Induction and Event Detection. Lifu Huang and Heng Ji. (paper) (code)

Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks. Viet Dac Lai, Tuan Ngo Nguyen and Thien Huu Nguyen. (paper) (code)

Event extraction via question answering:

(Findings) Event Extraction as Multi-turn Question Answering. Fayuan Li, Weihua Peng, Yuguang Chen, Quan Wang, Lu Pan, Yajuan Lyu and Yong Zhu. (paper) (code)

Event Extraction by Answering (Almost) Natural Questions. Xinya Du and Claire Cardie. (preprint) (paper) (code)

Event Extraction as Machine Reading Comprehension. Jian Liu, Yubo Chen, Kang Liu, Wei Bi and Xiaojiang Liu. (paper) (code)

Event arguments:

(Findings) Resource-Enhanced Neural Model for Event Argument Extraction. Jie Ma, Shuai Wang, Rishita Anubhai, Miguel Ballesteros and Yaser Al-Onaizan. (paper) (code)

(Discourse workshop) Joint Modeling of Arguments for Event Understanding. Yunmo Chen, Tongfei Chen, Benjamin Van Durme (paper) (code)

(Findings) Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction. Amir Pouran Ben Veyseh, Tuan Ngo Nguyen, Thien Huu Nguyen (paper)

++ (BlackboxNLP) BERTering RAMS: What and How Much does BERT Already Know About Event Arguments? - A Study on the RAMS Dataset. Varun Gangal, Eduard Hovy (paper)

Open Information Extraction

A Predicate-Function-Argument Annotation of Natural Language for Open-Domain Information eXpression. Mingming Sun et al. (paper) (code)

OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction. Keshav Kolluru et al. (paper) (code)

Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction. Patrick Hohenecker, Frank Mtumbuka, Vid Kocijan and Thomas Lukasiewicz. (paper) (code)

(Findings) Syntactic and Semantic-driven Learning for Open Information Extraction. Jialong Tang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Xinyan Xiao and Hua Wu. (paper) (code)

(Findings) MultiOIE: Multilingual Open Information Extraction Based on Multi-Head Attention with BERT. Youngbin Ro, Yukyung Lee and Pilsung Kang. (paper) (code)

Semantic Role Labeling

Graph Convolutions over Constituent Trees for Syntax-Aware Semantic Role Labeling. Diego Marcheggiani and Ivan Titov. (paper) (code)

SRLGRN: Semantic Role Labeling Graph Reasoning Network. Chen Zheng and Parisa Kordjamshidi. (paper) (code)

Semantic Role Labeling as Syntactic Dependency Parsing. Tianze Shi, Igor Malioutov and Ozan Irsoy. (paper) (code)

(Findings) High-order Semantic Role Labeling. Zuchao Li, Hai Zhao, Rui Wang and Kevin Parnow. (paper) (code)

Cross-lingual:

X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset. Angel Daza and Anette Frank. (paper) (code)

Alignment-free Cross-lingual Semantic Role Labeling. Rui Cai and Mirella Lapata. (paper) (code)

(Findings) CLAR: A Cross-Lingual Argument Regularizer for Semantic Role Labeling. Ishan Jindal, Yunyao Li, Siddhartha Brahma and Huaiyu Zhu. (paper) (code)

Semantic Parsing

See also the workshop on Interactive and Executable Semantic Parsing (Int-Ex).

Grounded Adaptation for Zero-shot Executable Semantic Parsing. Victor Zhong, Mike Lewis, Sida I. Wang and Luke Zettlemoyer. (paper) (code)

Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing. Xilun Chen, Asish Ghoshal, Yashar Mehdad, Luke Zettlemoyer and Sonal Gupta. (paper) (code)

(Findings+WS) An Instance Level Approach for Shallow Semantic Parsing in Scientific Procedural Text. Daivik Swarup et al. (paper) (code)

Conversational Semantic Parsing. Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Michael Haeger, Haoran Li, Yashar Mehdad, Veselin Stoyanov, Anuj Kumar, Mike Lewis and Sonal Gupta. (paper) (code)

Benchmarking Meaning Representations in Neural Semantic Parsing. Jiaqi Guo, Qian Liu, Jian-Guang LOU, Zhenwen Li, Xueqing Liu, Tao Xie and Ting Liu. (paper) (code)

An Imitation Game for Learning Semantic Parsers from User Interaction. Ziyu Yao, Yiqi Tang, Wen-tau Yih, Huan Sun and Yu Su. (paper) (code)

++ Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT. Rik van Noord, Antonio Toral and Johan Bos. (paper) (code)

Fast semantic parsing with well-typedness guarantees. Matthias Lindemann, Jonas Groschwitz and Alexander Koller. (paper) (code)

AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data. Silei Xu, Sina Semnani, Giovanni Campagna and Monica Lam. (paper) (code)

(Findings+WS) Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem. Shucheng Li et al. (paper) (code)

(Findings) Bootstrapping a Crosslingual Semantic Parser. Tom Sherborne, Yumo Xu and Mirella Lapata. (paper) (code)

(Findings) Accurate polyglot semantic parsing with DAG grammars. Federico Fancellu, Ákos Kádár, Ran Zhang and Afsaneh Fazly. (paper) (code)

Text to SQL

(Findings) On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries. Tianze Shi, Chen Zhao, Jordan Boyd-Graber, Hal Daumé III and Lillian Lee. (paper) (code)

(Findings) Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing. Xi Victoria Lin, Richard Socher and Caiming Xiong. (paper) (code)

IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generation. Yitao Cai and Xiaojun Wan. (paper) (code)

“What Do You Mean by That?” A Parser-Independent Interactive Approach for Enhancing Text-to-SQL. Yuntao Li, Bei Chen, Qian Liu, Yan Gao, Jian-Guang LOU, Yan Zhang and Dongmei Zhang. (paper) (code)

Semantic Evaluation for Text-to-SQL with Distilled Test Suites. Ruiqi Zhong, Tao Yu and Dan Klein. (paper) (code)

Re-examining the Role of Schema Linking in Text-to-SQL. Wenqiang Lei, Weixin Wang, Zhixin MA, Tian Gan, Wei Lu, Min-Yen Kan and Tat-Seng Chua. (paper) (code)

Mention Extraction and Linking for SQL Query Generation. Jianqiang Ma, ZEYU YAN, Shuai Pang, Yang Zhang and Jianping Shen. (paper) (code)

AMR Parsing

Improving AMR Parsing with Sequence-to-Sequence Pre-training. Dongqin Xu, Junhui Li, Muhua Zhu, Min Zhang and Guodong Zhou. (paper) (code)

XL-AMR: Enabling Cross-Lingual AMR Parsing with Transfer Learning Techniques. Rexhina Blloshmi, Rocco Tripodi and Roberto Navigli. (paper) (code)

(Findings) Pushing the Limits of AMR Parsing with Self-Learning. Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Revanth Gangi Reddy, Radu Florian and Salim Roukos. (paper) (code)

(Findings) The Role of Reentrancies in Abstract Meaning Representation Parsing. Marco Damonte, Ida Szubert, Shay B. Cohen and Mark Steedman. (paper) (code)

(Findings) Transition-based Parsing with Stack-Transformers. Ramón Fernandez Astudillo, Miguel Ballesteros, Tahira Naseem, Austin Blodgett and Radu Florian. (preprint) (paper) (code)

Semantically Inspired AMR Alignment for the Portuguese Language. Rafael Anchiêta and Thiago Pardo. (paper) (code)

Topic Models

(TACL) Topic Modeling in Embedding Spaces. Adji Bousso Dieng, Francisco Ruiz, David Blei. (paper) (code)

See also: Discovering Discrete Latent Topics with Neural Variational Inference (2016).

++ Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!. Suzanna Sia, Ayush Dalmia and Sabrina J. Mielke. (paper) (code)

++ (Findings) Rethinking Topic Modelling: From Document-Space to Term-Space. Magnus Sahlgren. (paper)

Sparse Parallel Training of Hierarchical Dirichlet Process Topic Models. Alexander Terenin, Måns Magnusson and Leif Jonsson. (paper) (code)

(Findings+WS) Unsupervised Few-Bits Semantic Hashing with Implicit Topics Modeling. Fanghua Ye, Jarana Manotumruksa and Emine Yilmaz. (paper) (code)

Neural topic modeling:

Neural Topic Modeling by Incorporating Document Relationship Graph. Deyu Zhou, Xuemeng Hu and Rui Wang. (paper)

Improving Neural Topic Models using Knowledge Distillation. Alexander Miserlis Hoyle, Pranav Goel and Philip Resnik. (paper)

(TACL) A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings (paper) (code)

Topic modeling for short texts:

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder. Xiaobao Wu, Chunping Li, Yan Zhu and Yishu Miao. (paper) (code)

Fact Checking

See also Factuality evaluation and correction under Summarization.

Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News. Nguyen Vo and Kyumin Lee. (paper) (code)

Neural Deepfake Detection with Factual Structure of Text. Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang and Jian Yin. (paper) (code)

++ Generating Fact Checking Briefs. Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes and Sebastian Riedel. (paper) (code)

++ Generating Label Cohesive and Well-Formed Adversarial Claims. Pepa Atanasova, Dustin Wright and Isabelle Augenstein. (paper) (code)

(Findings) Claim Check-Worthiness Detection as Positive Unlabelled Learning. Dustin Wright and Isabelle Augenstein. (paper) (code)

Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification. Shyam Subramanian and Kyumin Lee. (paper) (code)

Program Enhanced Fact Verification with Verbalization and Graph Attention Network. Xiaoyu Yang, Feng Nie, Yufei Feng, Quan Liu, Zhigang Chen and Xiaodan Zhu. (paper) (code)

++ Fact or Fiction: Verifying Scientific Claims. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan and Hannaneh Hajishirzi. (paper) (code)

Table Fact Verification with Structure-Aware Transformer. Hongzhi Zhang, Yingyao Wang, Sirui Wang, Xuezhi Cao, Fuzheng Zhang and Zhongyuan Wang. (paper) (code)

Constrained Fact Verification for FEVER. Adithya Pratapa, Sai Muralidhar Jayanthi and Kavya Nerella. (paper) (code)

(Findings) HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification. Yichen Jiang, Shikha Bordia, ZHENG ZHONG, Charles Dognin, Maneesh Singh and Mohit Bansal. (paper) (code)

(Findings) Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling. Zhenghao Liu et al. (paper) (code)

Explainable Automated Fact-Checking for Public Health Claims. Neema Kotonya and Francesca Toni. (paper) (code)

AnswerFact: Fact Checking in Product Question Answering. Wenxuan Zhang, Yang Deng, Jing Ma and Wai Lam. (paper) (code)

(Findings) Weakly- and Semi-supervised Evidence Extraction. Danish Pruthi, Bhuwan Dhingra, Graham Neubig and Zachary C. Lipton. (paper) (code)

Knowledge Graphs

(Findings+WS) Out-of-Sample Representation Learning for Knowledge Graphs. Marjan Albooyeh, Rishab Goel and Seyed Mehran Kazemi. (paper) (code)

(Findings+WS) AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding. Guanglin Niu, Bo Li, Yongfei Zhang, Shiliang Pu and Jingyang Li. (paper) (code)

Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning. Deren Lei, Gangrong Jiang, Xiaotao Gu, Kexuan Sun, Yuning Mao and Xiang Ren. (paper) (code)

(Findings) MCMH: Learning Multi-Chain Multi-Hop Rules for Knowledge Graph Reasoning. Lu Zhang, Mo Yu, Tian Gao and yue yu. (paper) (code)

Learning Physical Common Sense as Knowledge Graph Completion via BERT Data Augmentation and Constrained Tucker Factorization. Zhenjie Zhao et al. (paper) (code)

Knowledge Graph Alignment with Entity-Pair Embedding. Zhichun Wang, Jinjian Yang and Xiaoju Ye. (paper) (code)

TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion. Jiapeng Wu, Meng Cao, Jackie Chi Kit Cheung and William L. Hamilton. (paper) (code)

Message Passing for Hyper-Relational Knowledge Graphs. Mikhail Galkin, Priyansh Trivedi, Gaurav Maheshwari, Ricardo Usbeck and Jens Lehmann. (paper) (code)

H2KGAT: Hierarchical Hyperbolic Knowledge Graph Attention Network. Shen Wang, Xiaokai Wei, Cicero Nogueira dos Santos, Zhiguo Wang, Ramesh Nallapati, Andrew Arnold, Bing Xiang and Philip S. Yu. (paper) (code)

Knowledge Association with Hyperbolic Knowledge Graph Embeddings. Zequn Sun, Muhao Chen, Wei Hu, Chengming Wang, Jian Dai and Wei Zhang. (paper) (code)

Evaluating the Calibration of Knowledge Graph Embeddings for Trustworthy Link Prediction. Tara Safavi, Danai Koutra and Edgar Meij. (paper) (code)

Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs. Woojeong Jin, Meng Qu, Xisen Jin and Xiang Ren. (paper) (code)

CoDEx: A Comprehensive Knowledge Graph Completion Benchmark. Tara Safavi and Danai Koutra. (paper) (code)

Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph. Xin Lv, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Wei Zhang, YICHI ZHANG, Hao Kong and Suhui Wu. (paper) (code)

Structure Aware Negative Sampling in Knowledge Graphs. Kian Ahrabian, Aarash Feizi, Yasmin Salehi, William L. Hamilton and Avishek Joey Bose. (paper) (code)

(Findings) Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer. Xuelu Chen, Muhao Chen, Changjun Fan, Ankith Uppunda, Yizhou Sun and Carlo Zaniolo. (paper) (code)

(Findings) Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion. Rajarshi Das, Ameya Godbole, Nicholas Monath, Manzil Zaheer and Andrew McCallum. (paper) (code)

(Findings) A Greedy Bit-flip Training Algorithm for Binarized Knowledge Graph Embeddings. Katsuhiko Hayashi, Koki Kishimoto and Masashi Shimbo. (paper) (code)

Natural Language Inference

++ Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start. Wenpeng Yin, Nazneen Fatema Rajani, Dragomir Radev, Richard Socher and Caiming Xiong. (paper) (code)

New Protocols and Negative Results for Textual Entailment Data Collection. Samuel R. Bowman, Jennimaria Palomaki, Livio Baldini Soares and Emily Pitler. (paper) (code)

(Findings+WS) Thinking Like a Skeptic: Defeasible Inference in Natural Language. Rachel Rudinger et al. (paper) (code)

(Blackbox NLP) Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation (paper) (code)

What Can We Learn from Collective Human Opinions on Natural Language Inference Data?. Yixin Nie, Xiang Zhou and Mohit Bansal. (paper) (code)

Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference. Xiaoan Ding, Tianyu Liu, Baobao Chang, Zhifang Sui and Kevin Gimpel. (paper) (code)

ConjNLI: Natural Language Inference Over Conjunctive Sentences. Swarnadeep Saha, Yixin Nie and Mohit Bansal. (paper) (code)

An Analysis of Natural Language Inference Benchmarks through the Lens of Negation. Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei and Eduardo Blanco. (paper) (code)

(Findings) Mining Knowledge for Natural Language Inference from Wikipedia Categories. Mingda Chen, Zewei Chu, Karl Stratos and Kevin Gimpel. (paper) (code)

(Findings) Temporal Reasoning in Natural Language Inference. Siddharth Vashishtha, Adam Poliak, Yash Kumar Lal, Benjamin Van Durme and Aaron Steven White. (paper) (code)

(Findings) Enhancing Generalization in Natural Language Inference by Syntax. Qi He, Han Wang and Yue Zhang. (paper) (code)

NLI for languages other than English:

Data and Representation for Turkish Natural Language Inference. Emrah Budur, Rıza Özçelik, Tunga Gungor and Christopher Potts. (paper) (code)

(Findings) OCNLI: Original Chinese Natural Language Inference. Hai Hu, Kyle Richardson, Liang Xu, Lu Li, Sandra Kübler and Lawrence Moss. (paper) (code)

Emergent Language in Multi-Agent Communication

The Grammar of Emergent Languages. Oskar van der Wal, Silvan de Boer, Elia Bruni and Dieuwke Hupkes. (paper) (code)

Supervised Seeded Iterated Learning for Interactive Language Learning. Yuchen Lu, Soumye Singhal, Florian Strub, Olivier Pietquin and Aaron Courville. (paper) (code)

Applications

Cybersecurity

Introducing a New Dataset for Event Detection in Cybersecurity Texts. Hieu Man Duc Trong et al. (paper) (code)

Language and Code

PyMT5: multi-mode translation of natural language and Python code with transformers. Colin Clement, Dawn Drain, Jonathan Timcheck, Alexey Svyatkovskiy and Neel Sundaresan. (paper) (code)

(Findings) CodeBERT: A Pre-Trained Model for Programming and Natural Languages. Zhangyin Feng et al. (paper) (code)

(Findings) Towards Context-Aware Code Comment Generation. Xiaohan Yu, Quzhe Huang, Zheng Wang, Yansong Feng and Dongyan Zhao. (paper) (code)

Pandemics and other crises

(Findings+WS) A Corpus for Outbreak Detection of Diseases Prevalent in Latin America. Antonella Dellanzo, Viviana Cotik, Jose Ochoa-Luna (paper)

(Findings) An Empirical Methodology for Detecting and Prioritizing Needs during Crisis Events. M. Janina Sarol et al. (paper) (code)

COVID-19:

See also NLP for COVID-19 Workshop (Part 2). For COVID-related literature search, see under Scientific Text Mining.

Clinical and Biomedical

See also three workshops at this conference:

• 3rd Clinical Natural Language Processing Workshop (Clinical NLP 2020)
• LOUHI 2020 - 11th International Workshop on Health Text Mining and Information Analysis

(Findings+WS) Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization. Jiho Noh and Ramakanth Kavuluru. (paper) (code)

(Findings+WS) Characterizing the Value of Information in Medical Notes. Chao-Chun Hsu, Shantanu Karnwal, Sendhil Mullainathan, Ziad Obermeyer and Chenhao Tan. (paper) (code)

BioMegatron: Larger Biomedical Domain Language Model. Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi and Raghav Mani. (paper) (code)

Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition. Yun He, Ziwei Zhu, Yin Zhang, Qin Chen and James Caverlee. (paper) (code)

Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text. Dongfang Li, Baotian Hu, Qingcai Chen, Weihua Peng and Anqi Wang. (paper) (code)

(Findings+WS) Generating Accurate Electronic Health Assessment from Medical Graph. Zhichao Yang and hong yu. (paper) (code)

Predicting Clinical Trial Results by Implicit Evidence Integration. Qiao Jin, Chuanqi Tan, Mosha Chen, Xiaozhong Liu and Songfang Huang. (paper) (code)

Explainable Clinical Decision Support from Text. Jinyue Feng, Chantal Shaib and Frank Rudzicz. (paper) (code)

(Findings) Effective Crowd-Annotation of Participants, Interventions, and Outcomes in the Text of Clinical Trial Reports. Markus Zlabinger et al. (paper) (code)

Multimodal:

(Findings+WS) MedICaT: A Dataset of Medical Images, Captions, and Textual References. Sanjay Subramanian et al. (paper) (code)

(Findings+WS) Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays. Jianmo Ni, Chun-Nan Hsu, Amilcare Gentili and Julian McAuley. (paper) (code)

Radiology reports:

Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT. Akshay Smit et al. (preprint) (paper) (code)

Generating Radiology Reports via Memory-driven Transformer. Zhihong Chen, Yan Song, Tsung-Hui Chang, Xiang Wan (paper) (code)

(Findings+WS) Learning to Generate Clinically Coherent Chest X-Ray Reports. Justin Lovelace and Bobak Mortazavi. (paper) (code)

(WS) A Hybrid Deep Learning Approach for Spatial Trigger Extraction from Radiology Reports. Surabhi Datta, Kirk Roberts (paper) (code)

(WS) An Ensemble Approach to Automatic Structuring of Radiology Reports. Morteza Pourreza Shahri, Amir Tahmasebi, Bingyang Ye, Henghui Zhu, Javed Aslam, Timothy Ferris (paper) (code)

Not a cute stroke: Analysis of Rule- and Neural Network-based Information Extraction Systems for Brain Radiology Reports. Andreas Grivas, Beatrice Alex, Claire Grover, Richard Tobin, William Whiteley (paper) (code)

Mental Health

(Findings) Do Models of Mental Health Based on Social Media Data Generalize?. Keith Harrigian, Carlos Aguirre and Mark Dredze. (paper) (code)

A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support. Ashish Sharma, Adam Miner, David Atkins and Tim Althoff. (paper) (code)

Suicide risk detection:

Suicidal Risk Detection for Military Personnel. Sungjoon Park, Kiwoong Park, Jaimeen Ahn and Alice Oh. (paper) (code)

(Findings) Cross-Lingual Suicidal-Oriented Word Embedding toward Suicide Prevention. Daeun Lee, Soyoung Park, Jiwon Kang, Daejin Choi and Jinyoung Han. (paper) (code)

A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media. Ramit Sawhney, Harshit Joshi, Saumya Gandhi and Rajiv Ratn Shah. (paper) (code)

(WS) Assessing population-level symptoms of anxiety, depression, and suicide risk in real time using NLP applied to social media data.
Alex Fine, Patrick Crutchley, Jenny Blase, Joshua Carroll, Glen Coppersmith (paper)

Scientific Text Mining

See also the First Workshop on Scholarly Document Processing (SDP 2020) at this conference.

AxCell: Automatic Extraction of Results from Machine Learning Papers. Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor and Robert Stojnic. (paper) (code)

(Findings) Document Classification for COVID-19 Literature. Bernal Jimenez Gutierrez, Jucheng Zeng, Dongdong Zhang, Ping Zhang and Yu Su. (paper) (code)

SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search. Sean MacAvaney, Arman Cohan and Nazli Goharian. (paper) (code)

(Findings) Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora. Hancheng Cao et al. (paper) (code)

Essay Scoring

Multi-Stage Pre-training for Automated Chinese Essay Scoring. Wei Song, Kai Zhang, Ruiji Fu, Lizhen Liu, Ting Liu and Miaomiao Cheng. (paper) (code)

(Findings) Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking. Ruosong Yang et al. (paper) (code)

News Media

We Can Detect Your Bias: Predicting the Political Ideology of News Articles. Ramy Baly, Giovanni Da San Martino, James Glass and Preslav Nakov. (paper) (code)

(Findings) Privacy-Preserving News Recommendation Model Learning. Tao Qi, Fangzhao Wu, Chuhan Wu, Yongfeng Huang and Xing Xie. (paper) (code)

Weakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Media. Shamik Roy and Dan Goldwasser. (paper) (code)

(Findings) DeSMOG: Detecting Stance in Media On Global Warming. Yiwei Luo, Dallas Card and Dan Jurafsky. (paper) (code)

Social Media

Condolence and Empathy in Online Communities. Naitian Zhou and David Jurgens. (paper) (code)

Social Media Attributions in the Context of Water Crisis. Rupak Sarkar, Sayantan Mahinder, Hirak Sarkar and Ashiqur KhudaBukhsh. (paper) (code)

(Findings) Predicting Responses to Psychological Questionnaires from Participants’ Social Media Posts and Question Text Embeddings. Huy Vu et al. (paper) (code)

Hashtags, Emotions, and Comments: A Large-Scale Dataset to Understand Fine-Grained Social Emotions to Online Topics. Keyang Ding, Jing Li and Yuji Zhang. (paper) (code)

(Findings) Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media. Xiang Dai, Sarvnaz Karimi, Ben Hachey and Cecile Paris. (paper) (code)

Multi-resolution Annotations for Emoji Prediction. Weicheng Ma, Ruibo Liu, Lili Wang and Soroush Vosoughi. (paper) (code)

Rumor detection:

A State-independent and Time-evolving Network for Early Rumor Detection in Social Media. Rui Xia, Kaizhou Xuan and Jianfei Yu. (paper) (code)

Coupled Hierarchical Transformer for Stance-Aware Rumor Verification in Social Media Conversations. Jianfei Yu, Jing Jiang, Ling Min Serena Khoo, Hai Leong Chieu and Rui Xia. (paper) (code)

Twitter:

Sentiment Analysis of Tweets using Heterogeneous Multi-layer Network Representation and Embedding. Loitongbam Gyanendro Singh, Anasua Mitra and Sanasam Ranbir Singh. (paper) (code)

An Embedding Model for Estimating Legislative Preferences from the Frequency and Sentiment of Tweets. Gregory Spell, Brian Guay, Sunshine Hillygus and Lawrence Carin. (paper) (code)

On the Reliability and Validity of Detecting Approval of Political Actors in Tweets. Indira Sen, Fabian Flöck and Claudia Wagner. (paper) (code)

(Findings) TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification. Francesco Barbieri et al. (paper) (code)

Literature and Narrative

Measuring Information Propagation in Literary Social Networks. Matthew Sims and David Bamman. (paper) (code)

Novels:

Chapter Captor: Text Segmentation in Novels. Charuta Pethe, Allen Kim and Steve Skiena. (paper) (code)

What time is it? Temporal Analysis of Novels. Allen Kim, Charuta Pethe and Steve Skiena. (paper) (code)

Narrative:

Multi-view Story Characterization from Movie Plot Synopses and Reviews. Sudipta Kar, Gustavo Aguilar, Mirella Lapata and Thamar Solorio. (paper) (code)

Scene Restoring for Narrative Machine Reading Comprehension. Zhixing Tian, Yuanzhe Zhang, Kang Liu, Jun Zhao, Yantao Jia and Zhicheng Sheng. (paper) (code)

Modeling Protagonist Emotions for Emotion-Aware Storytelling. Faeze Brahman and Snigdha Chaturvedi. (paper) (code)

Movies

Joint Estimation and Analysis of Risk Behavior Ratings in Movie Scripts. Victor Martinez, Krishna Somandepalli, Yalda Tehranian-Uhls and Shrikanth Narayanan. (paper) (code)

Dictionaries

See also https://www.aclweb.org/anthology/W13-2507.pdf

(Workshop - Negative Results in NLP) Embedding Structured Dictionary Entries. Steven Wilson, Walid Magdy, Barbara McGillivray, Gareth Tyson (paper) (code)

(WMT20) Look It Up: Bilingual and Monolingual Dictionaries Improve Neural Machine Translation. Xing Jie Zhong, David Chiang (paper) (code)

Definition generation:

See also from NAACL 2019, Learning to Describe Unknown Phrases with Local and Global Contexts.

VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling. Machel Reid, Edison Marrese-Taylor and Yutaka Matsuo. (paper) (code)

Generationary or “How We Went beyond Word Sense Inventories and Learned to Gloss’’. Michele Bevilacqua, Marco Maru and Roberto Navigli. (paper) (code)

Reverse Dictionary:

(Findings) BERT for Monolingual and Cross-Lingual Reverse Dictionary. Hang Yan, Xiaonan Li, Xipeng Qiu, Bocao Deng.] (paper) (code)

(Demo) WantWords: An Open-source Online Reverse Dictionary System. Fanchao Qi, Lei Zhang, Yanhui Yang, Zhiyuan Liu, Maosong Sun (paper) (code) (demo)(https://wantwords.thunlp.org/)

Grammatical Error Correction

Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses. Simon Flachs, Ophélie Lacroix, Helen Yannakoudakis, Marek Rei and Anders Søgaard. (paper) (code)

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction. Mengyun Chen et al. (paper) (code)

(Findings) A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction. Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki and Kentaro Inui. (paper) (code)

(Findings) Improving Grammatical Error Correction with Machine Translation Pairs. Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei and Ming Zhou. (paper) (code)

Adversarial:

Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples. Lihao Wang and Xiaoqing Zheng. (paper) (code)

(Findings) Adversarial Grammatical Error Correction. Vipul Raheja and Dimitris Alikaniotis. (paper) (code)

Finance

(Findings) Event-Driven Learning of Systematic Behaviours in Stock Markets. Xianchao Wu. (paper) (code)

Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations. Ramit Sawhney, Shivam Agarwal, Arnav Wadhwa and Rajiv Ratn Shah. (paper) (code)

VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls. Ramit Sawhney, Piyush Khanna, Arshiya Aggarwal, Taru Jain, Puneet Mathur and Rajiv Ratn Shah. (paper) (code)

Law

De-Biased Court’s View Generation with Causality. Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si and Fei Wu. (paper) (code)

An Element-aware Multi-representation Model for Law Article Prediction. Huilin Zhong, Junsheng Zhou, Weiguang QU, Yunfei Long and Yanhui Gu. (paper) (code)

(Findings) Contract Discovery: Dataset and a Few-Shot Semantic Retrieval Challenge with Competitive Baselines. Łukasz Borchmann et al. (paper) (code)

(Findings) LEGAL-BERT: The Muppets straight out of Law School. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos (paper) (code)

Customer Service

Conversational Document Prediction to Assist Customer Care Agents. Jatin Ganhotra et al. (paper) (code)

Hate speech and offensive language

See also The Fourth Workshop on Online Abuse and Harms (WOAH) a.k.a. ALW (list of papers here: https://www.aclweb.org/anthology/volumes/2020.alw-1/).

Fortifying Toxic Speech Detectors Against Veiled Toxicity. Xiaochuang Han and Yulia Tsvetkov. (paper) (code)

HABERTOR: An Efficient and Effective Deep Hatespeech Detector. Thanh Tran, Yifan Hu, Changwei Hu, Kevin Yen, Fei Tan, Kyumin Lee and Se Rim Park. (paper) (code)

Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets. Nedjma Ousidhoum, Yangqiu Song and Dit-Yan Yeung. (paper) (code)

(Findings+WS) A little goes a long way: Improving toxic language classification despite data scarcity. Mika Juuti, Tommi Gröndahl, Adrian Flanagan and N. Asokan. (paper) (code)

TNT: Text Normalization based Pre-training of Transformers for Content Moderation. Fei Tan, Yifan Hu, Changwei Hu, Keqian Li and Kevin Yen. (paper) (code)

Other applications

Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding. Jiaming Shen, Heng Ji and Jiawei Han. (preprint) (paper) (code)

++ Solving Historical Dictionary Codes with a Neural Language Model. Christopher Chu, Raphael Valenti and Kevin Knight. (paper) (code)

(Findings) Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness. Lingjuan Lyu, Xuanli He and Yitong Li. (paper) (code)

Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product. Tiangang Zhu, Yue Wang, Haoran Li, Youzheng Wu, Xiaodong He and Bowen Zhou. (paper) (code)

Modeling the Music Genre Perception across Language-Bound Cultures. Elena V. Epure, Guillaume Salha, Manuel Moussallam and Romain Hennequin. (paper) (code)

(Findings) What Can We Do to Improve Peer Review in NLP?. Anna Rogers and Isabelle Augenstein. (paper) (code)

(Findings+WS) Where to Submit? Helping Researchers to Choose the Right Venue. Konstantin Kobs, Tobias Koopmann, Albin Zehe, David Fernes, Philipp Krop and Andreas Hotho. (paper) (code)

(Findings) Research Replication Prediction Using Weakly Supervised Learning. Tianyi Luo, xingyu li, Hainan Wang and Yang Liu. (paper) (code)

Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection. Shaolei Wang, Zhongyuan Wang, Wanxiang Che and Ting Liu. (paper) (code)

Assessing the Helpfulness of Learning Materials with Inference-Based Learner-Like Agent. Yun-Hsuan Jen, Chieh-Yang Huang, MeiHua Chen, Ting-Hao Huang and Lun-Wei Ku. (paper) (code)

(Findings) Inferring about fraudulent collusion risk on Brazilian public works contracts in official texts using a Bi-LSTM approach. Marcos Lima et al. (paper) (code)

Natural Language Processing for Achieving Sustainable Development: the Case of Neural Labelling to Enhance Community Profiling. Costanza Conforti et al. (paper) (code)

Towards Modeling Revision Requirements in wikiHow Instructions. Irshad Bhat, Talita Anthonio and Michael Roth. (paper) (code)

NwQM: A neural quality assessment framework for Wikipedia. Bhanu Prakash Reddy Guda, Sasi Bhushan Seelaboyina, Soumya Sarkar and Animesh Mukherjee. (paper) (code)

Competence-Level Prediction and Resume & Job Description Matching Using Context-Aware Transformer Models. Changmao Li, Elaine Fisher, Rebecca Thomas, Steve Pittard, Vicki Hertzberg and Jinho D. Choi. (paper) (code)

Other

++ Grounded Compositional Outputs for Adaptive Language Modeling. Nikolaos Pappas, Phoebe Mulcaire and Noah A. Smith. (paper) (code)

++ (Honorable Mention) If beam search is the answer, what was the question?. Clara Meister, Ryan Cotterell and Tim Vieira. (paper) (code)

++ Substance over Style: Document-Level Targeted Content Transfer. Allison Hegel, Sudha Rao, Asli Celikyilmaz and Bill Dolan. (paper) (code)

++ PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge. Yun He, Zhuoer Wang, Yin Zhang, Ruihong Huang and James Caverlee. (paper) (code)

++ Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics. Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith and Yejin Choi. (paper) (code)

++ Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task. Dongyeop Kang and Eduard Hovy. (paper) (code)

++ Learning from Task Descriptions. Orion Weller, Nicholas Lourie, Matt Gardner and Matthew Peters. (paper) (code)

++ (Findings) RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi and Noah A. Smith. (paper) (code)

Papers, sort

Multilevel Text Alignment with Cross-Document Attention. Xuhui Zhou, Nikolaos Pappas and Noah A. Smith. (paper) (code)

Writing Strategies for Science Communication: Data and Computational Analysis. Tal August, Lauren Kim, Katharina Reinecke and Noah A. Smith. (paper) (code)

TeaForN: Teacher-Forcing with N-grams. Sebastian Goodman, Nan Ding and Radu Soricut. (paper) (code)

Enhancing Aspect Term Extraction with Soft Prototypes. Zhuang Chen and Tieyun Qian. (paper) (code)

Top-Rank-Focused Adaptive Vote Collection for the Evaluation of Domain-Specific Semantic Models. Pierangelo Lombardo, Alessio Boiardi, Luca Colombo, Angelo Schiavone and Nicolò Tamagnone. (paper) (code)

SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge. Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu and Minlie Huang. (paper) (code)

SetConv: A New Approach for Learning from Imbalanced Data. Yang Gao, Yi-Fan Li, Yu Lin, Charu Aggarwal and Latifur Khan. (paper) (code)

Latent Geographical Factors for Analyzing the Evolution of Dialects in Contact. Yugo Murawaki. (paper) (code)

Exploring and Predicting Transferability across NLP Tasks. Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji and Mohit Iyyer. (paper) (code)

Interpreting Open-Domain Modifiers: Decomposition of Wikipedia Categories into Disambiguated Property-Value Pairs. Marius Pasca. (paper) (code)

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness. Reina Akama, Sho Yokoi, Jun Suzuki and Kentaro Inui. (paper) (code)

DORB: Dynamically Optimizing Multiple Rewards with Bandits. Ramakanth Pasunuru, Han Guo and Mohit Bansal. (paper) (code)

The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions. Xiang Zhou, Yixin Nie, Hao Tan and Mohit Bansal. (paper) (code)

Text Segmentation by Cross Segment Attention. Michal Lukasik, Boris Dadachev, Kishore Papineni and Goncalo Simoes. (paper) (code)

Dissecting Span Identification Tasks with Performance Prediction. Sean Papay, Roman Klinger and Sebastian Padó. (paper) (code)

Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning. Tsvetomila Mihaylova, Vlad Niculae and André F. T. Martins. (paper) (code)

Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast—-Choose Three. Steven Reich, David Mueller and Nicholas Andrews. (paper) (code)

Efficient Meta Lifelong-Learning with Limited Memory. Zirui Wang, Sanket Vaibhav Mehta, Barnabas Poczos and Jaime Carbonell. (paper) (code)

IGT2P: From Interlinear Glossed Texts to Paradigms. Sarah Moeller, Ling Liu, Changbing Yang, Katharina Kann and Mans Hulden. (paper) (code)

MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics. Anthony Chen, Gabriel Stanovsky, Sameer Singh and Matt Gardner. (paper) (code)

Learning VAE-LDA Models with Rounded Reparameterization Trick. Runzhi Tian, Yongyi Mao and Richong Zhang. (paper) (code)

Disentangle-based Continual Graph Representation Learning. Xiaoyu Kou, Yankai Lin, Shaobo Liu, Peng Li, Jie Zhou and Yan Zhang. (paper) (code)

Annotating Temporal Dependency Graphs via Crowdsourcing. Jiarui Yao, Haoling Qiu, Bonan Min and Nianwen Xue. (paper) (code)

A Method for Building a Commonsense Inference Dataset based on Basic Events. Kazumasa Omura, Daisuke Kawahara and Sadao Kurohashi. (paper) (code)

Keeping Up Appearances: Computational Modeling of Face Acts in Persuasion Oriented Discussions. Ritam Dutt, Rishabh Joshi and Carolyn Rose. (paper) (code)

Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains. Weijie Yu, Chen Xu, Jun Xu, Liang Pang, Xiaopeng Gao, Xiaozhao Wang and Ji-Rong Wen. (paper) (code)

Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors. Sida Gao and Matthew R. Gormley. (paper) (code)

Introducing Syntactic Structures into Target Opinion Word Extraction with Deep Learning. Amir Pouran Ben Veyseh, Nasim Nouri, Franck Dernoncourt, Dejing Dou and Thien Huu Nguyen. (paper) (code)

Coding Textual Inputs Boosts the Accuracy of Neural Networks. Abdul Rafae Khan, Jia Xu and Weiwei Sun. (paper) (code)

Selection and Generation: Learning towards Multi-Product Advertisement Post Generation. Zhangming Chan, Yuchi Zhang, Xiuying Chen, Shen Gao, Zhiqiang Zhang, Dongyan Zhao and Rui Yan. (paper) (code)

An Empirical Investigation of Contextualized Number Prediction. Taylor Berg-Kirkpatrick and Daniel Spokoyny. (paper) (code)

Form2Seq : A Framework for Higher-Order Form Structure Extraction. Milan Aggarwal, Hiresh Gupta, Mausoom Sarkar and Balaji Krishnamurthy. (paper) (code)

Modularized Transfomer-based Ranking Framework. Luyu Gao, Zhuyun Dai and Jamie Callan. (paper) (code)

RNNs can generate bounded hierarchical languages with optimal memory. John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang and Christopher D. Manning. (paper) (code)

Toward Micro-Dialect Identification in Diaglossic and Code-Switched Environments. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany and Lyle Ungar. (paper) (code)

End-to-End Emotion-Cause Pair Extraction based on Sliding Window Multi-Label Learning. Zixiang Ding, Rui Xia and Jianfei Yu. (paper) (code)

Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols. Prachi Jain, Sushant Rathi, Mausam - and Soumen Chakrabarti. (paper) (code)

Small but Mighty: New Benchmarks for Split and Rephrase. Li Zhang, Huaiyu Zhu, Siddhartha Brahma and Yunyao Li. (paper) (code)

Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference. Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu and Changyou Chen. (paper) (code)

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space. Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang and Jianfeng Gao. (paper) (code)

Exploiting Sentence Order in Document Alignment. Brian Thompson and Philipp Koehn. (paper) (code)

Modularized Syntactic Neural Networks for Sentence Classification. Haiyan Wu, Ying Liu and Shaoyun Shi. (paper) (code)

MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification. Qianli Ma, Zhenxi Lin, Jiangyue Yan, Zipeng Chen and Liuhong Yu. (paper) (code)

Public Sentiment Drift Analysis Based on Hierarchical Variational Auto-encoder. Wenyue Zhang et al. (paper) (code)

Iterative Feature Mining for Constraint-Based Data Collection to Increase Data Diversity and Model Robustness. Stefan Larson et al. (paper) (code)

Exploring Contextualized Neural Language Models for Temporal Dependency Parsing. Hayley Ross, Jonathon Cai and Bonan Min. (paper) (code)

Beyond [CLS] through Ranking by Generation. Cicero Nogueira dos Santos, Xiaofei Ma, Ramesh Nallapati, zhiheng huang and Bing Xiang. (paper) (code)

SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling. Di Wu, Liang Ding, Fan Lu and Jian Xie. (paper) (code)

Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution. Yordan Yordanov, Oana-Maria Camburu, Vid Kocijan and Thomas Lukasiewicz. (paper) (code)

Incorporating Behavioral Hypotheses for Query Generation. Ruey-Cheng Chen and Chia-Jung Lee. (paper) (code)

Incomplete Utterance Rewriting as Semantic Segmentation. Qian Liu, Bei Chen, Jian-Guang LOU, Bin Zhou and Dongmei Zhang. (preprint) (paper) (code)