Constituency Parsing

Constituent

• constituent: can be a single word or a phrases as a single unit within a hierarchical structure.
• a phrase: is a sequence of two or more words built around a head lexical item and working as a unit within a sentence.
• to be a phrase, a group of words should:
  • come together to play a specific role in the sentence
  • can be moved together or replaced as a whole

• People interpret the meaning of large text units by semantic composition of smaller elements

  Constituency Parse Tree

• Non-terminals: types of phrases

• Terminals: the exact words

Represent the meaning of long phrases

Map phrases into Vector Space

• What we want is
  • constituency sentence parsing
  • learn structure and representation

    Recursive vs recurrent neural networks

Recursive	Recurrent
- require a tree structure	- cannot capture phrases without prefix context - often capture too much of last words in final vector

Structure Prediction

• 输入：两个候选子结点的表示
• 输出：
  • 两个候选结点合并后的语义表示
  • 新结点的可能性分数

Parsing a sentence (greedily)

• 每次从所有可能的候选组合中，选择可能性分数最高的，进行合并
• 合并后，重复上述过程，直到只剩一个根结点

• score of tree: the sum of the parsing decision scores at each node

• 其中 x 为句子，y 为 parse tree
  • 损失函数

• 问题：单一的矩阵不能处理过于复杂的组成

  • 两个输入之间没有交互
  • 对所有的语法类型，使用了相同的参数矩阵

    改进：Syntactically-United RNN

    

• 为不同的语法类型使用了不同的参数矩阵。

• 参数矩阵被初始化为单位矩阵：即对两个输入取平均
  • 模型可以学到哪个两个输入向量哪个更重要
  • 可以学到对向量进行哪些旋转或放缩可以提高性能
• 通过可视化学习后的参数矩阵，可以看出学到了什么。可以发现学到了 soft head words

• 问题：

  • 没有学到二者之间的交互
  • 如一个词对另一个词的影响，very good

    改进：Matrix-Vector Recursive Neural Network

• 每个单词包含两个表示：

  • 单词向量：表示它的语义
  • 单词矩阵：表示它如何影响其他单词

改进：Recursive Neural Tensor Network

• 计算公式