PyTorch
提供几个pytorch中常用的向量相似度评估方法，并给出其源码实现，供大家参考。分别为以下六个。

  1. CosineSimilarity
  2. DotProductSimilarity
  3. ProjectedDotProductSimilarity
  4. BiLinearSimilarity
  5. TriLinearSimilarity
  6. MultiHeadedSimilarity

    1、余弦相似度

    余弦相似度用向量空间中两个向量夹角的余弦值作为衡量两个个体间差异的大小。余弦值越接近1，就表明夹角越接近0度，也就是两个向量越相似，称为”余弦相似性” ```python import torch import torch.nn as nn import math

class CosineSimilarity(nn.Module):

  1. def forward(self, tensor_1, tensor_2):
  2.     normalized_tensor_1 = tensor_1 / tensor_1.norm(dim=-1, keepdim=True)
  3.     normalized_tensor_2 = tensor_2 / tensor_2.norm(dim=-1, keepdim=True)
  4.     return (normalized_tensor_1 * normalized_tensor_2).sum(dim=-1)
  1. <a name="Ql9Lf"></a>
  2. ### 2、DotProductSimilarity 
  3. 这个相似度函数简单地计算每对向量之间的点积，并使用可选的缩放来减少输出的方差。
  4. ```python
  5. class DotProductSimilarity(nn.Module):
  7.     def __init__(self, scale_output=False):
  8.         super(DotProductSimilarity, self).__init__()
  9.         self.scale_output = scale_output
  11.     def forward(self, tensor_1, tensor_2):
  12.         result = (tensor_1 * tensor_2).sum(dim=-1)
  13.         if self.scale_output:
  14.             # TODO why allennlp do multiplication at here ?
  15.             result /= math.sqrt(tensor_1.size(-1))
  16.         return result

3、ProjectedDotProductSimilarity

这个相似度函数做一个投影，然后计算点积，计算公式为：
Pytorch 中六种常用的向量相似度评估方法 - 图1
计算后的激活函数。默认为不激活。

  1. class ProjectedDotProductSimilarity(nn.Module):
  3.     def __init__(self, tensor_1_dim, tensor_2_dim, projected_dim,
  4.                  reuse_weight=False, bias=False, activation=None):
  5.         super(ProjectedDotProductSimilarity, self).__init__()
  6.         self.reuse_weight = reuse_weight
  7.         self.projecting_weight_1 = nn.Parameter(torch.Tensor(tensor_1_dim, projected_dim))
  8.         if self.reuse_weight:
  9.             if tensor_1_dim != tensor_2_dim:
  10.                 raise ValueError('if reuse_weight=True, tensor_1_dim must equal tensor_2_dim')
  11.         else:
  12.             self.projecting_weight_2 = nn.Parameter(torch.Tensor(tensor_2_dim, projected_dim))
  13.         self.bias = nn.Parameter(torch.Tensor(1)) if bias else None
  14.         self.activation = activation
  16.     def reset_parameters(self):
  17.         nn.init.xavier_uniform_(self.projecting_weight_1)
  18.         if not self.reuse_weight:
  19.             nn.init.xavier_uniform_(self.projecting_weight_2)
  20.         if self.bias is not None:
  21.             self.bias.data.fill_(0)
  23.     def forward(self, tensor_1, tensor_2):
  24.         projected_tensor_1 = torch.matmul(tensor_1, self.projecting_weight_1)
  25.         if self.reuse_weight:
  26.             projected_tensor_2 = torch.matmul(tensor_2, self.projecting_weight_1)
  27.         else:
  28.             projected_tensor_2 = torch.matmul(tensor_2, self.projecting_weight_2)
  29.         result = (projected_tensor_1 * projected_tensor_2).sum(dim=-1)
  30.         if self.bias is not None:
  31.             result = result + self.bias
  32.         if self.activation is not None:
  33.             result = self.activation(result)
  34.         return result

4、BiLinearSimilarity

此相似度函数执行两个输入向量的双线性变换。这个函数有一个权重矩阵“W”和一个偏差“b”，以及两个向量之间的相似度，计算公式为：
Pytorch 中六种常用的向量相似度评估方法 - 图2
计算后的激活函数。 默认为不激活。

  1. class BiLinearSimilarity(nn.Module):
  3.     def __init__(self, tensor_1_dim, tensor_2_dim, activation=None):
  4.         super(BiLinearSimilarity, self).__init__()
  5.         self.weight_matrix = nn.Parameter(torch.Tensor(tensor_1_dim, tensor_2_dim))
  6.         self.bias = nn.Parameter(torch.Tensor(1))
  7.         self.activation = activation
  8.         self.reset_parameters()
  10.     def reset_parameters(self):
  11.         nn.init.xavier_uniform_(self.weight_matrix)
  12.         self.bias.data.fill_(0)
  14.     def forward(self, tensor_1, tensor_2):
  15.         intermediate = torch.matmul(tensor_1, self.weight_matrix)
  16.         result = (intermediate * tensor_2).sum(dim=-1) + self.bias
  17.         if self.activation is not None:
  18.             result = self.activation(result)
  19.         return result

5、TriLinearSimilarity

此相似度函数执行两个输入向量的三线性变换，计算公式为：
Pytorch 中六种常用的向量相似度评估方法 - 图3
计算后的激活函数。 默认为不激活。

  1. class TriLinearSimilarity(nn.Module):
  3.     def __init__(self, input_dim, activation=None):
  4.         super(TriLinearSimilarity, self).__init__()
  5.         self.weight_vector = nn.Parameter(torch.Tensor(3 * input_dim))
  6.         self.bias = nn.Parameter(torch.Tensor(1))
  7.         self.activation = activation
  8.         self.reset_parameters()
  10.     def reset_parameters(self):
  11.         std = math.sqrt(6 / (self.weight_vector.size(0) + 1))
  12.         self.weight_vector.data.uniform_(-std, std)
  13.         self.bias.data.fill_(0)
  15.     def forward(self, tensor_1, tensor_2):
  16.         combined_tensors = torch.cat([tensor_1, tensor_2, tensor_1 * tensor_2], dim=-1)
  17.         result = torch.matmul(combined_tensors, self.weight_vector) + self.bias
  18.         if self.activation is not None:
  19.             result = self.activation(result)
  20.         return result

6、MultiHeadedSimilarity

这个相似度函数使用多个“头”来计算相似度。也就是说，将输入张量投影到多个新张量中，并分别计算每个投影张量的相似度。这里的结果比典型的相似度函数多一个维度。

  1. class MultiHeadedSimilarity(nn.Module):
  3.     def __init__(self,
  4.                  num_heads,
  5.                  tensor_1_dim,
  6.                  tensor_1_projected_dim=None,
  7.                  tensor_2_dim=None,
  8.                  tensor_2_projected_dim=None,
  9.                  internal_similarity=DotProductSimilarity()):
  10.         super(MultiHeadedSimilarity, self).__init__()
  11.         self.num_heads = num_heads
  12.         self.internal_similarity = internal_similarity
  13.         tensor_1_projected_dim = tensor_1_projected_dim or tensor_1_dim
  14.         tensor_2_dim = tensor_2_dim or tensor_1_dim
  15.         tensor_2_projected_dim = tensor_2_projected_dim or tensor_2_dim
  16.         if tensor_1_projected_dim % num_heads != 0:
  17.             raise ValueError("Projected dimension not divisible by number of heads: %d, %d"
  18.                              % (tensor_1_projected_dim, num_heads))
  19.         if tensor_2_projected_dim % num_heads != 0:
  20.             raise ValueError("Projected dimension not divisible by number of heads: %d, %d"
  21.                              % (tensor_2_projected_dim, num_heads))
  22.         self.tensor_1_projection = nn.Parameter(torch.Tensor(tensor_1_dim, tensor_1_projected_dim))
  23.         self.tensor_2_projection = nn.Parameter(torch.Tensor(tensor_2_dim, tensor_2_projected_dim))
  24.         self.reset_parameters()
  26.     def reset_parameters(self):
  27.         torch.nn.init.xavier_uniform_(self.tensor_1_projection)
  28.         torch.nn.init.xavier_uniform_(self.tensor_2_projection)
  30.     def forward(self, tensor_1, tensor_2):
  31.         projected_tensor_1 = torch.matmul(tensor_1, self.tensor_1_projection)
  32.         projected_tensor_2 = torch.matmul(tensor_2, self.tensor_2_projection)
  34.         # Here we split the last dimension of the tensors from (..., projected_dim) to
  35.         # (..., num_heads, projected_dim / num_heads), using tensor.view().
  36.         last_dim_size = projected_tensor_1.size(-1) // self.num_heads
  37.         new_shape = list(projected_tensor_1.size())[:-1] + [self.num_heads, last_dim_size]
  38.         split_tensor_1 = projected_tensor_1.view(*new_shape)
  39.         last_dim_size = projected_tensor_2.size(-1) // self.num_heads
  40.         new_shape = list(projected_tensor_2.size())[:-1] + [self.num_heads, last_dim_size]
  41.         split_tensor_2 = projected_tensor_2.view(*new_shape)
  43.         # And then we pass this off to our internal similarity function. Because the similarity
  44.         # functions don't care what dimension their input has, and only look at the last dimension,
  45.         # we don't need to do anything special here. It will just compute similarity on the
  46.         # projection dimension for each head, returning a tensor of shape (..., num_heads).
  47.         return self.internal_similarity(split_tensor_1, split_tensor_2)