Normalization Layers

[nn.BatchNorm1d](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm1d.html#torch.nn.BatchNorm1d) Applies Batch Normalization over a 2D or 3D input (a mini-batch of 1D inputs with optional additional channel dimension) as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift .
[nn.BatchNorm2d](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html#torch.nn.BatchNorm2d) Applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift .
[nn.BatchNorm3d](https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm3d.html#torch.nn.BatchNorm3d) Applies Batch Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift .
[nn.GroupNorm](https://pytorch.org/docs/stable/generated/torch.nn.GroupNorm.html#torch.nn.GroupNorm) Applies Group Normalization over a mini-batch of inputs as described in the paper Group Normalization
[nn.SyncBatchNorm](https://pytorch.org/docs/stable/generated/torch.nn.SyncBatchNorm.html#torch.nn.SyncBatchNorm) Applies Batch Normalization over a N-Dimensional input (a mini-batch of [N-2]D inputs with additional channel dimension) as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift .
[nn.InstanceNorm1d](https://pytorch.org/docs/stable/generated/torch.nn.InstanceNorm1d.html#torch.nn.InstanceNorm1d) Applies Instance Normalization over a 3D input (a mini-batch of 1D inputs with optional additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.
[nn.InstanceNorm2d](https://pytorch.org/docs/stable/generated/torch.nn.InstanceNorm2d.html#torch.nn.InstanceNorm2d) Applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.
[nn.InstanceNorm3d](https://pytorch.org/docs/stable/generated/torch.nn.InstanceNorm3d.html#torch.nn.InstanceNorm3d) Applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization.
[nn.LayerNorm](https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html#torch.nn.LayerNorm) Applies Layer Normalization over a mini-batch of inputs as described in the paper Layer Normalization
[nn.LocalResponseNorm](https://pytorch.org/docs/stable/generated/torch.nn.LocalResponseNorm.html#torch.nn.LocalResponseNorm) Applies local response normalization over an input signal composed of several input planes, where channels occupy the second dimension. 
  2. class PixelNorm(nn.Module):
  3.     def __init__(self, num_channels=None):
  4.         super().__init__()
  5.         # num_channels is only used to match function signature with other normalization layers
  6.         # it has no actual use仅用于将函数签名与其他规范化层匹配 ,它没有实际用途
  7.     def forward(self, input):
  8.         return input / torch.sqrt(torch.mean(input ** 2, dim=1, keepdim=True) + 1e-5)
  11. def get_norm_layer(norm_type='instance'):
  12.     if norm_type == 'instance':
  13.         norm_layer = functools.partial(nn.InstanceNorm2d, affine=False)
  14.     elif norm_type == 'pixel':
  15.         norm_layer = PixelNorm
  16.     else:
  17.         raise NotImplementedError('normalization layer [%s] is not found' % norm_type)
  18.     return norm_layer