1 Parameter和Tensor有何不同

[torch/nn/parameter.py]
class Parameter(torch.Tensor):
    r"""A kind of Tensor that is to be considered a module parameter.
    Parameters are :class:`~torch.Tensor` subclasses, that have a
    very special property when used with :class:`Module` s - when they're
    assigned as Module attributes they are automatically added to the list of
    its parameters, and will appear e.g. in :meth:`~Module.parameters` iterator.
    Assigning a Tensor doesn't have such effect. This is because one might
    want to cache some temporary state, like last hidden state of the RNN, in
    the model. If there was no such class as :class:`Parameter`, these
    temporaries would get registered too.
    Args:
        data (Tensor): parameter tensor.
        requires_grad (bool, optional): if the parameter requires gradient. See
            :ref:`excluding-subgraphs` for more details. Default: `True`
    """
    ...

Parameter是Tensor的子类，Parameter不同于Tensor的地方在于Parameter被定义为Module的参数（Module是所有模型的基类，例如Linear、Conv都要继承Module）。当Parameter被赋值为Module的属性时，Parameter将被自动注册到Module的_parameters有序字典中。

• 例如 ```python import torch import torch.nn as nn

class Model(nn.Module): def init(self): super(Model, self).init() self.tensor = torch.zeros(1) self.param = nn.Parameter(torch.zeros(1))

当我们这样定义模型时，第7行把`torch.zeros(1)`赋值给`self.tensor`，第8行把`nn.Parameter(torch.zeros(1))`赋值给`self.param`
```python
model = Model()
print(model._parameters)
"""
OrderedDict([('param',
              Parameter containing:
              tensor([0.], requires_grad=True))])
"""

我们可以看到，self.param被注册到_parameters中了，而self.tensor并不在其中，而这就是Parameter和Tensor的不同之处，即Parameter可以被自动注册到_parameters中。

2 Parameter是如何被注册到Module

既然Parameter被注册的行为是在赋值时发生的，那么注册的行为就可以推测是在__setattr__()方法中进行的，下面分析Module类的__setattr__()方法。

[torch/nn/modules/module.py]
class Module:
    ...
    def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:
        ...
        params = self.__dict__.get('_parameters')
        if isinstance(value, Parameter):
            if params is None:
                raise AttributeError(
                    "cannot assign parameters before Module.__init__() call")
            remove_from(self.__dict__, self._buffers, self._modules, self._non_persistent_buffers_set)
            self.register_parameter(name, value)
        elif params is not None and name in params:
            if value is not None:
                raise TypeError("cannot assign '{}' as parameter '{}' "
                                "(torch.nn.Parameter or None expected)"
                                .format(torch.typename(value), name))
            self.register_parameter(name, value)
        else:
            ...

可以看到，第12行或者第18行代码调用了register_parameter()

[torch/nn/modules/module.py]
class Module:
    ...
    def register_parameter(self, name: str, param: Optional[Parameter]) -> None:
        r"""Adds a parameter to the module.
        The parameter can be accessed as an attribute using given name.
        Args:
            name (string): name of the parameter. The parameter can be accessed
                from this module using the given name
            param (Parameter): parameter to be added to the module.
        """
        ...
        self._parameters[name] = param
        ...

可以看到，当执行self.param = nn.Parameter(torch.zeros(1))最终会把param注册到_parameters中。