PyTorch：optim

原文：https://pytorch.org/tutorials/beginner/examples_nn/polynomial_optim.html#sphx-glr-beginner-examples-nn-polynomial-optim-py

经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测y = sin(x)从-pi到pi。

此实现使用来自 PyTorch 的nn包来构建网络。

与其像以前那样手动更新模型的权重，不如使用optim包定义一个优化器，该优化器将为我们更新权重。 optim包定义了许多深度学习常用的优化算法，包括 SGD + 动量，RMSProp，Adam 等。

  1. import torch
  2. import math
  4. # Create Tensors to hold input and outputs.
  5. x = torch.linspace(-math.pi, math.pi, 2000)
  6. y = torch.sin(x)
  8. # Prepare the input tensor (x, x^2, x^3).
  9. p = torch.tensor([1, 2, 3])
  10. xx = x.unsqueeze(-1).pow(p)
  12. # Use the nn package to define our model and loss function.
  13. model = torch.nn.Sequential(
  14.     torch.nn.Linear(3, 1),
  15.     torch.nn.Flatten(0, 1)
  16. )
  17. loss_fn = torch.nn.MSELoss(reduction='sum')
  19. # Use the optim package to define an Optimizer that will update the weights of
  20. # the model for us. Here we will use RMSprop; the optim package contains many other
  21. # optimization algorithms. The first argument to the RMSprop constructor tells the
  22. # optimizer which Tensors it should update.
  23. learning_rate = 1e-3
  24. optimizer = torch.optim.RMSprop(model.parameters(), lr=learning_rate)
  25. for t in range(2000):
  26.     # Forward pass: compute predicted y by passing x to the model.
  27.     y_pred = model(xx)
  29.     # Compute and print loss.
  30.     loss = loss_fn(y_pred, y)
  31.     if t % 100 == 99:
  32.         print(t, loss.item())
  34.     # Before the backward pass, use the optimizer object to zero all of the
  35.     # gradients for the variables it will update (which are the learnable
  36.     # weights of the model). This is because by default, gradients are
  37.     # accumulated in buffers( i.e, not overwritten) whenever .backward()
  38.     # is called. Checkout docs of torch.autograd.backward for more details.
  39.     optimizer.zero_grad()
  41.     # Backward pass: compute gradient of the loss with respect to model
  42.     # parameters
  43.     loss.backward()
  45.     # Calling the step function on an Optimizer makes an update to its
  46.     # parameters
  47.     optimizer.step()
  49. linear_layer = model[0]
  50. print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3')

脚本的总运行时间：（0 分钟 0.000 秒）

下载 Python 源码：polynomial_optim.py

下载 Jupyter 笔记本：polynomial_optim.ipynb

由 Sphinx 画廊生成的画廊