代码 - PyTorch 学习（一）| 代码 - 《CS》

数据处理
定义网络结构
训练
测试
可视化
- BUG
模型保存与读取
变量测试
格式转换

import 模块

from __future__ import print_function

将 future 看做是一个 Python 专门存放新特性的模块
import print_function 之后，在 Python2.x 中也需要使用 Python3.x 的 print() 来输出

import torch
import torchvision
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import matplotlib.pyplot as plt
import numpy as np
from torchvision import datasets, transforms
from torch.utils.data import Dataset, DataLoader

引入 root package
The torchvision package consists of popular datasets, model architectures, and common image transformations for computer vision
- torchvision.datasets : MNIST, CIFAR, etc.
- torchvision.transforms : torchvision.transforms are common image transformations. They can be chained together using torchvision.transforms.Compose(transforms).
torch.nn 神经网络工具箱（layer是由class定义） : Conv2d, MaxPool2d, etc.
torch.nn.functional 与 torch.nn 相似，但它的layer由def定义。
引入 optimizer
引入绘图模块
引入 numpy 模块
引入图片转换模块
引入数据模块

torch.utils.data.Dataset 是数据集的抽象类，所有的 torchvision.datasets 都是 torch.utils.data.Dataset 的子类。

torch.nn.functional是确定不变的运算公式，如果模型有可学习的参数，最好使用torch.nn中nn.Module对应的相关layer。比如Relu其实没有可学习的参数，只是进行一个运算而已，所以使用的就是functional中的relu函数，而卷积层和全连接层都有可学习的参数，所以用的是nn.Module中的类。

数据处理

batch_size = 4
epoch_size = 3

# 对数据进行预处理
transform = transforms.Compose(
    [transforms.ToTensor(),  # range [0, 255] -> [0.0,1.0]
     transforms.Normalize((0.5, 0.5, 0.5),  # 以均值和标准差进行标准化（C=3）
                          (0.5, 0.5, 0.5))])
trainset = datasets.CIFAR10(root='./data/cifar10',  # 相对目录
                            train=True,  # True 则为训练集
                            download=True,  # 目录中数据集不存在则下载
                            transform=transform)
trainloader = DataLoader(trainset,
                         batch_size=batch_size,
                         shuffle=True,  # 将数据打乱
                         num_workers=0)  # 进程数
testset = datasets.CIFAR10(root='./data/cifar10',
                           train=False,
                           download=True,
                           transform=transform)
testloader = DataLoader(testset,
                        batch_size=batch_size,
                        shuffle=False,
                        num_workers=0)
classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

Files already downloaded and verified
Files already downloaded and verified

`torchvision.transforms` 中的函数

Resize：把给定的图片resize到given size
Normalize：Normalized an tensor image with mean and standard deviation
ToTensor：convert a PIL image to tensor (HWC) in range [0,255] to a torch.Tensor(CHW) in the range [0.0,1.0]
ToPILImage: convert a tensor to PIL image
Scale：目前已经不用了，推荐用Resize
CenterCrop：在图片的中间区域进行裁剪
RandomCrop：在一个随机的位置进行裁剪
RandomHorizontalFlip：以0.5的概率水平翻转给定的PIL图像
RandomVerticalFlip：以0.5的概率竖直翻转给定的PIL图像
RandomResizedCrop：将PIL图像裁剪成任意大小和纵横比
Grayscale：将图像转换为灰度图像
RandomGrayscale：将图像以一定的概率转换为灰度图像
FiceCrop：把图像裁剪为四个角和一个中心
TenCrop
Pad：填充
ColorJitter：随机改变图像的亮度对比度和饱和度

def imshow(img):
    img = img / 2 + 0.5  # 去标准化
    npimg = img.numpy()  # 将torch.FloatTensor 转换为numpy
#     plt.axis("off")  # 不显示坐标尺寸
    plt.imshow(np.transpose(npimg, (1, 2, 0)))  # 进行转置
    plt.show()  # 显示图片
# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()
# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%11s' % classes[labels[j]] for j in range(batch_size)))

bird        frog       horse       truck

错误：

[Errno 32] Broken pipe

原因：

trainloader = DataLoader(trainset,
                 batch_size=batch_size,
                 shuffle=True,  # 将数据打乱
                 num_workers=2)  # 进程数

方法:

设置 num_workers = 0

`numpy.transpose` 说明

如果 x = np.arange(24).reshape((2, 3, 4))(维度/通道为 2，行数为 3，列数为 4)，
对 x 按照某一顺序进行转置（numpy.transpose）：

np.transpose(x, (2, 0, 1)) : shape(2,3,4) -> shape(4,2,3)
np.transpose(x, (0, 1, 2)) : shape(2,3,4) -> shape(2,3,4)

查看列表的维度

np.array(images).shape #(batch_size, C, H, W)
(4, 3, 32, 32)

定义网络结构

class Net(nn.Module): # nn.Module 是所有神经网络的基类，自定义的网络应该继承自它
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
net = Net().to(device)

# 定义损失函数和优化器
criterion = nn.CrossEntropyLoss()  # 交叉熵损失
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9) # SGD with momentum

训练

import os
if os.path.exists('./model/learn0.pt'):
    net.load_state_dict(torch.load('./model/learn0.pt'))
    net.eval()
else:
    for epoch in range(epoch_size):  # loop over the dataset multiple times
        running_loss = 0.0
        for i, data in enumerate(trainloader, 0):
            # get the inputs; data is a list of [inputs, labels]
            inputs, labels = inputs, labels = data[0].to(device), data[1].to(device)
            # zero the parameter gradients
            optimizer.zero_grad()
            # forward + backward + optimize
            outputs = net(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
            # print statistics
            running_loss += loss.item()
            if i % 2000 == 1999:    # print every 2000 mini-batches
                print('[%d, %5d] loss: %.3f' %
                      (epoch + 1, i + 1, running_loss / 2000))
                running_loss = 0.0
    print('Finished Training')

# tensorboard
from torch.autograd import Variable
from tensorboardX import SummaryWriter
dumpy_input = Variable(torch.rand(1,3 ,32 ,32 ))
with SummaryWriter('runs/learn-0/') as W:
    W.add_graph(net,(dumpy_input,))
import torch
import torchvision
from tensorboardX import SummaryWriter
from torchvision import datasets, transforms
# Writer will output to ./runs/ directory by default
writer = SummaryWriter('runs/unimportant/')
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
trainset = datasets.MNIST('mnist_train', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
model = torchvision.models.resnet50(False)
# Have ResNet model take in grayscale rather than RGB
model.conv1 = torch.nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
images, labels = next(iter(trainloader))
grid = torchvision.utils.make_grid(images)
writer.add_image('images', grid, 0)
writer.add_graph(model, images)
writer.close()

测试

# 测试集准备
dataiter = iter(testloader)
images, labels = dataiter.next()
# print images
print('GroundTruth:')
imshow(torchvision.utils.make_grid(images))
print(' '.join('%11s' % classes[labels[j]] for j in range(4)))
# 测试一次
outputs = net(images.to(device))
_, predicted = torch.max(outputs, 1)
print('Predicted:\n', ' '.join('%11s' % classes[predicted[j]]
                              for j in range(4)))
# 在整个测试集上测试
correct = 0
total = 0
with torch.no_grad(): # 不进行梯度计算，减小了内存的占用，一般此时不进行BP的计算
    for data in testloader:
        images, labels = data[0].to(device),data[1].to(device)
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()
print('Accuracy of the network on the %d test images: %d %%' % (batch_size*len(testloader),
    100 * correct / total))

GroundTruth:

cat        ship        ship       plane
Predicted:
         cat        ship        ship       plane
Accuracy of the network on the 10000 test images: 57 %

# 模型对各个class分类的表现
class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
    for data in testloader:
        images, labels = data[0].to(device), data[1].to(device)
        outputs = net(images)
        _, predicted = torch.max(outputs, 1) # _ 表示不重要，可直接取[1]赋值
        c = (predicted == labels).squeeze()
        for i in range(4):
            label = labels[i]
            class_correct[label] += c[i].item()
            class_total[label] += 1
for i in range(10):
    print('Accuracy of %5s : %2d %%' % (
        classes[i], 100 * class_correct[i] / class_total[i]))

Accuracy of plane : 62 %
Accuracy of   car : 68 %
Accuracy of  bird : 51 %
Accuracy of   cat : 39 %
Accuracy of  deer : 51 %
Accuracy of   dog : 40 %
Accuracy of  frog : 62 %
Accuracy of horse : 52 %
Accuracy of  ship : 80 %
Accuracy of truck : 62 %

可视化

BUG

必须先安装软件 graphviz 整个流程才可以继续下去
安装软件 -> 添加环境变量 -> pip install package
因为需要添加环境变量，所以添加完之后要关闭打开 notebook 的 cmd 窗口，重新启动才能 work。

import tensorwatch as tw
# from torchviz import make_dot
tw.draw_model(net.to('cpu'),[1,3,32,32])

模型保存与读取

# 模型保存
torch.save(net.state_dict(),'./model/learn0.pt')

# 模型读取
model = Net().to('cuda')
model.load_state_dict(torch.load('./model/learn0.pt'))
model.eval()

变量测试

# 测试 python
# help(torch.no_grad)
# print(torch.cuda.device_count())
print(net.parameters())

# 测试 shell
!nvidia-smi

格式转换

# !jupyter nbconvert --to html --template full learn.ipynb
# !jupyter nbconvert --to markdown learn.ipynb
!jupyter nbconvert --to html --template full learn-0.ipynb
!jupyter nbconvert --to markdown learn-0.ipynb

PyTorch 学习（一）| 代码