1、tensor 基本介绍

  • Tensors are a specialized data structure that are very similar to arrays and matrices. In PyTorch, we use tensors to encode the inputs and outputs of a model, as well as the model’s parameters.
  • Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators.
    • In fact, tensors and NumPy arrays can often share the same underlying memory, eliminating the need to copy data.
    • Tensors are also optimized for automatic differentiation (we’ll see more about that later in the Autograd section). If you’re familiar with ndarrays, you’ll be right at home with the Tensor API.
  • Tensor Attributes
    • Tensor attributes describe their shape, datatype, and the device on which they are stored. ```python tensor = torch.rand(3, 4)

jy: Shape of tensor: torch.Size([3, 4])

print(f”Shape of tensor: {tensor.shape}”)

jy: Datatype of tensor: torch.float32

print(f”Datatype of tensor: {tensor.dtype}”)

jy: Device tensor is stored on: cpu

print(f”Device tensor is stored on: {tensor.device}”)

  1. <a name="M1u1L"></a>
  2. # 2、Tensor 相关操作
  4. - Over 100 tensor operations, including transposing, indexing, slicing, mathematical operations, linear algebra, random sampling, 更多相关操作参考:
  5.    - [https://pytorch.org/docs/stable/torch.html](https://pytorch.org/docs/stable/torch.html)
  6. <a name="T6Asi"></a>
  7. ## 1）创建、判断 tensor
  9. - Tensors can be initialized in various ways. 
  10. <a name="Q7da5"></a>
  11. ### （1）基于列表创建
  13. - Tensors can be created directly from data（如 python 列表）. The data type is automatically inferred.
  14. ```python
  15. data = [[1, 2], [3, 4]]
  16. x_data = torch.tensor(data)

（2）基于 np 数组创建

  • Tensors can be created from NumPy arrays (and vice versa).
  • Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other. ```python data = [[1, 2], [3, 4]] np_array = np.array(data) x_np = torch.from_numpy(np_array)

jy: Tensor to NumPy array ================================

t = torch.ones(5)

jy: t: tensor([1., 1., 1., 1., 1.])

print(f”t: {t}”)

n = t.numpy()

jy: n: [1. 1. 1. 1. 1.]

print(f”n: {n}”)

jy: A change in the tensor reflects in the NumPy array.

t.add_(1)

jy: t: tensor([2., 2., 2., 2., 2.])

print(f”t: {t}”)

jy: n: [2. 2. 2. 2. 2.]

print(f”n: {n}”)

jy: NumPy array to Tensor =================================

n = np.ones(5) t = torch.from_numpy(n)

Changes in the NumPy array reflects in the tensor.

np.add(n, 1, out=n) print(f”t: {t}”) print(f”n: {n}”)

“”” t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64) n: [2. 2. 2. 2. 2.] “””

<a name="cBXmF"></a>
### （3）基于另外的 tensor

- The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.
```python
# retains the properties of x_data
x_ones = torch.ones_like(x_data) 
print(f"Ones Tensor: \n {x_ones} \n")

# overrides the datatype of x_data
x_rand = torch.rand_like(x_data, dtype=torch.float) 
print(f"Random Tensor: \n {x_rand} \n")

"""
Ones Tensor:
 tensor([[1, 1],
        [1, 1]])

Random Tensor:
 tensor([[0.7763, 0.1989],
        [0.1045, 0.8398]])
"""

（4）创建固定值、随机值 tensor

  • shape is a tuple of tensor dimensions, it determines the dimensionality of the output tensor. ```python shape = (2, 3,) rand_tensor = torch.rand(shape) print(f”Random Tensor: \n {rand_tensor} \n”) “”” Random Tensor: tensor([[0.3705, 0.7403, 0.8303],
      [0.5314, 0.7303, 0.8654]])
    “””

ones_tensor = torch.ones(shape) print(f”Ones Tensor: \n {ones_tensor} \n”) “”” Ones Tensor: tensor([[1., 1., 1.], [1., 1., 1.]]) “””

zeros_tensor = torch.zeros(shape) print(f”Zeros Tensor: \n {zeros_tensor}”) “”” Zeros Tensor: tensor([[0., 0., 0.], [0., 0., 0.]]) “””

<a name="R6emW"></a>
### （5）判断一个对象是否是 tensor
```python
import torch

shape = (2, 3,)
rand_tensor = torch.rand(shape)
print(torch.is_tensor(rand_tensor))  # jy: True

2）将 tensor 转移到 GPU/CPU

  • Over 100 tensor operations（including transposing, indexing, slicing, mathematical operations, linear algebra, random sampling）Each of them can be run on the GPU (at typically higher speeds than on a CPU).
  • If you’re using Colab, allocate a GPU by going to Edit > Notebook Settings. ```python

    We move our tensor to the GPU if available

    if torch.cuda.is_available(): tensor = tensor.to(‘cuda’) print(f”Device tensor is stored on: {tensor.device}”)

“”” Device tensor is stored on: cuda:0 “””

<a name="NvObS"></a>
## 3）tensor 切片赋值

- Try out some of the operations from the list. If you're familiar with the NumPy API, you'll find the Tensor API a breeze to use.
- Standard numpy-like indexing and slicing:
```python
tensor = torch.ones(4, 4)
tensor[:, 1] = 0
print(tensor)
"""
tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])
"""

4）tensors 合并

  • torch.cat：concatenate a sequence of tensors along a given dimension.
  • torch.stack：another tensor joining op that is subtly different from torch.cat
    • torch.stack：https://pytorch.org/docs/stable/generated/torch.stack.html
      t1 = torch.cat([tensor, tensor, tensor], dim=1)
print(t1)
"""
tensor([[1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
   [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
   [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.],
   [1., 0., 1., 1., 1., 0., 1., 1., 1., 0., 1., 1.]])
"""

      5）tensors 相乘

      （1）同维度同位置数值相乘

      ```python

      This computes the element-wise product

      print(f”tensor.mul(tensor) \n {tensor.mul(tensor)} \n”) “”” tensor.mul(tensor) tensor([[1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.]]) “””

Alternative syntax:

print(f”tensor tensor \n {tensor tensor}”) “”” tensor * tensor tensor([[1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.]]) “””

<a name="KELUO"></a>
### （2）tensor 矩阵相乘

- This computes the matrix multiplication between two tensors
```python
print(f"tensor.matmul(tensor.T) \n {tensor.matmul(tensor.T)} \n")
"""
tensor.matmul(tensor.T)
 tensor([[3., 3., 3., 3.],
        [3., 3., 3., 3.],
        [3., 3., 3., 3.],
        [3., 3., 3., 3.]])
"""

# Alternative syntax:
print(f"tensor @ tensor.T \n {tensor @ tensor.T}")
"""
tensor @ tensor.T
 tensor([[3., 3., 3., 3.],
        [3., 3., 3., 3.],
        [3., 3., 3., 3.],
        [3., 3., 3., 3.]])
"""

6）In-place operations（不推荐使用）

  • Operations that have a _ suffix are in-place. For example: x.copy_(y), x.t_(), will change x. ```python print(tensor, “\n”) “”” tensor([[1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.], [1., 0., 1., 1.]]) “””

tensor.add_(5) print(tensor) “”” tensor([[6., 5., 6., 6.], [6., 5., 6., 6.], [6., 5., 6., 6.], [6., 5., 6., 6.]]) “””


- NOTE：In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. Hence, their use is discouraged.
<a name="kUm1a"></a>
## 7）常用操作汇总

- `torch.max(input, dim, keepdim=False, *, out=None)`：按维度 dim 返回最大值，并且返回索引
   - `input`：the input tensor.
   - `dim`： the dimension to reduce. （如果为 n 维向量，则 dim 的值范围为 `[-n, n-1]`）
   -  `keepdim`：whether the output tensor has dim retained or not（默认为 False）
   -  Keyword Arguments
      - `out (tuple, optional)`：the result tuple of two output tensors `(max, max_indices)`，返回的最大值和索引各是一个 tensor，分别表示该维度的最大值，以及该维度最大值的索引，一起构成元组`(Tensor, LongTensor)`
```python
import torch


print("\n生成一个维度为 [1, 3] 值随机的 tensor")
tensor_random = torch.randn(1, 3)
print(tensor_random)


print("\n将列表转为 tensor 类型, 维度与列表的维度相同")
a = torch.tensor([[1, 2, 3]])
print(a)
# jy: torch.Size([1, 3])
print(a.shape)

print("\n将二维 tensor 列表转换为三维 tensor")
ls_dim2_tensor = [torch.randn(1, 3) for i in range(5)]
print(ls_dim2_tensor)


print("\n将 tensor 列表堆叠为再多一维的 tensor")
dim3_tensor = torch.stack(ls_dim2_tensor)
print(dim3_tensor)
# jy: torch.Size([5, 1, 3])
print(dim3_tensor.shape)

print("\n对多维向量中第一列的结果取最大值")
print(torch.max(dim3_tensor, 0))
print(torch.max(dim3_tensor, -3))



print("=" * 88)

ls_dim1_tensor = [torch.randn(3) for i in range(5)]
print(ls_dim1_tensor)
print("\n对列表中的一维向量中的每一维数值进行加和")
print(sum(ls_dim1_tensor))
print("\n对列表中的一维向量中的每一维取平均")
print(sum(ls_dim1_tensor) / len(ls_dim1_tensor))


print("\n将 tensor 列表堆叠为再多一维的 tensor")
dim2_tensor = torch.stack(ls_dim1_tensor)
print(dim2_tensor)
print(dim2_tensor.shape)
print("\n对二维向量中每一列的结果取最大值")
print(torch.max(dim2_tensor, 0))
print(torch.max(dim2_tensor, -2))

print("\n对二维向量中每一行的结果取最大值")
print(torch.max(dim2_tensor, 1))
print(torch.max(dim2_tensor, -1))



print("\n取两个同维度的向量中同一维度位置上的最大值")
#tensor_a = torch.randn(3)
#tensor_b = torch.randn(3)
tensor_a = torch.randn(1, 3)
tensor_b = torch.randn(1, 3)
print(tensor_a)
print(tensor_b)
tensor_max = torch.max(tensor_a, tensor_b)
print(tensor_max)

2、transforms

  • Data does not always come in its final processed form that is required for training machine learning algorithms. We use transforms to perform some manipulation of the data and make it suitable for training.
  • All TorchVision datasets have two parameters（that accept callables containing the transformation logic.）：
    • transform to modify the features
    • target_transform to modify the labels
  • The torchvision.transforms module offers several commonly-used transforms out of the box.
  • The FashionMNIST features are in PIL Image format, and the labels are integers. For training, we need the features as normalized tensors, and the labels as one-hot encoded tensors. To make these transformations, we use ToTensor and Lambda.

ds = datasets.FashionMNIST( root=”data”, train=True, download=True, transform=ToTensor(), targettransform=Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter(0, torch.tensor(y), value=1)) ) ```