04 Containerd CLI

ctr 使用

ctr 目前很多功能做的还没有 docker 那么完善，但基本功能已经具备了。下面将围绕镜像和容器这两个方面来介绍其使用方法。

# ctr -h
NAME:
   ctr -
        __
  _____/ /______
 / ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/
containerd CLI
USAGE:
   ctr [global options] command [command options] [arguments...]
VERSION:
   v1.4.4
DESCRIPTION:
ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.
COMMANDS:
   plugins, plugin            provides information about containerd plugins
   version                    print the client and server versions
   containers, c, container   manage containers
   content                    manage content
   events, event              display containerd events
   images, image, i           manage images
   leases                     manage leases
   namespaces, namespace, ns  manage namespaces
   pprof                      provide golang pprof outputs for containerd
   run                        run a container
   snapshots, snapshot        manage snapshots
   tasks, t, task             manage tasks
   install                    install a new package
   oci                        OCI tools
   shim                       interact with a shim directly
   help, h                    Shows a list of commands or help for one command
GLOBAL OPTIONS:
   --debug                      enable debug output in logs
   --address value, -a value    address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
   --timeout value              total timeout for ctr commands (default: 0s)
   --connect-timeout value      timeout for connecting to containerd (default: 0s)
   --namespace value, -n value  namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
   --help, -h                   show help
   --version, -v                print the version

镜像

镜像下载

# ctr i pull docker.io/library/nginx:alpine
docker.io/library/nginx:alpine:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:e20c21e530f914fb6a95a755924b1cbf71f039372e94ac5ddcf8c3b386a44615:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:96419b83f29b198ae9f63670d5a28325a8bc9ebaf76c1260cf15eca3a521ebd0: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:5b4dcb4d3646b218b442697b56990ca56997efa87f6861388164cc46d353659a:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:5fd75c905b52a175660818ab9f318cfab375da0b90d23690e00871c0db1ed3a4:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ba3557a56b150f9b813f9d02274d62914fd8fce120dd374d9ee17b87cf1d277d:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:468d8ccebf7a1512bde09f06975206a7f474c714f96020dabb6e3437a3bc426b:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:b7f67c5d6ce97f864346b65ed55571f7d191c0f9518ac59bb1b7291fbae0f716:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ed91f01a4fcb8de36c14f0048c15971b7a2d8eaf4740c9d2855de94e19467cd7:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:8051568c89ac729bba391cc909f7eb97ad3e0bc6991db52ad3b5a7bb4cc6c000:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 15.5s                                                                    total:  8.7 Mi (574.3 KiB/s)
unpacking linux/amd64 sha256:e20c21e530f914fb6a95a755924b1cbf71f039372e94ac5ddcf8c3b386a44615...
done

本地镜像列表查询

root@i-gqqsb62d:/etc/containerd# ctr i ls
REF                            TYPE                                                      DIGEST                                                                  SIZE    PLATFORMS     
                                                                           LABELS
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:e20c21e530f914fb6a95a755924b1cbf71f039372e94ac5ddcf8c3b386a44615 9.4 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -

这里需要注意PLATFORMS，它是镜像的能够运行的平台标识。

将镜像挂载到主机目录：

 ctr i mount docker.io/library/nginx:alpine /mnt/nginx_tmp/
sha256:22dd1c631d282d9c89a77fb4592d8f32dcc4e946228cf0b671a2d37c2511e1dc
/mnt/nginx_tmp/

# tree /mnt/nginx_tmp/ -L 1
/mnt/nginx_tmp/
├── bin
├── dev
├── docker-entrypoint.d
├── docker-entrypoint.sh
├── etc
├── home
├── lib
├── media
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin
├── srv
├── sys
├── tmp
├── usr
└── var

讲镜像从主机卸载

# ctr i unmount /mnt/nginx_tmp/
/mnt/nginx_tmp/

镜像包导出为压缩包

root@i-gqqsb62d:~# ctr i export nginx.tar.gz docker.io/library/nginx:alpine
root@i-gqqsb62d:~# ls nginx.tar.gz 
nginx.tar.gz

删除

root@i-gqqsb62d:~# ctr i remove docker.io/library/nginx:alpine
docker.io/library/nginx:alpine
root@i-gqqsb62d:~# ctr i ls
REF TYPE DIGEST SIZE PLATFORMS LABELS

导入

root@i-gqqsb62d:~# ctr i import nginx.tar.gz 
unpacking docker.io/library/nginx:alpine (sha256:e20c21e530f914fb6a95a755924b1cbf71f039372e94ac5ddcf8c3b386a44615)...done
root@i-gqqsb62d:~# ctr i ls
REF                            TYPE                                                      DIGEST                                                                  SIZE    PLATFORMS     
                                                                           LABELS
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:e20c21e530f914fb6a95a755924b1cbf71f039372e94ac5ddcf8c3b386a44615 9.4 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -

其他

root@i-gqqsb62d:~# ctr i -h
NAME:
   ctr images - manage images
USAGE:
   ctr images command [command options] [arguments...]
COMMANDS:
   check       check that an image has all content available locally
   export      export images
   import      import images
   list, ls    list images known to containerd
   mount       mount an image to a target path
   unmount     unmount the image from the target
   pull        pull an image from a remote
   push        push an image to a remote
   remove, rm  remove one or more images by reference
   tag         tag an image
   label       set and clear labels for an image
OPTIONS:
   --help, -h  show help

容器

创建容器

root@i-gqqsb62d:~# ctr c create docker.io/library/nginx:alpine
ctr: container id must be provided: invalid argument
root@i-gqqsb62d:~# ctr c create docker.io/library/nginx:alpine  nginx
root@i-gqqsb62d:~# ctr c ls
CONTAINER    IMAGE                             RUNTIME
nginx        docker.io/library/nginx:alpine    io.containerd.runc.v2

查看容器详细配置

# ctr c info nginx
{
    "ID": "nginx",
    "Labels": {
        "io.containerd.image.config.stop-signal": "SIGQUIT"
    },
    "Image": "docker.io/library/nginx:alpine",
    "Runtime": {
        "Name": "io.containerd.runc.v2",
        "Options": {
            "type_url": "containerd.runc.v1.Options"
        }
    },
    "SnapshotKey": "nginx",
    "Snapshotter": "overlayfs",
    "CreatedAt": "2021-03-23T06:00:08.021700023Z",
    "UpdatedAt": "2021-03-23T06:00:08.021700023Z",
    "Extensions": null,
    "Spec": {
        "ociVersion": "1.0.2-dev",
        "process": {
            "user": {
                "uid": 0,
                "gid": 0,
                "additionalGids": [
                    1,
                    2,
                    3,
                    4,
                    6,
                    10,
                    11,
                    20,
                    26,
                    27
                ]
            },
            "args": [
                "/docker-entrypoint.sh",
                "nginx",
                "-g",
                "daemon off;"
            ],
            "env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NGINX_VERSION=1.19.8",
                "NJS_VERSION=0.5.2",
                "PKG_RELEASE=1"
            ],
            "cwd": "/",
            "capabilities": {
                "bounding": [
                    "CAP_CHOWN",
                    "CAP_DAC_OVERRIDE",
                    "CAP_FSETID",
                    "CAP_FOWNER",
                    "CAP_MKNOD",
                    "CAP_NET_RAW",
                    "CAP_SETGID",
                    "CAP_SETUID",
                    "CAP_SETFCAP",
                    "CAP_SETPCAP",
                    "CAP_NET_BIND_SERVICE",
                    "CAP_SYS_CHROOT",
                    "CAP_KILL",
                    "CAP_AUDIT_WRITE"
                ],
                "effective": [
                    "CAP_CHOWN",
                    "CAP_DAC_OVERRIDE",
                    "CAP_FSETID",
                    "CAP_FOWNER",
                    "CAP_MKNOD",
                    "CAP_NET_RAW",
                    "CAP_SETGID",
                    "CAP_SETUID",
                    "CAP_SETFCAP",
                    "CAP_SETPCAP",
                    "CAP_NET_BIND_SERVICE",
                    "CAP_SYS_CHROOT",
                    "CAP_KILL",
                    "CAP_AUDIT_WRITE"
                ],
                "inheritable": [
                    "CAP_CHOWN",
                    "CAP_DAC_OVERRIDE",
                    "CAP_FSETID",
                    "CAP_FOWNER",
                    "CAP_MKNOD",
                    "CAP_NET_RAW",
                    "CAP_SETGID",
                    "CAP_SETUID",
                    "CAP_SETFCAP",
                    "CAP_SETPCAP",
                    "CAP_NET_BIND_SERVICE",
                    "CAP_SYS_CHROOT",
                    "CAP_KILL",
                    "CAP_AUDIT_WRITE"
                ],
                "permitted": [
                    "CAP_CHOWN",
                    "CAP_DAC_OVERRIDE",
                    "CAP_FSETID",
                    "CAP_FOWNER",
                    "CAP_MKNOD",
                    "CAP_NET_RAW",
                    "CAP_SETGID",
                    "CAP_SETUID",
                    "CAP_SETFCAP",
                    "CAP_SETPCAP",
                    "CAP_NET_BIND_SERVICE",
                    "CAP_SYS_CHROOT",
                    "CAP_KILL",
                    "CAP_AUDIT_WRITE"
                ]
            },
            "rlimits": [
                {
                    "type": "RLIMIT_NOFILE",
                    "hard": 1024,
                    "soft": 1024
                }
            ],
            "noNewPrivileges": true
        },
        "root": {
            "path": "rootfs"
        },
        "mounts": [
            {
                "destination": "/proc",
                "type": "proc",
                "source": "proc",
                "options": [
                    "nosuid",
                    "noexec",
                    "nodev"
                ]
            },
            {
                "destination": "/dev",
                "type": "tmpfs",
                "source": "tmpfs",
                "options": [
                    "nosuid",
                    "strictatime",
                    "mode=755",
                    "size=65536k"
                ]
            },
            {
                "destination": "/dev/pts",
                "type": "devpts",
                "source": "devpts",
                "options": [
                    "nosuid",
                    "noexec",
                    "newinstance",
                    "ptmxmode=0666",
                    "mode=0620",
                    "gid=5"
                ]
            },
            {
                "destination": "/dev/shm",
                "type": "tmpfs",
                "source": "shm",
                "options": [
                    "nosuid",
                    "noexec",
                    "nodev",
                    "mode=1777",
                    "size=65536k"
                ]
            },
            {
                "destination": "/dev/mqueue",
                "type": "mqueue",
                "source": "mqueue",
                "options": [
                    "nosuid",
                    "noexec",
                    "nodev"
                ]
            },
            {
                "destination": "/sys",
                "type": "sysfs",
                "source": "sysfs",
                "options": [
                    "nosuid",
                    "noexec",
                    "nodev",
                    "ro"
                ]
            },
            {
                "destination": "/run",
                "type": "tmpfs",
                "source": "tmpfs",
                "options": [
                    "nosuid",
                    "strictatime",
                    "mode=755",
                    "size=65536k"
                ]
            }
        ],
        "linux": {
            "resources": {
                "devices": [
                    {
                        "allow": false,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 1,
                        "minor": 3,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 1,
                        "minor": 8,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 1,
                        "minor": 7,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 5,
                        "minor": 0,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 1,
                        "minor": 5,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 1,
                        "minor": 9,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 5,
                        "minor": 1,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 136,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 5,
                        "minor": 2,
                        "access": "rwm"
                    },
                    {
                        "allow": true,
                        "type": "c",
                        "major": 10,
                        "minor": 200,
                        "access": "rwm"
                    }
                ]
            },
            "cgroupsPath": "/default/nginx",
            "namespaces": [
                {
                    "type": "pid"
                },
                {
                    "type": "ipc"
                },
                {
                    "type": "uts"
                },
                {
                    "type": "mount"
                },
                {
                    "type": "network"
                }
            ],
            "maskedPaths": [
                "/proc/acpi",
                "/proc/asound",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/sys/firmware",
                "/proc/scsi"
            ],
            "readonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        }
    }
}

other

root@i-gqqsb62d:~#  ctr c -h
NAME:
   ctr containers - manage containers
USAGE:
   ctr containers command [command options] [arguments...]
COMMANDS:
   create           create container
   delete, del, rm  delete one or more existing containers
   info             get info about a container
   list, ls         list containers
   label            set and clear labels for a container
   checkpoint       checkpoint a container
   restore          restore a container from checkpoint
OPTIONS:
   --help, -h  show help

任务

上面 create 的命令创建了容器后，并没有处于运行状态，只是一个静态的容器。所以还需要启动容器，task 代表任务的意思：

root@i-gqqsb62d:~# ctr task ls
TASK    PID    STATUS    
root@i-gqqsb62d:~# ctr task start -d nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
root@i-gqqsb62d:~# ctr task ls
TASK     PID     STATUS    
nginx    7931    RUNNING

当然，也可以一步到位直接创建并运行容器：

root@i-gqqsb62d:~#  ctr run -d docker.io/library/nginx:alpine nginx1
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
root@i-gqqsb62d:~# ctr task ls
TASK      PID     STATUS    
nginx1    8075    RUNNING
nginx     7931    RUNNING

进入容器

和 docker 的操作类似，但必须要指定 —exec-id，这个 id 可以随便写，只要唯一就行

root@i-gqqsb62d:~# ctr task exec --exec-id 0 -t nginx sh
/ # 
/ # ls
bin                   media                 srv
dev                   mnt                   sys
docker-entrypoint.d   opt                   tmp
docker-entrypoint.sh  proc                  usr
etc                   root                  var
home                  run
lib                   sbin

暂停容器

root@i-gqqsb62d:~# ctr task ls
TASK      PID     STATUS    
nginx     7931    RUNNING
nginx1    8075    RUNNING
root@i-gqqsb62d:~# ctr task pause nginx
root@i-gqqsb62d:~# ctr task ls
TASK      PID     STATUS    
nginx     7931    PAUSED
nginx1    8075    RUNNING

恢复

root@i-gqqsb62d:~# ctr task resume nginx
root@i-gqqsb62d:~# ctr task ls
TASK      PID     STATUS    
nginx     7931    RUNNING
nginx1    8075    RUNNING

杀死容器

root@i-gqqsb62d:~# ctr task kill nginx
root@i-gqqsb62d:~# ctr task ls
TASK      PID     STATUS    
nginx     7931    STOPPED
nginx1    8075    RUNNING

获取cgroup信息

root@i-gqqsb62d:~# ctr task metrics nginx
ID       TIMESTAMP
nginx    2021-03-23 06:08:00.601588973 +0000 UTC
METRIC                   VALUE
memory.usage_in_bytes    716800
memory.limit_in_bytes    9223372036854771712
memory.stat.cache        61440
cpuacct.usage            61135700
cpuacct.usage_percpu     [61135700]
pids.current             0
pids.limit               0
root@i-gqqsb62d:~# ctr task metrics nginx1
ID        TIMESTAMP
nginx1    2021-03-23 06:08:07.026595123 +0000 UTC    
METRIC                   VALUE
memory.usage_in_bytes    2162688
memory.limit_in_bytes    9223372036854771712
memory.stat.cache        57344
cpuacct.usage            46920481
cpuacct.usage_percpu     [46920481]
pids.current             2
pids.limit               0
root@i-gqqsb62d:~#

查看容器中所有进程的 PID：

root@i-gqqsb62d:~# ctr task ps nginx
PID    INFO
root@i-gqqsb62d:~# ctr task ps nginx1
PID     INFO
8075    -
8112    -

注意：这里的 PID 是宿主机看到的 PID，不是容器中看到的 PID。

命名空间

除了 k8s 有命名空间以外，Containerd 也支持命名空间。

root@i-gqqsb62d:~# ctr ns ls
NAME    LABELS 
default