是一种为Linux、FreeBSD和NetBSD操作系统设计的,把其他文件系统联合到一个联合挂载点的文件系统,它是以branch把不同文件系统的和目录透明地覆盖,形成单一一致的文件系统,这些branch可能是只读或读写的。
unionfs使用了copy-on-write机制来保证当创建、修改、删除原有的文件时,并没有真正地变更原有文件,而是写入到一个新的文件。

AUFS

Advanced Multi-Layered Unification Filesystem。
AUFS完全重写了早期的UnionFS 1.x,并引入了新功能,是Docker选用的第一种存储驱动。

Overview

image.png

Docker是如何使用AUFS的(以centos镜像为例)

/var/lib/docker
├── aufs
├── builder
├── buildkit
├── containers
├── image
├── network
├── overlay
├── plugins
├── runtimes
├── swarm
├── tmp
├── trust
└── volumes

先看一下Docker是如何存储镜像的:

【存储驱动无关】配置基目录:/var/lib/docker/image/

/var/lib/docker/image/aufs
├── distribution
│ ├── diffid-by-digest
│ │ └── sha256
│ │ └── a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17
│ └── v2metadata-by-diffid
│ └── sha256
│ └── 071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956
├── imagedb
│ ├── content
│ │ └── sha256
│ │ └── 1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb
│ └── metadata
│ └── sha256
├── layerdb
│ ├── sha256
│ │ └── 071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956
│ │ ├── cache-id
│ │ ├── diff
│ │ ├── size
│ │ └── tar-split.json.gz
│ └── tmp
└── repositories.json

可以看出有imagedb和layerdb两个目录,前者存储了每个镜像的json描述,后者存储每层的信息。

./repositories.json

存储了镜像name与id之间的映射关系。

  1. {
  2. "Repositories": {
  3. "centos": {
  4. "centos:latest": "sha256:1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb",
  5. "centos@sha256:184e5f35598e333bfa7de10d8fb1cebb5ee4df5bc0f970bf2b1e7c7345136426": "sha256:1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb"
  6. }
  7. }
  8. }

./imagedb/content/sha256/$image_id(文件)

查看镜像名为centos,image_id为1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb的内容:

  1. {
  2. "architecture": "amd64",
  3. "config": {
  4. "Hostname": "",
  5. "Domainname": "",
  6. "User": "",
  7. "AttachStdin": false,
  8. "AttachStdout": false,
  9. "AttachStderr": false,
  10. "Tty": false,
  11. "OpenStdin": false,
  12. "StdinOnce": false,
  13. "Env": [
  14. "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  15. ],
  16. "Cmd": [
  17. "/bin/bash"
  18. ],
  19. "ArgsEscaped": true,
  20. "Image": "sha256:b3a68d99a4a4195c6c97c2345b83cb2d6cfd1661247816ac403cf0b584437ad7",
  21. "Volumes": null,
  22. "WorkingDir": "",
  23. "Entrypoint": null,
  24. "OnBuild": null,
  25. "Labels": {
  26. "org.label-schema.build-date": "20181205",
  27. "org.label-schema.license": "GPLv2",
  28. "org.label-schema.name": "CentOS Base Image",
  29. "org.label-schema.schema-version": "1.0",
  30. "org.label-schema.vendor": "CentOS"
  31. }
  32. },
  33. "container": "1fdbb0fcc184eb795364f7aa5fdc00299d0a2b90d8e26b4696217c22da7f983f",
  34. "container_config": {
  35. "Hostname": "1fdbb0fcc184",
  36. "Domainname": "",
  37. "User": "",
  38. "AttachStdin": false,
  39. "AttachStdout": false,
  40. "AttachStderr": false,
  41. "Tty": false,
  42. "OpenStdin": false,
  43. "StdinOnce": false,
  44. "Env": [
  45. "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  46. ],
  47. "Cmd": [
  48. "/bin/sh",
  49. "-c",
  50. "#(nop) ",
  51. "CMD [\"/bin/bash\"]"
  52. ],
  53. "ArgsEscaped": true,
  54. "Image": "sha256:b3a68d99a4a4195c6c97c2345b83cb2d6cfd1661247816ac403cf0b584437ad7",
  55. "Volumes": null,
  56. "WorkingDir": "",
  57. "Entrypoint": null,
  58. "OnBuild": null,
  59. "Labels": {
  60. "org.label-schema.build-date": "20181205",
  61. "org.label-schema.license": "GPLv2",
  62. "org.label-schema.name": "CentOS Base Image",
  63. "org.label-schema.schema-version": "1.0",
  64. "org.label-schema.vendor": "CentOS"
  65. }
  66. },
  67. "created": "2018-12-06T00:21:07.135655444Z",
  68. "docker_version": "17.06.2-ce",
  69. "history": [
  70. {
  71. "created": "2018-12-06T00:21:06.488375516Z",
  72. "created_by": "/bin/sh -c #(nop) ADD file:6f877549795f4798a38b318c0f63f6646dbf10d3c249c7f4b73cc7cfe42dc0f5 in / "
  73. },
  74. {
  75. "created": "2018-12-06T00:21:06.910715944Z",
  76. "created_by": "/bin/sh -c #(nop) LABEL org.label-schema.schema-version=1.0 org.label-schema.name=CentOS Base Image org.label-schema.vendor=CentOS org.label-schema.license=GPLv2 org.label-schema.build-date=20181205",
  77. "empty_layer": true
  78. },
  79. {
  80. "created": "2018-12-06T00:21:07.135655444Z",
  81. "created_by": "/bin/sh -c #(nop) CMD [\"/bin/bash\"]",
  82. "empty_layer": true
  83. }
  84. ],
  85. "os": "linux",
  86. "rootfs": {
  87. "type": "layers",
  88. "diff_ids": [
  89. "sha256:071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956"
  90. ]
  91. }
  92. }

镜像与layer之间是通过diff_ids关联的,注意观察镜像json文件中的rootfs.diff_ids字段。
“diff_ids”: [“sha256:071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956”]

./distribution/v2metadata-by-diffid/sha256/$diff_id(文件)

layer有一个diff_id和一个digest,image中存储的是diff_id,要找到layer的digest需要到distribution目录下找。

/var/lib/docker/image/aufs/distribution/
├── diffid-by-digest
│ └── sha256
│ └── a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17
└── v2metadata-by-diffid
└── sha256
└── 071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956
4 directories, 2 files

diff_id为071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956的layer的digest文件内容:

  1. {
  2. "Digest": "sha256:a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17",
  3. "SourceRepository": "docker.io/library/centos",
  4. "HMAC": ""
  5. }

./layerdb/sha256/$chain_id(目录)

计算chainid时,用到了所有祖先layer的信息,从而能保证根据chainid得到的rootfs是唯一的。比如我在debian和ubuntu的image基础上都添加了一个同样的文件,那么commit之后新增加的这两个layer具有相同的内容,相同的diffid,但由于他们的父layer不一样,所以他们的chainid会不一样,从而根据chainid能找到唯一的rootfs。计算chainid的方法请参考image spec

  1. ChainID(L₀) = DiffID(L₀)
  2. ChainID(L₀|...|Lₙ₋₁|Lₙ) = Digest(ChainID(L₀|...|Lₙ₋₁) + " " + DiffID(Lₙ))

最上层(对外暴露)的layer的chain_id和diff_id相同。

/var/lib/docker/image/aufs/layerdb
├── sha256
│ └── 071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956
│ ├── cache-id
│ ├── diff
│ ├── size
│ └── tar-split.json.gz
└── tmp
3 directories, 4 files

可以看出每一个layer由几部分组成:

  • cache-id:cache-id是docker下载layer的时候在本地生成的一个随机uuid,指向真正存放layer文件的地方
  • diff:diff_id
  • size:layer大小
  • tar-split.json.gz:layer压缩包的split文件,通过这个文件可以还原layer的tar包,在docker save导出image的时候会用到,详情可参考https://github.com/vbatts/tar-split

查到的cache-id为b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb

【存储驱动相关->AUFS】数据基目录:/var/lib/docker/

/var/lib/docker/aufs/
├── diff
│ └── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
├── layers
│ └── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
└── mnt
└── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
发现这个layer_cache_id在三个目录中都有一份,分别存放diff、layers和mnt信息。

每一个Docker image都是由一系列read-only layer组成的。image layer的内容都存储在/var/lib/docker/aufs/diff目录下,而/var/lib/docker/aufs/layers目录,则存储着image layer如何堆栈这些layer的metadata。

./$layer_cache_id(目录)

以centos的layer_cache_id b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb为例:

diff(目录,rootfs)

/var/lib/docker/aufs/diff/b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb/
├── anaconda-post.log
├── bin -> usr/bin
├── dev
├── etc
├── home
├── lib -> usr/lib
├── lib64 -> usr/lib64
├── media
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/sbin
├── srv
├── sys
├── tmp
├── usr
└── var

layers(文件)

空文件

mnt(目录)

空目录

对centos添加一个Layer

只是加一个新的文件,文件内容为Hello World。

  1. [root@localhost ~]# vi Dockerfile
  2. [root@localhost ~]# cat Dockerfile
  3. FROM centos
  4. RUN echo "Hello World" > /tmp/newfile
  5. [root@localhost ~]# docker build -t changed-centos .
  6. Sending build context to Docker daemon 152.8MB
  7. Step 1/2 : FROM centos
  8. ---> 1e1148e4cc2c
  9. Step 2/2 : RUN echo "Hello World" > /tmp/newfile
  10. ---> Running in 6979002d42a9
  11. Removing intermediate container 6979002d42a9
  12. ---> b549ea076ea9
  13. Successfully built b549ea076ea9
  14. Successfully tagged changed-centos:latest
  15. [root@localhost ~]# docker images
  16. REPOSITORY TAG IMAGE ID CREATED SIZE
  17. changed-centos latest b549ea076ea9 8 seconds ago 202MB
  18. centos latest 1e1148e4cc2c 2 months ago 202MB
  19. [root@localhost ~]# docker history changed-centos
  20. IMAGE CREATED CREATED BY SIZE COMMENT
  21. b549ea076ea9 23 seconds ago /bin/sh -c echo "Hello World" > /tmp/newfile 12B
  22. 1e1148e4cc2c 2 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
  23. <missing> 2 months ago /bin/sh -c #(nop) LABEL org.label-schema.sc… 0B
  24. <missing> 2 months ago /bin/sh -c #(nop) ADD file:6f877549795f4798a… 202MB

先查看一下镜像的描述信息:

  1. {
  2. "architecture": "amd64",
  3. "config": {
  4. "Hostname": "",
  5. "Domainname": "",
  6. "User": "",
  7. "AttachStdin": false,
  8. "AttachStdout": false,
  9. "AttachStderr": false,
  10. "Tty": false,
  11. "OpenStdin": false,
  12. "StdinOnce": false,
  13. "Env": [
  14. "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  15. ],
  16. "Cmd": [
  17. "/bin/bash"
  18. ],
  19. "ArgsEscaped": true,
  20. "Image": "sha256:1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb",
  21. "Volumes": null,
  22. "WorkingDir": "",
  23. "Entrypoint": null,
  24. "OnBuild": null,
  25. "Labels": {
  26. "org.label-schema.build-date": "20181205",
  27. "org.label-schema.license": "GPLv2",
  28. "org.label-schema.name": "CentOS Base Image",
  29. "org.label-schema.schema-version": "1.0",
  30. "org.label-schema.vendor": "CentOS"
  31. }
  32. },
  33. "container": "6979002d42a94debbc1bee77216e1cad3b33563b9ae91638b9a04c718504bfdb",
  34. "container_config": {
  35. "Hostname": "",
  36. "Domainname": "",
  37. "User": "",
  38. "AttachStdin": false,
  39. "AttachStdout": false,
  40. "AttachStderr": false,
  41. "Tty": false,
  42. "OpenStdin": false,
  43. "StdinOnce": false,
  44. "Env": [
  45. "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  46. ],
  47. "Cmd": [
  48. "/bin/sh",
  49. "-c",
  50. "echo \"Hello World\" > /tmp/newfile"
  51. ],
  52. "ArgsEscaped": true,
  53. "Image": "sha256:1e1148e4cc2c148c6890a18e3b2d2dde41a6745ceb4e5fe94a923d811bf82ddb",
  54. "Volumes": null,
  55. "WorkingDir": "",
  56. "Entrypoint": null,
  57. "OnBuild": null,
  58. "Labels": {
  59. "org.label-schema.build-date": "20181205",
  60. "org.label-schema.license": "GPLv2",
  61. "org.label-schema.name": "CentOS Base Image",
  62. "org.label-schema.schema-version": "1.0",
  63. "org.label-schema.vendor": "CentOS"
  64. }
  65. },
  66. "created": "2019-02-15T08:42:01.794578059Z",
  67. "docker_version": "18.09.2",
  68. "history": [
  69. {
  70. "created": "2018-12-06T00:21:06.488375516Z",
  71. "created_by": "/bin/sh -c #(nop) ADD file:6f877549795f4798a38b318c0f63f6646dbf10d3c249c7f4b73cc7cfe42dc0f5 in / "
  72. },
  73. {
  74. "created": "2018-12-06T00:21:06.910715944Z",
  75. "created_by": "/bin/sh -c #(nop) LABEL org.label-schema.schema-version=1.0 org.label-schema.name=CentOS Base Image org.label-schema.vendor=CentOS org.label-schema.license=GPLv2 org.label-schema.build-date=20181205",
  76. "empty_layer": true
  77. },
  78. {
  79. "created": "2018-12-06T00:21:07.135655444Z",
  80. "created_by": "/bin/sh -c #(nop) CMD [\"/bin/bash\"]",
  81. "empty_layer": true
  82. },
  83. {
  84. "created": "2019-02-15T08:42:01.794578059Z",
  85. "created_by": "/bin/sh -c echo \"Hello World\" > /tmp/newfile"
  86. }
  87. ],
  88. "os": "linux",
  89. "rootfs": {
  90. "type": "layers",
  91. "diff_ids": [
  92. "sha256:071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956",
  93. "sha256:7124c7620988bfb6353327c13c4221608a0d869c444ec653b977ef0e630df004"
  94. ]
  95. }
  96. }

此时diff_ids变为了两个:

  • 07…是centos镜像的底层layer的diff_id
  • 71…是刚才新添加文件的layer的diff_id

然后根据diff_id去查cache-id。

/var/lib/docker/image/aufs/layerdb/sha256/
├── 071d8bd765171080d01682844524be57ac9883e53079b6ac66707e192ea25956
│ ├── cache-id b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
│ ├── diff
│ ├── size
│ └── tar-split.json.gz
└── 1c22d04a9a7faa9ca3e6da3721b4d8fb5c9f50d804f7b36418f9e0c0dc20f75f
├── cache-id 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
├── diff
├── parent
├── size
└── tar-split.json.gz
2 directories, 9 files

注意07…是centos镜像layer的chain_id,而1c…是刚才新添加文件的layer的chain_id(只有最底层/基础的layer,chain_id = diff_id,上层的并不相同)。

此时数据基目录是这样的:
/var/lib/docker/aufs
├── diff
│ ├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
│ └── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
├── layers
│ ├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
│ └── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
└── mnt
├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
└── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
7 directories, 2 files

b2…是centos镜像的layer,3e…是我们添加的文件。

diff

/var/lib/docker/aufs/diff/3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26/
└── tmp
└── newfile
1 directory, 1 file

  1. [root@localhost ~]# cat /var/lib/docker/aufs/diff/3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26/tmp/newfile
  2. Hello World

layers(下层,所依赖的layer的cache_id)

  1. [root@localhost ~]# cat /var/lib/docker/aufs/layers/3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
  2. b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb

mnt

空目录

结论

从上述示例中可见,当我们为镜像添加了一个文件时,会加一个新的layer,这个新的layer中只会存储着一个新的文件;如果是删除一个文件,那么会在新的layer中加一个标记文件(.wh.deleted_filename),标记该文件已被删除;如果是修改一个文件,则会将修改后的文件放到新的layer中,合并layer时以上层layer的文件为准(当然大文件只改一小部分也会导致整个文件拷贝)。

动态

在启动一个容器时,Docker会为其创建一个只读的init-layer,用来存储于整个容器内环境相关的内容;还会为其创建一个读写的layer来执行所有写操作。
容器的读写layer会存储在/var/lib/docker/aufs/diff/目录下,即使容器停止,这个layer也是存在的,因此重启容器不会丢失数据。只有当删掉容器的时候,才会删除这个读写layer。
image.png

启动changed-centos容器,返回的容器ID为bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8。

1) 会在/var/lib/docker/containers/$container_id下创建目录

/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8
├── bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8-json.log
├── checkpoints
├── config.v2.json
├── hostconfig.json
├── hostname
├── hosts
├── mounts
│ └── shm
├── resolv.conf
└── resolv.conf.hash

config.v2.json

  1. {
  2. "StreamConfig": {},
  3. "State": {
  4. "Running": false,
  5. "Paused": false,
  6. "Restarting": false,
  7. "OOMKilled": false,
  8. "RemovalInProgress": false,
  9. "Dead": false,
  10. "Pid": 0,
  11. "ExitCode": 2,
  12. "Error": "",
  13. "StartedAt": "2019-02-15T09:14:35.663470204Z",
  14. "FinishedAt": "2019-02-15T09:14:35.632614616Z",
  15. "Health": null
  16. },
  17. "ID": "bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8",
  18. "Created": "2019-02-15T09:14:35.427819187Z",
  19. "Managed": false,
  20. "Path": "bash",
  21. "Args": [
  22. "--name=changed-centos"
  23. ],
  24. "Config": {
  25. "Hostname": "bf1b6932346b",
  26. "Domainname": "",
  27. "User": "",
  28. "AttachStdin": false,
  29. "AttachStdout": false,
  30. "AttachStderr": false,
  31. "Tty": true,
  32. "OpenStdin": true,
  33. "StdinOnce": false,
  34. "Env": [
  35. "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  36. ],
  37. "Cmd": [
  38. "bash",
  39. "--name=changed-centos"
  40. ],
  41. "Image": "changed-centos",
  42. "Volumes": null,
  43. "WorkingDir": "",
  44. "Entrypoint": null,
  45. "OnBuild": null,
  46. "Labels": {
  47. "org.label-schema.build-date": "20181205",
  48. "org.label-schema.license": "GPLv2",
  49. "org.label-schema.name": "CentOS Base Image",
  50. "org.label-schema.schema-version": "1.0",
  51. "org.label-schema.vendor": "CentOS"
  52. }
  53. },
  54. "Image": "sha256:b549ea076ea925d4134586b5075c19033c6516dea68211eac8e0bcee050ff0bd",
  55. "NetworkSettings": {
  56. "Bridge": "",
  57. "SandboxID": "35c624057f9d6ac795532dc14f061074313b268f09912a637d5b0e548c2807ab",
  58. "HairpinMode": false,
  59. "LinkLocalIPv6Address": "",
  60. "LinkLocalIPv6PrefixLen": 0,
  61. "Networks": {
  62. "bridge": {
  63. "IPAMConfig": null,
  64. "Links": null,
  65. "Aliases": null,
  66. "NetworkID": "3c967ec4468ae5379835d2140449395c45f0aa9e6b2217f40634611f0b638683",
  67. "EndpointID": "",
  68. "Gateway": "",
  69. "IPAddress": "",
  70. "IPPrefixLen": 0,
  71. "IPv6Gateway": "",
  72. "GlobalIPv6Address": "",
  73. "GlobalIPv6PrefixLen": 0,
  74. "MacAddress": "",
  75. "DriverOpts": null,
  76. "IPAMOperational": false
  77. }
  78. },
  79. "Service": null,
  80. "Ports": null,
  81. "SandboxKey": "/var/run/docker/netns/35c624057f9d",
  82. "SecondaryIPAddresses": null,
  83. "SecondaryIPv6Addresses": null,
  84. "IsAnonymousEndpoint": true,
  85. "HasSwarmEndpoint": false
  86. },
  87. "LogPath": "/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8-json.log",
  88. "Name": "/vibrant_villani",
  89. "Driver": "aufs",
  90. "OS": "linux",
  91. "MountLabel": "",
  92. "ProcessLabel": "",
  93. "RestartCount": 0,
  94. "HasBeenStartedBefore": true,
  95. "HasBeenManuallyStopped": false,
  96. "MountPoints": {},
  97. "SecretReferences": null,
  98. "ConfigReferences": null,
  99. "AppArmorProfile": "",
  100. "HostnamePath": "/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/hostname",
  101. "HostsPath": "/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/hosts",
  102. "ShmPath": "/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/mounts/shm",
  103. "ResolvConfPath": "/var/lib/docker/containers/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/resolv.conf",
  104. "SeccompProfile": "",
  105. "NoNewPrivileges": false
  106. }

可以看到image字段是存储了镜像的ID。

2) 会在/var/lib/docker/image/aufs/layerdb/mounts/$container_id下创建目录

/var/lib/docker/image/aufs/layerdb/mounts/bf1b6932346bfd4793f4ebea51ca7a0ce5d460620b07bea3b3a6e1beb9e502b8/
├── init-id d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
├── mount-id d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910
└── parent 1c22d04a9a7faa9ca3e6da3721b4d8fb5c9f50d804f7b36418f9e0c0dc20f75f
0 directories, 3 files

在这里存放了容器与只读的layer(init-id)和读写的layer(mount-id)的id的映射关系。
parent存放了最上层layer(我们刚才添加了一个新文件的layer)的diff_id。

3) 查看只读layer与读写layer

在diff、layers、mnt下都多了两个layer。
/var/lib/docker/aufs/
├── diff
│ ├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
│ ├── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
│ ├── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910
│ └── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
├── layers
│ ├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
│ ├── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
│ ├── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910
│ └── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
└── mnt
├── 3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
├── b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb
├── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910
└── d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
11 directories, 4 files

diff

read_only_layer(带init的)

/var/lib/docker/aufs/diff/d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
├── dev
│ ├── console
│ ├── pts
│ └── shm
└── etc
├── hostname
├── hosts
├── mtab -> /proc/mounts
└── resolv.conf
4 directories, 5 files

read_write_layer

/var/lib/docker/aufs/diff/d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910
空目录

layers

read_only_layer(带init的)

d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910

文件内容(下层,依赖的layer):
3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb

read_write_layer(不带init的)

d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
文件内容(下层,依赖的layer):
d0ff844478a02c0f054411
5e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init
3e5dad646755b80ceeb2776db26d93cd8b2a45b66c670173eb0a5def51b27a26
b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb

4) 权限

/sys/fs/aufs/si_

当启动一个容器时,会在/sys/fs/aufs下创建一个目录
/sys/fs/aufs/si_c2d3541b6775f5e4/
├── br0
├── br1
├── br2
├── brid0
├── brid1
├── brid2
└── xi_path

br0就是branch0,以此类推
[root@localhost ~]# cat /sys/fs/aufs/si_c2d3541b6775f5e4/br0
/var/lib/docker/aufs/diff/d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910=rw
[root@localhost ~]# cat /sys/fs/aufs/si_c2d3541b6775f5e4/br1
/var/lib/docker/aufs/diff/d0ff844478a02c0f0544115e77626cf4f0c4a7996a6a6764eea2a2dd29201910-init=ro+wh
[root@localhost ~]# cat /sys/fs/aufs/si_c2d3541b6775f5e4/br2
/var/lib/docker/aufs/diff/b2fe7fefbb164b8a8adb84273b94eeccc6794c231fc3958bc98a35cc72225cbb=ro+wh

可以看出diff下面的layer目录的权限,比如rw,ro+wh
一般来说ro的分支都会有wh的属性,比如 “[dir]=ro+wh”。所谓whiteout的意思,如果在union中删除的某个文件,实际上是位于一个readonly的分支(目录)上,那么,在mount的union这个目录中你将看不到这个文件,但是read-only这个层上我们无法做任何的修改,所以,我们就需要对这个readonly目录里的文件作whiteout。AUFS的whiteout的实现是通过在上层的可写的目录下建立对应的whiteout隐藏文件来实现的。

实践

在一个目录下创建五个layer,一个叫containerlayer,其他叫image_layer{n},并且每个layer下都放着同名文件,文件内容不同。然后将五个layer以aufs方式挂载到一个目录中(container_layer为rw,其他为ro),此时这个目录就是5个layer合并后的视图。
然后可以去修改视图,然后查看反映到layer的影响。

  1. [root@localhost ~]# mkdir aufs
  2. [root@localhost ~]# mkdir aufs_view
  3. [root@localhost ~]# cd aufs
  4. [root@localhost aufs]# mkdir container_layer
  5. [root@localhost aufs]# echo "I am container layer" > ./container_layer/container_layer"
  6. > ^C
  7. [root@localhost aufs]# echo "I am container layer" > ./container_layer/container_layer
  8. [root@localhost aufs]# mkdir image_layer_1
  9. [root@localhost aufs]# echo "I am image layer 1" > ./image_layer_1/image_layer_1
  10. [root@localhost aufs]# mkdir image_layer_2
  11. [root@localhost aufs]# echo "I am image layer 2" > ./image_layer_2/image_layer_2
  12. [root@localhost aufs]# mkdir image_layer_3
  13. [root@localhost aufs]# echo "I am image layer 3" > ./image_layer_3/image_layer_3
  14. [root@localhost aufs]# mkdir image_layer_4
  15. [root@localhost aufs]# echo "I am image layer 4" > ./image_layer_3/image_layer_4

查看layer

/root/aufs
├── container_layer
│ ├── container_layer
│ └── image_layer_4
├── image_layer_1
│ └── image_layer_1
├── image_layer_2
│ └── image_layer_2
├── image_layer_3
│ └── image_layer_3
└── image_layer_4
└── image_layer_4
5 directories, 6 files

mount

[root@localhost aufs]# mount -t aufs -o dirs=./container_layer:./image_layer_4:./image_layer_3:./image_layer_2:./image_layer_1 none ../aufs_view
mount: 文件系统类型错误、选项错误、aufs 上有坏超级块、
缺少代码页或助手程序,或其他错误
有些情况下在 syslog 中可以找到一些有用信息- 请尝试
dmesg | tail 这样的命令看看。
[root@localhost aufs]# dmesg | tail
[17270.804810] aufs au_xino_create:212:mount[13603]: xino doesn’t support /var/tmp/.xino(xfs)

[root@localhost aufs]# mount -t aufs -o dirs=./container_layer:./image_layer_4:./image_layer_3:./image_layer_2:./image_layer_1,xino=/dev/shm/aufs.xino aufs ../aufs_view

查看mount结果

/root/aufs_view/
├── container_layer
├── image_layer_1
├── image_layer_2
├── image_layer_3
└── image_layer_4
0 directories, 5 files

[root@localhost aufs]# cat /sys/fs/aufs/si_c2d3541b1a03cde4/*
/root/aufs/container_layer=rw
/root/aufs/image_layer_4=ro
/root/aufs/image_layer_3=ro
/root/aufs/image_layer_2=ro
/root/aufs/image_layer_1=ro
64
65
66
67
68
/dev/shm/aufs.xino

修改视图

echo -e "\nwrite to mnt's image-layer" >> /root/aufs_view/image_layer_4
此时查看/root/aufs/image_layer_4/image_layer_4,内容无变化
但是看/root/aufs/container_layer下多了一个image_layer_4。

  1. [root@localhost aufs]# cat ../aufs_view/image_layer_4
  2. I am image layer 4
  3. write to mnt's image-layer
  4. [root@localhost aufs]# ls container_layer/
  5. container_layer image_layer_4
  6. [root@localhost aufs]# cat container_layer/image_layer_4
  7. I am image layer 4
  8. write to mnt's image-layer