Swarm 三节点集群搭建

Swarm 三节点集群搭建

创建3节点swarm cluster的方法

https://labs.play-with-docker.com/ play with docker 网站，优点是快速方便，缺点是环境不持久，4个小时后环境会被重置
在本地通过虚拟化软件搭建Linux虚拟机，优点是稳定，方便，缺点是占用系统资源，需要电脑内存最好8G及其以上
在云上使用云主机，亚马逊，Google，微软Azure，阿里云，腾讯云等，缺点是需要消耗金钱（但是有些云服务，有免费试用）

多节点的环境涉及到机器之间的通信需求，所以防火墙和网络安全策略组是大家一定要考虑的问题，特别是在云上使用云主机的情况，下面这些端口记得打开 防火墙 以及 设置安全策略组

TCP port 2376
TCP port 2377
TCP and UDP port 7946
UDP port 4789

为了简化，以上所有端口都允许节点之间自由访问就行。

swarm集群创建

主节点初始化swarm

[root@master ~]# docker swarm init
Swarm initialized: current node (l557hn1xrjh807syfm330buj8) is now a manager.
To add a worker to this swarm, run the following command:
 docker swarm join --token SWMTKN-1-0n7svwv1rx7mw2vh72ftmhiszal5hub9grttmqcsjuptfpx7bz-8xaeyf6ubsg5l6bntx95q66ua 192.168.226.130:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

另外两个节点创建
```
docker swarm join --token SWMTKN-1-0n7svwv1rx7mw2vh72ftmhiszal5hub9grttmqcsjuptfpx7bz-8xaeyf6ubsg5l6bntx95q66ua 192.168.226.130:2377
```
使用Vagrant + Virtualbox搭建集群
下载安装 [VirtualBox](https://www.virtualbox.org/)
下载安装 [Vagarnt](https://www.vagrantup.com/)

Windows环境需要关闭Microsoft-Hyper-V，进入powershell执行以下命令 Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-All

Vagrant入门系列视频

本节Vagrant搭建的文件下载

[Centos 版 vagrant相关文件](https://dockertips.readthedocs.io/en/latest/_downloads/01803e0f19cfb47a524c08f4ed905771/vagrant-setup.zip)
[Ubuntu 版 vagrant相关文件](https://dockertips.readthedocs.io/en/latest/_downloads/b8bf8b42569184d1cae3e4a6987441e4/vagrant-setup-ubuntu.zip)
压缩包中的文件：

setup.sh

#/bin/sh
# install some tools
sudo yum install -y git vim gcc glibc-static telnet psmisc bridge-utils ipvsadm
# install docker
curl -fsSL get.docker.com -o get-docker.sh
sh get-docker.sh
if [ ! $(getent group docker) ]; then
 sudo groupadd docker
else
 echo "docker user group already exists"
fi
sudo gpasswd -a $USER docker
sudo systemctl start docker
sudo systemctl enable docker
rm -rf get-docker.sh
# open password auth for backup if ssh key doesn't work, bydefault, username=vagrant password=vagrant
sudo sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config
sudo systemctl restart sshd

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.require_version ">= 1.6.0"
boxes = [
 {
     :name => "swarm-manager",
     :eth1 => "192.168.200.10",
     :mem => "1024",
     :cpu => "1"
 },
 {
     :name => "swarm-worker1",
     :eth1 => "192.168.200.11",
     :mem => "1024",
     :cpu => "1"
 },
 {
     :name => "swarm-worker2",
     :eth1 => "192.168.200.12",
     :mem => "1024",
     :cpu => "1"
 }
]
Vagrant.configure(2) do |config|
config.vm.box = "centos/7"
boxes.each do |opts|
 config.vm.define opts[:name] do |config|
   config.vm.hostname = opts[:name]
   config.vm.provider "vmware_fusion" do |v|
     v.vmx["memsize"] = opts[:mem]
     v.vmx["numvcpus"] = opts[:cpu]
   end
   config.vm.provider "virtualbox" do |v|
     v.customize ["modifyvm", :id, "--memory", opts[:mem]]
     v.customize ["modifyvm", :id, "--cpus", opts[:cpu]]
   end
   config.vm.network :private_network, ip: opts[:eth1]
 end
end
config.vm.provision "shell", privileged: false, path: "./setup.sh"
end

下载完成后解压完成，然后启动virtualbox，之后在Vegrantfile所在目录执行vagrant up即可创建三个虚拟机，账号密码均为 vagrant

Vagrant的基本操作请参考上面的B站或者Youtube视频
虚拟机的启动：vagrant up
虚拟机的停止：vagrant halt
虚拟机的删除：vagrant destroy
启动完成后virtualbox中的界面
四、Docker-Swarm-三节点集群搭建 - 图1

ssh进入到每个节点，账号密码均为 vegrant，最好设置下docker开机启动systemctl enable docker，以免服务器重启后docker没启动。后续服务器的启停都可通过virtualbox进行控制。

创建集群swarm

在manager节点初始化swarm

[vagrant@swarm-manager ~]$ docker swarm init --advertise-addr=192.168.200.10
Swarm initialized: current node (j25pbulr2bm7esaee0uy6ewya) is now a manager.
To add a worker to this swarm, run the following command:
 docker swarm join --token SWMTKN-1-1rxgbocrvd3kn3mzsdvvt01c6e5l58ybg7w3qznn8i1hd20112-460bplsbpwhworso626ft2ebq 192.168.200.10:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
[vagrant@swarm-manager ~]$

添加其余两个节点到此集群 ``` [vagrant@swarm-worker1 ~]$ docker swarm join —token SWMTKN-1-1rxgbocrvd3kn3mzsdvvt01c6e5l58ybg7w3qznn8i1hd20112-460bplsbpwhworso626ft2ebq 192.168.200.10:2377 This node joined a swarm as a worker. [vagrant@swarm-worker1 ~]$

[vagrant@swarm-worker2 ~]$ docker swarm join —token SWMTKN-1-1rxgbocrvd3kn3mzsdvvt01c6e5l58ybg7w3qznn8i1hd20112-460bplsbpwhworso626ft2ebq 192.168.200.10:2377 This node joined a swarm as a worker. [vagrant@swarm-worker2 ~]$


3. 检查所有node

[vagrant@swarm-manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j25pbulr2bm7esaee0uy6ewya * swarm-manager Ready Active Leader 20.10.12 mzn9gxzakytpgnl9k7m42fj96 swarm-worker1 Ready Active 20.10.12 1otz7ws0zjo5tw6axc4t6as5a swarm-worker2 Ready Active 20.10.12

<a name="Jnaow"></a>
## 构建服务
1. 构建服务的一个replicas

[vagrant@swarm-manager ~]$ docker service create —name web nginx 3y0o0tfgeda5bybmrjqo0ifew overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS 3y0o0tfgeda5 web replicated 1/1 nginx:latest [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS c40apkgoku95 web.1 nginx:latest swarm-manager Running Running 52 seconds ago [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c0d86fd60604 nginx:latest “/docker-entrypoint.…” About a minute ago Up About a minute 80/tcp web.1.c40apkgoku95cb7ucs2fl7iu5


2. 构建三个replicas，发现三个分别不同的节点，且删除其中一个容器会马上恢复

[vagrant@swarm-manager ~]$ docker service update web —replicas=3 web overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS 3y0o0tfgeda5 web replicated 3/3 nginx:latest [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS c40apkgoku95 web.1 nginx:latest swarm-manager Running Running 3 minutes ago szec49wrmasz web.2 nginx:latest swarm-worker1 Running Running about a minute ago vwvu91oourun web.3 nginx:latest swarm-worker2 Running Running 57 seconds ago [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker container ls CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c0d86fd60604 nginx:latest “/docker-entrypoint.…” 3 minutes ago Up 3 minutes 80/tcp web.1.c40apkgoku95cb7ucs2fl7iu5 [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker container rm -f c0 c0 [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS s9d0imneryj3 web.1 nginx:latest swarm-manager Running Running 8 seconds ago c40apkgoku95 _ web.1 nginx:latest swarm-manager Shutdown Failed 13 seconds ago “task: non-zero exit (137)” szec49wrmasz web.2 nginx:latest swarm-worker1 Running Running about a minute ago vwvu91oourun web.3 nginx:latest swarm-worker2 Running Running about a minute ago

<a name="G5Pzl"></a>
## docker service 命令

[vagrant@swarm-manager ~]$ docker service —help Usage: docker service COMMAND Manage services Commands: create Create a new service inspect Display detailed information on one or more services logs Fetch the logs of a service or task ls List services ps List the tasks of one or more services rm Remove one or more services rollback Revert changes to a service’s configuration scale Scale one or multiple replicated services update Update a service Run ‘docker service COMMAND —help’ for more information on a command.


- scale：指定实例个数

[vagrant@swarm-manager ~]$ docker service scale web=4 web scaled to 4 overall progress: 4 out of 4 tasks 1/4: running [==================================================>] 2/4: running [==================================================>] 3/4: running [==================================================>] 4/4: running [==================================================>] verify: Service converged [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS 3y0o0tfgeda5 web replicated 4/4 nginx:latest [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS s9d0imneryj3 web.1 nginx:latest swarm-manager Running Running 4 minutes ago c40apkgoku95 _ web.1 nginx:latest swarm-manager Shutdown Failed 4 minutes ago “task: non-zero exit (137)” szec49wrmasz web.2 nginx:latest swarm-worker1 Running Running 6 minutes ago vwvu91oourun web.3 nginx:latest swarm-worker2 Running Running 6 minutes ago 115v70hxk9g3 web.4 nginx:latest swarm-manager Running Running 20 seconds ago [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service scale web=3 web scaled to 3 overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged [vagrant@swarm-manager ~]$ [vagrant@swarm-manager ~]$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS 3y0o0tfgeda5 web replicated 3/3 nginx:latest


- inspect：查看service细节
- logs：获取service的日志，加上`-f`参数可以实时打印日志

[vagrant@swarm-manager ~]$ docker service logs web web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh web.3.vwvu91oourun@swarm-worker2 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf web.3.vwvu91oourun@swarm-worker2 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh web.3.vwvu91oourun@swarm-worker2 | /docker-entrypoint.sh: Configuration complete; ready for start up web.3.vwvu91oourun@swarm-worker2 | 2021/12/29 13:08:18 [notice] 1#1: using the “epoll” event method web.3.vwvu91oourun@swarm-worker2 | 2021/12/29 13:08:18 [notice] 1#1: nginx/1.21.4 ```

四、Docker-Swarm-三节点集群搭建

Swarm 三节点集群搭建

swarm集群创建

使用Vagrant + Virtualbox搭建集群

创建集群swarm