title: keepalived配置详解 #标题tags: keepalived #标签
date: 2020-09-23
categories: linux大杂烩 # 分类

最近项目上用到了keepalived，之前搞过，但只是记了个随笔，今日抽出时间来，把此文档整理下。

环境准备

项目背景: 两个tomcat服务器，要使用keepalived配置VIP，能够实现tomcat服务异常时VIP自动漂到备机。

hostname IP service VIP
tomcat-01 192.168.20.10 tomcat、keepalived 192.168.20.15
tomcat-02 192.168.20.6 tomcat、keepalived 192.168.20.15

部署tomcat

两台机器都需要安装tomcat。

  1. # 安装java环境，为了方便，直接yum安装了
  2. $ yum -y install java
  4. # 安装tomcat
  5. $ wget https://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-9/v9.0.38/bin/apache-tomcat-9.0.38.tar.gz
  6. $ tar zxf apache-tomcat-9.0.38.tar.gz && mv apache-tomcat-9.0.38 /opt/tomcat-9.0.38
  8. # 准备首页文件（两个机器的首页内容自定义，测试时最好不要定义为一样的，后期好分辨）
  9. $ mkdir /web/webapp1/ -p
  10. $ cat > /web/webapp1/index.jsp << EOF
  11. <%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
  12. <html>
  13.  <head>
  14.         <title>JSP test1 page</title>
  15.  </head>
  16.  <body>
  17.         <% out.println("This is 192.168.20.10...");%>
  18.  </body>
  19. </html>
  20. EOF
  22. # <% out.println..... 后面写的就是首页内容
  23. $ vim /opt/tomcat-9.0.38/conf/server.xml       # 修改主配置文件
  24.                    ......................................
  25.  <Host name="localhost"  appBase="webapps"
  26.             unpackWARs="true" autoDeploy="true">     #定位到该行，然后添加下面两行内容
  27.         <Context docBase="/web/webapp1" path="" reloadable="false">
  28.         </Context>
  31. # 启动tomcat
  32. $ /opt/tomcat-9.0.38/bin/startup.sh
  33. # 确定端口已监听
  34. $ ss -lnput | grep 8080
  35. tcp    LISTEN     0      100      :::8080                 :::*                   users:(("java",pid=33918,fd=56))
  37. # 调整防火墙策略
  38. $ firewall-cmd --add-port=8080/tcp --permanent
  39. $ firewall-cmd --reload

访问测试:

keepalived配置详解 - 图1

部署keepalived

若部署最新版本，则会在make时出错，怀疑是需要升级内核，没有过多纠结，而是选择编译了2.0.20版本。

两台机器都需要安装keepalived。

  1. $ wget https://keepalived.org/software/keepalived-2.0.20.tar.gz
  2. $ yum install -y gcc openssl-devel openssl libnl libnl-devel libnfnetlink-devel
  3. $ tar zxf keepalived-2.0.20.tar.gz && cd keepalived-2.0.20
  4. $ ./configure --prefix=/opt/keepalived-2.0.20
  5. $ make && make install
  8. # 添加为系统服务并开机自启
  9. $ mkdir /etc/keepalived
  10. $ cp keepalived/etc/init.d/keepalived /etc/init.d/
  11. $ cp keepalived/etc/sysconfig/keepalived /etc/sysconfig/
  12. $ cp keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
  13. $ cd /etc/init.d/
  14. $ chkconfig --add keepalived
  15. $ systemctl enable keepalived

配置keepalived

配置文件中有一些无关紧要的配置，我也没耐心写，就这样吧，能过满足业务需求即可。

配置tomcat01主机

  1. # 先看配置含义，下面有完整无注释的配置
  2. $ cat /etc/keepalived/keepalived.conf
  3. global_defs {
  4.    script_user root
  5.    router_id TOMCAT-01    # route_id 必须唯一，可以写为主机名
  6. }
  8. # 下面是定义一个健康检测机制，可被多次调用
  9. vrrp_script chk_tomcat {
  10.         script "/etc/keepalived/chk_tomcat.sh"       # 指定命令或者脚本路径，脚本需要有执行权限
  11.         interval 2             # 执行间隔时间，默认为 1s
  12. }
  14. # 下面是定义一个VRRP实例，实例名为 VI_1
  15. vrrp_instance VI_1 {
  16.     state BACKUP         # 状态为backup
  17.     interface ens33    # 指定VIP绑定到哪块网卡
  18.     unicast_src_ip 192.168.20.10     # 指定为本机IP
  19.     unicast_peer {
  20.     192.168.20.6               # 指定对端IP
  21.                 }
  23.     virtual_router_id 23     # 虚拟路由ID，参与主备的机器路由ID都应该是一样的
  24.     priority 100             # 指定优先级
  25.     nopreempt               # 不抢占，即允许一个priority比较低的节点作为master
  26.     advert_int 1            # 发vrrp包的时间间隔，即多久进行一次master选举（可认为是健康检查时间间隔）
  27.     authentication {       # 认证区域，参与主备的机器，此字段应保持一致
  28.         auth_type PASS
  29.         auth_pass 1234
  30.     }
  31.     virtual_ipaddress {        # 指定VIP
  32.         192.168.20.15/24
  33.     }
  34. # 调用上面定义的健康检测模块    
  35.     track_script {
  36.         chk_tomcat
  37.     }
  38. }
  41. ################    完整无注释的配置文件如下    #################
  43. $ cat /etc/keepalived/keepalived.conf        # keepalived.conf 配置文件如下
  44. global_defs {
  45.    script_user root
  46.    router_id TOMCAT-01
  47. }
  50. vrrp_script chk_tomcat {
  51.         script "/etc/keepalived/chk_tomcat.sh"
  52.         interval 2
  53. }
  55. vrrp_instance VI_1 {
  56.     state BACKUP
  57.     interface ens33
  58.     unicast_src_ip 192.168.20.10
  59.     unicast_peer {
  60.     192.168.20.6             
  61.                 }
  63.     virtual_router_id 23
  64.     priority 100
  65.     nopreempt
  66.     advert_int 1
  67.     authentication {
  68.         auth_type PASS
  69.         auth_pass 1234
  70.     }
  71.     virtual_ipaddress {
  72.         192.168.20.15/24
  73.     }
  74.     track_script {
  75.         chk_tomcat
  76.     }
  77. }
  80. $ cat /etc/keepalived/chk_tomcat.sh             # 健康检测脚本如下
  81. #!/usr/bin/env bash
  82. keepalived_log=/etc/keepalived/vip.log
  83. tomcat_pid=$(ps -ef | grep tomcat-9.0.38  | grep -v grep  | wc -l)    # 确保此处过滤出来的是你的进程，并且尽可能精准匹配你的进程
  84. if [[ ${tomcat_pid} -eq 0 ]];then
  85.     cat >> ${keepalived_log} << EOF
  86.         Tomcat stopped running at $(date '+%F %T')
  87.         Stopping keepalived ...
  88. EOF
  89.     systemctl stop keepalived
  90. fi
  92. $ chmod +x /etc/keepalived/chk_tomcat.sh      # 脚本需要有执行权限
  95. # 将配置文件同步到tomcat-02
  96. $ rsync -az /etc/keepalived/keepalived.conf 192.168.20.6:/etc/keepalived/
  97. $ rsync -az /etc/keepalived/chk_tomcat.sh 192.168.20.6:/etc/keepalived/

配置tomcat02主机

$ cat /etc/keepalived/keepalived.conf      # 修改配置文件如下
global_defs {
   script_user root
   router_id TOMCAT-02       # 修改route_id
}


vrrp_script chk_tomcat {
        script "/etc/keepalived/chk_tomcat.sh"
        interval 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    unicast_src_ip 192.168.20.6          # 修改为本机IP
    unicast_peer {
    192.168.20.10                        # 修改为对端IP
                }

    virtual_router_id 23
    priority 100
    nopreempt
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        192.168.20.15/24
    }
    track_script {
        chk_tomcat
    }
}

接下来两台机器就都可以启动keepalived了。在启动之前，先配置防火墙规则，防止两台机器通信受阻，导致出现脑裂现象。

调整防火墙策略

注：两台主机都需要执行以下命令，以便放行相关流量。224.0.0.18是keepalived的组播地址，使用的是vrrp协议。

# 注意修改网卡名称和你主机网卡名的一致
$ firewall-cmd --direct --permanent --add-rule ipv4 filter OUTPUT 0 --in-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT

$ firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface ens33 --destination 224.0.0.18 --protocol vrrp -j ACCEPT

$ firewall-cmd --reload

启动keepalived

两台机器都启动keepalived。

$ systemctl start keepalived
# 确认进程存在
$ ps -ef | grep keepalived | grep -v grep
root      34325      1  0 02:59 ?        00:00:00 /opt/keepalived-2.0.20/sbin/keepalived -D
root      34326  34325  0 02:59 ?        00:00:00 /opt/keepalived-2.0.20/sbin/keepalived -D
root      34327  34325  0 02:59 ?        00:00:00 /opt/keepalived-2.0.20/sbin/keepalived -D

确认VIP已存在

注：VIP只能存在一台机器上，并且只能用ip 命令查看到VIP。

# 查看VIP（一般VIP会在先启动keepalived的那个机器上）
$ ip a    # 或 ip a show ens33
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:0a:3a:ad brd ff:ff:ff:ff:ff:ff
    inet 192.168.20.10/24 brd 192.168.20.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.20.15/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::1707:f528:803d:73b9/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

至此，配置已经完成了，自行验证VIP漂移效果即可。

注意事项

如果发生过服务不可用，记得在服务恢复后，重新启动下keepalived（上面定义的配置文件中，只要服务不可用，就会杀掉keepalived进程），以便keepalived继续正常工作（也可以将脚本中的systemctl stop keepalived改为systemctl restart keepalived，但这样的话，在你服务进程恢复前，keepalived会一直进行重启，从而产生大量的日志，keepalived的日志在/var/log/messages）。