一、架构说明

分别位于三台机器上，搭建三个一主二从的主从，三个一主二从的节点形成一个高可用的集群

ip Redis节点
192.168.137.130 8001
8004
8007
192.168.137.131 8002
8005
8008
192.168.137.132 8003
8006
8009

二、搭建过程

2.1 在对应的机器上，创建不同的目录，存放对应端口的redis配置文件

  1. mkdir /opt/module/redis-cluster
  2. cd /opt/module/redis-cluster
  3. mkdir 8001 8004 8007 # 其他节点根据规划，创建不同的目录
  4. cp redis.conf 8001 # 复制redis.conf

2.2 修改redis.conf

主要修改内容如下

  1. 1. daemonize yes
  2. 2. port 8001 # 不同的节点改成不同的端口
  3. 3. dir /usr/local/redis-cluster/8001/ #不同的节点，存放数据的目录
  4. 4. cluster-enabled yes   # 启动集群模式
  5. 5. cluster-config-file nodes-8001.conf  # 设置集群的配置文件，用于存放集群的节点信息
  6. 6. cluster-node-timeout 5000   # 集群之间节点通讯的超时时间
  7. 7. bind 0.0.0.0 
  8. 8. protected-mode  no  关闭保护模式
  9. # 如果想要设置密码，可以添加如下的两个配置
  10. requirepass your_password   #设置redis客户端访问时的密码
  11. masterauth your_password    #设置访问master时携带的密码

2.3 将此份修改完成之后的配置文件，复制到8004/8007中

修改2，3，5这三个配置即可

2.4 其他机器中，重复2.1-2.3创建出不同的目录即可

2.5 使用脚本启动redis

不同的机器，请修改对应的目录

  1. /opt/module/redis-5.0.8/src/redis-server /opt/module/redis-cluster/8001/redis.conf
  2. /opt/module/redis-5.0.8/src/redis-server /opt/module/redis-cluster/8004/redis.conf
  3. /opt/module/redis-5.0.8/src/redis-server /opt/module/redis-cluster/8007/redis.conf

执行完成，在三台机器中可以看到如下的结果，则证明，redis实例已经启动完毕

  1. [ifan@hadoop130 redis-cluster]$ ps -ef|grep redis
  2. ifan       8096      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8001 [cluster]
  3. ifan       8101      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8004 [cluster]
  4. ifan       8106      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8007 [cluster]
  5. ifan       8118   7377  0 20:19 pts/0    00:00:00 grep --color=auto redis
  6. -------------------------------------------------------------------------
  7. [ifan@hadoop131 redis-cluster]$ ps -ef|grep redis
  8. ifan       2849      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8002 [cluster]
  9. ifan       2851      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8005 [cluster]
  10. ifan       2856      1  0 20:19 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8008 [cluster]
  11. ifan       2866   2252  0 20:19 pts/0    00:00:00 grep --color=auto redis
  12. -------------------------------------------------------------------------
  13. [ifan@hadoop132 redis-cluster]$ ps -ef|grep redis
  14. ifan       2763      1  0 20:20 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8003 [cluster]
  15. ifan       2765      1  0 20:20 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8006 [cluster]
  16. ifan       2770      1  0 20:20 ?        00:00:00 /opt/module/redis-5.0.8/src/redis-server 0.0.0.0:8009 [cluster]
  17. ifan       2778   2315  0 20:20 pts/0    00:00:00 grep --color=auto redis

2.6 启动实例完成之后，需要通过redis-cli，来创建整个集群

通过 redis-cli 查看帮助，可以看到其中有个—cluster，在通过—cluster的帮助，可以看到如下的内容

  1. [ifan@hadoop130 redis-cluster]$ /opt/module/redis-5.0.8/src/redis-cli --cluster help
  2. Cluster Manager Commands:
  3.   create         host1:port1 ... hostN:portN
  4.                  --cluster-replicas <arg>
  5.   check          host:port
  6.                  --cluster-search-multiple-owners
  7.   info           host:port
  8.   fix            host:port
  9.                  --cluster-search-multiple-owners
  10.   reshard        host:port
  11.                  --cluster-from <arg>
  12.                  --cluster-to <arg>
  13.                  --cluster-slots <arg>
  14.                  --cluster-yes
  15.                  --cluster-timeout <arg>
  16.                  --cluster-pipeline <arg>
  17.                  --cluster-replace
  18.   rebalance      host:port
  19.                  --cluster-weight <node1=w1...nodeN=wN>
  20.                  --cluster-use-empty-masters
  21.                  --cluster-timeout <arg>
  22.                  --cluster-simulate
  23.                  --cluster-pipeline <arg>
  24.                  --cluster-threshold <arg>
  25.                  --cluster-replace
  26.   add-node       new_host:new_port existing_host:existing_port
  27.                  --cluster-slave
  28.                  --cluster-master-id <arg>
  29.   del-node       host:port node_id
  30.   call           host:port command arg arg .. arg
  31.   set-timeout    host:port milliseconds
  32.   import         host:port
  33.                  --cluster-from <arg>
  34.                  --cluster-copy
  35.                  --cluster-replace
  36.   help           
  38. For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

最终，通过如下的命令，搭建redis集群(注意，—cluster-replicas后面的数量，为从节点的数量)

  1. /opt/module/redis-5.0.8/src/redis-cli --cluster create --cluster-replicas 2 192.168.137.130:8001 192.168.137.131:8002 192.168.137.132:8003 192.168.137.130:8004 192.168.137.131:8005 192.168.137.132:8006 192.168.137.130:8007 192.168.137.131:8008 192.168.137.132:8009

可以看到输出结果如下

  1. [ifan@hadoop130 redis-cluster]$ ./joinmaster.sh 
  2. >>> Performing hash slots allocation on 9 nodes...
  3. Master[0] -> Slots 0 - 5460
  4. Master[1] -> Slots 5461 - 10922
  5. Master[2] -> Slots 10923 - 16383
  6. Adding replica 192.168.137.131:8005 to 192.168.137.130:8001
  7. Adding replica 192.168.137.132:8006 to 192.168.137.130:8001
  8. Adding replica 192.168.137.130:8007 to 192.168.137.131:8002
  9. Adding replica 192.168.137.132:8009 to 192.168.137.131:8002
  10. Adding replica 192.168.137.131:8008 to 192.168.137.132:8003
  11. Adding replica 192.168.137.130:8004 to 192.168.137.132:8003
  12. M: 8be4fa220f77092bccd95843d68ff82f4ac952e3 192.168.137.130:8001
  13.    slots:[0-5460] (5461 slots) master
  14. M: be7638ba83837f13c3bd610dae5dd252feee2ffd 192.168.137.131:8002
  15.    slots:[5461-10922] (5462 slots) master
  16. M: 8ff4e94320635a059f61fc8b339b0e34bb0a2422 192.168.137.132:8003
  17.    slots:[10923-16383] (5461 slots) master
  18. S: 2ea09b98a09393f170e39070156bc8c87f199153 192.168.137.130:8004
  19.    replicates 8ff4e94320635a059f61fc8b339b0e34bb0a2422
  20. S: 3e6b1162265a0439ef30b766f273bd8a9416f076 192.168.137.131:8005
  21.    replicates 8be4fa220f77092bccd95843d68ff82f4ac952e3
  22. S: d79a98f7db5540b5116a9bfe96ded010694319ea 192.168.137.132:8006
  23.    replicates 8be4fa220f77092bccd95843d68ff82f4ac952e3
  24. S: 5b301de776eba740131b2b92776725d74896dbcf 192.168.137.130:8007
  25.    replicates be7638ba83837f13c3bd610dae5dd252feee2ffd
  26. S: e8da5bfaec54b133d95339d0c354a3ac15daed6c 192.168.137.131:8008
  27.    replicates 8ff4e94320635a059f61fc8b339b0e34bb0a2422
  28. S: 9f15c0ba29671d5405f8a87a78cad92574ed1d8d 192.168.137.132:8009
  29.    replicates be7638ba83837f13c3bd610dae5dd252feee2ffd
  30. Can I set the above configuration? (type 'yes' to accept): yes
  31. >>> Nodes configuration updated
  32. >>> Assign a different config epoch to each node
  33. >>> Sending CLUSTER MEET messages to join the cluster
  34. Waiting for the cluster to join
  35. .....
  36. >>> Performing Cluster Check (using node 192.168.137.130:8001)
  37. M: 8be4fa220f77092bccd95843d68ff82f4ac952e3 192.168.137.130:8001
  38.    slots:[0-5460] (5461 slots) master
  39.    2 additional replica(s)
  40. S: d79a98f7db5540b5116a9bfe96ded010694319ea 192.168.137.132:8006
  41.    slots: (0 slots) slave
  42.    replicates 8be4fa220f77092bccd95843d68ff82f4ac952e3
  43. S: 3e6b1162265a0439ef30b766f273bd8a9416f076 192.168.137.131:8005
  44.    slots: (0 slots) slave
  45.    replicates 8be4fa220f77092bccd95843d68ff82f4ac952e3
  46. S: 2ea09b98a09393f170e39070156bc8c87f199153 192.168.137.130:8004
  47.    slots: (0 slots) slave
  48.    replicates 8ff4e94320635a059f61fc8b339b0e34bb0a2422
  49. S: 9f15c0ba29671d5405f8a87a78cad92574ed1d8d 192.168.137.132:8009
  50.    slots: (0 slots) slave
  51.    replicates be7638ba83837f13c3bd610dae5dd252feee2ffd
  52. S: e8da5bfaec54b133d95339d0c354a3ac15daed6c 192.168.137.131:8008
  53.    slots: (0 slots) slave
  54.    replicates 8ff4e94320635a059f61fc8b339b0e34bb0a2422
  55. S: 5b301de776eba740131b2b92776725d74896dbcf 192.168.137.130:8007
  56.    slots: (0 slots) slave
  57.    replicates be7638ba83837f13c3bd610dae5dd252feee2ffd
  58. M: 8ff4e94320635a059f61fc8b339b0e34bb0a2422 192.168.137.132:8003
  59.    slots:[10923-16383] (5461 slots) master
  60.    2 additional replica(s)
  61. M: be7638ba83837f13c3bd610dae5dd252feee2ffd 192.168.137.131:8002
  62.    slots:[5461-10922] (5462 slots) master
  63.    2 additional replica(s)
  64. [OK] All nodes agree about slots configuration.
  65. >>> Check for open slots...
  66. >>> Check slots coverage...
  67. [OK] All 16384 slots covered.

通过上面的输出，可以看出，实际上，通过redis-cli组成了三个一主二丛的架构

  1. [ifan@hadoop130 redis-cluster]$ cat 8001/nodes-8001.conf 
  2. 节点的id                                                                        节点的ip和端口                          节点的角色   slave跟随的主节点id                                                        节点状态   
  3. d79a98f7db5540b5116a9bfe96ded010694319ea 192.168.137.132:8006@18006 slave 8be4fa220f77092bccd95843d68ff82f4ac952e3 0 1604541789000 6 connected
  5. 3e6b1162265a0439ef30b766f273bd8a9416f076 192.168.137.131:8005@18005 slave 8be4fa220f77092bccd95843d68ff82f4ac952e3 0 1604541788533 5 connected
  6. 2ea09b98a09393f170e39070156bc8c87f199153 192.168.137.130:8004@18004 slave 8ff4e94320635a059f61fc8b339b0e34bb0a2422 0 1604541788000 4 connected
  7. 9f15c0ba29671d5405f8a87a78cad92574ed1d8d 192.168.137.132:8009@18009 slave be7638ba83837f13c3bd610dae5dd252feee2ffd 0 1604541789139 9 connected
  8. e8da5bfaec54b133d95339d0c354a3ac15daed6c 192.168.137.131:8008@18008 slave 8ff4e94320635a059f61fc8b339b0e34bb0a2422 0 1604541788125 8 connected
  9. 5b301de776eba740131b2b92776725d74896dbcf 192.168.137.130:8007@18007 slave be7638ba83837f13c3bd610dae5dd252feee2ffd 0 1604541789000 7 connected
  10.                                                                                                                                                                                                                  节点的slot范围
  11. 8ff4e94320635a059f61fc8b339b0e34bb0a2422 192.168.137.132:8003@18003 master - 0 1604541788000 3 connected 10923-16383
  12. 8be4fa220f77092bccd95843d68ff82f4ac952e3 192.168.137.130:8001@18001 myself,master - 0 1604541787000 1 connected 0-5460
  13. be7638ba83837f13c3bd610dae5dd252feee2ffd 192.168.137.131:8002@18002 master - 0 1604541788125 2 connected 5461-10922
  14. vars currentEpoch 9 lastVoteEpoch 0

redis-cli设置值测试，会直接通过hash算法，重定向到对应的节点中。

  1. [ifan@hadoop130 8001]$ /opt/module/redis-5.0.8/src/redis-cli -c -p 8004
  2. 127.0.0.1:8004> set k2 v1
  3. -> Redirected to slot [449] located at 192.168.137.130:8001
  4. OK
  5. 192.168.137.130:8001> keys *
  6. 1) "k2"