参照这个https://blog.51cto.com/14143894/2463392?source=drh
calico 有个博客 https://www.tigera.io/blog/configuring-route-reflectors-in-calico/
但我遇到了问题，请见之前一篇日志，以后再搞
具体步骤如下：
1、关闭 node-to-node BGP网格
添加 default BGP配置，调整 nodeToNodeMeshEnabled和asNumber：

  1. [root@k8s-master1 calico]# cat bgp.yaml 
  2.  apiVersion: projectcalico.org/v3
  3.  kind: BGPConfiguration
  4.  metadata:
  5.    name: default
  6.  spec:
  7.    logSeverityScreen: Info
  8.    nodeToNodeMeshEnabled: false  
  9.    asNumber: 63400

直接应用一下，当我们禁用node-to-node mesh的时候，网络立马就会断，所以断的话要提前做好影响的范围，也就是切换这个网络是需要断网的，使用node-to-node BGP这种也是建议100个节点以下，当超过100台节点一定要使用路由反射RR模式

  1. [root@k8s-master1 calico]# calicoctl apply -f bgp.yaml 
  2. Successfully applied 1 'BGPConfiguration' resource(s)

查看bgp网络配置情况，false为关闭

  1. [root@k8s-master1 calico]# calicoctl get bgpconfig
  2. NAME      LOGSEVERITY   MESHENABLED   ASNUMBER   
  3. default   Info          false         63400

去查看pod的网络测试已经断开了，这里是因为我们使用caclico的配置禁用了node-to-node mesh了

  1. [root@k8s-master1 calico]# ping 10.244.245.2
  2. PING 10.244.245.2 (10.244.245.2) 56(84) bytes of data.

ASN号可以通过获取 # calicoctl get nodes —output=wide
这里有个编号，ASN64300，一个编号就是一个自治系统

  1. [root@k8s-master1 calico]# calicoctl get nodes --output=wide
  2. NAME                ASN       IPV4           IPV6   
  3. k8s-master1   (63400)   10.4.7.11/24          
  4. k8s-node1   (63400)   10.4.7.12/24          
  5. k8s-node2   (63400)   10.4.7.21/24

2、配置指定节点充当路由反射器
为方便让BGPPeer轻松选择节点，通过标签选择器匹配，也就是可以去调用k8s里面的标签进行关联，我们可以给哪个节点作为路由发射器打个标签
给路由器反射器节点打标签，我这将node1打上标签
[root@k8s-master1 calico]# kubectl label node k8s-node1 route-reflector=true
查看node BJP的节点状态，因为禁用了网格，所以这里都关闭了，所以也就不通了。

  1. [root@k8s-master1 calico]# calicoctl node status
  2. Calico process is running.
  3. IPv4 BGP status
  4. No IPv4 peers found.
  5. IPv6 BGP status
  6. No IPv6 peers found.

然后配置路由器反射器节点routeReflectorClusterID，增加一个集群节点的ID
下面的可以通过-o yaml输出出来

  1. [root@k8s-master1 calico]# calicoctl get node k8s-node2 -o yaml > node.yaml
  2. apiVersion: projectcalico.org/v3
  3. kind: Node
  4. metadata:
  5.   annotations:
  6.     projectcalico.org/kube-labels: '{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"k8s-node2","kubernetes.io/os":"linux"}'
  7.   creationTimestamp: null
  8.   labels:
  9.     beta.kubernetes.io/arch: amd64
  10.     beta.kubernetes.io/os: linux
  11.     kubernetes.io/arch: amd64
  12.     kubernetes.io/hostname: k8s-node2
  13.     kubernetes.io/os: linux
  14.   name: k8s-node2
  15. spec:
  16.   bgp:
  17.     ipv4Address: 10.4.7.12/24
  18.     routeReflectorClusterID: 244.0.0.1   # 集群ID
  19.   orchRefs:
  20.   - nodeName: k8s-node2
  21.     orchestrator: k8s

应用一下
[root@k8s-master1 calico]# calicoctl apply -f node.yaml
现在，很容易使用标签选择器将路由反射器节点与其他非路由反射器节点配置为对等：现在也就是将其他的节点去连接这个k8s-node1打标签的路由发射器

  1. [root@k8s-master1 calico]# cat bgp1.yaml 
  2. apiVersion: projectcalico.org/v3
  3. kind: BGPPeer
  4. metadata:
  5.   name: peer-with-route-reflectors
  6. spec:
  7.   nodeSelector: all()    #所以的节点
  8.   peerSelector: route-reflector == 'true'

就是带route-reflector的都去连接匹配这个，刚才我们不是打上标签了嘛，所以需要我们去连接这个路由反射器
查看节点的BGP规则与连接状态，这样的话就显示一个路由反射器的节点

  1. [root@k8s-master1 calico]# calicoctl apply -f bgp1.yaml 
  2. Successfully applied 1 'BGPPeer' resource(s)
  3. [root@k8s-master1 calico]# calicoctl get bgppeer
  4. NAME                         PEERIP   NODE    ASN   
  5. peer-with-route-reflectors            all()   0     
  6. [root@k8s-master1 calico]# calicoctl node status
  7. Calico process is running.
  8. IPv4 BGP status
  9. +--------------+---------------+-------+----------+-------------+
  10. | PEER ADDRESS |   PEER TYPE   | STATE |  SINCE   |    INFO     |
  11. +--------------+---------------+-------+----------+-------------+
  12. | 10.4.7.12    | node specific | up    | 08:22:22 | Established |
  13. +--------------+---------------+-------+----------+-------------+
  14. IPv6 BGP status
  15. No IPv6 peers found.

查看容器网络联通性

  1. [root@k8s-master1 calico]# ping 10.244.203.80
  2. PING 10.244.203.80 (10.244.203.80) 56(84) bytes of data.
  3. 64 bytes from 10.244.203.80: icmp_seq=1 ttl=63 time=1.71 ms

添加多个路由反射器
现在进行对路由反射器添加多个，100个节点以内建议2-3个路由反射器
1）进行对集群节点打标签

  1. [root@k8s-master1 calico]# kubectl label node k8s-node2 route-reflector=true
  2. node/k8s-node2 labeled

2）对k8s-node2添加然后配置路由器反射器节点
[root@k8s-master1 calico]# calicoctl get node k8s-node2 -o yaml
3)查看节点状态

  1. [root@k8s-master1 calico]# calicoctl node status
  2. Calico process is running.
  3. IPv4 BGP status
  4. +--------------+---------------+-------+----------+-------------+
  5. | PEER ADDRESS |   PEER TYPE   | STATE |  SINCE   |    INFO     |
  6. +--------------+---------------+-------+----------+-------------+
  7. | 10.4.7.12    | node specific | up    | 08:22:22 | Established |
  8. | 10.4.7.21    | node specific | up    | 08:44:44 | Established |
  9. +--------------+---------------+-------+----------+-------------+
  10. IPv6 BGP status
  11. No IPv6 peers found.

4）测试网络连通性

  1. [root@k8s-master1 calico]# ping 10.244.203.81
  2. PING 10.244.203.81 (10.244.203.81) 56(84) bytes of data.
  3. 64 bytes from 10.244.203.81: icmp_seq=1 ttl=63 time=12.7 ms
  4. 64 bytes from 10.244.203.81: icmp_seq=2 ttl=63 time=1.40 ms

所以这是使用路由反射器来解决节点增多BGP带来的消耗

我的问题 解决了
问题在于 node120 上的/etc/calico/calicoctl.cfg datastoreType原来用的是etcd的，没法正常连接，现在换成了kubernetes，都可以正常运行了。
**
cat /etc/calico/calicoctl.cfg
apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
datastoreType: “kubernetes”
kubeconfig: “/root/.kube/config”

cat bgp_disable.yaml
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
name: default
spec:
logSeverityScreen: Info
nodeToNodeMeshEnabled: false
asNumber: 64512

apiVersion: projectcalico.org/v3
kind: Node
metadata:
annotations:
projectcalico.org/kube-labels: ‘{“beta.kubernetes.io/arch”:”amd64”,”beta.kubernetes.io/os”:”linux”,”kubernetes.io/arch”:”amd64”,”kubernetes.io/hostname”:”node121”,”kubernetes.io/os”:”linux”,”route-reflector”:”true”}’
creationTimestamp: “2020-12-06T15:29:03Z”
labels:
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/os: linux
kubernetes.io/arch: amd64
kubernetes.io/hostname: node121
kubernetes.io/os: linux
route-reflector: “true”
name: node121
resourceVersion: “136548”
uid: a77e209d-a0f5-44c0-9c71-45e6a9a8d55a
spec:
addresses:
- address: 10.10.10.121/24
- address: 10.10.10.121
bgp:
ipv4Address: 10.10.10.121/24
routeReflectorClusterID: 244.0.0.1
orchRefs:
- nodeName: node121
orchestrator: k8s
status:
podCIDRs:
- 192.168.1.0/24

cat bgp1.yaml
apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
name: peer-with-route-reflectors
spec:
nodeSelector: all() #所以的节点
peerSelector: route-reflector == ‘true’

最后一次执行这给 这3个文件 就可以了 ，得到的结果是这这样的
calicoctl get bgppeers
NAME PEERIP NODE ASN
peer-with-route-reflectors all() 0

calicoctl node status
Calico process is running.
IPv4 BGP status
+———————+———————-+———-+—————+——————-+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+———————+———————-+———-+—————+——————-+
| 10.10.10.121 | node specific | up | 07:03:12 | Established |
+———————+———————-+———-+—————+——————-+
IPv6 BGP status
No IPv6 peers found.