前言说明

对于 Kubernetes 的 Service，无论是 Cluster Ip 或 NodePort 均是四层的负载，Ingress可实现集群内服务七层负载均衡。

Ingress 是从 Kuberenets 集群外部访问集群的一个入口，将外部的请求转发到集群内 Service 上，相当于 nginx、haproxy 等负载均衡代理服务器。那为啥不直接用 nginx 实现，因为直接用 ng 当有新服务加入时无法动态配置。

Ingress Controller 可以理解为一个监听器，通过不断地监听 kube-apiserver，实时的感知后端 Service、Pod 的变化，当这些信息变化后，Ingress Controller 再结合 Ingress 的配置，更新反向代理负载均衡器，达到服务发现的作用。

Ingress控制器的实现方式有很多，使用场景较多的是 traefik 和 nginx-ingress，traefik 的性能较 nginx-ingress 差，但配置使用简单。

Ingress-nginx是7层的负载均衡器 ，负责统一管理外部对 k8s 中 Service 的请求。主要包含：

  • ingress-nginx-controller：根据用户编写的ingress规则（创建的ingress的yaml文件），动态的去更改nginx服务的配置文件，并且reload重载使其生效（是自动化的，通过lua脚本来实现）；
  • Ingress资源对象：将Nginx的配置抽象成一个Ingress对象

image.png

实现逻辑

1）ingress controller通过和kubernetes api交互，动态的去感知集群中ingress规则变化
2）读取ingress规则，按照自定义的规则，生成一段nginx配置
3）再写到nginx-ingress-controller的pod里，这个Ingress controller的pod里运行着一个Nginx服务，控制器把生成的nginx配置写入/etc/nginx/nginx.conf文件中
4）然后reload使配置生效。以此达到域名分别配置和动态更新。

服务安装

参考官档：https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md

  1. # 下载资源
  2. $ wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
  4. # 修改部署节点
  5. $ grep -n5 nodeSelector mandatory.yaml
  6. 212-    spec:
  7. 213-      hostNetwork: true  # 添加为host模式
  8. 214-      # wait up to five minutes for the drain of connections
  9. 215-      terminationGracePeriodSeconds: 300
  10. 216-      serviceAccountName: nginx-ingress-serviceaccount
  11. 217:      nodeSelector:
  12. 218-        kubernetes.io/os: linux  # 根据标签选择可以部署的机器
  13.             #kubernetes.io/hostname: master  # 仅仅部署在hostname为master的机器
  14. 219-      containers:
  15. 220-        - name: nginx-ingress-controller
  16. 221-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
  17. 222-          args:
  19. # 查看 node 节点标签，包含 kubernetes.io/os: linux ，能够适配部署需求
  20. kubectl get node --show-labels
  21. NAME    STATUS   ROLES    AGE   VERSION   LABELS
  22. master1 Ready    master   88d   v1.19.8   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=m1,kubernetes.io/os=linux,node-role.kubernetes.io/master=
  24. # 资源应用
  25. $ kubectl apply -f mandatory.yaml
  27. # 查看
  28. $ kubectl get pod -ningress-nginx -o wide
  29. NAME                                        READY   STATUS    RESTARTS   AGE   IP          NODE    NOMINATED NODE   READINESS GATES
  30. nginx-ingress-controller-7d4544b644-hdws6   1/1     Running   0          17m   10.23.34.9  master1 <none>           <none>
  32. # 验证，在 nginx-ingress-controller 的 pod 所在节点（master1）执行
  33. $ curl 127.0.0.1
  34. <html>
  35. <head><title>404 Not Found</title></head>
  36. <body>
  37. <center><h1>404 Not Found</h1></center>
  38. <hr><center>nginx/1.17.8</center>
  39. </body>
  40. </html>
  42. 说明：出现了上面的信息证明 Ingress Controller 已经安装成功。

服务使用

Base Used

示例：nginx 应用 ingress 资源 （ngdemo.yaml）

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      app: my-nginx
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    app: my-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: my-nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-nginx
  annotations:
    kubernetes.io/ingress.class: "nginx"  #指定此Ingress通过nginx-ingress来处理
spec:
  rules:
  - host: ngdemo.crab.com  # 将域名映射到 my-nginx 服务
    http:
      paths:
      - path: /
        backend:
          serviceName: my-nginx  # 将所有请求发送到my-nginx的80端口
          servicePort: 80     

#大部分Ingress controller不是直接转发到svc。是直接转发到Pod（通过svc获取后端Endpoints列表），此举可减少网络跳转，提高性能

#应用
$ kubectl apply -f ngdemo.yaml

#验证
##在本地hosts添加解析
 $ tail -1 /etc/hosts
10.23.34.9 ngdemo.crab.com  # ip可以是ingress-nginx pod所在的任何node节点ip

浏览器访问
image.png

Https

证书生成

#自签名证书
$ openssl req -x509 -nodes -days 36500 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=*.crab.com/O=ingress-nginx"

# 证书信息保存到secret对象中，ingress-nginx会读取secret对象解析出证书加载到nginx配置中
$ kubectl create secret tls tls-ngdemo --key tls.key --cert tls.crt

应用

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      app: my-nginx
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    app: my-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: my-nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-nginx
spec:
  rules:
  - host: ngdemo.crab.com  # 将域名映射到 my-nginx 服务
    http:
      paths:
      - path: /
        backend:
          serviceName: my-nginx
          servicePort: 80
  tls:
  - hosts:
    -  ngdemo.crab.com  # 将域名映射到 my-nginx 服务
    secretName: tls-ngdemo

Multipath forwarding

目标：

myblog.crab.com -> 172.21.51.143 -> /foo/aaa   service1:4200/foo/aaa
                                    /bar       service2:8080
                                    /               service3:80/

实现：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-fanout-example
spec:
  rules:
  - host: myblog.crab.com
    http:
      paths:
      - path: /foo
        pathType: Prefix
        backend:
          service: 
            name: service1
            port:
              number: 4200
      - path: /bar
        pathType: Prefix
        backend:
          service: 
            name: service2
            port:
              number: 8080
      - path: /
        pathType: Prefix
        backend:
          service: 
            name: service3
            port:
              number: 80

URL Rewrite

目标：

nginx.crab.com -> 172.21.51.143 ->  /api/v1   -> nginx-v1 service
                                    /api/v2   -> nginx-v2 service

实现：

$ cat nginx-v1-dpl.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: luffy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-v1
  template:
    metadata:
      labels:
        app: nginx-v1
    spec:
      containers:
      - image: nginx:alpine
        name: nginx-v1
        command: ["/bin/sh", "-c", "echo 'this is nginx-v1'>/usr/share/nginx/html/index.html;nginx -g 'daemon off;'"]

$ cat nginx-v1-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
  namespace: luffy
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-v1
  type: ClusterIP

# 创建nginx-v2

$ kubectl apply -f .


$ cat ingress-rewrite.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-rewrite
  namespace: luffy
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
  - host: nginx.luffy.com
    http:
      paths:
      - path: /api/v1/(.*)
        pathType: Prefix
        backend:
          service:
            name: nginx-v1
            port:
              number: 80
      - path: /api/v2/(.*)
        pathType: Prefix
        backend:
          service:
            name: nginx-v2
            port:
              number: 80



# 访问测试
$ http://nginx.luffy.com/api/v1/
$ http://nginx.luffy.com/api/v2/