date: 2020-07-20title: k8s之ingress controller #标题
tags: ingress #标签
categories: k8s # 分类

记录下k8s中暴露服务的方式——ingress controller。
前言: ingress只是k8s中的一种配置信息，而ingress controller才是监听 80/443端口，并根据ingress配置的路由信息执行http路由转发的组件。

ingress controller有多种实现的方式，比较常用的有 Traefic 、 Nginx Ingress Controller for Kubernetes 等。这里使用 Nginx Ingress Controller for Kubernetes来进行实现。

创建nginx及tomcat应用

nginx服务的yaml文件如下:

  1. $ cat nginx-deployment.yaml 
  2. apiVersion: apps/v1
  3. kind: Deployment
  4. metadata:
  5.   name: nginx-deployment
  6.   labels:
  7.     app: nginx
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: nginx
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: nginx
  17.     spec:
  18.       containers:
  19.       - name: nginx
  20.         image: nginx:latest
  21.         imagePullPolicy: IfNotPresent
  23. ---
  24. apiVersion: v1
  25. kind: Service
  26. metadata:
  27.   name: nginx-service
  28.   labels:
  29.     app: nginx
  30. spec:
  31.   selector:
  32.     app: nginx
  33.   ports:
  34.   - name: nginx-port
  35.     protocol: TCP
  36.     port: 80
  37.     nodePort: 32600
  38.     targetPort: 80
  39.   type: NodePort
  41. $ kubectl apply -f nginx-deployment.yaml    # 执行yaml文件

访问service:

k8s之ingress controller - 图1

tomcat服务的yaml文件如下:

  1. cat tomcat-deployment.yaml 
  2. apiVersion: apps/v1
  3. kind: Deployment
  4. metadata:
  5.   name: tomcat-deployment
  6.   labels:
  7.     app: tomcat
  8. spec:
  9.   replicas: 1
  10.   selector:
  11.     matchLabels:
  12.       app: tomcat
  13.   template:
  14.     metadata:
  15.       labels:
  16.         app: tomcat
  17.     spec:
  18.       containers:
  19.       - name: tomcat
  20.         image: tomcat:latest
  21.         imagePullPolicy: IfNotPresent
  23. ---
  24. apiVersion: v1
  25. kind: Service
  26. metadata:
  27.   name: tomcat-service
  28.   labels:
  29.     app: tomcat
  30. spec:
  31.   selector:
  32.     app: tomcat
  33.   ports:
  34.   - name: tomcat-port
  35.     protocol: TCP
  36.     port: 8080
  37.     nodePort: 32601
  38.     targetPort: 8080
  39.   type: NodePort
  43. $ kubectl apply -f tomcat-deployment.yaml   # 执行此yaml文件

访问service:

k8s之ingress controller - 图2

创建ingress-controller

  1. $ cat nginx-ingress.yaml        # 文件内容如下
  3. # 如果打算用于生产环境，请参考 https://github.com/nginxinc/kubernetes-ingress/blob/v1.5.5/docs/installation.md 并根据自己的情况做进一步定制
  5. apiVersion: v1
  6. kind: Namespace
  7. metadata:
  8.   name: nginx-ingress
  10. ---
  11. apiVersion: v1
  12. kind: ServiceAccount
  13. metadata:
  14.   name: nginx-ingress 
  15.   namespace: nginx-ingress
  17. ---
  18. apiVersion: v1
  19. kind: Secret
  20. metadata:
  21.   name: default-server-secret
  22.   namespace: nginx-ingress
  23. type: Opaque
  24. data:
  25.   tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN2akNDQWFZQ0NRREFPRjl0THNhWFhEQU5CZ2txaGtpRzl3MEJBUXNGQURBaE1SOHdIUVlEVlFRRERCWk8KUjBsT1dFbHVaM0psYzNORGIyNTBjbTlzYkdWeU1CNFhEVEU0TURreE1qRTRNRE16TlZvWERUSXpNRGt4TVRFNApNRE16TlZvd0lURWZNQjBHQTFVRUF3d1dUa2RKVGxoSmJtZHlaWE56UTI5dWRISnZiR3hsY2pDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUwvN2hIUEtFWGRMdjNyaUM3QlBrMTNpWkt5eTlyQ08KR2xZUXYyK2EzUDF0azIrS3YwVGF5aGRCbDRrcnNUcTZzZm8vWUk1Y2Vhbkw4WGM3U1pyQkVRYm9EN2REbWs1Qgo4eDZLS2xHWU5IWlg0Rm5UZ0VPaStlM2ptTFFxRlBSY1kzVnNPazFFeUZBL0JnWlJVbkNHZUtGeERSN0tQdGhyCmtqSXVuektURXUyaDU4Tlp0S21ScUJHdDEwcTNRYzhZT3ExM2FnbmovUWRjc0ZYYTJnMjB1K1lYZDdoZ3krZksKWk4vVUkxQUQ0YzZyM1lma1ZWUmVHd1lxQVp1WXN2V0RKbW1GNWRwdEMzN011cDBPRUxVTExSakZJOTZXNXIwSAo1TmdPc25NWFJNV1hYVlpiNWRxT3R0SmRtS3FhZ25TZ1JQQVpQN2MwQjFQU2FqYzZjNGZRVXpNQ0F3RUFBVEFOCkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWpLb2tRdGRPcEsrTzhibWVPc3lySmdJSXJycVFVY2ZOUitjb0hZVUoKdGhrYnhITFMzR3VBTWI5dm15VExPY2xxeC9aYzJPblEwMEJCLzlTb0swcitFZ1U2UlVrRWtWcitTTFA3NTdUWgozZWI4dmdPdEduMS9ienM3bzNBaS9kclkrcUI5Q2k1S3lPc3FHTG1US2xFaUtOYkcyR1ZyTWxjS0ZYQU80YTY3Cklnc1hzYktNbTQwV1U3cG9mcGltU1ZmaXFSdkV5YmN3N0NYODF6cFErUyt1eHRYK2VBZ3V0NHh3VlI5d2IyVXYKelhuZk9HbWhWNThDd1dIQnNKa0kxNXhaa2VUWXdSN0diaEFMSkZUUkk3dkhvQXprTWIzbjAxQjQyWjNrN3RXNQpJUDFmTlpIOFUvOWxiUHNoT21FRFZkdjF5ZytVRVJxbStGSis2R0oxeFJGcGZnPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  26.   tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdi91RWM4b1JkMHUvZXVJTHNFK1RYZUprckxMMnNJNGFWaEMvYjVyYy9XMlRiNHEvClJOcktGMEdYaVN1eE9ycXgrajlnamx4NXFjdnhkenRKbXNFUkJ1Z1B0ME9hVGtIekhvb3FVWmcwZGxmZ1dkT0EKUTZMNTdlT1l0Q29VOUZ4amRXdzZUVVRJVUQ4R0JsRlNjSVo0b1hFTkhzbysyR3VTTWk2Zk1wTVM3YUhudzFtMApxWkdvRWEzWFNyZEJ6eGc2clhkcUNlUDlCMXl3VmRyYURiUzc1aGQzdUdETDU4cGszOVFqVUFQaHpxdmRoK1JWClZGNGJCaW9CbTVpeTlZTW1hWVhsMm0wTGZzeTZuUTRRdFFzdEdNVWozcGJtdlFmazJBNnljeGRFeFpkZFZsdmwKMm82MjBsMllxcHFDZEtCRThCay90elFIVTlKcU56cHpoOUJUTXdJREFRQUJBb0lCQVFDZklHbXowOHhRVmorNwpLZnZJUXQwQ0YzR2MxNld6eDhVNml4MHg4Mm15d1kxUUNlL3BzWE9LZlRxT1h1SENyUlp5TnUvZ2IvUUQ4bUFOCmxOMjRZTWl0TWRJODg5TEZoTkp3QU5OODJDeTczckM5bzVvUDlkazAvYzRIbjAzSkVYNzZ5QjgzQm9rR1FvYksKMjhMNk0rdHUzUmFqNjd6Vmc2d2szaEhrU0pXSzBwV1YrSjdrUkRWYmhDYUZhNk5nMUZNRWxhTlozVDhhUUtyQgpDUDNDeEFTdjYxWTk5TEI4KzNXWVFIK3NYaTVGM01pYVNBZ1BkQUk3WEh1dXFET1lvMU5PL0JoSGt1aVg2QnRtCnorNTZud2pZMy8yUytSRmNBc3JMTnIwMDJZZi9oY0IraVlDNzVWYmcydVd6WTY3TWdOTGQ5VW9RU3BDRkYrVm4KM0cyUnhybnhBb0dCQU40U3M0ZVlPU2huMVpQQjdhTUZsY0k2RHR2S2ErTGZTTXFyY2pOZjJlSEpZNnhubmxKdgpGenpGL2RiVWVTbWxSekR0WkdlcXZXaHFISy9iTjIyeWJhOU1WMDlRQ0JFTk5jNmtWajJTVHpUWkJVbEx4QzYrCk93Z0wyZHhKendWelU0VC84ajdHalRUN05BZVpFS2FvRHFyRG5BYWkyaW5oZU1JVWZHRXFGKzJyQW9HQkFOMVAKK0tZL0lsS3RWRzRKSklQNzBjUis3RmpyeXJpY05iWCtQVzUvOXFHaWxnY2grZ3l4b25BWlBpd2NpeDN3QVpGdwpaZC96ZFB2aTBkWEppc1BSZjRMazg5b2pCUmpiRmRmc2l5UmJYbyt3TFU4NUhRU2NGMnN5aUFPaTVBRHdVU0FkCm45YWFweUNweEFkREtERHdObit3ZFhtaTZ0OHRpSFRkK3RoVDhkaVpBb0dCQUt6Wis1bG9OOTBtYlF4VVh5YUwKMjFSUm9tMGJjcndsTmVCaWNFSmlzaEhYa2xpSVVxZ3hSZklNM2hhUVRUcklKZENFaHFsV01aV0xPb2I2NTNyZgo3aFlMSXM1ZUtka3o0aFRVdnpldm9TMHVXcm9CV2xOVHlGanIrSWhKZnZUc0hpOGdsU3FkbXgySkJhZUFVWUNXCndNdlQ4NmNLclNyNkQrZG8wS05FZzFsL0FvR0FlMkFVdHVFbFNqLzBmRzgrV3hHc1RFV1JqclRNUzRSUjhRWXQKeXdjdFA4aDZxTGxKUTRCWGxQU05rMXZLTmtOUkxIb2pZT2pCQTViYjhibXNVU1BlV09NNENoaFJ4QnlHbmR2eAphYkJDRkFwY0IvbEg4d1R0alVZYlN5T294ZGt5OEp0ek90ajJhS0FiZHd6NlArWDZDODhjZmxYVFo5MWpYL3RMCjF3TmRKS2tDZ1lCbyt0UzB5TzJ2SWFmK2UwSkN5TGhzVDQ5cTN3Zis2QWVqWGx2WDJ1VnRYejN5QTZnbXo5aCsKcDNlK2JMRUxwb3B0WFhNdUFRR0xhUkcrYlNNcjR5dERYbE5ZSndUeThXczNKY3dlSTdqZVp2b0ZpbmNvVlVIMwphdmxoTUVCRGYxSjltSDB5cDBwWUNaS2ROdHNvZEZtQktzVEtQMjJhTmtsVVhCS3gyZzR6cFE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
  28. ---
  29. kind: ConfigMap
  30. apiVersion: v1
  31. metadata:
  32.   name: nginx-config
  33.   namespace: nginx-ingress
  34. data:
  35.   server-names-hash-bucket-size: "1024"
  38. ---
  39. kind: ClusterRole
  40. apiVersion: rbac.authorization.k8s.io/v1beta1
  41. metadata:
  42.   name: nginx-ingress
  43. rules:
  44. - apiGroups:
  45.   - ""
  46.   resources:
  47.   - services
  48.   - endpoints
  49.   verbs:
  50.   - get
  51.   - list
  52.   - watch
  53. - apiGroups:
  54.   - ""
  55.   resources:
  56.   - secrets
  57.   verbs:
  58.   - get
  59.   - list
  60.   - watch
  61. - apiGroups:
  62.   - ""
  63.   resources:
  64.   - configmaps
  65.   verbs:
  66.   - get
  67.   - list
  68.   - watch
  69.   - update
  70.   - create
  71. - apiGroups:
  72.   - ""
  73.   resources:
  74.   - pods
  75.   verbs:
  76.   - list
  77. - apiGroups:
  78.   - ""
  79.   resources:
  80.   - events
  81.   verbs:
  82.   - create
  83.   - patch
  84. - apiGroups:
  85.   - extensions
  86.   resources:
  87.   - ingresses
  88.   verbs:
  89.   - list
  90.   - watch
  91.   - get
  92. - apiGroups:
  93.   - "extensions"
  94.   resources:
  95.   - ingresses/status
  96.   verbs:
  97.   - update
  98. - apiGroups:
  99.   - k8s.nginx.org
  100.   resources:
  101.   - virtualservers
  102.   - virtualserverroutes
  103.   verbs:
  104.   - list
  105.   - watch
  106.   - get
  108. ---
  109. kind: ClusterRoleBinding
  110. apiVersion: rbac.authorization.k8s.io/v1beta1
  111. metadata:
  112.   name: nginx-ingress
  113. subjects:
  114. - kind: ServiceAccount
  115.   name: nginx-ingress
  116.   namespace: nginx-ingress
  117. roleRef:
  118.   kind: ClusterRole
  119.   name: nginx-ingress
  120.   apiGroup: rbac.authorization.k8s.io
  122. ---
  123. apiVersion: apps/v1
  124. kind: DaemonSet
  125. metadata:
  126.   name: nginx-ingress
  127.   namespace: nginx-ingress
  128.   annotations:
  129.     prometheus.io/scrape: "true"
  130.     prometheus.io/port: "9113"
  131. spec:
  132.   selector:
  133.     matchLabels:
  134.       app: nginx-ingress
  135.   template:
  136.     metadata:
  137.       labels:
  138.         app: nginx-ingress
  139.     spec:
  140.       serviceAccountName: nginx-ingress
  141.       containers:
  142.       - image: nginx/nginx-ingress:1.5.3
  143.         imagePullPolicy: IfNotPresent
  144.         name: nginx-ingress
  145.         ports:
  146.         - name: http
  147.           containerPort: 80
  148.           hostPort: 80
  149.         - name: https
  150.           containerPort: 443
  151.           hostPort: 443
  152.         - name: prometheus
  153.           containerPort: 9113
  154.         env:
  155.         - name: POD_NAMESPACE
  156.           valueFrom:
  157.             fieldRef:
  158.               fieldPath: metadata.namespace
  159.         - name: POD_NAME
  160.           valueFrom:
  161.             fieldRef:
  162.               fieldPath: metadata.name
  163.         args:
  164.           - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
  165.           - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
  166.          #- -v=3 # Enables extensive logging. Useful for troubleshooting.
  167.          #- -report-ingress-status
  168.          #- -external-service=nginx-ingress
  169.          #- -enable-leader-election
  170.           - -enable-prometheus-metrics
  171.          #- -enable-custom-resources
  174. $ kubectl apply -f nginx-ingress.yaml

创建ingress

  1. $ cat ingress-v1.yaml 
  2. apiVersion: networking.k8s.io/v1beta1
  3. kind: Ingress
  4. metadata:
  5.   name: my-ingress-for-nginx  # Ingress 的名字，仅用于标识
  6. spec:
  7.   rules:                      # Ingress 中定义 L7 路由规则
  8.   - host: www.lvnginx.cn   # 根据 virtual hostname 进行路由（请使用自己的域名）
  9.     http:
  10.       paths:                  # 按路径进行路由
  11.       - path: /
  12.         backend:
  13.           serviceName: nginx-service  # 指定后端的 Service 为之前创建的 nginx-service
  14.           servicePort: 80
  15.   - host: www.lvtomcat.cn   # 根据 virtual hostname 进行路由（请使用自己的域名）
  16.     http:
  17.       paths:                  # 按路径进行路由
  18.       - path: /
  19.         backend:
  20.           serviceName: tomcat-service  # 指定后端的 Service 为之前创建的 nginx-service
  21.           servicePort: 8080
  23. $ kubectl apply -f ingress-v1.yaml

域名访问测试

创建后自己做域名解析（需解析到work节点，不能解析到master节点，具体原因及解决办法参考我之前的博文: K8s之Ingress-nginx原理及配置），可以解析后，分别访问 www.lvtomcat.cn 和 www.lvnginx.cn，可以看到如下页面，则表示成功。

k8s之ingress controller - 图3

k8s之ingress controller - 图4