背景

服务器配置

软件版本

镜像版本

创建初始文件夹

#/Users/keyboardone/同步空间/software
mkdir -p /opt/software
cd /opt/software/k8s/
chmod 755 /opt/software/k8s/*.sh

前置工作

安装docker

docker安装

安装k8s

局域网离线部署k8s集群

安装helm

helm安装
helm部署私库

安装seatunnel

参考官网：https://seatunnel.apache.org/docs/2.1.1/start/kubernetes

seatunnel版本：2.1.1

下载镜像

编辑Dockerfile

vi Dockerfile

FROM flink:1.13
ENV SEATUNNEL_VERSION="2.1.0"
RUN wget https://archive.apache.org/dist/incubator/seatunnel/${SEATUNNEL_VERSION}/apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
RUN tar -xzvf apache-seatunnel-incubating-${SEATUNNEL_VERSION}-bin.tar.gz
RUN mkdir -p $FLINK_HOME/usrlib
RUN cp apache-seatunnel-incubating-${SEATUNNEL_VERSION}/lib/seatunnel-core-flink.jar $FLINK_HOME/usrlib/seatunnel-core-flink.jar
RUN rm -fr apache-seatunnel-incubating-${SEATUNNEL_VERSION}*

构建

docker build -t seatunnel:2.1.0-flink-1.13 -f Dockerfile .

加载镜像到minikube

minikube image load seatunnel:2.1.0-flink-1.13

k8s安装cert文件

#下载
wget https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml
#部署
kubectl create -f cert-manager.yaml

helm安装Flink Kubernetes Operator

helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-0.1.0/
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator

验证

验证pod进程

#alias kubectl="minikube kubectl --"
kubectl get pod -A

验证seatunnel应用

构造虚拟数据流

vi flink.streaming.conf
#新增如下内容
env {
 execution.parallelism = 1
}
source {
   FakeSourceStream {
     result_table_name = "fake"
     field_name = "name,age"
   }
}
transform {
   sql {
     sql = "select name,age from fake"
   }
}
sink {
 ConsoleSink {}
}

拷贝配置文件到nfs目录下

编写seatunnel的yaml部署文件

vi seatunnel-flink.yaml

新增如下内容

apiVersion: flink.apache.org/v1alpha1
kind: FlinkDeployment
metadata:
  namespace: default
  name: seatunnel-flink-streaming-example
spec:
  #image: seatunnel:2.1.0-flink-1.13
  image: ren:8088/keyboardone/seatunnel:2.1.0-flink-1.13
  flinkVersion: v1_14
  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "2"
  serviceAccount: flink
  jobManager:
    replicas: 1
    resource:
      memory: "2048m"
      cpu: 1
  taskManager:
    resource:
      memory: "2048m"
      cpu: 2
  podTemplate:  
    spec:
      containers:
        - name: flink-main-container
          volumeMounts:
            - mountPath: /data
              name: config-volume
      volumes:
        - name: config-volume
          hostPath:
            path: "/nfs/data"
            type: Directory
  job:
    jarURI: local:///opt/flink/usrlib/seatunnel-core-flink.jar
    entryClass: org.apache.seatunnel.SeatunnelFlink
    args: ["--config", "/data/flink.streaming.conf"]
    parallelism: 2
    upgradeMode: stateless

执行部署

kubectl apply -f seatunnel-flink.yaml

验证

验证服务

kubectl get svc

查看日志

kubectl logs -f deploy/seatunnel-flink-streaming-example

暴露端口

kubectl port-forward svc/seatunnel-flink-streaming-example-rest 8081

Now the Flink Dashboard is accessible at localhost:8081.

打印日志

kubectl logs \
-l 'app in (seatunnel-flink-streaming-example), component in (taskmanager)' \
--tail=-1 \
-f

删除部署

kubectl delete -f seatunnel-flink.yaml