https://plugins.jenkins.io/kubernetes/

inbound-agent镜像 : https://hub.docker.com/r/jenkins/inbound-agent

1. 静态节点实践

首先,在Jenkins 中创建一个jnlp类型的agent, 拿到连接所需要的认证和秘钥信息。

  1. java -jar agent.jar -jnlpUrl http://192.168.1.200:8080/computer/buildtest01/jenkins-agent.jnlp -secret 047a2627be5ee128c1a93f281051acc7bbbbd5221dcb56ae9fa408a494e72c2f -workDir "/home/jenkins"

1.1 Docker静态节点

启动镜像,传递认证参数。

  1. ## 测试
  2. docker run --init jenkins/inbound-agent -webSocket -url http://192.168.1.200:8080 -workDir=/home/jenkins 047a2627be5ee128c1a93f281051acc7bbbbd5221dcb56ae9fa408a494e72c2f buildtest01
  3. ## 运行
  4. docker run -itd --name buildtest01 jenkins/inbound-agent -webSocket -url http://192.168.1.200:8080 -workDir=/home/jenkins 047a2627be5ee128c1a93f281051acc7bbbbd5221dcb56ae9fa408a494e72c2f buildtest01

1.2 K8s部署静态节点

  1. java -jar agent.jar -jnlpUrl http://192.168.1.200:8080/computer/buildtest02/jenkins-agent.jnlp -secret f8b870a1f3783261375485aa28053ad22c9289ba7bc32e4167712ce57fb6cab4 -workDir "/home/jenkins"
  1. ---
  2. kind: Deployment
  3. apiVersion: apps/v1
  4. metadata:
  5. labels:
  6. k8s-app: jenkinsagent
  7. name: jenkinsagent
  8. namespace: devops
  9. spec:
  10. replicas: 1
  11. revisionHistoryLimit: 10
  12. selector:
  13. matchLabels:
  14. k8s-app: jenkinsagent
  15. template:
  16. metadata:
  17. labels:
  18. k8s-app: jenkinsagent
  19. namespace: devops
  20. name: jenkinsagent
  21. spec:
  22. containers:
  23. - name: jenkinsagent
  24. image: jenkins/inbound-agent
  25. securityContext:
  26. privileged: true
  27. imagePullPolicy: IfNotPresent
  28. resources:
  29. limits:
  30. cpu: 1000m
  31. memory: 2Gi
  32. requests:
  33. cpu: 500m
  34. memory: 512Mi
  35. env:
  36. - name: JENKINS_URL
  37. value: http://192.168.1.200:8080
  38. - name: JENKINS_SECRET
  39. value: f8b870a1f3783261375485aa28053ad22c9289ba7bc32e4167712ce57fb6cab4
  40. - name: JENKINS_AGENT_NAME
  41. value: buildtest02
  42. - name: JENKINS_AGENT_WORKDIR
  43. value: /home/jenkins/workspace

如果遇到了agent启动时的链接问题, 可以参考在yaml中加上这个变量。即使用websocket的方式链接。

  1. - name: JENKINS_WEB_SOCKET
  2. value: true

2. 动态节点实践

2.1 Docker 动态节点

docker插件: https://plugins.jenkins.io/docker-plugin/
项目地址: https://github.com/jenkinsci/docker-plugin
image.png
Jenkins动态slave的配置,需要Jenkins调用Docker的接口完成的。我们需要开启Docker远程访问

MacOS

docker 开启API远程访问 (mac) 参考文档: https://juejin.im/entry/5bdf04b06fb9a049e41223f1

  1. docker pull bobrik/socat
  2. docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 2376:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock

CentOS
编辑/usr/lib/systemd/system/docker.service

  1. [Service]
  2. Type=notify
  3. # the default is not to use systemd for cgroups because the delegate issues still
  4. # exists and systemd currently does not support the cgroup feature set required
  5. # for containers run by docker
  6. ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock
  1. systemctl daemon-reload
  2. systemctl restart docker

当我们把上面的配置完成后,可以通过Curl命令进行基本的测试API。出现一下信息表示成功开启。

  1. #curl -XGET http://127.0.0.1:2375/version
  2. {"Platform":{"Name":"Docker Engine - Community"},"Components":[{"Name":"Engine","Version":"19.03.5","Details":{"ApiVersion":"1.40","Arch":"amd64","BuildTime":"2019-11-13T07:29:19.000000000+00:00","Experimental":"false","GitCommit":"633a0ea","GoVersion":"go1.12.12","KernelVersion":"4.19.76-linuxkit","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"v1.2.10","Details":{"GitCommit":"b34a5c8af56e510852c35414db4c1f4fa6172339"}},{"Name":"runc","Version":"1.0.0-rc8+dev","Details":{"GitCommit":"3e425f80a8c931f88e6d94a8c831b9d5aa481657"}},{"Name":"docker-init","Version":"0.18.0","Details":{"GitCommit":"fec3683"}}],"Version":"19.03.5","ApiVersion":"1.40","MinAPIVersion":"1.12","GitCommit":"633a0ea","GoVersion":"go1.12.12","Os":"linux","Arch":"amd64","KernelVersion":"4.19.76-linuxkit","BuildTime":"2019-11-13T07:29:19.000000000+00:00"}

Jenkins配置

我们需要安装插件 Docker plugin

方式1: 启动容器进行构建(无需连接master)

此处课程中没有测试出来…. 可能与环境有关….这种方式了解即可。

  1. docker pull maven:3.8.1-openjdk-8-slim
  1. dockerNode(dockerHost: 'tcp://192.168.1.200:2375', image: 'maven:3.8.1-openjdk-8-slim') {
  2. // some block
  3. sh "mvn -v"
  4. sh "sleep 5"
  5. }

image.png


方式2: 使用CLoud

配置云: 节点管理-> CLoud-> Docker

image.png

Docker Cloud details信息填写:

  • Docker Host URI Docker主机信息(需要开启Docker配置)
    • tcp://192.168.1.200:2375
    • unix:///var/run/docker.sock
  • Connection Timeout 连接超时时间
  • Read Timeout 读操作超时时间 (调大些,容易出现超时的情况)
  • Enabled 是否启用?默认否
  • Error Duration 错误的持续时间 默认300 5分钟
  • Container Cap 容器数量 负值或零,或2147483647都意味着“无限制” ,默认值100。

image.png


Docker Agent templates

  • Labels 节点标签
  • Enabled 是否启动 默认否
  • Name 节点名称
  • Docker Image 镜像标签
  • Remote File System Root 远程文件系统根目录
  • 用法 自定义指定项目运行
  • Connect method 连接方式
    • JNLP 推荐
      • User 运行用户
      • Jenkins URL jenkins地址
      • Idle timeout 空闲时间多少秒后杀死slave
  • SSH
  • Docker Container
  • Pull strategy 镜像下载策略
  • Pull timeout 镜像下载超时时间 单位秒
  • Instance Capacity 实例数量

image.png

image.png

image.png


测试

  1. node("jenkins-node"){
  2. echo "Hello world!"
  3. sh "sleep 20"
  4. }

image.png


2.2 K8s部署动态节点

https://plugins.jenkins.io/kubernetes/

安装kubernetes插件(安装完成后最好重启一下)。
配置插件信息 节点管理 > Cloud ->增加一个云。

制作Kubernetes证书

  • 进入集群服务器 .kube/config
  • 复制config文件中的certificate-authority-data内容,生成base64文件 ca.crt
    • echo xxxxxx | base64 –d > ca.crt

Jenkins新建凭据
新建凭据 , Secret file.类型, 将.kube/config文件上传上去。
image.png

将ca.crt 内容复制到 服务证书key,选择证书凭据,测试连接。

image.png

测试流水线的模板:

  1. pipeline{
  2. agent{
  3. kubernetes{
  4. label "test01"
  5. cloud 'kubernetes'
  6. yaml '''
  7. ---
  8. kind: Pod
  9. apiVersion: v1
  10. metadata:
  11. labels:
  12. k8s-app: jenkinsagent
  13. name: jenkinsagent
  14. namespace: devops
  15. spec:
  16. containers:
  17. - name: jenkinsagent
  18. image: jenkins/inbound-agent
  19. imagePullPolicy: IfNotPresent
  20. '''
  21. }
  22. }
  23. options {
  24. skipDefaultCheckout()
  25. }
  26. stages{
  27. stage("test"){
  28. steps{
  29. script{
  30. sh "sleep 3"
  31. }
  32. }
  33. }
  34. }
  35. }

运行后能够,正常启动一个agent节点则正常。

3. 测试流水线

docker

安装 docker-pipeline 插件 https://plugins.jenkins.io/docker-workflow/
未安装插件导致的报错:

  1. WorkflowScript: 3: Invalid agent type "docker" specified. Must be one of [any, label, none] @ line 3, column 9.
  2. docker {
  3. ^

docker 启动的Jenkins ,默认容器中没有docker 命令。需要配置

  1. + docker inspect -f . maven:3.6.3-jdk-8
  2. /var/jenkins_home/workspace/build-test-pipeline@tmp/durable-4746e7b2/script.sh: line 1: docker: command not found
  1. docker run -itd --name jenkins \
  2. -u root \
  3. -p 8080:8080 \
  4. -p 50000:50000 \
  5. --privileged=true \
  6. -v /data/cicd/jenkins:/var/jenkins_home \
  7. -v /var/run/docker.sock:/var/run/docker.sock \
  8. -v /usr/bin/docker:/usr/bin/docker \
  9. docker.io/jenkins/jenkins:2.277.1-lts-centos7

image, alwaysPull, args, containerPerStageRoot, customWorkspace, label, registryCredentialsId, registryUrl, reuseNode

  1. pipeline {
  2. agent {
  3. node {
  4. label "master"
  5. }
  6. }
  7. options {
  8. skipDefaultCheckout true
  9. }
  10. stages {
  11. stage('Maven Build') {
  12. agent {
  13. docker {
  14. image 'maven:3-alpine'
  15. args '-v $HOME/.m2:/root/.m2'
  16. }
  17. }
  18. steps {
  19. sh 'mvn clean package'
  20. }
  21. }
  22. }
  23. }