1. 准备
kubectl versiondocker versionflanneld --versionhelm version
2. 安装 mysql 集群
2.1 需要的镜像
docker.io/mysql:latest
docker.io/ist0ne/xtrabackup:latest
2.2 准备后端存储
由于mysql的helm包需要使用pv，而开源的k8s集群默认没有配置pv后端存储，因此需要手动添加nfs服务器(如何搭建nfs服务器请参考百度)为后端存储
创建 pv 的 yaml 文件. 这里只是声明一下 PV 的地址
[root@master hrr]# cat pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv003
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce #accessModes需要和pvc的模式一样
nfs:
path: /nfsdata/pv003
server: 192.168.222.129
[root@docker hrr]# kubectl create -f pv.yml
persistentvolume "pv003" created
[root@master hrr]# kubectl get pv #Available表示没有被PVC使用
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
pv001 10Gi RWO Retain Available 26s
pv002 10Gi RWO Retain Available 14s
pv003 10Gi RWO Retain Available 3s
[root@master hrr]# kubectl get pv #Bound表示被PVC使用
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
pv001 10Gi RWO Retain Bound default/data-mysql-mysqlha-0 32s
pv002 10Gi RWO Retain Bound default/data-mysql-mysqlha-1 17s
pv003 10Gi RWO Retain Bound default/data-mysql-mysqlha-2 4s
2.3 准备docker image
在安装mysql的helm包时需要mysql与xtrabackup的docker镜像。我并不知道 xtrabackup 是干啥的
[root@docker docker]# docker pull docker.io/ist0ne/xtrabackup #下载xtrabackup镜像
2.4 安装 mysql helm 包
helm install mysql apphub/mysqlha
NOTES:
The MySQL cluster is comprised of 3 MySQL pods: 1 master and 2 slaves. Each instance is accessible within the cluster through:
<pod-name>.mysql-mysqlha
`mysql-mysqlha-0.mysql-mysqlha` is designated as the master and where all writes should be executed against. Read queries can be executed against the `mysql-mysqlha-readonly` service which distributes connections across all MySQL pods.
To connect to your database:
1. Obtain the root password:
kubectl get secret --namespace default mysql-mysqlha -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo
2. Run a pod to use as a client:
kubectl run mysql-client --image=mysql:5.7.13 -it --rm --restart='Never' --namespace default -- /bin/sh
3. To connect to Master service (read/write):
mysql -h mysql-mysqlha-0.mysql-mysqlha -u root -p
4. To connect to slave service (read-only):
mysql -h mysql-mysqlha-readonly -u root -p
安装完后，在本地会有 mysql 的 helm 包:
[root@master archive]# pwd
/root/.helm/cache/archive
[root@master archive]# ls
guestbook-0.2.0.tgz mysqlha-1.0.0.tgz
[root@master archive]# tar -tvf mysqlha-1.0.0.tgz
-rwxr-xr-x 0/0 378 2019-08-22 13:20 mysqlha/Chart.yaml
-rwxr-xr-x 0/0 2561 2019-08-22 13:20 mysqlha/values.yaml
-rwxr-xr-x 0/0 1216 2019-08-22 13:20 mysqlha/templates/NOTES.txt
-rwxr-xr-x 0/0 961 2019-08-22 13:20 mysqlha/templates/_helpers.tpl
-rwxr-xr-x 0/0 756 2019-08-22 13:20 mysqlha/templates/configmap.yaml
-rwxr-xr-x 0/0 967 2019-08-22 13:20 mysqlha/templates/secret.yaml
-rwxr-xr-x 0/0 10905 2019-08-22 13:20 mysqlha/templates/statefulset.yaml
-rwxr-xr-x 0/0 1203 2019-08-22 13:20 mysqlha/templates/svc.yaml
-rwxr-xr-x 0/0 56 2019-08-22 13:20 mysqlha/OWNERS
-rwxr-xr-x 0/0 6724 2019-08-22 13:20 mysqlha/README.md
将 mysqlha-1.0.0.tgz 解压，查看文件内容:
[root@master mysqlha]# ls
Chart.yaml OWNERS README.md templates values.yaml
[root@master mysqlha]# cat README.md
# MySQL - Single Master, Multiple Slaves
[MySQL](https://MySQL.org) is one of the most popular database servers in the world. Notable users include Wikipedia, Facebook and Google.
## Introduction
This chart bootstraps a single master and multiple slave MySQL deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. Largely inspired by this [tutorial](https://kubernetes.io/docs/tutorials/stateful-application/run-replicated-stateful-application/), further work was made to 'production-ize' the example.
## Prerequisites
- Kubernetes 1.6+
- PV provisioner support in the underlying infrastructure
## Installing the Chart
To install the chart with the release name `my-release`:
$ helm install --name my-release <helm-repo>/mysqlha
The command deploys MySQL cluster on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation.
### Uninstall
To uninstall/delete the `my-release` deployment:
$ helm delete my-release
## Configuration
The following table lists the configurable parameters of the MySQL chart and their default values.
| Parameter | Description | Default |
| ----------------------------------------- | ------------------------------------------------- | -------------------------------------- |
| `mysqlImage` | `mysql` image and tag. | `mysql:5.7.13` |
| `xtraBackupImage` | `xtrabackup` image and tag. | `gcr.io/google-samples/xtrabackup:1.0` |
| `imagePullPolicy` | Image pull policy. | `IfNotPresent` |
| `nameOverride` | `String to partially override mysqlha.fullname template with a string (will prepend the release name)` | `nil` |
| `fullnameOverride` | `String to fully override mysqlha.fullname template with a string` | `nil` |
| `replicaCount` | Number of MySQL replicas | 3 |
| `mysqlRootPassword` | Password for the `root` user. | Randomly generated |
| `mysqlUser` | Username of new user to create. | `nil` |
| `mysqlPassword` | Password for the new user. | Randomly generated |
| `mysqlReplicationUser` | Username for replication user | `repl` |
| `mysqlReplicationPassword` | Password for replication user. | Randomly generated |
| `mysqlDatabase` | Name of the new Database to create | `nil` |
| `configFiles.master.cnf` | Master configuration file | See `values.yaml` |
| `configFiles.slave.cnf` | Slave configuration file | See `values.yaml` |
| `podAnnotations` | Pod annotations | `nil` |
| `schedulerName` | Name of the k8s scheduler (other than default) | `nil` |
| `persistence.enabled` | Create a volume to store data | true |
| `persistence.size` | Size of persistent volume claim | 10Gi |
| `persistence.storageClass` | Type of persistent volume claim | `nil` |
| `persistence.accessModes` | Persistent volume access modes | `[ReadWriteOnce]` |
| `persistence.annotations` | Persistent volume annotations | `{}` |
| `resources` | CPU/Memory resource requests/limits | Memory: `128Mi`, CPU: `100m` |
| `metrics.enabled` | Start a side-car prometheus exporter | false |
| `metrics.image` | Exporter image | `prom/mysqld-exporter` |
| `metrics.imageTag` | Exporter image | `v0.10.0` |
| `metrics.imagePullPolicy` | Exporter image pull policy | `IfNotPresent` |
| `metrics.resources` | Exporter resource requests/limit | See `values.yaml` |
| `metrics.livenessProbe.initialDelaySeconds` | Delay before metrics liveness probe is initiated | 15 |
| `metrics.livenessProbe.timeoutSeconds` | When the probe times out | 5 |
| `metrics.readinessProbe.initialDelaySeconds` | Delay before metrics readiness probe is initiated | 5 |
| `metrics.readinessProbe.timeoutSeconds` | When the probe times out | 1 |
Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
## Persistence
The [MySQL](https://hub.docker.com/_/mysql/) image stores the MySQL data and configurations at the `/var/lib/mysql` path of the container.
By default persistence is enabled, and a PersistentVolumeClaim is created and mounted in that directory. As a result, a persistent volume will need to be defined:
# https://kubernetes.io/docs/user-guide/persistent-volumes/#azure-disk
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/azure-disk
parameters:
skuName: Premium_LRS
location: westus
In order to disable this functionality you can change the values.yaml to disable persistence and use an emptyDir instead.
查看 values.yaml 文件:
[root@master mysqlha]# cat values.yaml
## mysql image version
## ref: https://hub.docker.com/r/library/mysql/tags/
##
mysqlImage: mysql:5.7.13 #前面下载的mysql镜像
xtraBackupImage: registry.cn-hangzhou.aliyuncs.com/kubeapps/gcr-xtrabackup:1.0 #前面下载的xtrabackup镜像
## Specify an imagePullPolicy (Required)
## It's recommended to change this to 'Always' if the image tag is 'latest'
## ref: http://kubernetes.io/docs/user-guide/images/#updating-images
##
imagePullPolicy: IfNotPresent
## String to partially override orangehrm.fullname template (will maintain the release name)
##
# nameOverride: ""
## String to fully override orangehrm.fullname template
##
# fullnameOverride: ""
mysqlha:
replicaCount: 3
## Password for MySQL root user
##
# mysqlRootPassword: ## Default: random 10 character string
## Username/password for MySQL replication user
##
mysqlReplicationUser: repl
# mysqlReplicationPassword:
## Create a database user
##
# mysqlUser:
# mysqlPassword: ## Default: random 10 character string
## Allow unauthenticated access, uncomment to enable
##
# mysqlAllowEmptyPassword: true
## Create database with name and grant all permissions to user on startup, if needed
# mysqlDatabase:
## Configuration files for the master and slaves
##
configFiles:
master.cnf: |
# Apply this config only on the master.
[mysqld]
log-bin
skip_name_resolve
slave.cnf: |
# Apply this config only on slaves.
[mysqld]
super-read-only
skip_name_resolve
podAnnotations: {}
## Use an alternate scheduler, e.g. "stork".
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
# schedulerName:
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
enabled: true
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, azure-disk on
## Azure, standard on GKE, AWS & OpenStack)
##
# storageClass: "-"
accessModes:
- ReadWriteOnce
size: 10Gi
annotations: {}
resources:
requests:
cpu: 100m
memory: 128Mi
metrics:
enabled: false
image: prom/mysqld-exporter
imageTag: v0.10.0
annotations: {}
livenessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
readinessProbe:
initialDelaySeconds: 5
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 100Mi
2.5 查看mysql的有状态应用、pod、svc、pvc等k8s资源
[root@master hrr]# kubectl get statefulset
NAME DESIRED CURRENT AGE
mysql-mysqlha 3 1 18s
[root@master hrr]# kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.254.0.1 <none> 443/TCP 4d
mysql-mysqlha None <none> 3306/TCP 27s
mysql-mysqlha-readonly 10.254.54.188 <none> 3306/TCP 27s
[root@master hrr]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-mysqlha-0 2/2 Running 0 45s
mysql-mysqlha-1 2/2 Running 0 24s
mysql-mysqlha-2 1/2 Running 0 4s
[root@master hrr]# kubectl get pv
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
pv001 10Gi RWO Retain Bound default/data-mysql-mysqlha-0 2m
pv002 10Gi RWO Retain Bound default/data-mysql-mysqlha-2 2m
pv003 10Gi RWO Retain Bound default/data-mysql-mysqlha-1 1m
[root@master hrr]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES AGE
data-mysql-mysqlha-0 Bound pv001 10Gi RWO 51s
data-mysql-mysqlha-1 Bound pv003 10Gi RWO 51s
data-mysql-mysqlha-2 Bound pv002 10Gi RWO 51s
[root@master hrr]# kubectl get configmap
NAME DATA AGE
mysql-mysqlha 4 54s
[root@master hrr]# kubectl get secret
NAME TYPE DATA AGE
mysql-mysqlha Opaque 2 51m
mysql.v1 helm.sh/release 1 51m
[root@master hrr]# kubectl get pod -owide
NAME READY STATUS RESTARTS AGE IP NODE
mysql-mysqlha-0 2/2 Running 0 1m 172.17.93.2 node3
mysql-mysqlha-1 2/2 Running 0 41s 172.17.78.2 node2
mysql-mysqlha-2 2/2 Running 0 21s 172.17.86.2 master
查看pvc里的数据
[root@master hrr]# ll /nfsdata/pv001
total 4
drwxr-xr-x 6 polkitd input 4096 Aug 26 03:54 mysql
[root@master hrr]# ll /nfsdata/pv001/mysql/
total 177164
-rw-r----- 1 polkitd input 56 Aug 26 03:54 auto.cnf
-rw-r----- 1 polkitd input 3091158 Aug 26 03:54 binlog.000001
-rw-r----- 1 polkitd input 155 Aug 26 03:54 binlog.000002
-rw-r----- 1 polkitd input 32 Aug 26 03:54 binlog.index
-rw------- 1 polkitd input 1676 Aug 26 03:54 ca-key.pem
-rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 ca.pem
-rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 client-cert.pem
-rw------- 1 polkitd input 1680 Aug 26 03:54 client-key.pem
-rw-r----- 1 polkitd input 5419 Aug 26 03:54 ib_buffer_pool
-rw-r----- 1 polkitd input 12582912 Aug 26 03:54 ibdata1
-rw-r----- 1 polkitd input 50331648 Aug 26 03:54 ib_logfile0
-rw-r----- 1 polkitd input 50331648 Aug 26 03:54 ib_logfile1
-rw-r----- 1 polkitd input 12582912 Aug 26 03:55 ibtmp1
drwxr-x--- 2 polkitd input 187 Aug 26 03:54 #innodb_temp
drwxr-x--- 2 polkitd input 143 Aug 26 03:54 mysql
-rw-r----- 1 polkitd input 29360128 Aug 26 03:54 mysql.ibd
drwxr-x--- 2 polkitd input 8192 Aug 26 03:54 performance_schema
-rw------- 1 polkitd input 1680 Aug 26 03:54 private_key.pem
-rw-r--r-- 1 polkitd input 452 Aug 26 03:54 public_key.pem
-rw-r--r-- 1 polkitd input 1112 Aug 26 03:54 server-cert.pem
-rw------- 1 polkitd input 1680 Aug 26 03:54 server-key.pem
drwxr-x--- 2 polkitd input 28 Aug 26 03:54 sys
-rw-r----- 1 polkitd input 12582912 Aug 26 03:54 undo_001
-rw-r----- 1 polkitd input 10485760 Aug 26 03:54 undo_002
若有收获，就点个赞吧
0 人点赞
