es压测
参考链接:
https://blog.csdn.net/BearStarX/article/details/103038171
https://blog.csdn.net/BearStarX/article/details/103047725
https://esrally.readthedocs.io/en/stable/install.html
https://cloud.tencent.com/developer/article/1595367
集群主机配置:
mem 1536G
cpu 80c
disk 12*1.8T ssd
节点信息:共3台机器,每台上面4个实例
10.118.25.27
10.118.25.28
10.118.25.38
压测工具:esrally
一. esrally环境准备:esrally我是单独部署在一台机器,和es集群分割开
1.将es5.4.1按照生产环境配置部署ok
2.部署esrally环境:
0>
yum install bzip2-devel bzip2
1> 安装python3.5
tar -zxf Python-3.5.7.tgz
cd Python-3.5.7
./configure —prefix=/usr/local/python3.5
make
make install
ln -s /usr/local/python3.5/bin/python3.5 /usr/bin/python3.5
ln -s /usr/bin/python3.5 /usr/bin/python3
python3 —version
2> 安装pip3
tar -zxf pip-9.0.3.tar.gz
cd pip-9.0.3
python3 setup.py install
ln -s /usr/local/python3.5/bin/pip3 /usr/bin/pip3
pip3 —version
3> 设置py3独立环境(此步骤可有可无)
pip3 install virtualenv
ln -s /usr/local/python3.5/bin/virtualenv /usr/bin/virtualenv
mkdir python3.5_virtualenv
cd python3.5_virtualenv/
virtualenv —python=python3 .
source ./bin/activate
4> 安装esrally
pip3 install esrally
5> 装git1.9+
安装依赖
yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel asciidoc
yum install gcc perl-ExtUtils-MakeMaker
卸载旧版本
yum remove git
安装新版本
tar -xvf git-2.23.0.tar.xz
cd git-2.23.0/
make prefix=/usr/local/git all
make prefix=/usr/local/git install
echo “export PATH=$PATH:/usr/local/git/bin” >> /etc/profile
source /etc/profile
版本验证
[root@localhost ~]# git —version
git version 2.23.0
6> 装jdk8
二. 离线数据源准备:
1、下载数据包allCountries.zip
2、解压:allCountries.txt
3、txt转json(脚本参考官网)
python3 toJSON.py > documents.json
4、创建index.json (数据相关的)
5、创建track.json (定义要压哪些指标)delete-index,create-index,cluster-healt,bulk,force-merge,index-append,index-stats,default,query-match-all
根据实际情况修改document-count、uncompressed-bytes值
6、验证
esrally list tracks —track-path=~/rally-tracks/tutorial
三.压测5.4.1
1.按照生产配置部署5.4.1集群OK
2.开始压测
esrally —track-path=~/rally-tracks/tutorial —offline —pipeline=benchmark-only —target-hosts=10.118.25.27:9200
—report-file=/home/elastic/5.4.1_result.csv —report-format=csv
四.压测7.6.0
1.按照生产配置部署7.6.0集群OK
2.开始压测
esrally —client-options=”basic_auth_user:’elastic’,basic_auth_password:’Bdp1@elastic’” —track-path=~/rally-tracks/tutorial —offline
—pipeline=benchmark-only —target-hosts=10.118.25.27:9200 —report-file=/home/elastic/7.6_result.csv —report-format=csv
结果比较
esrally list races
esrally compare —baseline=xxxxx —contender=xxxx