二、ES的安装

2.1 安装Java

运行ES需要JAVA环境
各个版本对Java的以来

1. ES5需要Java8以上的版本
2. ES6.5开始支持Java 11
3. ES7.0开始内置Java环境
4. https://www.elastic.co/cn/support/matrix#matrix_jvm

   2.2 获取ES安装包

下载https://www.elastic.co/cn/downloads/past-releases#elasticsearch

这个文档写的是依据7.1

2.2.1 目录结构

2.2.2 JVM配置

• 修改JVM - - config/jvm.options
  • 7.1 默认的是1GB
• 配置建议
  • Xmx和Xms设置一样
  • Xmx不要超过机器内存的50%
  • 不要超过30GB https://www.elastic.co/cn/blog/a-heap-of-trouble

2.2.3 启动 bin/elasticsearch

访问 http://localhost:9200/ 响应

{
"name": "192.168.0.103",
"cluster_name": "elasticsearch",
"cluster_uuid": "DtJanm4HRyin88L4HTO5_A",
"version": {
"number": "7.1.0",
"build_flavor": "default",
"build_type": "tar",
"build_hash": "606a173",
"build_date": "2019-05-16T00:43:15.323135Z",
"build_snapshot": false,
"lucene_version": "8.0.0",
"minimum_wire_compatibility_version": "6.8.0",
"minimum_index_compatibility_version": "6.0.0-beta1"
},
"tagline": "You Know, for Search"

• 查看已安装插件 bin/elasticsearch-plugin list
• 安装插件 bin/elasticsearch-plugin install analysis-icu 一个国际化分词插件
• 访问 http://localhost:9200/_cat/plugins 查看插件是否安装成功

2.2.4 在本机启动ES集群

bin/elasticsearch -E node.name=node1 -E cluster.name=geektime -E path.data=node1_data -d
bin/elasticsearch -E node.name=node2 -E cluster.name=geektime -E path.data=node2_data -d
bin/elasticsearch -E node.name=node3 -E cluster.name=geektime -E path.data=node3_data -d

查看集群是否启动http://localhost:9200/_cat/nodes

2.4 安装Kibana

Kibana 7.10.0

# 启动
bin/kibana -d # 后台启动
# 汉化
# 只需要在配置文件 kibana.yml 中加入
i18n.locale: "zh-CN"

访问http://localhost:5601/app/kibana

2.4.1 dev_tool

http://localhost:5601/app/kibana#/dev_tools/console?_g=(refreshInterval:(pause:!f,value:900000),time:(from:now-7d,to:now),time:(from:now-7d,to:now))) ,可以在这个界面内直接查看访问ES里的数据

一些快捷键

cmd + / （查看API帮助文档）

cmd + option + 0

cmd + option +shift + 0

2.4.2 Kibana Plugins

Known Plugins | Kibana Guide [7.1] | Elastic

查询所有Kibana支持的插件

bin/kibana-plugin install plugin_location #安装插件
bin/kibana-plugin list # 查看所有插件
bin/kibana remove # 删除插件

安装logtrail

./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.1.0-0.1.31.zip

2.4.3 在Docker安装ElasticSerach

Compose: https://docs.docker.com/compose/install/

docker-compose.yml

1. 放到指定目录下然后执行docker-compose up 启动

如果执行失败请参考 https://www.elastic.co/guide/en/elasticsearch/reference/7.2/docker.html#docker-cli-run-prod-mode
MACOS :stty -echo -icanon && nc -U ~/Library/Containers/com.docker.docker/Data/debug-shell.sock && stty sane # ls -al /var/lib/docker/overlay2/

1. http://localhost:9000/#/connect 可以查看集群状态【cerebro】

• 安装docker https://www.docker.com/products/docker-desktop
• 安装 docker-compose https://docs.docker.com/compose/install/
• 如何创建自己的Docker Image - https://www.elastic.co/cn/blog/how-to-make-a-dockerfile-for-elasticsearch
• 如何在为docker image安装 Elasticsearch 插件 - https://www.elastic.co/cn/blog/elasticsearch-docker-plugin-management
• 如何设置 Docker 网络 - https://www.elastic.co/cn/blog/docker-networking
• Cerebro 源码 https://github.com/lmenezes/cerebro
• 一个开源的 ELK（Elasticsearch + Logstash + Kibana） docker-compose 配置 https://github.com/deviantony/docker-elk
• Install Elasticsearch with Docker https://www.elastic.co/guide/en/elasticsearch/reference/7.2/docker.html

2.5 安装Logstash

Logstash 7.1.0

vim bin/logstash.conf
==================
input {
  file {
    path => "/Users/weibin/personal/Learn/applications/ElasticStack/logstash-7.1.0/bin/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }
  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }
  mutate {
    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }
  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }
}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

启动

sudo ./logstash -f logstash.conf

movies.csv