部署结构

image.png
方案一：
开发环境，由于我们应用和机器都部署在一台EC2实例上。
日志平台的架构： Logstash —- > Elasticsearch —- > Kibana

logstash 安装 版本：6.2.3

  3. yum install -y unzip
  4. cd /opt/elk
  5. wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.3.zip
  6. unzip logstash-6.2.3.zip
  7. touch /opt/elk/logstash-6.2.3/test.conf
  10. echo "input {" >> /opt/elk/logstash-6.2.3/test.conf
  11. echo "   file {" >> /opt/elk/logstash-6.2.3/test.conf
  12. echo "       path => \"/usr/local/app/dev_module/nohup.out\"" >> /opt/elk/logstash-6.2.3/test.conf
  13. echo "       start_position => beginning" >> /opt/elk/logstash-6.2.3/test.conf
  14. echo "   }" >> /opt/elk/logstash-6.2.3/test.conf
  15. echo "}" >> /opt/elk/logstash-6.2.3/test.conf
  16. echo "filter {" >> /opt/elk/logstash-6.2.3/test.conf
  17. echo "}" >> /opt/elk/logstash-6.2.3/test.conf
  18. echo "output {" >> /opt/elk/logstash-6.2.3/test.conf
  19. echo "   elasticsearch {" >> /opt/elk/logstash-6.2.3/test.conf
  20. echo "   hosts => \"xxx.xxx.xxx.xxx:9200\"" >> /opt/elk/logstash-6.2.3/test.conf
  21. echo "   }" >> /opt/elk/logstash-6.2.3/test.conf
  22. echo "}" >> /opt/elk/logstash-6.2.3/test.conf
  24. # 启动
  25. nohup /opt/elk/logstash-6.2.3/bin/logstash -f /opt/elk/logstash-6.2.3/test.conf  --path.data=/opt/elk/logstash-6.2.3/logs > nohup.out 2>&1 &

ElasticSearch 安装 版本：6.4.3

  1. # 修改环境配置
  2. sed -i 's/vm.max_map_count=.*/vm.max_map_count=6553600/g' /etc/sysctl.conf
  3. /sbin/sysctl -p
  4. # 添加elsearch用户
  5. groupadd elsearch
  6. useradd elsearch -g elsearch -p elasticsearch
  8. # 安装
  9. cd /opt
  10. mkdir elk
  11. cd elk
  12. wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.3.tar.gz
  13. tar -zxvf elasticsearch-6.4.3.tar.gz
  14. chown -R elsearch:elsearch elasticsearch-6.4.3
  15. cd elasticsearch-6.4.3
  16. echo "network.host: 0.0.0.0" >> config/elasticsearch.yml
  17. echo "http.port: 9200" >> config/elasticsearch.yml
  18. echo "http.cors.enabled: true" >> config/elasticsearch.yml
  19. echo "http.cors.allow-origin: \"*\"" >> config/elasticsearch.yml
  21. su elsearch
  22. cd /opt/elk/elasticsearch-6.4.3/
  23. nohup /opt/elk/elasticsearch-6.4.3/bin/elasticsearch > nohup.out 2>&1 &
  24. tail -200f nohup.out

Kibana 安装 版本：6.4.3

  1. cd /opt/elk
  2. wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.3-linux-x86_64.tar.gz
  3. tar -zxvf kibana-6.4.3-linux-x86_64.tar.gz
  4. cd kibana-6.4.3-linux-x86_64
  6. echo "server.port: 5601" >> config/kibana.yml
  7. echo "server.host: \"0.0.0.0\"" >> config/kibana.yml
  8. echo "elasticsearch.url: \"http://localhost:9200\"" >> config/kibana.yml
  9. echo "kibana.index: \".kibana\"" >> config/kibana.yml
  11. nohup /opt/elk/kibana-6.4.3-linux-x86_64/bin/kibana  > nohup.out 2>&1 &
  12. tail -200f nohup.out
  14. http://xxx.xxx.xxx.xxx:5601/app/kibana

logstash收集、解析和转换日志
ElasticSearch 对日志建立索引
kibana 针对索引日志数据进行图形化展示，搜索。
架构缺点： logtash作用于日志收集过重，和应用服务器部署一台，过于占用资源。

方案二

日志平台的架构：Filebeat —- > Logstash —- > Elasticsearch —- > Kibana
Filebeat是日志收集的agent应用，采集日志通过异步的方式发送给Logstash，而且filebeat占用系统资源消耗少。验证启动两个软件，filebeat大概占用20M内存，logtash大概在600M+。

Filebeat 安装 版本：7.8.0

  1. cd /opt/elk
  2. wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.8.0-linux-x86_64.tar.gz
  3. tar -zxvf filebeat-7.8.0-linux-x86_64.tar.gz
  4. cd filebeat-7.8.0-linux-x86_64
  5. mv filebeat.yml filebeat.yml.bak
  6. touch filebeat.yml
  8. echo "filebeat.inputs:" >> filebeat.yml
  9. echo "- type: log" >> filebeat.yml
  10. echo "  enabled: true" >> filebeat.yml
  11. echo "  paths:" >> filebeat.yml
  12. echo "    - /usr/local/app/dev-module/nohup.out" >> filebeat.yml
  13. echo "  fields:" >> filebeat.yml
  14. echo "    module: acct" >> filebeat.yml
  15. echo "  fields_under_root: true" >> filebeat.yml
  16. echo "" >> filebeat.yml
  17. echo "output.logstash:" >> filebeat.yml
  18. echo "  hosts: ["xxx.xxx.xxx.xxx:5044"]" >> filebeat.yml
  19. echo "" >> filebeat.yml
  20. echo "filebeat.config.modules:" >> filebeat.yml
  21. echo "  path: ${path.config}/modules.d/*.yml" >> filebeat.yml
  22. echo "  reload.enabled: true" >> filebeat.yml
  25. /opt/elk/filebeat-7.8.0-linux-x86_64
  26. nohup ./filebeat -e -c filebeat.yml  > nohup.out 2>&1 &
  27. tail -200f nohup.out

Logstash 安装 版本：7.8.0

  1. cd /opt/elk
  2. wget https://artifacts.elastic.co/downloads/logstash/logstash-7.8.0.tar.gz
  3. tar -zxvf logstash-7.8.0.tar.gz
  4. touch /opt/elk/logstash-7.8.0/test.conf
  6. # 接收Filebeat的日志输入
  7. input {
  8.   beats {
  9.     port => 5044
  10.     ssl  => false
  11.   }
  12. }
  13. filter {
  14.     grok {
  15.         #match => {  "message" => "(\s*%{TIMESTAMP_ISO8601:timestamp}\s*\[.*\]\s*\[.*\]\s*%{LOGLEVEL:loglevel}\s*---\s*%{GREEDYDATA:content}\s)"  }
  16.         match => {  "message" => "(\s*%{TIMESTAMP_ISO8601:timestamp}\s*\[%{GREEDYDATA:traceId},%{GREEDYDATA:spanId}\]\s*\[.*\]\s*%{LOGLEVEL:loglevel}\s*---\s*%{GREEDYDATA:content}\s)"  }
  17.     }
  18.     ruby {
  19.         code => "event.set('collection_time', event.get('@timestamp'))"
  20.     }
  21.     date {
  22.         match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
  23.         timezone => "Asia/Shanghai"
  24.         target => "@timestamp"
  25.     }
  26.     mutate {
  27.       remove_field => [ "host","timestamp" ]
  28.     }
  29. }
  30. output {
  31.    elasticsearch {
  32.        hosts => "xxxxx:9200"
  33.        index => "Index-%{[module]}-%{+YYYY.MM.dd}" #索引名称,按年月日切分
  34.    }
  35. }
  37. cd /opt/elk/logstash-7.8.0
  38. nohup /opt/elk/logstash-7.8.0/bin/logstash -f /opt/elk/logstash-7.8.0/test.conf  --path.data=/opt/elk/logstash-7.8.0/logs > nohup.out 2>&1 &
  39. tail -200f nohup.out

ElasticSearch 安装 版本：7.8.0

  1. # 修改环境配置
  2. sed -i 's/vm.max_map_count=.*/vm.max_map_count=6553600/g' /etc/sysctl.conf
  3. /sbin/sysctl -p
  5. # 添加elsearch用户
  6. groupadd elsearch
  7. useradd elsearch -g elsearch -p elasticsearch
  9. # 安装
  10. cd /opt
  11. mkdir elk
  12. cd elk
  13. wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.8.0-linux-x86_64.tar.gz
  14. tar -zxvf elasticsearch-7.8.0-linux-x86_64.tar.gz
  15. chown -R elsearch:elsearch elasticsearch-7.8.0
  16. cd elasticsearch-7.8.0
  18. echo "network.host: 0.0.0.0" >> config/elasticsearch.yml
  19. echo "http.port: 9200" >> config/elasticsearch.yml
  20. echo "http.cors.enabled: true" >> config/elasticsearch.yml
  21. echo "http.cors.allow-origin: \"*\"" >> config/elasticsearch.yml
  22. echo "bootstrap.memory_lock: false" >> config/elasticsearch.yml
  23. echo "bootstrap.system_call_filter: false" >> config/elasticsearch.yml
  24. echo "node.name: node-1" >> config/elasticsearch.yml
  25. echo "cluster.initial_master_nodes: [\"node-1\"]" >> config/elasticsearch.yml
  27. su elsearch
  28. cd /opt/elk/elasticsearch-7.8.0/
  29. nohup /opt/elk/elasticsearch-7.8.0/bin/elasticsearch > nohup.out 2>&1 &
  30. tail -200f nohup.out

Kibana 安装 版本：7.8.0

  1. cd /opt/elk
  2. wget https://artifacts.elastic.co/downloads/kibana/kibana-7.8.0-linux-x86_64.tar.gz
  3. tar -zxvf kibana-7.8.0-linux-x86_64.tar.gz
  4. cd kibana-7.8.0-linux-x86_64
  6. echo "server.port: 5601" >> config/kibana.yml
  7. echo "server.host: \"0.0.0.0\"" >> config/kibana.yml
  8. echo "elasticsearch.hosts: [\"http://localhost:9200/\"]" >> config/kibana.yml
  9. echo "kibana.index: \".kibana\"" >> config/kibana.yml
  11. cd /opt/elk/kibana-7.8.0-linux-x86_64
  12. nohup /opt/elk/kibana-7.8.0-linux-x86_64/bin/kibana --allow-root > nohup.out 2>&1 &
  13. tail -200f nohup.out
  15. http://xxx.xxx.xxx.xxx:5601/app/kibana

扩展：

  1. # 日志格式
  2. ${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr([%X{X-B3-TraceId:-},%X{X-B3-SpanId:-}] [%thread]){cyan} %clr(%-5level) %clr(---){faint} %clr(%m){faint} %clr([%logger{50}:%line]){cyan} %n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}