1、前置环境

1.1下载地址

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-7.10.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.10.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.0-linux-x86_64.tar.gz

1.2 系统优化

useradd elastic
#增加最大文件打开数
echo "* soft nofile 65536" >> /etc/security/limits.conf
echo "* hard nofile 65535" >> /etc/security/limits.conf
#增加最大进程数
echo "* soft nproc 65536" >> /etc/security/limits.conf
#增加最大内存映射数
echo "vm.max_map_count=262144" >> /etc/sysctl.conf

sysctl -p
#退出当前会话，重新连接服务器

2、部署elasticsearch

2.1 解压elasticsearch

tar xf elasticsearch-7.10.0-linux-x86_64.tar.gz -C /data
mv /data/elasticsearch-7.10.0 /data/elasticsearch
mkdir -p /data/elasticsearch/data
chown -R elastic:elastic /data/elasticsearch

2.2 编辑yml配置文件

vim /data/elasticsearch/config/elasticsearch.yml

cluster.name: dev-elk
node.name: ELK
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
#单节点模式，集群模式为cluster.initial_master_nodes: ["172.16.10.132","172.16.10.xxx","172.16.10.xxx"] 
cluster.initial_master_nodes: ["172.16.10.132"]
http.cors.enabled: true
http.cors.allow-origin: "*"

2.3 启动elasticsearch

su elastic -c "/data/elasticsearch/bin/elasticsearch &"

2.4 验证elasticsearch

3、部署logstash

3.1 解压logstash

tar xf logstash-7.10.0-linux-x86_64.tar.gz -C /data/
mv /data/logstash-7.10.0 /data/logstash
mkdir -p /data/logstash/data/sincedb
cp /data/logstash/config/logstash-sample.conf /data/logstash/config/logstash.conf 

3.2 编辑conf配置文件

vim /data/logstash/config/logstash.conf

input {
  file {
    #测试监听es的日志
    path => "/data/elasticsearch/logs/*.log"
    #从文件末尾开始，只读取新增的内容（适合已经运行的日志）
    start_position => "beginning"
    sincedb_path => "/data/logstash/data/sincedb"
  }
}

output {
  elasticsearch {
    hosts => ["http://172.16.10.132:9200"]
    index => "dev-logs-%{+YYYY.MM.dd}"
    #user => "elastic"
    #password => "changeme"
  }
}

3.3 启动logstash

/data/logstash/bin/logstash -f /data/logstash/config/logstash.conf

3.4 验证logstash

正常启动

验证索引

curl -X GET "http://172.16.10.132:9200/_cat/indices?v"

#可以看到日志索引已经成功生成了

4、部署kibana

4.1 解压kibana

tar xf kibana-7.10.0-linux-x86_64.tar.gz -C /data/
mv /data/kibana-7.10.0-linux-x86_64 /data/kibana

4.2 编辑yml配置文件

vim /data/kibana/config/kibana.yml

server.port: 5601
server.host: "172.16.10.132"
elasticsearch.hosts: ["http://172.16.10.132:9200"]
kibana.index: ".kibana"
i18n.locale: "zh-CN"

4.3 启动kibana

/data/kibana/bin/kibana --allow-root &

4.4 验证kibana

浏览器访问

http://172.16.10.132:5601/

5、验证采集日志索引

5.1 创建索引模式

5.2 选择可视化索引

此时，已经可以看到采集的elasticsearch所有日志

5.3 比对日志一致性

写入一条任意文本

echo "test txt" >> /data/elasticsearch/logs/dev-elk.log

点击kibana刷新