CenterOS ELK环境搭建
登录 / 注册
标签搜索
侧边栏壁纸
博主昵称
jupiter

  • 累计撰写 360 篇文章
  • 累计收到 121 条评论
linux使用 学习笔记

CenterOS ELK环境搭建

jupiter
jupiter
2025-06-06 / 0 评论 / 6 阅读 / 正在检测是否收录...

1.准备工作

1.1 安装jdk8(可以省略)

  • 下载安装包
wget https://mirrors.tuna.tsinghua.edu.cn/Adoptium/8/jdk/x64/linux/OpenJDK8U-jdk_x64_linux_hotspot_8u422b05.tar.gz
  • 解压并移动到目标路径
tar xzvf OpenJDK8U-jdk_x64_linux_hotspot_8u422b05.tar.gz
mv jdk8u422-b05 jdk8
mv jdk8 /software/
  • 配置环境变量
 vim ~/.bashrc
export JAVA_HOME=/software/jdk8
export PATH=$PATH:$JAVA_HOME/bin
 source ~/.bashrc
  • 验证
[root@localhost ~]# java -version

2.Elasticsearch 部署

2.1 源码部署[单节点]

下载源码包并解压

下载地址：https://www.elastic.co/cn/downloads/elasticsearch

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.14.3-linux-x86_64.tar.gz
tar xzvf elasticsearch-8.14.3-linux-x86_64.tar.gz
mkdir /data/elasticsearch
mv elasticsearch-8.14.3 /data/elasticsearch/
cd /data/elasticsearch/elasticsearch-8.14.3/

创建es启动用户

# 创建用户
useradd es
# 目录授权
chown es:es -R /data/elasticsearch/elasticsearch-8.14.3/

ES 不能用root启动，否则会出现报错

cd bin
[root@localhost bin]# ./elasticsearch
warning: ignoring JAVA_HOME=/software/jdk8; using bundled JDK
Jul 28, 2024 12:07:26 AM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
[2024-07-28T00:07:26,940][INFO ][o.e.n.NativeAccess       ] [localhost.localdomain] Using [jdk] native provider and native methods for [Linux]
[2024-07-28T00:07:26,953][ERROR][o.e.b.Elasticsearch      ] [localhost.localdomain] fatal exception while booting Elasticsearchjava.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.server@8.14.3/org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:286)
        at org.elasticsearch.server@8.14.3/org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:169)
        at org.elasticsearch.server@8.14.3/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)

See logs for more details.

ERROR: Elasticsearch did not exit normally - check the logs at /software/elasticsearch-8.14.3/logs/elasticsearch.log

ERROR: Elasticsearch died while starting up, with exit code 1

系统参数配置

#1、设置系统参数 *表示所有用户生效
echo '* soft nofile 100001' >> /etc/security/limits.conf
echo '* hard nofile 100002' >> /etc/security/limits.conf
echo '* soft nproc 100001' >> /etc/security/limits.conf
echo '* hard nproc 100002' >> /etc/security/limits.conf
 
#2、设置内存设置
echo 'vm.max_map_count=655360' >> /etc/sysctl.conf
 
#3、加载sysctl配置，执行命令
sysctl -p
# 重启生效
reboot

不配置系统参数启动会出现如下报错

[2024-07-28T02:28:31,731][ERROR][o.e.b.Elasticsearch      ] [es-node1] node validation exception
[2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch. For more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.14/bootstrap-checks.html]
bootstrap check failure [1] of [2]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.14/_file_descriptor_check.html]
bootstrap check failure [2] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.14/_maximum_map_count_check.html]
ERROR: Elasticsearch did not exit normally - check the logs at /software/elasticsearch-8.14.3/logs/es.log
[2024-07-28T02:28:31,735][INFO ][o.e.n.Node               ] [es-node1] stopping ...
[2024-07-28T02:28:31,749][INFO ][o.e.n.Node               ] [es-node1] stopped
[2024-07-28T02:28:31,750][INFO ][o.e.n.Node               ] [es-node1] closing ...
[2024-07-28T02:28:31,756][INFO ][o.e.n.Node               ] [es-node1] closed
[2024-07-28T02:28:31,758][INFO ][o.e.x.m.p.NativeController] [es-node1] Native controller process has stopped - no new native processes can be started

ERROR: Elasticsearch died while starting up, with exit code 78

修改配置文件

vim config/elasticsearch.yml
  • 修改数据和日志目录(这里可以不用修改，如果不修改，默认放在elasticsearch根目录下)
# 数据目录位置
path.data: /data/elasticsearch/data 
# 日志目录位置
path.logs: /data/elasticsearch/logs
  • 修改绑定的ip允许远程访问
#默认只允许本机访问，修改为0.0.0.0后则可以远程访问
# 绑定到0.0.0.0，允许任何ip来访问
network.host: 0.0.0.0
  • 初始化节点名称
cluster.name: es 
node.name: es-node1
cluster.initial_master_nodes: ["es-node1"]
  • 开启xpack 认证功能
# cd 到 elasticsearch文件夹下
# 创建一个证书颁发机构
#会要求输入密码直接回车即可 
#执行完成之后会在bin目录的同级目录生成一个文件elastic-stack-ca.p12
./bin/elasticsearch-certutil ca
# 为节点生成证书和私钥
#会要求输入密码直接回车即可 
#执行完成之后会在bin目录的同级目录生成一个文件elastic-certificates.p12
./bin/elasticsearch-certutil cert --ca ./elastic-stack-ca.p12
# 移动到config/certs目录下 可以手动创建
mkdir config/certs
mv *.p12 config/certs/
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

xpack.security.http.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

xpack 认证功能认证未开启会出现问题

  • 访问http://IP:9200/测试，页面无法加载，后台日志出现报错
[2024-07-28T02:51:56,319][WARN ][o.e.h.n.Netty4HttpServerTransport] [es-node1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.168.124.16:9200, remoteAddress=/192.168.124.16:40472}
[2024-07-28T02:52:05,731][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [es-node1] received plaintext traffic on an encrypted channel, closing connection Netty4TcpChannel{localAddress=/192.168.124.16:9300, remoteAddress=/192.168.124.16:57560, profile=default}
^[[B^[[B^[[B[2024-07-28T03:03:25,366][WARN ][o.e.h.n.Netty4HttpServerTransport] [es-node1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/192.168.124.16:9200, remoteAddress=/192.168.124.16:40476}

是因为ES8默认开启了 SSL 认证,解决办法

1、使用 https 发送请求,需要完成https证书配置等，暂时跳过

2、修改elasticsearch.yml配置文件将xpack.security.enabled设置为false[生产环境下不建议这么使用]

cd /software/elasticsearch-8.14.3/conf/
vim elasticsearch.yml

xpack.security.enabled: false
  • 再次重启访问访问即可正常

切换用户启动测试

# 目录授权
chown es:es -R /data/elasticsearch

# 切换用户
su es

cd /data/elasticsearch/elasticsearch-8.14.3/bin/
./elasticsearch

# -d 后台启动
━
✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  ys42G-eSmGL*jqZF7iqL


❌ Unable to generate an enrollment token for Kibana instances, try invoking `bin/elasticsearch-create-enrollment-token -s kibana`.

❌ An enrollment token to enroll new nodes wasn't generated. To add nodes and enroll them into this cluster:
• On this node:
  ⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.
  ⁃ Restart Elasticsearch.
• On other nodes:
  ⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.

image-20250606234416646

用户密码重置

# 或者之前设置过忘记了，可以重新设置密码
./bin/elasticsearch-reset-password -u elastic
./bin/elasticsearch-reset-password -u kibana

3.Kibana部署

3.1 源码部署

下载源码包并解压

下载地址：Download Kibana Free | Get Started Now | Elastic

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.14.3-linux-x86_64.tar.gz
tar xzvf kibana-8.14.3-linux-x86_64.tar.gz
mv kibana-8.14.3 /data/elasticsearch
cd /data/elasticsearch/kibana-8.14.3/

修改配置文件

vim config/kibana.yml
# 修改绑定的ip允许远程访问
server.host: "0.0.0.0"

# Kibana汉化页面
i18n.locale: "zh-CN"

# 配置 elasticsearch 登录用户
elasticsearch.username: "kibana"
elasticsearch.password: "上面设置的密码"

启动测试

# 目录授权给es用户
chown es:es -R /data/elasticsearch/kibana-8.14.3/
# 通过es用户启动
su es
cd /data/elasticsearch/kibana-8.14.3/

./bin/kibana

# 后台启动
nohup ./bin/kibana > /dev/null 2>&1 &

访问测试

http:// 172.21.58.47:5601/

4.Logstash 部署

4.1 源码部署

下载源码包并解压

下载地址：Download Logstash Free | Get Started Now | Elastic

wget https://artifacts.elastic.co/downloads/logstash/logstash-8.14.3-linux-x86_64.tar.gz
tar xzvf logstash-8.14.3-linux-x86_64.tar.gz
mv logstash-8.14.3 /software/
cd /software/logstash-8.14.3

参考资料

  1. Index of /Adoptium/8/jdk/x64/linux/ | 清华大学开源软件镜像站 | Tsinghua Open Source Mirror
  2. ELK介绍、Elasticsearch单节点部署、Elasticsearch集群部署_systemctl 管理elsearch-CSDN博客
  3. [ES错误：max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]-CSDN博客](https://blog.csdn.net/weixin_43950568/article/details/122459088)
  4. [vm.max_map_count [65530] is too low 问题解决（Windows 10、WSL 2、Docker Desktop）_容器化vm.max map count [65530] istoo low-CSDN博客](https://blog.csdn.net/Pointer_v/article/details/112395425)
  5. ELasticsearch基本使用——基础篇_elasticsearch使用-CSDN博客
  6. Elasticsearch 8.0报错：received plaintext http traffic on an https channel, closing connection_closing connection -1-CSDN博客
  7. ES 8.x 系列教程：ES 8.0 服务安装（可能是最详细的ES 8教程）-阿里云开发者社区 (aliyun.com)
  8. 【ES三周年】吊打ElasticSearch和Kibana（入门保姆级教程-2）-腾讯云开发者社区-腾讯云 (tencent.com)
  9. SpringBoot整合Logstash,实现日志统计_springboot 整合 logstash-CSDN博客
  10. Logstash 安装与部署（无坑版）-腾讯云开发者社区-腾讯云 (tencent.com)
0
暂无标签
版权属于： jupiter
本文链接： http://blog.inat.top/archives/568/
作品采用： 《 署名-非商业性使用-相同方式共享 4.0 国际 (CC BY-NC-SA 4.0) 》许可协议授权

评论 (0)

打卡
取消