A Step-by-Step Guide to Installing and Configuring Filebeat on CentOS 8
In modern IT infrastructure, managing and analyzing log files from various sources is critical for monitoring, troubleshooting, and security. The Elastic Stack (often called the ELK Stack) provides a powerful suite for this, and Filebeat is a cornerstone of its data collection process. As a lightweight, open-source log shipper, Filebeat efficiently forwards log data from your servers to a central location like Elasticsearch or Logstash.
This comprehensive guide will walk you through the entire process of installing and configuring Filebeat on a CentOS 8 server, enabling you to start centralizing your logs effectively.
Prerequisites
Before we begin, ensure you have the following:
- A running CentOS 8 server.
- Access to a user with
sudoor root privileges. - An operational Elasticsearch or Logstash instance to receive the logs.
Step 1: Import the Elastic GPG Key
First, you need to import the official Elastic GPG key. This is a crucial security step that allows your system to verify the authenticity of the Filebeat package you are about to install, ensuring it hasn’t been tampered with.
Execute the following command in your terminal:
sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
Step 2: Add the Elastic Repository
Your system’s package manager (dnf) needs to know where to find the Filebeat software. We can do this by creating a new repository file.
Create a file named elastic.repo in the /etc/yum.repos.d/ directory:
sudo vi /etc/yum.repos.d/elastic.repo
Add the following content to the file. This configuration points to the official Elastic repository for version 7.x of the stack, which is compatible with most modern deployments.
[elastic-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
Save and exit the editor. Your system is now configured to pull packages directly from the Elastic source.
Step 3: Install Filebeat
With the repository in place, installing Filebeat is as simple as running a single command. The dnf package manager will handle the download and installation process.
sudo dnf install filebeat -y
This command installs the Filebeat binary and its associated configuration files. However, the service will not be started or enabled automatically.
Step 4: Configure Filebeat
This is the most critical part of the setup, where you tell Filebeat what logs to collect and where to send them. The main configuration file is located at /etc/filebeat/filebeat.yml.
Open the file for editing:
sudo vi /etc/filebeat/filebeat.yml
Here, we will configure two main sections: inputs and outputs.
A. Configure Inputs
The filebeat.inputs section defines which files to monitor. By default, it may be disabled. You need to enable it and specify the paths to your log files.
For example, to collect system logs from /var/log/messages, find the filebeat.inputs section and modify it to look like this:
#=========================== Filebeat inputs =============================
filebeat.inputs:
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched.
paths:
- /var/log/messages
- /var/log/secure
# - /var/log/*.log
You can add multiple paths to collect logs from different locations, including application logs.
B. Configure Outputs
The output section determines the destination for your logs. You must configure one of the two primary options: Elasticsearch or Logstash.
Option 1: Send Logs Directly to Elasticsearch
If you want to send data directly to Elasticsearch, find the output.elasticsearch section. Comment out the output.logstash section by adding a # at the beginning of each line, and configure your Elasticsearch details.
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["your-elasticsearch-host:9200"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "your-user"
#password: "your-password"
Remember to replace your-elasticsearch-host with the actual IP address or hostname of your Elasticsearch server.
Option 2: Send Logs to Logstash for Processing
If you use Logstash for additional parsing or enrichment, configure the output.logstash section instead. Make sure the output.elasticsearch section is completely commented out.
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["your-logstash-host:5044"]
Replace your-logstash-host with the IP address or hostname of your Logstash server. The default port for Beats input in Logstash is 5044.
Step 5: Enable and Start the Filebeat Service
After saving your configuration, you are ready to start the Filebeat service. First, it’s a good practice to test your configuration file for syntax errors.
sudo filebeat test config -e
If you see “Config OK”, you can proceed.
Now, enable Filebeat to start automatically on system boot and then start the service immediately.
sudo systemctl enable filebeat
sudo systemctl start filebeat
Step 6: Verify Filebeat is Running
Finally, verify that the service is active and running without errors.
Check the service status:
sudo systemctl status filebeat
You should see an “active (running)” status. If there are any issues, you can inspect the logs for more detailed error messages:
sudo journalctl -u filebeat
At this point, log data should be flowing from your CentOS 8 server to your Elasticsearch or Logstash instance. You can confirm this by checking for new documents in Kibana or observing the output from Logstash.
Key Security and Performance Tips
- Principle of Least Privilege: If possible, create a dedicated user for Filebeat that has read-only access to the necessary log files, rather than running it as root.
- Secure Communication: For production environments, always use HTTPS/TLS to encrypt data in transit between Filebeat and Elasticsearch/Logstash. This can be configured under the
sslsettings in yourfilebeat.ymloutput section. - Use Filebeat Modules: For common applications like Nginx, Apache, or MySQL, Filebeat offers pre-configured modules that automatically handle parsing and dashboard creation in Kibana. You can enable them with the
filebeat modules enable <module_name>command.
By following these steps, you have successfully deployed a powerful and efficient log shipping agent, taking a significant step towards a robust, centralized logging architecture.
Source: https://kifarunix.com/install-and-configure-filebeat-on-centos-8/
