A Comprehensive Guide to Installing Filebeat 9 on Ubuntu & Debian
Effectively managing and analyzing log files is a critical task for any system administrator, DevOps engineer, or security professional. As systems grow in complexity, centralizing logs becomes essential for monitoring, troubleshooting, and security analysis. This is where Filebeat, a lightweight and powerful log shipper from the Elastic Stack, excels.
This step-by-step guide will walk you through the entire process of installing and configuring Filebeat 9 on modern Ubuntu and Debian systems, such as Ubuntu 22.04 and Debian 12. By the end, you will have a fully functional Filebeat agent shipping your logs to Elasticsearch or Logstash.
Prerequisites
Before we begin, ensure you have the following:
- A running instance of Ubuntu or Debian.
- Access to a user account with
sudoor root privileges. - An active internet connection to download the necessary packages.
- The IP address or hostname of your Elasticsearch or Logstash instance.
Step 1: Add the Official Elastic Repository
To ensure you are installing the latest and most secure version of Filebeat, it’s best practice to use the official Elastic package repository. This process involves adding the repository’s GPG key to trust its packages and then adding the repository source itself.
First, import the Elastic GPG key to authenticate the packages. This security step prevents package tampering.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Next, add the repository definition to your system’s source list. This command tells your package manager where to find the Filebeat package.
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list
Step 2: Install the Filebeat Package
With the repository configured, you can now proceed with the installation.
First, update your system’s package index to refresh the list of available packages from all configured repositories, including the one you just added.
sudo apt-get update
Now, install the Filebeat package using the apt-get command.
sudo apt-get install filebeat
This command will download and install Filebeat along with its dependencies.
Step 3: Configure Filebeat to Ship Logs
After installation, you must configure Filebeat by telling it where to find your logs (inputs) and where to send them (outputs). The main configuration file is located at /etc/filebeat/filebeat.yml.
Pro Tip: Before making any changes, create a backup of the original configuration file. This allows you to easily revert if something goes wrong.
sudo cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.bak
Now, open the configuration file with your preferred text editor, like nano or vim:
sudo nano /etc/filebeat/filebeat.yml
Configuring the Output
Scroll down to the Output section. The two most common outputs are Elasticsearch and Logstash.
To send data directly to Elasticsearch, locate the output.elasticsearch: section and configure it with your cluster’s details.
# ================================== Output ===================================
# Configure the output where events will be sent.
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["your-elasticsearch-host:9200"]
# API key for authentication.
# api_key: "id:api_key"
# Username and password for basic authentication.
username: "your-elastic-user"
password: "your-password"
# Protocol - either http or https.
protocol: "https"
Security Best Practice: Avoid using the elastic superuser. Instead, create a dedicated user role in Elasticsearch with the minimum required permissions for Filebeat to write data.
Configuring the Input
Next, tell Filebeat which log files to monitor. By default, the filebeat.inputs section might be disabled. You need to enable it and specify the paths.
For example, to collect all log files from /var/log/, you can use the following configuration:
# ================================== Inputs ===================================
filebeat.inputs:
- type: filestream
id: my-filestream-id
enabled: true
paths:
- /var/log/*.log
Filebeat also offers powerful modules for popular services like Nginx, Apache, MySQL, and more. To enable a module, use the filebeat modules command. For example, to enable the system module for collecting system and authorization logs:
sudo filebeat modules enable system
Step 4: Enable and Start the Filebeat Service
Once your configuration is complete, you can start the Filebeat service. We will use systemd to manage it.
First, reload the systemd manager configuration to ensure it recognizes the Filebeat service.
sudo systemctl daemon-reload
Next, enable the Filebeat service to start automatically on boot. This is crucial for ensuring continuous log collection after a server reboot.
sudo systemctl enable filebeat
Finally, start the service immediately.
sudo systemctl start filebeat
Step 5: Verify Filebeat is Running Correctly
The final step is to verify that everything is working as expected.
Check the status of the Filebeat service to ensure it is active and running without errors.
sudo systemctl status filebeat
Look for the active (running) status in the output. If you see errors, check the Filebeat logs for clues using journalctl. This command shows you a live stream of the logs, which is helpful for real-time troubleshooting.
sudo journalctl -u filebeat -f
The most important verification step is to confirm that your logs are appearing in Kibana. Log in to your Kibana instance, navigate to the Discover tab, and you should see incoming log data from your server.
By following these steps, you have successfully deployed a robust log shipping agent that forms a foundational part of a centralized logging and observability strategy.
Source: https://kifarunix.com/install-filebeat-9-on-ubuntu-debian/
