How to Install and Configure Logstash 9 on Ubuntu & Debian: A Step-by-Step Guide
Logstash is a powerful, open-source data processing pipeline that allows you to collect data from a multitude of sources, transform it, and send it to your preferred destination. As a core component of the Elastic Stack (ELK Stack), it is essential for centralizing logs, parsing metrics, and enriching data before indexing it in Elasticsearch.
This guide provides a comprehensive walkthrough for installing and configuring Logstash 9 on modern Ubuntu (22.04, 20.04) and Debian (12, 11) systems.
Prerequisites: Verifying Your Java Installation
Before you begin, it’s crucial to understand that Logstash is a Java application. Logstash 9 requires a compatible Java Development Kit (JDK), specifically version 17 or 21. You can verify your installed Java version with the following command:
java -version
If you don’t have a suitable JDK installed, you can easily install OpenJDK 17, a widely supported version, using your system’s package manager.
sudo apt update
sudo apt install openjdk-17-jdk -y
Once installed, confirm the version again to ensure your system is ready.
Step 1: Add the Elastic APT Repository
To ensure you receive official and up-to-date versions of Logstash, you must configure your system to use the official Elastic repository. This process involves two main actions: importing the GPG key and adding the repository source.
First, download and install the Public Signing Key for the Elastic repository. This key is used by apt to verify that the packages you are downloading are authentic and have not been tampered with.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Next, add the repository definition to your system’s sources list. This command creates a new source file that points apt to the official Logstash 9 repository.
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list
Step 2: Install Logstash on Your System
With the repository successfully configured, you can now install Logstash. First, refresh your system’s package list to include the newly added Elastic repository.
sudo apt update
Now, proceed with the Logstash installation command.
sudo apt install logstash
This command will download and install the Logstash package along with its dependencies. The Logstash service will be set up but will not be started or enabled by default.
Step 3: Create Your First Logstash Configuration
Logstash is useless without a configuration file that defines its data pipeline. This pipeline consists of three main sections: inputs, filters, and outputs.
- Input: Where data comes from (e.g., files, syslog, beats).
- Filter: How data is processed and transformed (e.g., parsing with grok, adding fields).
- Output: Where the processed data is sent (e.g., Elasticsearch, a file, the console).
Configuration files are stored in the /etc/logstash/conf.d/ directory. Let’s create a simple test configuration that takes input from your terminal (stdin) and outputs it to the console (stdout).
Create a new configuration file:
sudo nano /etc/logstash/conf.d/01-basic-pipeline.conf
Add the following content to the file. This configuration is excellent for verifying that your Logstash installation is working correctly.
# /etc/logstash/conf.d/01-basic-pipeline.conf
# A basic pipeline for testing purposes.
# Input from the command line, output to the command line.
input {
stdin { }
}
filter {
# Filters are optional, so we'll leave this empty for now.
}
output {
stdout {
codec => rubydebug
}
}
Save and close the file (press CTRL+X, then Y, then Enter in nano). The rubydebug codec provides detailed and structured output, which is helpful for debugging.
Step 4: Start and Enable the Logstash Service
Now that Logstash is installed and configured, it’s time to manage its service using systemd.
First, reload the systemd daemon to ensure it recognizes the new Logstash service file.
sudo systemctl daemon-reload
Next, enable the Logstash service to start automatically on boot. This is a critical step for production environments.
sudo systemctl enable logstash.service
Finally, start the Logstash service.
sudo systemctl start logstash.service
Logstash can take a moment to initialize as it starts the Java Virtual Machine (JVM). You can check its status to confirm it is running correctly.
sudo systemctl status logstash.service
Look for the line Active: active (running) in the output. If you see any errors, you can investigate the logs for more details using the command: sudo journalctl -u logstash.service.
Essential Management and Security Tips
- Configuration Validation: Before restarting Logstash with a new configuration, always test your configuration files to avoid service failures. You can do this with the following command:
bash
sudo -u logstash /usr/share/logstash/bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/
- Managing Memory (JVM Heap): For production workloads, you will need to adjust the JVM heap size. By default, Logstash allocates 1GB. This is configured in the
/etc/logstash/jvm.optionsfile. Modify the-Xms(initial size) and-Xmx(maximum size) settings according to your server’s available RAM and workload. For example:
-Xms4g
-Xmx4g
- Checking Logs: The primary Logstash logs are located at
/var/log/logstash/logstash-plain.log. This file is your first stop for troubleshooting pipeline errors or performance issues. - Applying Changes: After modifying a configuration file, you must restart the Logstash service for the changes to take effect.
bash
sudo systemctl restart logstash.service
By following these steps, you have successfully installed, configured, and launched a Logstash 9 instance. You are now ready to build more complex pipelines to collect, enrich, and visualize your data with the full power of the Elastic Stack.
Source: https://kifarunix.com/install-logstash-9-on-ubuntu-debian/
