Step-by-Step Guide: Installing the ELK Stack 9.x on Ubuntu 24.04
The ELK Stack—comprising Elasticsearch, Logstash, and Kibana—is a powerful, open-source platform for searching, analyzing, and visualizing log data in real-time. Whether you’re managing system logs, application metrics, or security events, mastering the ELK Stack is a critical skill for any DevOps engineer, system administrator, or developer.
This comprehensive guide will walk you through the entire process of installing and configuring the latest ELK Stack (version 9.x) on a fresh Ubuntu 24.04 server.
Prerequisites
Before we begin, ensure your system meets the following requirements:
- An Ubuntu 24.04 server.
- At least 4GB of RAM and 2 CPU cores are recommended for a basic setup.
- Root or sudo privileges to install packages and modify configuration files.
- Java Development Kit (JDK). Elasticsearch is built on Java and requires it to run. We will install OpenJDK, a trusted open-source implementation.
First, let’s update your package index and install the required Java version:
sudo apt update
sudo apt install openjdk-21-jdk -y
You can verify the Java installation with the following command, which should display the installed version:
java -version
Step 1: Install and Configure Elasticsearch
Elasticsearch is the heart of the stack, providing a distributed search and analytics engine. We’ll install it from the official Elastic repository to ensure we get the latest stable version.
Add the Elastic Repository
First, import the Elastic GPG key to authenticate the packages:
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Next, add the repository to your system’s source list:
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list
Now, update your package list again to include the new repository:
sudo apt update
Install Elasticsearch
With the repository configured, installing Elasticsearch is straightforward:
sudo apt install elasticsearch
Configure Elasticsearch
After installation, you need to configure Elasticsearch by editing its main configuration file, /etc/elasticsearch/elasticsearch.yml.
sudo nano /etc/elasticsearch/elasticsearch.yml
For a single-node setup, you need to make a few key changes:
Set the Network Host: To allow access from other machines (like the one you’ll use to access Kibana), change
network.host. For testing, you can set it to0.0.0.0. For production, always bind this to a private IP address for security.network.host: 0.0.0.0Configure Discovery: For a single node, you must specify the discovery type.
discovery.type: single-nodeJVM Heap Size (Optional but Recommended): For performance, it’s best practice to set the initial and maximum heap size to the same value, typically no more than 50% of your system’s RAM. You can configure this in
/etc/elasticsearch/jvm.options.d/jvm.options.
Save the file and exit the editor.
Start and Enable Elasticsearch
Now, reload the systemd daemon, start the Elasticsearch service, and enable it to launch on boot:
sudo systemctl daemon-reload
sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch
The service may take a moment to start. You can verify it’s running correctly with:
curl -X GET "localhost:9200"
If successful, you will see a JSON response with details about your Elasticsearch node.
Important Security Note: On the first start, Elasticsearch 9.x automatically enables security features and generates a password for the elastic superuser. To retrieve this password, run:
sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
Save this password securely, as you will need it to configure Kibana and manage your cluster.
Step 2: Install and Configure Kibana
Kibana is the visualization layer of the ELK Stack. It provides a web interface for exploring and visualizing the data stored in Elasticsearch.
Install Kibana
Since you already added the Elastic repository, you can install Kibana directly:
sudo apt install kibana
Configure Kibana
Next, configure Kibana to connect to your Elasticsearch instance. Open the configuration file:
sudo nano /etc/kibana/kibana.yml
Make the following adjustments:
Set Server Host: To access the Kibana dashboard from your browser, uncomment and set
server.hostto0.0.0.0.server.host: "0.0.0.0"Set Elasticsearch Hosts: Point Kibana to your Elasticsearch instance. Since it’s on the same server,
localhostis sufficient.elasticsearch.hosts: ["http://localhost:9200"]
Save the file and exit.
Start and Enable Kibana
Start the Kibana service and enable it for boot:
sudo systemctl daemon-reload
sudo systemctl start kibana
sudo systemctl enable kibana
Actionable Security Tip: If you have a firewall enabled (like UFW), you must allow traffic on port 5601 to access the Kibana dashboard.
sudo ufw allow 5601/tcp
You can now access the Kibana web interface by navigating to http://your_server_ip:5601 in your browser. Log in using the username elastic and the password you generated in the previous step.
Step 3: Install and Configure Logstash
Logstash is the data processing pipeline that ingests data from various sources, transforms it, and sends it to a “stash,” like Elasticsearch.
Install Logstash
Install Logstash using the apt package manager:
sudo apt install logstash
Configure Logstash
Logstash’s configuration defines the data pipeline. We will create a simple configuration to listen for Beats input (like from Filebeat) and send the output to Elasticsearch.
Create a new configuration file:
sudo nano /etc/logstash/conf.d/02-beats-input.conf
Add the following configuration. This sets up Logstash to listen on port 5044 for data from Beats agents.
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
user => "elastic"
password => "YOUR_ELASTIC_PASSWORD"
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}
Crucially, replace "YOUR_ELASTIC_PASSWORD" with the actual password for the elastic user you saved earlier. This configuration ensures that logs are properly indexed by date in Elasticsearch.
Save and close the file.
Start and Enable Logstash
Finally, start and enable the Logstash service:
sudo systemctl start logstash
sudo systemctl enable logstash
Logstash can take a minute or two to initialize fully as it starts the Java Virtual Machine.
Conclusion
You have successfully installed and configured a fully functional ELK Stack 9.x on your Ubuntu 24.04 server. Elasticsearch is storing your data, Kibana is ready to visualize it, and Logstash is prepared to process incoming logs.
Your next step is to start shipping data. A common approach is to install Filebeat on client servers to collect log files and forward them to your Logstash instance on port 5044. From there, you can begin building powerful dashboards in Kibana to gain deep insights into your systems and applications.
Source: https://kifarunix.com/install-elk-stack-9-x-on-ubuntu-24-04/
