Secure Your ELK Stack: A Step-by-Step Guide to Filebeat and Logstash SSL/TLS Encryption
In any modern data pipeline, security is not an afterthought—it’s a foundational requirement. When using the Elastic Stack to ship logs and metrics, the connection between Filebeat and Logstash represents a critical pathway. Leaving this channel unencrypted exposes sensitive data to potential interception and eavesdropping. Implementing SSL/TLS encryption is the definitive solution to protect this data in transit.
This guide provides a clear, actionable walkthrough for securing the communication channel between Filebeat and Logstash, ensuring your logging pipeline is robust, secure, and compliant.
Why Encrypt the Filebeat to Logstash Pipeline?
By default, data sent from Filebeat to Logstash travels in plain text. This creates a significant security vulnerability, especially if your beats and Logstash instances communicate over untrusted networks. Encrypting this connection with SSL/TLS provides three core security benefits:
- Confidentiality: Encryption ensures that log data cannot be read by unauthorized parties if it is intercepted. This is crucial for protecting sensitive information like personal data, access credentials, or proprietary application logs.
- Integrity: SSL/TLS guarantees that the data received by Logstash is exactly what Filebeat sent. It protects against man-in-the-middle attacks where an adversary could alter logs in transit.
- Authentication: The configuration ensures that Filebeat is sending data to a trusted Logstash server, and conversely, Logstash only accepts data from authorized Filebeat clients. This prevents rogue clients from flooding your pipeline or unauthorized servers from impersonating your Logstash instance.
The Foundation: Generating Your SSL/TLS Certificates
Before configuring the services, you need the necessary security certificates. For this process, you will typically need a Certificate Authority (CA), which is used to sign and validate the server and client certificates. For internal infrastructure, creating your own self-signed CA is a common and effective practice.
You will need to generate the following components:
- A Root Certificate Authority (CA): This is the master certificate used to sign other certificates. It establishes a root of trust for your environment.
- A Server Certificate and Key: This pair is used by Logstash to identify itself to clients. The certificate must be signed by your CA.
- A Client Certificate and Key: This pair is used by Filebeat to authenticate itself to the Logstash server. This certificate must also be signed by your CA.
Tools like OpenSSL or the elasticsearch-certutil utility provided with the Elastic Stack can be used to generate these files. Ensure you store your private keys securely and restrict their file permissions.
Configuring Logstash for Secure Ingestion (Server-Side)
The first step is to configure your Logstash instance to listen for incoming Beats connections over a secure SSL/TLS channel. This is done within your Logstash pipeline configuration file (e.g., beats-input.conf).
You need to modify the beats input plugin with the appropriate SSL settings.
input {
beats {
port => 5044
ssl => true
ssl_certificate => "/path/to/your/logstash.crt"
ssl_key => "/path/to/your/logstash.key"
ssl_certificate_authorities => ["/path/to/your/ca.crt"]
ssl_verify_mode => "force_peer"
}
}
Let’s break down these critical parameters:
ssl => true: This is the essential flag that enables SSL/TLS encryption for the Beats input.ssl_certificate: This specifies the path to the public certificate file for your Logstash server.ssl_key: This points to the corresponding private key for your Logstash server certificate. This file must be kept secure and readable only by the Logstash user.ssl_certificate_authorities: This optional but highly recommended setting provides the path to your CA’s public certificate. It is used to verify the authenticity of incoming client certificates.ssl_verify_mode => "force_peer": This setting enforces client authentication. When set, Logstash will only accept connections from clients (Filebeat) that present a valid certificate signed by the specified CA. This prevents unauthorized clients from connecting.
Once you have updated the configuration, restart your Logstash service for the changes to take effect.
Configuring Filebeat for Secure Shipping (Client-Side)
Next, you must configure your Filebeat clients to send data using the encrypted connection. This is done in the filebeat.yml configuration file, specifically within the Logstash output section.
output.logstash:
hosts: ["your-logstash-server.com:5044"]
ssl.enabled: true
ssl.certificate_authorities: ["/path/to/your/ca.crt"]
ssl.certificate: "/path/to/your/filebeat.crt"
ssl.key: "/path/to/your/filebeat.key"
Here is an explanation of the client-side configuration:
ssl.enabled: true: This enables SSL/TLS communication from Filebeat to the Logstash output.ssl.certificate_authorities: This is one of the most important settings. It provides the path to your CA’s public certificate. Filebeat uses this to verify that it is connecting to the correct Logstash server and not an imposter.ssl.certificate: This is the path to the client certificate for this specific Filebeat instance.ssl.key: This points to the corresponding private key for the Filebeat client certificate. As with the server key, ensure this file’s permissions are properly restricted.
After saving the filebeat.yml file, restart the Filebeat service. It will now attempt to establish a secure, mutually authenticated connection to your Logstash server.
Verification and Security Best Practices
To confirm that your setup is working correctly, check the logs for both Logstash and Filebeat. A successful connection will be established without any SSL handshake errors. If you see errors related to “certificate signed by unknown authority” or “handshake failure,” double-check your certificate paths and ensure all certificates were signed by the same CA.
For a truly robust and secure pipeline, consider these additional tips:
- Use Strong File Permissions: Your private key files (
.key) contain sensitive information. Set their file permissions so they are only readable by the root user and the service user (e.g.,logstashorfilebeat). A permission setting of600is highly recommended. - Implement a Certificate Rotation Policy: Certificates have an expiration date. Establish a process for renewing and redeploying your certificates before they expire to avoid unexpected service outages.
- Do Not Share Client Certificates: Each client or logical group of clients should have its own unique certificate. This allows for more granular control and easier revocation if a specific client is compromised.
By taking the time to properly configure SSL/TLS, you transform your logging pipeline from a potential liability into a secure and resilient asset. Encrypting data in transit is a non-negotiable step for protecting your organization’s sensitive information and maintaining a strong security posture.
Source: https://kifarunix.com/easy-way-to-configure-filebeat-logstash-ssl-tls-connection/
