Setting up LibModsecurity with Nginx on CentOS 8

13/08/2025

0 Views 0

SaveSavedRemoved 0

Setting up LibModsecurity with Nginx on CentOS 8

A Step-by-Step Guide to Installing ModSecurity 3 with Nginx on CentOS 8

Protecting your web applications from malicious attacks is no longer optional—it’s a critical necessity. A Web Application Firewall (WAF) acts as a powerful shield, filtering and monitoring HTTP traffic between a web application and the internet. One of the most robust and widely respected open-source WAFs is ModSecurity.

This guide will walk you through the complete process of compiling and setting up LibModsecurity (v3) with Nginx on a CentOS 8 server. By integrating ModSecurity, you can defend your applications against a wide range of threats, including the OWASP Top 10 vulnerabilities like SQL injection and Cross-Site Scripting (XSS).

A Quick Note on CentOS 8: As of December 31, 2021, CentOS 8 has reached its End-of-Life (EOL). While these instructions are written for CentOS 8, they are highly applicable to its derivatives like AlmaLinux 8 and Rocky Linux 8, which are recommended for new deployments.

What is ModSecurity and Why Do You Need It?

ModSecurity is a toolkit for real-time web application monitoring, logging, and access control. When integrated with your Nginx web server, it inspects all incoming requests and their responses based on a predefined set of rules.

Think of it as a dedicated security guard for your web server. It can:

Block known attack patterns: Defend against common exploits out-of-the-box.
Implement virtual patching: Protect against vulnerabilities before official patches are released.
Provide detailed logging: Gain deep insight into the traffic hitting your server.
Prevent data leakage: Stop sensitive information from being exposed in error messages.

By the end of this tutorial, you will have a fully functional WAF protecting your Nginx server.

Prerequisites: Preparing Your System

Before we begin compiling, we need to install the necessary development tools and libraries from the CentOS repositories.

Open your terminal and run the following commands to install all dependencies:

sudo dnf install -y epel-release
sudo dnf install -y git gcc-c++ flex bison yajl yajl-devel curl-devel curl GeoIP-devel doxygen zlib-devel pcre-devel

These packages provide the compilers, build tools, and development headers required to compile both LibModsecurity and Nginx from source.

Step 1: Compile and Install LibModsecurity

The first major step is to compile the core ModSecurity library. We will clone the official repository to ensure we have the latest stable version.

Clone the LibModsecurity Repository
Navigate to a suitable directory (e.g., /usr/local/src/) and clone the project from GitHub.
```
cd /usr/local/src/
sudo git clone --depth 1 -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity
```
Compile the Library
Move into the newly created directory and run the build scripts.
```
cd ModSecurity
sudo ./build.sh
sudo ./configure
sudo make
sudo make install
```

This process will compile and install the libmodsecurity library onto your system, making it available for other applications like the Nginx connector.

Step 2: Compile Nginx with the ModSecurity v3 Connector

Standard Nginx installations from repositories don’t typically include the ModSecurity module. Therefore, we must compile Nginx from source and explicitly include the ModSecurity v3 Connector as a dynamic module.

Download the Nginx Source Code and the Connector
First, download the Nginx connector.
```
cd /usr/local/src/
sudo git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git
```
Next, visit the official Nginx website to find the latest stable version number. Download and extract the source code. For this example, we’ll use version 1.22.0.
```
sudo wget http://nginx.org/download/nginx-1.22.0.tar.gz
sudo tar -xvzf nginx-1.22.0.tar.gz
```
Configure and Compile Nginx
Navigate into the Nginx source directory. Now, we’ll run the ./configure script, telling it to build the ModSecurity connector as a dynamic module.
```
cd nginx-1.22.0
sudo ./configure --with-compat --add-dynamic-module=/usr/local/src/ModSecurity-nginx
```
- --with-compat: Ensures compatibility with dynamic modules.
- --add-dynamic-module: Points to the location of the connector’s source code.
Once the configuration is complete, compile and install Nginx.
```
sudo make
sudo make install
```
After this completes, a compiled module file named ngx_http_modsecurity_module.so will be created in the objs/ directory. We need to copy this to the Nginx modules directory.
```
sudo cp objs/ngx_http_modsecurity_module.so /usr/share/nginx/modules/
```

Step 3: Configure Nginx to Load and Use ModSecurity

Now that Nginx and the module are installed, we need to tell Nginx how to use them.

Load the Dynamic Module
Edit your main Nginx configuration file, typically located at /usr/local/nginx/conf/nginx.conf. At the top of the file, add the following line to load our new module:
```
load_module /usr/share/nginx/modules/ngx_http_modsecurity_module.so;
```

Enable ModSecurity in Your Server Block
Inside the same nginx.conf file, locate the server block where you want to enable the WAF. Add the following directives inside it:

server { # ... other server directives like listen, server_name, etc.modsecurity on; modsecurity_rules_file /usr/local/nginx/conf/modsecurity.conf;

}

modsecurity on;: This switch activates ModSecurity for this server block.
modsecurity_rules_file: This points to the main ModSecurity configuration file, which we will create next.

Step 4: Integrate the OWASP Core Rule Set (CRS)

ModSecurity is the engine, but the OWASP Core Rule Set (CRS) provides the intelligence. It’s a highly refined set of rules designed to block a vast array of common attacks.

Download and Set Up CRS
First, download the ruleset into the Nginx configuration directory.
```
cd /usr/local/nginx/conf/
sudo git clone https://github.com/coreruleset/coreruleset.git
```
This creates a coreruleset directory. We need to create a main modsecurity.conf file and link the CRS rules.

Create the Main modsecurity.conf File
Create the modsecurity.conf file we referenced earlier in the Nginx server block.

sudo nano /usr/local/nginx/conf/modsecurity.conf

Paste the following configuration into the file. This tells ModSecurity to enable its rule engine and include the OWASP CRS files.

# This is a basic configuration to load the OWASP Core Rule Set.
# The SecRuleEngine directive will be set by the CRS setup file.

Include coreruleset/crs-setup.conf.example
Include coreruleset/rules/*.conf

Activate the Rule Engine
The CRS comes with an example setup file. Let’s rename it to activate it.
```
cd /usr/local/nginx/conf/coreruleset/
sudo mv crs-setup.conf.example crs-setup.conf
```
By default, the crs-setup.conf file sets the rule engine to DetectionOnly. This is a crucial safety feature that logs potential attacks without blocking them. It allows you to check for false positives before fully enabling blocking mode.

Step 5: Testing and Verifying Your WAF

It’s time to see our work in action.

Check Nginx Configuration and Restart
First, test your Nginx configuration for any syntax errors.
```
sudo /usr/local/nginx/sbin/nginx -t
```
If it returns “syntax is ok” and “test is successful,” you can safely start or restart the Nginx service.
```
sudo /usr/local/nginx/sbin/nginx
# or if it's already running:
# sudo /usr/local/nginx/sbin/nginx -s reload
```
Simulate a Basic Attack
From your local machine or another server, use curl to send a request with a common attack vector to your server’s IP address or domain.
```
curl "http://YOUR_SERVER_IP/?param=<script>alert(1)</script>"
```
Check the Logs
Since we are in DetectionOnly mode, the request will succeed (you’ll get a 200 OK response). However, ModSecurity should have logged the attempt. Check your Nginx error log:
```
sudo tail -f /usr/local/nginx/logs/error.log
```
You should see an entry from ModSecurity that identifies the malicious pattern, including details like the matched rule ID from the OWASP CRS. This confirms your WAF is working correctly.

Security Best Practices and Next Steps

Run in DetectionOnly Mode: Always start with SecRuleEngine DetectionOnly. Monitor your error.log for a few days or weeks to ensure that legitimate user traffic is not being flagged (false positives).
Enable Blocking Mode: Once you are confident there are no false positives, edit /usr/local/nginx/conf/coreruleset/crs-setup.conf and change SecRuleEngine DetectionOnly to SecRuleEngine On. Reload Nginx to apply the change. Now, ModSecurity will actively block malicious requests with a 403 Forbidden error.
Keep Rules Updated: The threat landscape is constantly evolving. Periodically git pull the latest updates for the OWASP Core Rule Set to stay protected against new threats.
Monitor and Tune: Regularly review your logs to understand the threats your server is facing and to fine-tune any rules if necessary.

By following this guide, you have successfully fortified your Nginx web server with a world-class Web Application Firewall, adding a critical layer of defense to your overall security posture.

Source: https://kifarunix.com/configure-libmodsecurity-with-nginx-on-centos-8/