How to Install ModSecurity with Nginx on Oracle Linux 8

Nginx is a free, open-source, and widely used web server. It is essential to secure the Nginx web server to defend it from DDoS and other attacks, and ModSecurity can help.

ModSecurity is a free and open-source web application firewall used to protect web servers from external threats. It is used by many websites, cPanel, and other hosting control panels to protect their web applications and servers from SQL injection, cross-site scripting, and local file inclusion attack. Originally it was designed to secure only the Apache web servers. Now it can also work with the Nginx web server.

In this post, we will show you how to install ModSecurity with Nginx on Oracle Linux 8.

Step 1 – Install Required Dependencies

First, you will need to install some required dependencies needed to compile Nginx with ModSecurity. You can install all of them with the following command:

dnf install gcc-c++ flex bison yajl curl-devel curl zlib-devel pcre-devel autoconf automake git curl make libxml2-devel pkgconfig libtool httpd-devel redhat-rpm-config git wget openssl openssl-devel vim

Next, install other packages that are not available in Oracle Linux:

 dnf install https://rpmfind.net/linux/centos/8-stream/PowerTools/x86_64/os/Packages/doxygen-1.8.14-12.el8.x86_64.rpm
dnf install https://yum.oracle.com/repo/OracleLinux/OL8/codeready/builder/x86_64/getPackage/yajl-devel-2.1.0-10.el8.x86_64.rpm

Next, install the EPEL and Remi repositories using the following command:

dnf install epel-release https://rpms.remirepo.net/enterprise/remi-release-8.rpm -y

Next, install GeoIP using the following command:

dnf --enablerepo=remi install GeoIP-devel -y

Step 2 – Install ModSecurity On Oracle Linux 8

First, download the latest version of ModSecurity from the Git Hub repository using the following command:

git clone --depth 1 -b v3/master --single-branch https://github.com/SpiderLabs/ModSecurity

Once the download is completed, navigate to the ModSecurity directory and install other modules with the following command:

cd ModSecurity
git submodule init
git submodule update

Next, compile and install ModSecurity with the following command:

./build.sh
./configure
make
make install

Step 3 – Compile Nginx with LibModsecurity Support

In order to enable LibModsecurity support in Nginx, you will need to compile Nginx with support for LibModsecurity.

First, download the ModSecurity-nginx connector using the following command:

cd /root
git clone https://github.com/SpiderLabs/ModSecurity-nginx.git

Next, download the latest stable version of Nginx with the following command:

wget http://nginx.org/download/nginx-1.19.10.tar.gz

Next, extract the downloaded file using the following command:

tar xzf nginx-1.19.10.tar.gz

Next, create an Nginx user with the following command:

useradd -r -M -s /sbin/nologin -d /usr/local/nginx nginx

Next, change the directory to the Nginx source and compile it using the following command:

cd nginx-1.19.10
./configure --user=nginx --group=nginx --with-pcre-jit --with-debug --with-http_ssl_module --with-http_realip_module --add-module=/root/ModSecurity-nginx

Next, install it using the following command:

make
make install

Next, copy a sample ModSecurity configuration file and Unicode mapping file with the following command:

cp /root/ModSecurity/modsecurity.conf-recommended /usr/local/nginx/conf/modsecurity.conf
cp /root/ModSecurity/unicode.mapping /usr/local/nginx/conf/

Next, back up the Nginx configuration file:

cp /usr/local/nginx/conf/nginx.conf{,.bak}

Next, edit the Nginx configuration file with the following command:

nano /usr/local/nginx/conf/nginx.conf

Remove all lines and add the following lines:

user  nginx;
worker_processes  1;
pid        /run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  nginx.example.com;
        modsecurity  on;
        modsecurity_rules_file  /usr/local/nginx/conf/modsecurity.conf;
        access_log  /var/log/nginx/access_kifarunix-demo.log;
        error_log  /var/log/nginx/error_kifarunix-demo.log;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

Save and close the file, then create an Nginx log directory:

mkdir /var/log/nginx

Step 4 – Create a Systemd Service File for Nginx

Next, you will need to create a systemd service file to manage the Nginx service. You can create it using the following command:

nano /etc/systemd/system/nginx.service

Add the following lines:

[Unit]
Description=The nginx HTTP and reverse proxy server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/bin/rm -f /run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=mixed
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Save and close the file, then create a symlink of Nginx binary using the following command:

ln -s /usr/local/nginx/sbin/nginx /usr/sbin/

Next, reload the systemd daemon to apply the changes:

systemctl daemon-reload

Next, start the Nginx service and enable it to start at system reboot:

systemctl enable --now nginx

You can check the status of Nginx with the following command:

systemctl status nginx

You will get the following output:

● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/etc/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2022-06-29 12:22:20 EDT; 7s ago
  Process: 52249 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 52247 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
  Process: 52246 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
 Main PID: 52251 (nginx)
    Tasks: 2 (limit: 11409)
   Memory: 3.6M
   CGroup: /system.slice/nginx.service
           ├─52251 nginx: master process /usr/sbin/nginx
           └─52252 nginx: worker process

Jun 29 12:22:20 oraclelinux8 systemd[1]: Starting The nginx HTTP and reverse proxy server...
Jun 29 12:22:20 oraclelinux8 nginx[52247]: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
Jun 29 12:22:20 oraclelinux8 nginx[52247]: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
Jun 29 12:22:20 oraclelinux8 systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argument
Jun 29 12:22:20 oraclelinux8 systemd[1]: Started The nginx HTTP and reverse proxy server.

Step 5 – Enable ModSecurity Rule

By default, ModSecurity has been configured for detection-only mode, so you must enable the ModSecurity rule in the modsecurity.conf file.

To enable it, run the following command:

sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /usr/local/nginx/conf/modsecurity.conf

Also enable the audit log with the following command:

sed -i 's#/var/log/modsec_audit.log#/var/log/nginx/modsec_audit.log#' /usr/local/nginx/conf/modsecurity.conf

Step 6 – Install OWASP ModSecurity Core Rule Set

OWASP provides generic attack detection rules for ModSecurity, so it is recommended to download and integrate with ModSecurity for better security.

git clone https://github.com/SpiderLabs/owasp-modsecurity-crs.git /usr/local/nginx/conf/owasp-crs

Next, rename the OWASP rule configuration file using the following command:

cp /usr/local/nginx/conf/owasp-crs/crs-setup.conf{.example,}

Next, define the OWASP rule in the ModSecurity configuration file:

echo -e "Include owasp-crs/crs-setup.conf\nInclude owasp-crs/rules/*.conf" >> /usr/local/nginx/conf/modsecurity.conf

Next, restart the Nginx service to apply the changes:

systemctl restart nginx

Step 7 – Verify ModSecurity

At this point, Nginx is installed and configured with ModSecurity support. Now, it’s time to test it.

Execute the code injection using the curl command:

curl localhost/index.html?exec=/bin/bash

If ModSecurity is working fine, you should get the “403 Forbidden” error as shown below:

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.19.10</center>
</body>
</html>

You can also verify the ModSecurity log for more information:

tail -100 /var/log/nginx/modsec_audit.log

You should see the following output:

ModSecurity: Warning. Matched "Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:exec' (Value: `/bin/bash' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "496"] [id "932160"] [rev ""] [msg "Remote Command Execution: Unix Shell Code Found"] [data "Matched Data: bin/bash found within ARGS:exec: /bin/bash"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION"] [tag "WASCTC/WASC-31"] [tag "OWASP_TOP_10/A1"] [tag "PCI/6.5.2"] [hostname "127.0.0.1"] [uri "/index.html"] [unique_id "165651983136.006138"] [ref "o1,8v21,9t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/nginx/conf/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "127.0.0.1"] [uri "/index.html"] [unique_id "165651983136.006138"] [ref ""]

---cnJj4Dnx---I--

---cnJj4Dnx---J--

---cnJj4Dnx---Z--

Conclusion

In this guide, we learned how to install ModSecurity with Nginx on Oracle Linux 8. Your server is not fully protected with ModSecurity and it can able to protect from a wide range of attacks. Give it a try on your dedicated server from Atlantic.Net!