Conquering the web. Application instruction for OWASP Testing Guide v4

Web security is a very broad term. It includes bugs in old protocols, usage of dangerous techniques, trivial human errors made by developers, and more. It is difficult to test products in such a broad area without a plan. The Open Web Application Security Project (OWASP) made the life of pentesters easier by producing the OWASP Testing Guide.

Out of the several pentesting methodologies, only one was developed specifically for the web. This article is not about the compliance with PCI DSS standards (which is a highly specialized area); it is dedicated to the universal testing methodology. Let’s go together through the OWASP Testing Guide v4 and see what sections of this document contain hidden pitfalls and, accordingly, require special attention.

HackMag has recently published an article explaining how to check web sites for vulnerabilities; this material briefly mentions OWASP and its field of application. At the time of writing, the actual version of the OWASP Testing Guide (PDF) was v.4, but recently OWASP released v.4.1. Version 5 is under development, and you can make commits in its public repository on GitHub. Even though the guide is pretty voluminous and seemingly comprehensive, it should be considered just the basis for your research (i.e. not a universal manual suitable for all situations). Below is a brief instruction on how to use the OWASP Testing Guide.

All that glitters is not gold

Albert Einstein once said: “order is for idiots, genius can handle chaos.” However, in pentesting, careful planning is a prerequisite for success. Still, even the most detailed plan only describes a tentative sequence of actions as it’s impossible to forecast all possible nuances.

This is because new technologies emerge much faster than updates to the methodology, and web applications can be used virtually for every purpose: from creating a simple website to developing an admin panel to control physical devices. This is why any methodology must be supplemented by creative thinking and practical experience.

I strongly recommend using all available tools: different utilities can provide different results when you perform tests under the same section of the Testing Guide, while the overlapping sections allow to eliminate potential flaws not detected by other testers and automated tools.

One might think that the methodology is primarily designed for black box testing (even though the gray box and white box are mentioned in the testing guide, too); but generally speaking, it can be applied to any testing type after adding the required methods and tools.

Testing Guide Introduction

In the beginning of the guide, its authors say that automated black box testing is not efficient by itself and must be supplemented by manual testing. This is correct, and the guide provides examples involving the Nessus scanner; however, it does not say a word about the OpenVAS scanner that is not much inferior to Nessus.

It is beneficial to use all the available scanners and other features of commercial products (e.g. Burp Pro) because different tools often bring different results. Never disregard false positives: sometimes such results turn out to be true.

Testing for Information Gathering

Conduct search engine discovery/reconnaissance for information leakage

Open-source intelligence (OSINT) is the first phase of any pentesting research, including testing of web applications. It is performed prior to commencing the main works; its purpose is to check whether the tested objects indeed belong to the customer and estimate the scope of work and labor costs.

According to the methodology, this phase is performed using various search engines that offset each others’ shortfalls. I suggest reviewing the information available on DuckDuckGo and Google Dorks.

But of course, OSINT is not limited to searches in publicly available sources due to the existence of nonindexed forums (for instance, those hidden in darknet) and data provided only by personalized requests (e.g. the requester’s IP address must be located in a certain country). If you encounter a resource that needs a personalized request, try this website. At any pentesting stage, keep in mind that the tested system may provide some valuable information by a personalized request. To obtain data required to make such a request, use passive information collection techniques (e.g. FOCA) to extract metadata from documents that are likely present on the tested resource.

IT websites and resources are useful as well. Keep in mind that the testing guide must be treated just as a starting point, not a step-by-step instruction.

Enumerate applications on webserver

This section is dedicated to web applications available either on secret subdomains or through relative paths accessible from the outside (some website resources can be accessed only by users who know their secret URLs). The section can be supplemented by the following tricks:

• sites can be similar to each other (e.g. created by the same developer). Special tools, such as publicwww.com/, can be used to search for identical code fragments (comments, various identifier in JS libraries, authors’ names in comments, etc.) in the hope that these sites have been indexed;
• you can search for subdomains using various tools or manually, applying either the search by certificates or DNS requests. Also, you can use custom-made or publicly available wordlists for brute-forcing and employ tons of other utilities that are continuously updated and improved.

Map execution paths through application

This pertains to the web application ‘mapping’ (i.e. depiction of all website sections in the text or graphic form). This process can be automated using special tools; in the end, you get a scheme of the web application or site and use it in your research. For instance, such a scheme allows to match website sections against the methodology sections. In addition, the automated utilities can find something you have missed at the information collection stage.

Configuration and Deployment Management Testing

This section describes the testing of the web application’s infrastructure. The guide primarily refers to the web server and DBMS that constitute the basis of any application. However, I would also recommend to keep in mind other infrastructure components such as CI/CD systems and message brokers – provided that your research plan covers these items.

Authentication testing / Authorization testing

When you test the authentication and authorization mechanisms, never forget about OAuth, SSO, and OpenID. You may even encounter an SSL certificate-based authentication system.

There are plenty of various authentication schemes; it is physically impossible to master them all in a short period of time; so, your primary goal is to get an idea what testing section is applicable in each particular case.

Input validation testing

The first two chapters of this section address vulnerabilities stemming from the reflected and stored cross-site scripting (XSS). However, XSS vulnerabilities constitute a subclass of a larger group of vulnerabilities: reflected/stored HTML injections. You may encounter a situation with no XSS, but with an HTML injection is in place. In addition to XSS/HTML injections, never forget about template injections that constitute a separate subclass of attacks resulting in remote code execution. SSRF attacks form a separate subgroup of RCE attacks.

Another important aspect is the web application environment. Your goal is to find out how can a potential malefactor exploit it. Here is an example showing how hashes can be leaked from a Windows server due to a single vulnerability stemming from the poor filtration of input data.

The section also addresses binary vulnerabilities, including overflow and format string. Generally speaking, this topic includes the entire spectrum of binary vulnerabilities, tricks used to exploit them, and remote attack techniques. The theme is so broad that it deserves a separate article or even book.

Business logic testing

Technically, a section dedicated to the business logic can include anything. Problems in this sphere may lead to DDOS attacks and disruptions of the information integrity, confidentiality, and accessibility. You cannot take precautions against every contingency and have to act according to the situation. Therefore, this section is mostly theoretical because the practical testing techniques depend on the architecture and internal structure of the tested object.

Client side testing

The first two chapters of this section address the above-mentioned XSS-related vulnerabilities – this time, on the client side. In this regard, I would like to stress two important circumstances. First, never forget about HTML injections (they can be performed on the client side as well). Second, there are four types of XSS-related vulnerabilities: server-side reflected, server-side stored, client-side reflected, and client-side stored ones. In the last case, the XSS payload is kept in the browser storage (either in the session storage or local storage). In my opinion, such vulnerabilities should be divided into client-side reflected and client-side stored ones because DOM-based XSS attacks and arbitrary JS injections can be performed in the context of both these vulnerabilities.

In a similar way, a client-side template injection (CSTI) attack is a ‘relative’ of a server-side template injection (SSTI) attack: their principle is the same, and the only difference between them is that the CSTI attack is performed on the client side.

Conclusions

Web technologies have plenty of nuances no one can keep in mind. The main weapons of a pentester are search engines and carefully selected tools.

Experience comes with practice, while theoretical knowledge is available in books, articles and reports, as well as on forums and big-bounty portals. You can even create and maintain your personal knowledge base – this is convenient if you attend conferences, such as PHDays, ZeroNights, RuCTF, and OffensiveCon, or take video lessons. Remember: in each specific case, the testing methodology is just a starting point for further research.

Always Google everything pertaining to the security of the web application’s component you are testing. For instance, if you have encountered SOAP, research JWT in relation to JAVA and Web Services; or, if you are dealing with XML documents, review available information on XXE and XSLT. Be prepared to situations not addressed in the testing guide (for instance, the customer may want you to test its own security systems).

Remember: creative approach is your key to success because not a single guide can describe all possible eventualities. Good luck!