Pentest in your own way. How to create a new testing methodology using OSCP and Hack The Box machines

Write-ups for HTB or CTF machines can be as fascinating as good detective novels. An experienced pentester quickly finds a vulnerability, writes a script to exploit it, and grabs the much-desired flag. However, in real life, this process, including all the successful and unsuccessful attempts, can take from several hours to several weeks. Normally, pentesters don’t mention such things in write-ups because descriptions of long fruitless searches are not that interesting to readers. So, how to identify the ‘right places’ in reality and how to find out what must be paid attention to? Seasoned pentesters use for this purpose their own methodologies and sets of custom-made checklists.

Normally, penetration testing of any computer involves a number of stages:

1. Scanning and information collection;
2. Enumeration;
3. Exploitation;
4. Privilege Escalation;
5. Post-Exploitation; and 
6. House Cleaning.

Let’s examine them in more detail to understand how to act more effectively at each stage.

Scanning and information collection

The first thing you have to do is revert the test PC to its original state. All services have a Revert or Reset option. Information collection without restoring the original state of the target VM is useless: the required files may be missing, or a ready-made exploit can be stored in a public folder on FTP spoiling all your pleasure and ruining the opportunity to gain experience.

Furthermore, some exploits (e.g. buffer overflow) can be used only once, and you won’t find the required service when you scan such a VM. Therefore, always restore the target machine to its original condition! HTB tries to avoid using such services, but you still can vote for Revert there or restart the VM yourself if you have a VIP status.

Port scan

Arguments about the right way to run a port scan will never end. There are plenty of various scripts written for this purpose (e.g. Autorecon and nmapAutomator), and they are really useful – but I don’t recommend starting with them. Why? Because it’s always better to understand how a specific utility works rather than rely on it blindly. Of course, you can download such a script and review its source code, but in the meantime, use oldie-goodie Nmap.

First of all, quickly scan the most popular ports:

If Nmap returns an error claiming that the host cannot be reached, try the -Pn option. While you are reviewing the received information, run a full scan of all TCP ports.

And don’t forget about UDP ports! I have no doubt that you can write a command line for scanning them yourself.

Ultimately, you will get full information on ports available on the target machine.

Don’t rush to search for an exploit for VMware or Samba! This is the biggest mistake a pentester can make; it involves the risk to fall into one of the so-called ‘rabbit holes’. These dead ends are your main nightmare at an OSCP exam: you can spend hours trying to hack, for instance, WordPress and end up with nothing since the vulnerability was in a different service – but you simply did not get to it.

Your objective is to compile the complete list of open ports and potential services running behind them. Then continue with information collection.

To enhance your scanning skills, read write-ups, study tools mentioned in them, and accumulate knowledge. I suggest creating a separate repository for documents and use it for systematic development of your own methodology. For instance, I currently have a dozen of different scanning methods in the ‘port scanning’ section (Nmap, masscan, scripts, etc.).

Pay special attention to ports that Nmap cannot clearly identify. This also applies to ports previously unknown to you and familiar services (e.g. web servers and FTP) running on nonstandard ports (e.g. 9000, 5000, etc.). Such services can be potentially vulnerable to exploration and, therefore, hidden from quick scans by the authors. This is why I recommend to perform a full scan of all TCP ports and a fast scan of basic UDP ports. I had encountered VMs whose vulnerable services were hidden in the UDP.

Services

In this section, I’ll describe several popular ports and explain what should you pay attention to when you examine each of them.

Port 21

I always check all found ports (especially unknown ones) on speedguide.net. Once it even helped me to hack a test VM at the exam!

As you can see, port 21 is usually responsible for FTP. But it can also be used for any of the FTP services or not for FTP at all. For instance, some cunning admin could use it for SSH. Therefore, I always try to connect to the service using netcat.

In most cases, you will see an FTP server banner like the one shown below.

Should you encounter SSH or other service there, just write this down and go to the respective section of your checklist to find out what else you can get from there.

The examination of an FTP server must give you answers to the following questions:

• what is name of the server program and what is its version?
• is anonymous access allowed or not?
• is there anything of interest in server folders?
• is any folder writable? and 
• does this server version contain any vulnerabilities?

1. The FTP version can be identified using nc or Nmap:

In addition, there are plenty of programs on the Internet that try to heuristically determine the FTP server version; feel free to use them.

1. The next step is to connect to the server using a standard FTP client and check anonymous access (usually it’s guest, ftp, or anonymous). If it works, great. If not, don’t worry: you may find the required credentials later, or the service may be not required for your purposes at all.
2. If you manage to get on the FTP server, take a look around. It might be just an empty folder or a web server directory (if so, write this down). If you see some files, store them for further research. By the way, it’s now time to find and write down a command for recursive download of all files from the FTP. After all, you aren’t going to download them one by one, right?
3. Check whether you can write something to the folder. The most convenient way is to use the put command. But remember, sometimes the server allows you to write files, but not delete them! Accordingly, don’t put anything of value there: lab VMs are available to everyone.
4. Time to look for vulnerabilities. The easiest (and permitted at the OSCP exam) way is to use Nmap scripts. Always begin with them, regardless of the port you are exploring. Find scripts for FTP and run them all at once. (By the way, nmapAutomator can automatically do this for you.)

Any vulnerabilities identified? Excellent! Right them down but don’t rush with exploitation! There are still plenty of unchecked ports. You don’t want to fall into a rabbit hole, don’t you? But if you found nothing, it doesn’t really matter! In the “Searching for vulnerabilities” section, I will explain where else you can search for security holes.

While dealing with FTP in OSCP labs and on HTB machines, I had encountered the following exciting things:

• FTP servers vulnerable to buffer overflow;
• full access to the C drive containing configuration files with logins to other services;
• possibility to write a file to the web server directory for subsequent code execution;
• directory traversal (i.e. the possibility to read files not only from the FTP server folder); and 
• much more! Don’t forget to add successful tricks to your methodology.

Port 22

Normally, this port is used by SSH, but pentesters don’t trust numbers. Check it with nc:

Great, this is definitely SSH! The package version is OpenSSH_8.2p1. The words Ubuntu-4ubuntu0.1 indicate that this is most likely Ubuntu. After searching Google for “openssh 8.2p1 4ubuntu0.1 version”, you can confidently say that the Ubuntu version is Focal 20.04. Write down this information, it may come in handy later.

Now you need an SSH server key fingerprint.

Why is this information important? Because, using it, you can check whether a known SSH private key is available for this fingerprint. Make sure to review this info about SSH Bad Keys.

The next task is the most interesting and difficult one: understanding the extended SSH output. To display it, run SSH with the -v key.

At first, you may be shocked by the volume of the displayed text, but the info it contains is of utmost importance and must be carefully reviewed.

Why is it so important? There are plenty of VMs with old versions of SSH servers in OSCP labs and on HTB. New versions of Kali consider their encryption algorithm outdated and refuse to connect to such servers via SSH; while the server released in 2014 considers this algorithm perfectly safe. Without the -v option, you will just get the “Access Denied” message without any explanations. To find out the reason for rejection, you have to add a couple of options to the connection string. I had encountered such situations, and this is one of the popular topics on the student forum. So, make sure to carefully review the SSH output displayed with the -v key and make sure you understand it!

If you have a list of logins and passwords, you may try to guess them (e.g. with Hydra). As always, write down the obtained information and incorporate it into your methodology! Should you encounter some interesting SSH-related info in somebody else’s write-up, make sure to research it thoroughly and add to your repository of knowledge.

While dealing with SSH, I had encountered the following exciting things:

• above-mentioned ‘bad’ SSH keys that can be identified and obtained by fingerprints;
• vulnerable keys that can be obtained by public key fingerprint;
• a vulnerability present in earlier SSH versions and enabling the attacker to enumerate local users;
• different client and server versions requiring special flags; and 
• information on the OS and server versions enabling the attacker to use path traversal.

Port 80

This port accommodates all web services and, accordingly, is the most popular and most promising one from the exploitation perspective. But prior to launching the browser, make sure to methodically check all the available information!

Generally speaking, nc is useless in this situation. But I still recommend to run it first.

Who knows, may be an SSH server is running on port 80? Next, check the server headers:

They likely contain the server type and version, PHP version (if it’s used), and other useful info.

As you can see, this server protected by Qrator (a DDOS protection service). The string Link: <https://xakep.ru/wp-json/>; rel=https://api.w.org/ indicates that the site likely uses the WordPress engine.

Time to review the page source code! The -L option enables the correct processing of HTTP redirects.

Here you can practice in reading HTML and searching with grep and regular expressions. First of all, you have to locate and identify the following items:

• CMS version and its name;
• template engine that can be used by the site;
• any comments and temporarily hidden links to other sites or sections;
• links to other subdomains (this is especially actual for HTB); and 
• versions of the JS libraries (sometimes this allows to indirectly identify the CMS version).

Now you can launch the browser and check if there is anything of interest. This is the most creative part that requires practical experience. Read write-ups to learn what seasoned pentesters do in such situations. The following items must attract your attention:

• any robots.txt orsitemap.xml files;
• any feedback or authentication forms; and 
• any closed sections and registration or password recovery options.

Check the Internet for the CMS source code. Plenty of CMSs are available on GitHub. There you can also get an idea what files to pay attention to and find lists of fixed vulnerabilities.

I use a handy Firefox plugin called Wappalyzer: it allows to identify the used frameworks and other features of the web server.

Generally speaking, the list of potential actions is endless; to address all of them in detail, a dozen of articles are required.

Nothing of interest there? Then take a wordlist and try to search for files and hidden directories. The best tools for this are nikto, Gobuster, and DIRB.

If you see that brute-forcing would take five or six hours, then, most probably, this is a wrong way. Such things happened to me on lab machines. Just make a note that you haven’t checked this option and move on.

While dealing with web services, I had encountered the following exciting things:

• SQL and command injections;
• information about dates of birth that allows to guess a password or answer a secret question in the password recovery form;
• XSS allowing to get admin sessions;
• web server configuration errors;
• developer scripts that make it possible (in combination with path traversal) to find files with passwords and download them; and 
• etc., etc., etc.

I suggest using one of the largest payload collections available on GitHub to exploit XSS vulnerabilities, bypass command filtering, and make other injections.

Ports 139 and 445

This is Samba. There are plenty of enumeration tools for it, and I will list just a few of them.

This utility allows to find out the hostname, its domain and workgroup membership, available resources, possible usernames, and much more. Feel free to experiment with it.

To get the list of available folders, use the command:

If you need to recursively view all folders, type:

Use Nmap to detect known vulnerabilities in the SMB. Another powerful tool is EternalBlue.

To execute a command remotely with SMB, type:

Overall, the recommendations for these ports are similar to those provided for FTP. Find out what resources are available to you, what users are on the server (they can be used for brute-forcing), and whether the available folders contain any useful info. And, as always, carefully read the write-ups, note the utilities used, understand how their work, and add them to your collection. Important: always try to figure out why that very utility (i.e. not its analogues) was used in each particular case.

My personal methodology includes plenty of other ports, but I won’t describe them in detail here. Instead, I refer you to the above-mentioned port checklist.

Searching for vulnerabilities

Now you have a list of programs running on the host and known their versions. Time to look for known vulnerabilities and ready-made exploits. In OSCP labs, you normally don’t have to write your own exploits: in most cases, it’s sufficient to slightly adjust the ready-made ones. By contrast, hardcore HTB machines often require manually written (i.e. custom-made) exploits.

Where to look for exploits? First of all, keep in mind that ready-made exploits exist not for all vulnerabilities; furthermore, for some of them, it’s simply impossible to write an exploit. The best way is to start your searches from the largest vulnerability database: cvedetails.com.

This database allows you to search for exploits using such criteria as vendor, program, version, etc. Important: never degrade to a script kiddie who can only run programs and wait for the results. Read the CVE, understand why the program is vulnerable, check if there is an exploit and whether it requires compilation. Can you build it yourself? Will it work on your system? What are its prerequisites? Etc., etc.

In many situations, there is a long list of great exploits, but all of them require admin rights in the CMS or on the website. If so, you have two options: find the admin credentials or look for another way to penetrate into the target VM. In the OSCP lab, I encountered a VM invincible to a ready-made privilege escalation exploit; but to make it functional, all I had to do was read the CVE and replace just one word in it!

Where to get exploits? The most famous exploit repository is exploit-db.com and its console variant, searchsploit. The exploits stored there are rather safe to use. Search Google for “program name exploit version” – sometimes you may find interesting freshly minted exploits this way.

Here are some recommendations on finding exploits and using them.

1. Master searchsploit. You can practice on an Apache server: learn how to display only remote access exploits (i.e. not DOS attacks) and separate the Apache httpd output from Apache Tomcat;
2. Use Google to search for exploits on GitHub. Learn how to do this in the best way and how to properly filter the results;
3. Learn to build exploits not only for x86_64, but also for the pure i386 architecture. In many situations, this skill is extremely useful; and 
4. Learn how to find and use exploits in Metasploit.

Privilege escalation

This section could be huge (up to several articles in size); so, I am just going to give you the link to a great article explaining in detail how to gain root privileges in Linux. In Windows, the situation is slightly more complicated, but plenty of practical materials on this topic are available online as well.

Below is a list of useful tips and tricks I had dealt with in my pentesting endeavors.

• Generally speaking, such tools as linPEAS and the like are useful, but they find absolutely nothing on VMs used at the OSCP exam! Just figure out how they work and compile a list of commands you may need (e.g. finding SUID);
• Install and thoroughly examine ‘clean’ Windows and Linux systems. You must understand what services are present in the system out of the box and what elements require additional installation;
• Found an unknown program? Write it down! Very often it can be used for privilege escalation;
• Carefully examine any oddities in the sbin andbin folders. Once I had eximXX there (the version was clearly indicated in the name), but it wasn’t on the list of installed packages. And of course, it could be used to gain root privileges! and 
• Review the ports available from the inside. Pretty often vulnerable or misconfigured services can be found on 127.0.0.1.

An interesting example: once I encountered a lab VM with Windows NT 4.0 and with access via VNC. The point is that the admin password was written on the desktop wallpaper and hidden behind the login window. So, to read the password, I just had to move the login window.

As always, add all interesting solutions to your methodology. For me, it’s usually more difficult to gain access to a machine than to elevate my privileges on it. Apparently, this is because of my extensive experience as a solution architect and system engineer…

Post-Exploitation

This stage is rarely required for HTB VMs (except for intro-level ones), but it’s very important in labs.

The list of mandatory actions at the post-exploitation stage is as follows:

1. Collect all available password hashes. It can be mimikatz, the /etc/shadow file, service settings, and database settings;
2. Examine all network connections. Sometimes you can detect interesting communications with other servers and find ways to intercept them;
3. Review all information available in the databases. There may be logins/passwords to other servers or interesting backups;
4. Never disregard backups: they may contain important info;
5. User folders may contain the same backups, command execution history (e.g. a password used to connect to a database or another server), etc.; and 
6. Don’t forget to check the Windows registry. It may contain passwords to VNC and other similar services.

Make sure to document all findings. Very often you have information required to access a new VM but cannot find it. If you stumble upon a document or an RDP file with a password to a new computer in the lab, make a note of it! And keep in mind that in OSCP labs, it’s normal to use the same password many times.

House cleaning

In any lab, a rule of good form is to clean the VM from your traces. This includes removing any exploits you had used, returning the machine to its original state in the OSCP lab, and removing all Metasploit shells. By the way, there is a separate section in the OSCP exam report dedicated to this topic.

Conclusions

The conclusion is simple. If you want to be a good pentester, introduce consistency into your actions. Explore operating systems and networks, continuously study new materials, and update your checklists and technique lists. Ultimately, you will develop a certain ‘sense’ telling you what to pay attention to when dealing with a new target. This ‘sense’ is extremely useful if you intend to take the OSCP exam. It is not uncommon that a person taking the exam simply forgets to check the content of robots.txt due to the overwhelming stress and consequently fails to hack the target VM. Remember: an efficient personal checklist helps to reduce stress and boosts your performance.