How to provide process isolation and not destroy Windows

How can we isolate suspicious processes in Windows and not destroy the OS? How can we create a reliable and Windows-compatible sandbox without hardware virtualization and kernel function hooking, but with the use of documented default OS security mechanisms? In this article we will be discussing the most common problems faced by sandbox developers (and, as a result, consumers). And of course we will also offer our own solutions.

Introduction, or how bad life is without a sandbox

Professionals all know a few axioms that they don't like to talk about. And what is there to say about axioms, anyway? They simply exist, just as clear as day. For example, here's one of them: a signature-based antivirus doesn't offer any protection. Okay, so it just doesn't protect anything, and that's that. People have gone over it a thousand times, complete with examples, nice presentations, and so on and so forth. And wide-spread nasties such as Ransomware are proof of the ineffectiveness of signature-based and heuristic technologies. All kinds of cryptors and obfuscators successfully solve the problem of defending long-known malware from being detected, so that for some time this malware is not detected by antivirus software. This is ample time for someone to come to harm and for someone else to profit.

It's not even about 0day. You can take old well-known malware, morph it, remove behavioral signatures (a couple days of work for a lazy person), and use it again and again until you get bored or get caught. At the same time, the people who sold the cure, so people would never come to harm again, act like they've got nothing to do with things anymore. They publish some newsletter and talk about Internet hygiene in all seriousness, forgetting to say that if the hygienic procedures would be fully complied with, there would be almost no need in antivirus software, especially commercial.

Sandboxes and aspects of their implementation

So, antivirus software doesn't save and sometimes even destroys what's already there. "Let's approach security from the opposite angle and isolate processes from one another," someone really smart once said. And indeed, it's nice when suspicious processes are executed in a somewhat isolated environment called a sandbox. When running inside a sandbox, malware can't escape and damage the system. This could be the ultimate solution, but using existing sandboxes is somewhat complicated…

Later on we are going to discuss all the details of building a sandbox, and they certainly will come in handy when you need to choose a means to isolate processes or HIPS (Host-based Intrusion Prevention System.)

Nuance 1, or one sandbox for all

Most sandboxes actually don't allow users to isolate processes. The truth is that in most uses the protected system is divided into two parts: the trusted and the untrusted. Typical processes are executed in the trusted part, and the untrusted part is reserved for isolated processes. This means that all isolated processes are executed in the same sandbox, have access to each other and each other's resources, and use the same registry and the same file system.

Malware can therefore gain a foothold in the sandbox and get its start with just one of the isolated applications (or several isolated applications, or any of them). Given that, sandboxes often don't log isolated process actions. Actions that are alarming to HIPS pass freely through sandboxes with just an isolation mark, and that is not very good.
How can we check that isolation indeed works this way? It's actually quite simple! Start two applications in a sandbox. For example, notepad.exe and wordpad.exe. Using notepad.exe, create a text file 1.txt.

Fig. 1. Create a text file

Fig. 1. Create a text file

Of course, the file will not be saved on the desktop, but in the "virtual" directory. Try to open it with Wordpad (fig. 3).

Fig. 2. It seems that one isolated environment is created for all isolated applications

Fig. 2. It seems that one isolated environment is created for all isolated applications

Fig. 3. The file is opening for reading

Fig. 3. The file is opening for reading

So, a file created with one isolated application can be opened with another isolated application. Put bluntly, it's not very isolating at all. But there is some protection from recording, isn't there? Let's change the contents (fig. 4).

Fig. 4. And change the contents of the text file

Fig. 4. And change the contents of the text file

Then save it. Now let's try to open 1.txt with notepad.exe. Of course, we open up notepad.exe in the sandbox (fig. 5).

Fig. 5. Modified text file

Fig. 5. Modified text file

Now here's what we were talking about. Two isolated applications are not actually isolated from one another. It's not really very clear what this isolation is for. Even an encryptor that does not have access to local folders on your computer can encrypt everything in a virtualized directory and, if its lucky, on network resources as well, because sandbox settings are the same for all isolated applications.

Nuance 2, or poor isolation

No, the isolated processes can't reach a trusted part of the system… but in most implementations only recording is restricted. Meaning they can read almost anything without limits and often have network access. This was probably done for better compatibility, but it cannot be called isolation.
Try and conduct this simple experiment with your sandbox. Create a directory on your hard drive. Like this: E:\Photos. Put, let's say, a photo in it (fig. 6).

Fig. 6. Private pic

Fig. 6. Private pic

Start Internet Explorer in the sandbox and try to send this picture, for example, to rghost.

Fig. 7. The isolated browser has the read permission by default

Fig. 7. The isolated browser has the read permission by default

Fig. 8. The application isolated in the sandbox managed to steal the private photo

Fig. 8. The application isolated in the sandbox managed to steal the private photo

So? Did it work? It's not a good sign if the experiment was successful. It's even worse if you can't specify the directory an isolated application cannot access. And it's not good at all if isolated applications have the read permission for the data in directories of the current user.

The virtualization of the file system and registry is built on the principle of "copy on demand" in most implementations. So if a user tries to read a file, it is read from a source directory if there's no analogue in the virtual directory. And if there is such a file in the virtual directory, then the isolated application will work with it. The same can be said about virtual registries. And it's clear that when you try to record a file in a real directory, it is also recorded in the virtual file system. Almost always. If the malware is therefore "isolated" in a sandbox, it has full access to all the other "isolated" processes, to almost all data in the system for reading, and to virtualized (created by isolated applications) data (which is often open to all isolated applications) for recording.

Nuance 3, or "let's reinvent the wheel, it's fun"

Please subscribe to read full article

1 year

for only $29

With subscription you are free to read all of the materials of Hackmag.com.
Read more about the project


Please subscribe to view comments

Only subscribers can participate in the discussions. You may login in to your account or sign up to Hackmag and pay a subscription to access the discussions.