Modologic. Dissecting the Pathologic virtual machine

Information collection

Scripts are stored in the file Scripts.vfs. The format is makeshift and very simple. The easiest way to unpack this archive is to use ready-made software VFS Explorer.

The archive contains plenty of binary files with the .bin extension. The first four bytes don’t contain any noticeable signature.

Normally, developers use time-tested scripting languages (e.g. Lua or Python) in their products. In “Pathologic”, you are dealing with binary files, which indicates that its scripts have been compiled. However, the sparse (full of zeros) bytecode leaves no doubt that the script format is proprietary. Only makeshift languages use file space so wastefully compared to optimized processor opcodes. Zeros in the bytecode can indicate that the instruction takes up a fixed number of bytes, but no arguments were passed to it.

First of all, I searched Google for mods and found a partial description of the file format. It helped at the initial stage, but the document ends in the middle of the header list…. So, I had no choice but to start the oldie-goodie debugger and analyze the code myself.

This research became possible thanks to the recent publication of the alpha version of “Pathologic”. Its main advantage is that the executable files are accompanied by .pdb files containing prototypes of functions and all structures used in the game. These debug symbols made it possible to reduce the research time from several months to a couple of weeks.

Searching for an entry point

To find out what fields are used in this proprietary format, you have to find the place where scripts are read in the game code. For this purpose, I suggest to run “Pathologic” in x64dbg, a modern alternative to OllyDbg. The debugger automatically loads symbols from disk and assigns original names to addresses in memory.

But first, let’s disable full-screen mode by editing config.ini:

Now you can interact with the game in the debugger without worries that it can crash any second leaving you no choice but to restart the system. To find out what files are opened by the game, conditional breakpoints can be used. Go to the WinAPI function CreateFileA and create the following breakpoint:

The following data can be added to the log: string formatting and expression functions. In this particular case, let’s instruct the debugger to add an entry to the log for each opened file; the path to this file can be obtained via the stack (in fact, you just take the string from the first argument of the CreateFileA function).

Start the game, and you’ll see zillions of game assets in the log:

Hmm, this seems to be interesting. The scripts are packed in an archive, but for debugging or patching purposes, the game first tries to read them from the disk. And only if they aren’t there, it reads the contents of these files from the game archive. This significantly simplifies modding.

Let’s say I want to pause the debugger on a specific file. To do so, I slightly edit the breakpoint:

The strstr function returns one if the string returned by ansi contains the fire.bin substring. Then I restart the debugger, and a breakpoint is triggered on the required file. Let’s look at the stack to find out the address from where the call was made:

Looks like a script start point! Time to open Engine.dll in IDA Pro.

Examining headers

IDA shows not very many incoming references to CreateFileA, and I quickly found a piece of code inside RunScript that reads the script file via the virtual file system (VFS).

The script body is passed to the constructor of the CScript object. IDA’s capacity with regards to programs written in C++ is very limited; so, CScript::CScript is just a function name with no direct links to its class. Many classes have virtual function tables that can be used to identify the class of a given function (even if you don’t have symbols at hand).

According to the function prototype, the first argument is a script file, and it also acts as the class instance body. Such a solution can often be found in old games. Their developers tried to save on serialization and simply dumped class bodies to disk (e.g. to create savegame files). Let’s look at the definition of the CScript class:

The structure was read from the PDB file. IDA can’t handle templates properly, and these scary long strings with multiple commas and angle brackets are nothing more than names of C++ structures. The class body is similar to the script structure described on the fan site. It can be used in further analysis.

The first four bytes represent the m_ulGlobalVarCount field followed by the m_pGlobalVarTypes byte array. Next comes boost::scoped_array. After it is the std::map dictionary. It’s not clear from the definitions what bytes represent them in the class body. The easiest way is to record values of each field for any known file in the debugger. One way or another, all these fields should be reduced to simple C structures in the class dump.

For this purpose, I enable in IDA addressing relative to the function beginning, and then enable synchronization between the assembler and pseudocode windows. Now you can find a specified address in any running debugger by entering the string CScript::CScript + E3 and view real values of the fields in dynamics.

Calls to functions belonging to the qmemcpy type are very helpful: the source data are stored in the ESI register; while the number of bytes to be copied, in ECX. This makes it possible to understand where a long string (not a field of the int type!) is supposed to be located in the file.

Some fields (e.g. m_ulGlobalVarCount) can be null, which means that the structure you are interested in doesn’t follow them. To detect interaction with files that contain this structure, conditional breakpoints like EAX != 0 can be used.

Using this technique, I restored file headers in 010 Editor. The procedure is simple: you make a guess, reflect it in the template, and check the values in the debugger. Then repeat until the picture comes together. The restored template is too large to list it here; it’s available in my repository on GitHub. Below is a simplified version:

The file begins with defining global variables. Next comes the ‘pool’ consisting of strings in Unicode format. It’s followed by definitions of the functions used (in fact, this is the import section). The field m_ulRunOp specifies the command used to run the script. Tasks and events serve the same purpose. The headers end with an array of instructions, which is great: the file format analysis was actually performed to get them!

Examining bytecode

I decided to look at other methods of the CScript class and came across the following function:

After setting a breakpoint on it, I discovered that the ulType argument takes an opcode number; while the reader object refers to the current position in the read fire.bin file. To find out where the numbers are and where the arguments are, I decided to collect numbers using a conditional breakpoint:

The following list appears in the log:

Finding these numbers in the file body:

The opcode number comes first followed by arguments of variable length. So, I have no choice but to examine each number separately and produce a map.

Inside the CreateInstruction function, there is a huge switch with handlers for each number. The first objective is to write a length disassembler. It’s not really important how the opcode works; what’s important is how much space it takes up in the file. Then you can read the rest of the stuff correctly.

So, how much space does it take? The class body can be used as a hint:

Inheritance of the IInstruction interface ensures that the first four bytes act as a reference to vftable. Then go two fields: m_ulVarIn and m_ulVarOut. Looking at the code, you can see that the cursor of the reader object is shifted twice by four bytes, which exactly corresponds to the two fields in the class body. This means that eight bytes are read. Let’s create the same structure in the template:

CreateInstruction processes 87 opcodes, and a few liters of coffee later, I managed to examine them all. All that remained was to compare opcode numbers in the template with the list of intercepted numbers in the debugger. After a while, I identified and fixed the errors, and my template became applicable to all scripts.

Parsing the code without errors is half the battle. The next step is to understand what it does.

Writing decompiler

I already have a list of opcodes whose names indicate what they do. But in the ideal case, I need readable pseudocode clearly showing what the script does. Following the logic of object-oriented programming, an opcode object should contain the execution code that is called when the script is executed. And indeed, searches by name bring the Execute method belonging to the class with the same name:

As you can see, the code increments the global instruction counter m_ulCurOp, a local analogue of the EIP register.

Logically, the Set method should be called, but in reality, bytes at the address that I dubbed vSet are ‘called’. The point is that IVariableEx is an interface, and its specific implementation depends on the variable type. 24 bytes are left for six references to six different types of variables. The general interface leaves space for a specific reference, but doesn’t know its exact location. So, this can be considered a call to the Set method. The code uses it to pass the value of the m_ulVarIn variable to the m_ulVarOut variable. In pseudocode, this can be represented as follows:

The m_Stack object represents std::vector, a dimensionless array from a standard C++ library. It implements the stack of the virtual machine written in a makeshift scripting language. This stack operates on the LIFO (Last In, First Out) principle (i.e. adds the last element to the end of the list and takes the next element from the end). This is why the code uses negative numbers: they represent negative indices from the end of the list.

I decided to write my decompiler in Python since this language is perfectly suited for binary parsers. First, I had to create my own reader class to read the byte stream in a convenient way:

Thanks to this class, adding a new opcode became a very simple operation, and the code is easy to read:

In the code produced by IDA, it’s not always clear what a complex instruction does. The easiest way is to check the state of the stack (where all variables are stored) before and after the instruction is executed. To do this, I had to write another script for the debugger (but this time for IDA Pro).

When you fall inside an instruction like CInstructionMov, the second argument is CScriptRun::Data *data: a reference to a structure with global variables. Let’s read it:

IDAPython makes it possible to read register and memory values during debugging:

I take a reference from the stack and unwind it to the required structures located at known offsets:

An interesting trick is used here. The stack contains references to objects: classes derived from CVariableBase. The first four bytes in the body of any object represent a reference to vftable. Its address can be used to get the class name provided by debugging symbols. And if you know the class of a variable, you can easily read its values:

By dumping values before and after the instruction is executed, I managed to parse the opcodes without digging through tons of C++ code decompiled with errors. Of course, if an instruction behaved unexpectedly, I had to look into the decompiler, but most instructions were parsed without any hassle.

All I had to do was parse the code of the remaining 86 opcodes in a similar way. About a week later, I got an operational decompiler. If I try to describe each opcode separately, this article would turn into a book. The full decompiler code is available on GitHub.

Migrating from the alpha version

The alpha version stores in its archive 343 scripts dated July 2004. The official release of “Pathologic” contains 978 scripts dated up to October 2005.

I am interested in scripts of the full version, but when I try to open them using the 010 Editor template created for the alpha version, a template engine error occurs. The headers are parsed properly, which means that their format hasn’t changed, but the list of opcodes breaks the template execution. To find out what’s the matter, I have no choice but to compare both engines.

To do this, a plugin for IDA Pro called Diaphora is required. It’s designed to compare two versions of the same file. The patch diffing technique makes it possible to find the place fixed by a security patch and write an exploit for the old version. The same method can be used to determine the location of known functions to understand what has changed in the new version of the engine.

IDA is renowned for its backward compatibility with its old versions; so, at the end of my research, the plugin crashed with an error.

OK, I’m examining the source code of the file C:\Program Files\IDA Professional 9.0 SP1\python\idc.py (since I remember that the function should be there) and expectedly don’t find it there. Then I’m examining the code of the old version:

IDA developers like to change the names of functions while leaving old comments to them. I search the text for a copy of the comment and find the new name:

I make changes in the Diaphora code and restart the plugin. This time everything goes well. The plugin creates a SQLite database for both files and then compares them to find similar functions. At the output, I get several new windows displaying lists of similar functions. Then I take a snapshot of the database and click “Import all functions”.

Now I have an almost completely labeled database for the new version of the engine. But I still have to find changes in the code of the CreateInstruction function. I compare all its instructions one by one until I stumble upon the first differences:

Two new opcodes were added close to the end of the list! The first one takes up nine bytes and does nothing useful. But the second one transfers control and essentially repeats the CInstructionCall code (so I named it Call2 in the decompiler). The remaining opcodes operate the same as before.

Examining existing scripts

The decompiler is ready, and it works. Sometimes, the decompilation isn’t very accurate, but you still can understand what this or that script does. To prevent opcodes from merging into a single array of text, I decided to separate them into typical basic blocks by adding line breaks after control transfer instructions.

bed.bin

This script is responsible for the behavior of beds in the game.

When an object is created (i.e. when the player appears in the immediate vicinity), the code at the address RunOp is executed. The addresses represent ordinal numbers of opcodes (similar to line numbers in BASIC). The first (i.e. null) opcode passes the value 1 onto the stack. The second one calls the SetVisibility function with a single argument that is taken from the top of the stack. Then an infinite loop consisting of the Hold function calls starts.

The GlobalTasks header contains event addresses (something like callbacks that are called in response to certain events in the game). As far as I understand, the only ‘event’ in the life of a bed object is a situation when the player presses the action button. The code begins at offset 0x7 and executes the ActivateSleepMode function (i.e. opens the sleep time selection menu).

item_milk.bin

Below is the milk object code:

The list of commands is too long to quote, and I just provide a possible execution path:

At the beginning of the script, the drink sound is played using the PlaySound function. Then the value of the hunger attribute is checked, and if hunger exists, 0.07 is subtracted from it. Now you know how a bottle of milk works. And, as you can see, the decompiler operates without errors.

Existing Mods

First, I decided to check existing modifications for “Pathologic”. It turned out that most noteworthy mods are based on Rat Fog Editor. Some madman has methodically edited all bytes in the weather.bin script to identify a pattern in game’s reactions to changes.

Indeed, the script contains plenty of float values that are responsible for lighting, fog, and color correction.

The remaining decompiled scripts are available in my repository on GitHub; I assembled it using four different versions of the game.

Modding

The first thing that came to my mind was to change the contents of municipal urns. According to the game, the main function of an attending physician is to rummage through garbage. Let’s ease Bachelor Dankovsky’s burden by placing a cure in each urn. To do this, open the urna.bin script.

The script calculates the probability of an item to appear in the urn, and if the player is lucky, then the bottle_empty object is added to the urn using the AddItem function. In another script, you can see the name powder; so, all you have to do is replace a single line in this script with another one.

Conclusions

The debug information published together with the game made it possible to reconstruct its code and create modifications for many games.

To create a fully-functional SDK, plenty of work is still required, but a start has been made. The decompiler is far from perfection, and not all commands are displayed correctly. The next step is to decompile game scripts into C-like code (like Hex-Rays).

Congrats! Today you’ve reverse-engineered software written in C++; while the “Pathologic” community got an operational (albeit still raw) modding tool.