Hardcore debugging. Reverse-engineering an ARM application for QNX in QEMU

You’ve probably noticed that most of my publications discuss Windows tasks on Intel processors. That’s not good, and I’m trying my best to expand the scope of my studies by exploring exotic virtual machines and processors.

Virtual machines are covered in my articles to a lesser extent for a serious reason: they’re very difficult to debug dynamically, and the only solution is static analysis. However, an inquisitive mind will always find an unconventional approach (e.g. various emulators). Today, I’d like to draw your attention to the legendary QEMU emulator that can be used to analyze applications running on ARM QNX. Furthermore, you have the fortunate opportunity to examine ARM in dynamics on your favorite PC and even on Windows!

A device running the QNX operating system on an ARM processor will be used as a test system. For ethical reasons, I won’t disclose the program name or specific details of its operation. In addition, the application dump procedure, as well as QEMU installation and image loading, are beyond the scope of this article since they are addressed in detail in numerous publications.

Under the conditions of the problem, the device’s file system has already been loaded to QEMU and is operating there successfully. Your ultimate goal is to make the activation code (blue frame in the screenshot) in the activation string (red frame) valid using a patch or keygen. This activation string is fed to the program using a special Lua activation script. In the screenshot below, the validation result is shown in the yellow frame.

First, let’s connect the GDB debugger. This procedure is described in multiple articles, but, for some reason, all of them omit one important aspect: configuring the network connection. A novice user might get the false impression that the -s -S option is sufficient for GDB to detect QEMU on port 1234.

Of course, in reality, this isn’t true; so, let’s discuss in more detail how to connect the debugger via a virtual network adapter. In fact, the method presented on the QEMU website is the only way I was able to connect. I I’ll try to explain the step-by-step sequence of actions required to implement it in my own words since it’s not entirely obvious.

First, you have to create a new network connection named tap0 (Control Panel → Network and Internet → Network Connections).

It’s not that easy to create it: I managed to do this only after installing OpenVPN. After creating a virtual adapter, you specify the following settings.

Next, you add the -nic tap,ifname=tap0,id=hub0 option to the QEMU command line; after that, you can test connections between debuggers and the emulator. I understand that you’ve got accustomed to the handy and comfy x64dbg (and the funny thing is that it also can be used to debug applications in the emulator), but let’s save such exotica for a future article. This time, I’m going to use the handiest debugging technique involving the popular IDA disassembler.

You’re probably aware that this disassembler features its own debugger. Of course, universal tools are always worse than specialized ones, and the debugger embedded in IDA is significantly inferior to x64dbg in terms of convenience and reliability. But in this particular case, it’s the lesser of all evils.

The procedure required to connect IDA to QEMU via GDB is described in detail (and with color images) on the Hex-Rays website, but again, I’ll try to explain it in my own words as there are some important nuances.

After configuring QEMU as described above and starting it, you enter the command pdebug 1234 into its console. Then you open the Debugger → Attach → Remote GDB debugger menu in IDA, go to the connection settings window, and enter the following information (based on the above-selected settings).

Next, in the GDB configuration window (Debugger options → Set specific options), you have to specify the processor type (in this particular case, ARM Little-endian). Otherwise, IDA would likely crash during the connection.

When you attempt to attach, IDA requests the name or PID of the process running in the emulator. You don’t know it, and the selection list is empty.

Fortunately, if you select process 0, IDA successfully connects to the emulator, which indicates that you’ve done everything correctly.

Still, let’s try to find out which process is handling your request. First, let’s review the list of processes running in QEMU. Too bad, QNX doesn’t support the familiar ps command, but it supports pidin ar that outputs the data you’re looking for.

Fortunately, the number of processes isn’t large. Let’s figure out which one you’re interested in. In the very first screenshot, the validation result is shown in the yellow frame. The substring validState: is likely present in the desired module. Using grep, you can quickly find it. Let’s extract this module and load it to IDA — the string you’re looking for is indeed there.

Technically, the mission can be considered accomplished at this point since all you have to do is patch the check

so that it becomes an unconditional transition:

But let’s try to get to the initial check since this function could be just a debug status print. The listing indicates that code’s validity depends on the a2 parameter passed to the function: if it points to 0, the code is valid. The a2 parameter is passed to sub_1183F8 via the R1 register, but it’s difficult to understand where it comes from: there are no direct references to sub_1183F8.

This is an excellent opportunity to test the debugger! Let’s set a breakpoint (F2) at address 1183F8 and click Debugger → Attach to process. If everything was done correctly, IDA will successfully attach to the process, and it will be displayed in blue in the green bar at the top of the window.

If something goes wrong and IDA starts complaining about access rights and other issues, double-check the addresses and settings in the Debugger → Process options window.

Click Debugger → Continue process (F9), and the Running window with a single Suspend button will appear. After that, run the activation script in the QEMU console, and the process will pause at your breakpoint. In the Stack view window (the bottom right one), you can see the return address: 4FD1D4.

Let’s try to convert this fragment into pseudocode.

The sub_1183F8 function is called from sub_4FD0DC, and its second parameter is called v12 there. The a2 value is assigned to this address (coincidentally, it’s the second parameter of sub_4FD0DC). The message two lines above this assignment indicates that a2 is the result of activation, which seems quite logical: 0 is successful activation; otherwise an error number is returned (in this case, 2). If you move up the return stack a bit, you can find the reference to sub_4fd0dc.

It looks like you’ve found the error handler and the correct code path with successful activation (sub_4FD0DC(a1, 0);). There’s no need to go into further details since they are purely technical: you can either patch this function so that it takes the correct path, or find and reverse-engineer the activation code validation algorithm to create a keygen (a more professional approach).

Whatever you choose is irrelevant to the topic of this article, but in conclusion, I’d like to explain how to implement the above-described sequence of actions if IDA, for some reason, doesn’t see QEMU. Unfortunately, such a possibility is quire real, and I couldn’t find satisfactory explanations online; therefore, you should always be prepared for the real hardcore: debugging with bare GDB. Let’s try to reconstruct the process step by step.

Note that this path is difficult and thorny; difficulties start right at the beginning: you have to build the GDB debugger for ARM and QNX. GDB is free open-source software distributed as source code, but building a binary from this code is a separate quest, which is definitely beyond the scope of this article.

Numerous tips on this matter can be found on the Internet, but for the sake of brevity, let’s assume that you already have a compiled GDB debugger binary for your processor version. In this case, you configure the QEMU emulator as described above, start it, enter the command pdebug 1234 in the console, start GDB in another window, and enter the connection command:

If everything was done correctly, the following message should appear in the GDB window.

Now you again have to attach to the desired process from the GDB debugger. In the screenshot showing the process list, you’ve identified the PID of the desired process: 512024. To attach to it in the GDB console, enter the command:

For now, you can disregard numerous error messages: GDB has successfully attached to process 512024 and, similar to IDA, stopped at the intermediate breakpoint 1037878. The emulator also pauses at that moment, and you can take advantage of this to set a breakpoint at address 1183F8 (as described above). Now, if you don’t mind, I’d like to briefly outline distinctive features of the GDB debugger.

No doubt, debugging an application in the GDB debugger is a quite exotic process. GDB, as you’re already aware, has no GUI and is controlled by console commands. In this regard, it most closely resembles the DOS debugger debug.com (being an old-school hacker, I used to work with it extensively and painstakingly). A complete list of GDB commands can be found on sourceware.org. Right now, you’re interested in breakpoint-related commands.

Even a brief review of GDB’s breakpoint options shows that, despite its extreme user-unfriendliness, this is an amazingly flexible tool whose capabilities are on par with x64dbg’s conditional breakpoints (and even surpass them in some cases). Each breakpoint can be equipped with sophisticated scripts to customize program execution in the debugger in accordance with your needs and preferences. These scripts can even call functions and code blocks implemented within the application, but this topic deserves a separate article. For now, you just need simple breakpoints at address 1183F8:

Everything seems to be simple and clear. The asterisk before the address means that the following string, 0x1183F8, is a hexadecimal address, not an identifier from the debug symbol table (which you didn’t attach to the debugger as you don’t have one). Next, you use the c (continue) command to unfreeze QEMU (while the debugger, by contrast, freezes waiting for a breakpoint) and run the activation script in the QEMU console.

As you can see, the breakpoint was triggered at address 1183F8; you’ve paused on it in GDB; while the program execution in QEMU froze again. With the program paused, you can perform various operations (e.g. examine the code you’ve stopped on). To display the five disassembled instructions at the address pointed to from the PC register, enter the command x/5i $pc in GDB — and you’ll get the following output:

Are you interested in the return stack? Just enter x/4x $sp, and you’ll get the top four values on the stack:

To view the value of a given register, use the print $r0 command. To take a step forward, enter the stepi or nexti commands (not to be confused with the step and next commands that won’t work due to the absence of a loaded symbol table). If you want to continue your work after stopping at the address 118408 without setting an additional breakpoint, type:

Conclusions

As you can see, with certain skill and diligence, you can successfully perform debugging tasks without enlisting IDA. Neither exotic processors nor unusual hardware can prevent an inquisitive mind from debugging and dynamic analysis of applications in a familiar operating system.

Good luck!