Blindfold game. Manage your Android smartphone via ABD

The above-described issue was solved as follows: ADB managed to drop down the status bar and select the required mode in the settings menu. Then a script with ADB commands was created in /etc/rc.local folder to automate this process on each boot. But are there any other things you can implement with ADB in a similar way? Oh, sure!

In late 2021, World Chess Championship was held in Dubai. Interest in this event in Russia was fueled by the fact that the Russian Ian Nepomniachtchi had challenged the current world champion Magnus Carlsen. My friend and I closely watched the tournament and even decided to play in tandem against Magnus. But since he was busy with the championship, we had to play against his electronic clone: the Play Magnus app for Android.

The difficulty level in the app is set by the age of Magnus Carlsen whose professional skills are truly amazing: we never managed to reach the ’13 years’ level – at this age, Magnus became one of the youngest grandmasters in the world. Another impressive ability of professional chess players is playing without looking at the chessboard. The record belongs to Timur Gareev who simultaneously played blindfold on 48 boards! My friend was eager to master this skill, and I decided to help him by writing a script acting as a wrapper for the chess app.

The script logic is simple: you enter your move (e.g. ‘e2e4’) on the computer keyboard, and the program performs a sequence of taps on the screen of your phone via ADB. Then the entire history of the game is copied to the phone clipboard and transmitted to the computer. The last move (i.e. the countermove of the chess app) is displayed on the screen. For a better understanding of what’s going on, you can watch a demo of this scrip in action.

ADB

Android Debug Bridge (ADB) is a handy debugging tool for Android devices. Among other things, it allows you to move files from your computer to your phone and vice versa using the commands adb push filename and adb pull filename.

To install or remove an app, just type adb install program.apk or adb uninstall <package name>. To connect to the phone command shell, use the adb shell command. ADB supports plenty of other functions, but for my purposes, it was important that it can simulate interactions with the phone’s touchscreen (i.e. make taps and swipes). Overall, I strongly recommend using ADB if you are a hacker pentester specializing in Android.

Prior to using ADB for the first time, you have to enable USB debugging on your phone. To do this, open “Settings”, go to the “System” section, and then to “About phone”. Here, look for the item “Build Number” and tap it several times. An additional item, “Developer options”, will appear in the “System” section. Go there and activate “USB debugging”.

After enabling ADB on your phone, you have to install the ADB server on your computer. If you are a Linux user, enter the command sudo apt install adb in the command-line interpreter. On a Windows PC, you have to download the archive with utilities. Unzip it to any location you like (e.g. C:\platform-tools). Then open the command prompt by pressing Win + R and typing cmd and go to the folder containing the ADB utility by typing cd C:\platform-tools.

Now you can connect your phone to the computer using a USB cable. You may need to switch your phone from “Charge” mode to “File Transfer” mode. Then type adb devices in the command line. The first time you connect, you should grant debugging permissions to the new device. Check the “Always allow from this computer” box on your phone and click OK.

You can continue using ADB via a USB cable, but it’s not always handy. To operate remotely over network, enter the command adb tcpip 5555: ADB will wait for a connection from your PC on TCP port 5555. After that, you can disconnect the cable and enter the command adb connect <phone IP address> on your computer.

Now all ADB commands will work as if the phone were connected with a cable. This setting remains in effect until the phone is rebooted. Now you have to be careful: make sure not to allow an attacker to connect to your phone by accident. If an unknown device connects to the phone, it will notify you by displaying a message. You can “revoke USB debugging permissions” in the “Developer options” section next to the item enabling USB debugging. The adb usb command switches the phone back to the USB cable.

Writing code

I am going to write my script in Python 3. First of all, I have to import the required modules and define the path to the ADB utility in the system.

Writing the help function.

The main logic will be implemented in a class called Screen. I set the parameters board_x0, board_y0, board_x1, and board_y1 representing the x and y coordinates of the upper left and buttom right edges of the chessboard, respectively. The menu and close_menu represent positions of the buttons that open and close the menu. The save_game, ok_button and back_button stand for coordinates of the Export Game button, OK button, and Undo Last Move button, respectively (see the screenshot). Pawn is an array of coordinates of buttons used to select a new queen, rook, bishop, or knight to replace a pawn when it moves to its last rank.

Since coordinates of most elements depend on the screen resolution, they have to be set for each device individually. To determine coordinates, I suggest activating “Show taps” and “Pointer location” functions in “Developer options”. I add the game variable to store the progress of the game. I also define TAP_SLEEP (delay between taps on the screen) and MOVE_SLEEP (pause after each move). It should be large enough to let the program make a countermove.

When you press the “Export Game” button, all moves made since the beginning of the game are written to the clipboard using the special chess notation, PGN. But the problem is that ADB cannot directly get data from the clipboard. To solve this problem, I use the Clipper app that allows to read from the clipboard and write to it using broadcast intents. Clipper is an open-source app, but don’t worry: you don’t have to compile it for Android. The repository contains a ready-use apk file. Download and then install it by executing the adb install clipper.apk command in the folder containing the apk file.

Time to write the __init__() function: it will run the Clipper app and calculate the size of chessboard squares using the recently defined constants.

Then I define the _tap_screen() function that takes x and y coordinates and simulates tapping the phone screen at this location.

For convenience purposes, I create another function: _tap_board(). Its only difference from _tap_screen() is that its x and y coordinates are relative to the chessboard, not the screen, and the reference point is placed in the bottom left corner of the playing area.

The _copy_game() function is used to export the game progress to the clipboard. It simulates taps in the respective places of the screen. To get data from the buffer, the Clipper app is called by executing the command adb shell am broadcast -a clipper.get. The data I need are contained in the output of this command in the data field.

To make moves, I am going to write the move() function that takes as input a string in the ‘e2e4’ format ignoring case distinctions. If the ‘back’ string is passed instead of a move, the program will undo the last move I’ve made. To calculate the coordinates of the required chessboard square, I use the string method find. It searches for the move letter (or number) in the string abcdefgh (or 12345678) and returns its index.

This index represents the position of the required square on the chessboard to tap on. If I play the Black, the order of the letters abcdefgh (as well as the numbers 12345678) should be reversed, because the app flips the chessboard. If a pawn moves to its last rank, then an additional letter will determine the piece that replaces it. For instance, if you make the move a7a8r, then the pawn will be replaced by a rook.

Finally, I have to write the answer() function that receives Magnus’ countermoves. A situation when the game progress doesn’t change after the player’s move must be taken into account as well. It occurs if the player’s move was incorrect. Such a situation is quite possible when you play blindfold chess. In this case, the ‘ILLEGAL MOVE!’ message will be displayed. If the move was correct, then the last move in the game (i.e. countermove of the chess program) has to be written to the answer variable. The state variable contains the result of the game. The * character in state means that the game is ongoing (see the PGN description). At the end of the game, state will contain the White and Black scores.

The Screen class is ready now. It’s time to write the main loop of the program. Help will be displayed if the -h command line option is specified. The is_black flag is responsible for the color of the player’s pieces; it will be set by the -b parameter. Next, I create an s class instance object and launch an infinite loop with moves and countermoves; it runs until the game ends. The criterion for the end of the game is the absence of the * character in the state variable. In addition, if the player is playing the Black, the program must first read the initial move of the White. You can exit the game at any time by pressing Ctrl + C.

That’s it. Prior to running the script, make sure that your phone is on the adb devices list.

Conclusions

You may have noticed that I don’t control the execution of commands sent to the phone. Yet another ‘blindfold game’… Of course, this is a significant drawback of the approach used. If you want, you can add feedback to the above algorithm, e.g. by taking a screenshot with the following commands:

Note that the ADB server can also be installed on Android to control the target phone from another phone (or even from itself!). To do this, you can use Termux as a terminal emulator for Android. The program code is available on GitHub. Feel free to use it as a basis for your own scripts. Good luck!