Bot Development Challenges: Unraveling Those “Easy Projects”

In the world of mobile games, our team member, Nuno Aguiar, is on a mission to build a bot for a unique tower-building challenge during the training hours that Exaud offers to all its employees. Picture blocks dropping, taps on the screen, and a digital dance unfolding. This article tracks Nuno’s coding journey, crafting a virtual architect, with the goal of comprehending and mastering the game’s intricacies through the lens of artificial intelligence. Join Nuno in exploring the challenges of bot development and the nuances of seemingly ‘easy’ projects.

Solving the Timing Dilemma in Bot Development

I’m trying to develop a bot to play a mobile mini-game that involves constructing a tower with blocks that descend when you tap the screen. The block dropper moves horizontally on its own, requiring precise timing to erect the tower without any tilting, or it risks collapsing.

Problem: How does the bot “know” the exact time to tap?

Let’s analyze the problem and see how can we solve it.

At the beginning of the game and after every block drop, the block dropper moves from the center of the screen to the right at a fixed speed. After hitting the wall, it shifts left at an incremented speed based on the game level. The higher the level, the faster the speed. Upon hitting the left wall, it turns around and heads for the right wall, repeating this cycle until the user taps to drop the block. You can only drop a block when it’s moving at this incremented speed; otherwise, tapping instantly when it starts at the center of the screen would be ineffective.

Deciding on the Right Path
Because of this, the first solution that came to mind was to program the timing of the tap since we know the time it takes from the center to the left. after that, we could probably find a formula through analysis or trial and error.

The catch: The game doesn’t run perfectly consistently. there will be a point in time when, , for whatever reason, the game skips a frame, slows down somewhere, and all that perfect timing is useless.

Navigating the Complexity of Pixel Detection

So, another solution: we could see when the dropper reaches the center of the screen by looking at its pixels and tapping at that moment. However, with this solution, a different problem arises: we need to capture the screen at the same speed the game is displayed (30 fps), check if the pixels of the dropper are there, and then tap. This operation can be very CPU intensive. By the time we confirm if the dropper is at the center, meaning it’s time to tap, the dropper could have already moved a few pixels more, causing the block to drop in the wrong place.

Weighing the Pros and Cons

Okay, so what if we don’t search the entire area for the dropper but instead check if a single pixel from the center has changed, considering the background is “static”? This would be a less demanding process, and there should be fewer mistimed drops, right? Let’s explore this approach.

While scouring the Internet for a way to capture a single pixel from the screen quickly, we realize that the most available and straightforward methods involve capturing the entire screen first and then extracting the pixel from that image. This is a slow process and would be as fast as the solution of searching for the dropper in the center of the screen , which would also need to be captured . After further investigation, we uncovered a potential solution to swiftly retrieve a pixel from the screen that doesn’t entail capturing the entire screen. However, this is not guaranteed to work and demands a more complex implementation.

And here’s the moment when you need to choose a solution. Should you opt for the path of non-guaranteed solution that demands more development time but, ifsuccessful, could be the best approach? Or should you go with the method of simply timing the tap event, which is easier and quicker to implement but might lead to more mistimed drops? It all depends on the ultimate goal: is it to achieve the highest score possible in the game? To showcase a bot in action to your peers? Do you have the time to pursue a more complex solution, and have you considered exploring alternative solutions online?

For me it was to see how far the tower could grow, essentially aiming for the highest score. Therefore, the optimal choice was to implement the more complex solution to guarantee the best results. Did it work? Well, given that this solution requires additional time, the project in currently on hold since I haven’t allocated that time yet.

What about you? What approach would you take?