Google Chrome Dino Bot Using Image Recognition | python

What would you see in the Chrome browser when there is no internet connection? Yes, everyone knows that dinosaur game that appears on the screen. So, in this article, we are going to create a simple python bot that will play the Chrome Dino Game without user intervention. We are not using any machine learning or artificial intelligence here to counter this problem, but we will use simple image / screen processing.

We will be working with Pyautogui and PIL (Python Imaging Library) for implementation. This project is very simple and only contains about 50 lines of code, but the result will surprise you.

Some libraries used:

1. PIL : Python Imaging Library (PIL) — is a free library for the Python programming language that adds support for opening, manipulating and saving many different image file formats.
2. Pyautogui: PyAutoGUI — is a Python module for programmatically controlling mouse and keyboard without any user intervention.
3. Time: Python "Time" A module that allows us to perform various operations related to time, its transformations and the views that find its application in various applications in life.
4. Numpy: NumPy — is a library for the Python programming language that adds support for large multidimensional arrays and matrices, as well as a large collection of high-level math functions for working with these arrays.

Algorithm —

1. Click the reload button using the Pyautogui library using the coordinates "replaybutton".
2. Calculate the sum of all white pixel values ​​present in the field in front of the Dinosaur.
3. If the sum of the pixel values ​​present at any time in the field becomes less than the sum of the white pixel values, it means that a "bush" or "bird" is coming. So either we have to make our Dinosaur jump or bend over.
4. To protect Dino from the Bush, we take a jump.
5. To protect Dino from the Bird, we always suppress Dinosaur.

Below is the Python implementation —

# import the above libraries into # implement functionality from PIL import ImageGrab, ImageOps import pyautogui import time import numpy as np     class < code class = "plain"> cordinates ():     # coordinates of the play button to start the game replaybutton = ( 360 , 214 ) # these coordinates represent the top right coordinates   # which will be used to define the front panel   dinasaur = ( 149 , 239 )    def restartGame ():      # using the pyautogui library, we click on # button playback without any user interaction pyautogui.click (cordinates.replaybutton)   # we will keep our bot always at the bottom # prevent birds from hitting pyautogui.keyDown ( ’down’ )   def press_space ():   # release Down key pyautogui.keyUp ( ’ down’ )    # press space to get past the Bush   pyautogui.keyDown ( ’space’ )   # to easily recognize the space time.sleep ( 0 .05 )    # printout of the "Jump" statement to # terminal to see the current output print ( "jump " )   time.sleep ( 0.10 )   # release the spacebar pyautogui.keyUp ( ’ space’ )   # again pressing the Down key so that my bot is always at the bottom pyautog ui.keyDown ( ’down’ )    def imageGrab ():  # determining the coordinates of the box in front of the dinosaur box = (cordinates.dinasaur [ 0 ] + 30 , cordinates.dinasaur [ 1 ], cordinates.dinasaur [ 0 ] + 120 , cordinates.dinasaur [ 1 ] + 2 )   # capture all pixel values ​​as RGB tuples image = ImageGrab.grab (box)    # convert RGB to grayscale in   # make processing easier and faster result grayImage = ImageOps.grayscale (image)      # using numpy to get the sum of all pixels in grayscale   a = np.array (grayImage.getcolors ())     # return amount print (a. sum ())  return a. sum ()       # function to restart the game restartGame () < code class = "keyword"> while True :    # 435 - the sum of the values ​​of the white pixels in the block.   # You can get a different value if you take more # or less than the field accepted in this article. # if the value returned function imageGrab, not equal to 435, # this means that a bird or bush is approaching a dinosaur if (imageGrab ()! = 435 ):    press_space ()  # time to recognize the operation by the above function time.sleep ( 0.1 )

Output:

Improvements: Over time, Dino Bot gets faster. Birds and bushes begin to arrive very quickly. Thus, we do not force our Bot to learn all this, changing its speed based on past learning. Thus, our bot will run around 2000 points. To score more points, we must apply machine learning and artificial intelligence.