Google Chrome Dino Bot Using Image Recognition | python



What would you see in the Chrome browser when there is no internet connection? Yes, everyone knows that dinosaur game that appears on the screen. So, in this article, we are going to create a simple python bot that will play the Chrome Dino Game without user intervention. We are not using any machine learning or artificial intelligence here to counter this problem, but we will use simple image / screen processing.

We will be working with Pyautogui and PIL (Python Imaging Library) for implementation. This project is very simple and only contains about 50 lines of code, but the result will surprise you.

Some libraries used:

  1. PIL : Python Imaging Library (PIL) — is a free library for the Python programming language that adds support for opening, manipulating and saving many different image file formats.
  2. Pyautogui: PyAutoGUI — is a Python module for programmatically controlling mouse and keyboard without any user intervention.
  3. Time: Python “Time” A module that allows us to perform various operations related to time, its transformations and the views that find its application in various applications in life.
  4. Numpy: NumPy — is a library for the Python programming language that adds support for large multidimensional arrays and matrices, as well as a large collection of high-level math functions for working with these arrays.

Algorithm —

  1. Click the reload button using the Pyautogui library using the coordinates “replaybutton”.
  2. Calculate the sum of all white pixel values ​​present in the field in front of the Dinosaur.
  3. If the sum of the pixel values ​​present at any time in the field becomes less than the sum of the white pixel values, it means that a “bush” or “bird” is coming. So either we have to make our Dinosaur jump or bend over.
  4. To protect Dino from the Bush, we take a jump.
  5. To protect Dino from the Bird, we always suppress Dinosaur.

Below is the Python implementation —

# import the above libraries into
# implement functionality

from PIL import ImageGrab, ImageOps

import pyautogui

import time

import numpy as np 

  

class < code class = "plain"> cordinates ():

 

  # coordinates of the play button to start the game

replaybutton = ( 360 , 214 )

# these coordinates represent the top right coordinates

  # which will be used to define the front panel

  dinasaur = ( 149 , 239

 

def restartGame ():

  

  # using the pyautogui library, we click on

# button playback without any user interaction

pyautogui.click (cordinates.replaybutton)

 

# we will keep our bot always at the bottom

# prevent birds from hitting

pyautogui.keyDown ( `down` )

 

def press_space ():

 

# release Down key

pyautogui.keyUp ( ` down`

 

# press space to get past the Bush

  pyautogui.keyDown ( `space` )

 

# to easily recognize the space

time.sleep ( 0 .05

 

# printout of the "Jump" statement to

# terminal to see the current output

print ( "jump " )

  time.sleep ( 0.10 )

 

# release the spacebar

pyautogui.keyUp ( ` space` )

 

# again pressing the Down key so that my bot is always at the bottom

pyautog ui.keyDown ( `down` )

  

def imageGrab (): 

# determining the coordinates of the box in front of the dinosaur

box = (cordinates.dinasaur [ 0 ] + 30 , cordinates.dinasaur [ 1 ],

cordinates.dinasaur [ 0 ] + 120 , cordinates.dinasaur [ 1 ] + 2 )

 

# capture all pixel values ​​as RGB tuples

image = ImageGrab.grab (box)

  

# convert RGB to grayscale in

  # make processing easier and faster result

grayImage = ImageOps.grayscale (image)

  

  # using numpy to get the sum of all pixels in grayscale

  a = np.array (grayImage.getcolors ())

 

  # return amount

print (a. sum ()) 

return a. sum ()

 

 

 
# function to restart the game
restartGame ()

< code class = "keyword"> while True

  # 435 - the sum of the values ​​of the white pixels in the block.

  # You can get a different value if you take more

# or less than the field accepted in this article.

# if the value returned function imageGrab, not equal to 435,

# this means that a bird or bush is approaching a dinosaur

if (imageGrab ()! = 435 ): 

  press_space () 

# time to recognize the operation by the above function

time.sleep ( 0.1

Output:

Improvements: Over time, Dino Bot gets faster. Birds and bushes begin to arrive very quickly. Thus, we do not force our Bot to learn all this, changing its speed based on past learning. Thus, our bot will run around 2000 points. To score more points, we must apply machine learning and artificial intelligence.