Pattern matching using OpenCV in Python



Matching with pattern — it is a method of finding areas of an image similar to a patch (pattern). 
Patch — it is a small image with certain functions. The target of pattern matching — find the patch / pattern in the image. 
To find it, the user must provide two input images: original image (S) — the image in which to find the template, and the template image (T) — the image to be found in the original image.

  • This is basically a method for finding and finding the location of the image template in the enlarged image.
  • The idea here is to find identical areas of the image that match the pattern we provide by giving the threshold
    • The threshold depends on the precision with which we want to detect the pattern in the original image.
    • For example, if we apply face recognition and want detect human eyes, we can provide a random eye image as a template and search for the source (human face).
    • In this case, since the “eyes” show a lot of variation from person to person, even if we set threshold of 50% (0.5), the eye will be detected.
    • In cases where nearly identical patterns are to be searched, the threshold should be set high (t & gt; = 0.8)

      How does templating work?

    • The template image just slides over the input image (like in 2D convolution)
    • The template and patch of the input image below the image template are compared.
    • The result is compared to a threshold.
    • If the result exceeds the threshold, the portion will be marked as detected.
    • B function cv2.matchTemplate (img_gray, template, cv2.TM_CCOEFF_NORMED) first parameter — this is mainimage, the second parameter is — this is the pattern to match, and the third parameter — the method used for the mapping.

    # Python program for illustration
    # pattern matching

    import cv2

    import numpy as np

     
    # Read main image

    img_rgb = cv2.imread ( `mainimage.jpg` ). 

     
    # Convert it to grayscale

    img_gray = cv2.cvtColor (img_rgb, cv2.COLOR_BGR2GRAY)

     
    # Read the template

    template = cv2.imread ( `template` , 0 )

     
    # Save the width and height of the template in w and h

    w, h = template.shape [:: - 1 ]

      
    # Perform matching operations.

    res = cv2.matchTemplate (img_gray, template, cv2.TM_CCOEFF_NORMED)

     
    # Specify a threshold

    threshold = 0.8

      
    # Store the coordinates of the matching area in an array

    loc = np.where (res & gt; = threshold) 

     
    # Draw a rectangle around the appropriate area.

    for pt in z ip ( * loc [:: - 1 ]):

      cv2.rectangle (img_rgb, pt, (pt [ 0 ] + w, pt [ 1 ] + h), ( 0 , 255 , 255 ), 2 )

     
    # Show the final image from the corresponding area.

    cv2.imshow ( `Detected` , img_rgb)

    Template matching constraints:

    1. The occurrences of the template must preserve the orientation of the master image of the template (template)
    2. As a result, it does not work for rotated or scaled versions of the template because the change shape / size / shear, etc. A patterned object will give a false match.
    3. This method is ineffective in calculating a correlation image for medium to large images, since this process takes a long time.
    4. To avoid the problem caused by different template and source image sizes, we can use multiscale . In cases where the dimensions of your template do not match the dimensions of the area of ​​the image you want to match, that does not mean that you cannot apply template matching.

      The multiscaling mechanism in the match pattern

      The Multi Scaling process looks like this:

      1. Loop the input image at multiple scales (i.e. make the input image gradually all less and less).
      2. Apply pattern matching using cv2.matchTemplate and track the match with the highest correlation coefficient (along with the x, y coordinates of the region with the highest correlation coefficient).
      3. After looping through all the scales, take the region with the highest correlation coefficient and use it as the “matched” region.

      # Python program for illustration
      # multiscaling against pattern

      import cv2

      import numpy as np

       
      # Read main image

      img_rgb = cv2.imread ( ` mainimage.jpg` )

       
      # Convert to grayscale

      img_gray = cv2.cvtColor (img_rgb, cv2.COLOR_BGR2GRAY)

       
      # Read the template

      template = cv2.imread ( `template` , 0 )

        
      # Save the width and height of the template in w and h

      w, h = template.shape [:: - 1 ]

      found = None

       

      for scale in np.linspace ( 0.2 , 1.0 , 20 ) [:: - 1 ]:

        

      # resize image to scale and track

      # resize relation

      resized = imutils.resize (img_gray, width = int (img_gray.shape [ 1 ] * scale))

      r = img_gray .shape [ 1 ] / float (resized.shape [ 1 ])

        

        # if the resized image is smaller than the template, then a gap

      # from loop

        # define edges in the modified grayscale image and apply the template

      # match to find the pattern at the edge of the image

      # = cv2.Canny (resized, 50, 200) result = cv2.matchTemplate (edged, template,

      # cv2.TM_CCOEFF) (_, maxVal, _, maxLoc) = cv2.minMaxLoc (result etat)

      # if we find a new maximum correlation value, we will update

      # the variable found, if found, is None or maxVal & gt; found [0]:

      if resized.shape [ 0 ] & lt; h or resized.shape [ 1 ] & lt; w:

      break

      found = (maxVal, maxLoc, r)

       
      # unpack the found variable and calculate the coordinates (x, y)
      # bounding box based on resized ratio

      (_, maxLoc, r) = found

      (startX, startY) = ( int ( maxLoc [ 0 ] * r), int (maxLoc [ 1 ] * r))

      (endX, endY) = ( int ((maxLoc [ 0 ] + tW) * r), int ((maxLoc [ 1 ] + tH) * r))

       
      # draw a bounding box around detected result and display the image

      cv2.rectangle (image, (startX, startY), (endX, endY), ( 0 , 0 , 255 ), 2 )

      cv2.imshow ( "Image" , image)

      cv2.waitKey ( 0 )

      Step-by-step explanation of the above code:

      • After storing the width and height of the template in w and r, we initialize the variable we found to keep track of the area and scale of the image with the best match. From there, we start looping through multiple zoom levels using the np.linspace function. This function takes three arguments: a start value, an end value, and the number of equal slices of the piece in between. In this example, we will start at 100% of the original image size and work our way up to 20% of the original size in 20 percentage points of the same size.
      • We then resize the image to the current scale and calculate the ratio of the old width to new width — as you will see later, it is important to keep track of this ratio. We check that the input image is larger than our template. If the template is larger, then our call to cv2.matchTemplate will throw an error, so we`ll just break out of the loop if that`s the case.
      • At this point, we can apply template matching to our modified image:
        • The cv2.minMaxLoc function takes our correlation result and returns a 4-tuple that includes the minimum correlation value, the maximum correlation value, the (x, y) -coordinate of the minimum value, and the (x, y) -coordinate of the maximum value, respectively. We are only interested in the maximum value and the (x, y) -coordinate, so we keep the maximums and discard the minimums.
      • After that we check the image areas that are mapped at each iteration of the scale ... From there we update our found variable to keep track of the maximum found correlation value, the (x, y) -coordinate of the maximum value along with the ratio of the width of the original image to the current one, the width of the resized image,
      • After we looped through all scales, we unpack our found variable and then compute our start and end (x, y) -coordinates of our bounding box. Special care is taken to multiply the coordinates of the bounding box by a ratio to ensure that the coordinates match the original dimensions of the input image.
      • Finally, we draw our bounding box and display it on our screen.

      This article is courtesy of Pratima Upadhyay . If you are as Python.Engineering and would like to contribute, you can also write an article using contribute.python.engineering or by posting an article contribute @ python.engineering. See my article appearing on the Python.Engineering homepage and help other geeks.

      Please post comments if you find anything wrong or if you would like to share more information on the topic discussed above.