Python | Detecting a document field using a match pattern

In this article, we will learn how to use pattern matching to detect related fields in a document image.

Solution :
The above task can be achieved using pattern matching templates. Cropped margin images and apply template matching using cropped margin images and a document image. The algorithm is simple, but reproducible in complex versions to solve the problem of detecting and localizing fields for images of documents belonging to specific domains.

Approach :

  • Cut / crop field images from the main document and use them as separate templates.
  • Define / adjust thresholds for different fields.
  • Apply a template matching for each cropped field template using the OpenCV function cv2.matchTemplate ()
  • Draw bounding boxes using the coordinates of rectangles selected from pattern matching.
  • Optional: Complete templates margins and fine-tuning threshold to improve the result for different images in the document.

Input image:

Output image:

Below is the Python code:

# import libraries

import numpy as np

import imutils

import cv2

 

field_threshold = { "prev_policy_no" : 0.7 ,

"address"  : 0.6 ,

  }

  
# Function for generating constraints
# margins around detected fields

def getBoxed (img, img_gray, template, field_name = " policy_no " ):

  

  w, h = template.shape [:: - 1

 

  # Apply pattern matching

res = cv2.matchTemplate (img_gray, template,

cv2.TM_CCOEFF_NORMED)

  

  hits = np.where (res & gt; = field_threshold [field_name])

 

# Draw a rectangle around the appropriate area.

for pt in zip ( * hits [:: - 1 ]): 

cv2.rectangle (img, pt, (pt [ 0 ] + w, pt [ 1 ] + h),

( 0 , 255 , 255 ), 2 )

 

y = pt [ 1 ] - 10 if pt [ 1 ] - 10 & gt;  10 else pt [ 1 ] + h + 20

 

cv2.putText (img, field_name, (pt [ 0 ], y),

cv2.FONT_HERSHEY_SIMPLEX, 0.8 , ( 0 , 0 , 255 ), 1 )

 

re turn img

 

  
# Driver function

if __ name__ = = `__main__` :

 

# Read original document image

img = cv2.imread ( `doc.png` )

 

# 3rd to 2nd conversion

img_gray  = cv2.cvtColor (img, cv2.COLOR_BGR2GRAY)

 

# Field templates

template_add = cv2. imread ( `doc_address.png` , 0 )

template_prev = cv2.imread ( ` doc_prev_policy.png` , 0 )

 

img = getBoxe d (img.copy (), img_gray.copy (),

template_add, `address` )

  

img = getBoxed (img.copy (), img_gray.copy (),

template_prev, `prev_policy_no` )

  

cv2.imshow ( `Detected ` , img)

Benefits of using pattern matching :

  • Computationally inexpensive.
  • Easy to use and customizable for different use cases use.
  • Gives good results in case of lack of document data.

Disadvantages :

  • The result is not very accurate compared to deep learning segmentation methods.
  • No solution to overlapping pattern problems.