Python | Extracting foreground to image using Grabcut algorithm

Foreground Extrac is any method that allows the foreground of an image to be extracted for further processing like object recognition, tracking, etc. The algorithm used to extract the foreground is here GrabCut algorithm. In this algorithm, an area is drawn according to the foreground, a rectangle-method/">rectangle is drawn above it. This is the rectangle-method/">rectangle that surrounds our main object. The coordinates of the region are determined by the foreground mask. But this segmentation is not ideal as it can mark some foreground as background and vice versa. This problem can be avoided manually. This foreground extraction technique functions like a green screen in cinematography.

• The ROI is determined by the amount of foreground and background segmentation to be performed and selected by the user. Everything outside the ROI is treated as background and turns black. The elements inside the ROI are still unknown.
• Then rectangle-method/">rectangle, backgroundModel, foregroundModel, iterationCount [, mode])

  Parameters:

  • image: Input 8-bit 3-channel image.
  • mask : Input / output 8-bit single-channel mask. The mask is initialized by the function when mode is set to GC_INIT_WITH_RECT. Its elements may have one of the following values:
    • GC_BGD defines an obvious background pixels.
    • GC_FGD defines an obvious foreground (object) pixel.
    • GC_PR_BGD defines a possible background pixel.
    • GC_PR_FGD defines a possible foreground pixel.
  • rectangle-method/">rectangle: It is the region of interest containing a segmented object. The pixels outside of the ROI are marked as obvious background. The parameter is only used when mode == GC_INIT_WITH_RECT.
  • backgroundModel: Temporary array for the background model.
  • foregroundModel: Temporary array for the foreground model.
  • iterationCount: Number of iterations the algorithm should make before returning the result. Note that the result can be refined with further calls with mode == GC_INIT_WITH_MASK or mode == GC_EVAL.
  • mode: It defines the Operation mode. It can be one of the following:
    • GC_INIT_WITH_RECT : The function initializes the state and the mask using the provided rectangle-method/">rectangle. After that it runs iterCount iterations of the algorithm.
    • GC_INIT_WITH_MASK : The function initializes the state using the provided mask. Note that GC_INIT_WITH_RECT and GC_INIT_WITH_MASK can be combined. Then, all the pixels outside of the ROI are automatically initialized with GC_BGD.
    • GC_EVAL : The value means that the algorithm should just resume.

  Below is the implementation:

  # Python program for illustration # fetch foreground using # GrabCut algorithm   # arrange imports import numpy as np import cv2 from matplotlib import pyplot as plt   # the path to the specified input image and # the image is loaded using the imread command image = cv2.imread ( ’image.jpg’ )    # create a simple mask similar to the image # to the uploaded image, with # form and type of return mask = np.zeros (image.shape [: 2 ], np.uint8)   # specify foreground and background model # using numpy, the array consists of 1 line # and 65 columns and all elements of the array are 0 # Data type for the array np.float64 ( default) backgroundModel = np. zeros (( 1 , 65 ), np.float64) foregroundModel = np.zeros (( 1 , 65 ), np.float64)   # define the ROI # as rectangle-method/">rectangle coordinates # where values ​​are entered as # (start_x, start_y, width, heights a) # these coordinates correspond to the input image # may differ for different images rectangle-method/">rectangle = ( 20 , 100 , 150 , 150 )   # apply the capture algorithm with matching # values ​​as parameters, number of iterations = 3 # cv2.GC_INIT_WITH_RECT is used because # rectangle-method/">rectangle mode is used cv2.grabCut (image, mask, rectangle-method/">rectangle,    backgroundModel, foregroundModel,   3 , cv2.GC_INIT_WITH_RECT)   # In the new mask image, pixels will be # marked with four flags # four flags indicate background / foreground # mask changed, all 0 and 2 pixels # converted to background # mask changed, all 1 and 3 pixels # now part of the foreground # the return type is also mentioned, # this gives us the final mask mask2 = np.where ((mask = = 2 ) | (mask = = 0 ), 0 , 1 ). astype ( ’ uint8’ )   # The final mask is multiplied by # input image to give a segmented image . image = image * mask2 [:,:, np.newaxis]   # output a segmented image with a color scale plt.imsh ow (image) plt.colorbar () plt.show ()

  Input image:

  

  Output:

  

  Here we took the input image is 500X281 and coordinates for the rectangle-method/">rectangle have been determined accordingly. The output image shows how the object to the left of the image becomes part of the foreground and the background is subtracted.