Object Detection And Tracking using Python - Opencv

Let us discuss today how you can detect and track an object in real-time. We will be using Python language and Opencv library for this purpose. Check out the video below. If you have read my previous blogs, you can directly skip down to the contour part.

As usual, we need to make a few assumptions for the proper working of this application.

1. This background is always static i.e. there is no addition or subtraction of objects in the background scene.
2. The background-color is always constant. It does not change with time.
3. The object that will be used for writing/painting is of a different color than the background to give us sufficient contrast between foreground and background.

We are ready to begin now. Let us start by installing necessary python libraries for our project using pip install. We will be needing Numpy and Opencv libraries. Now create a python project and create a new script. Import the required libraries into python script as shown below.

import cv2
import numpy as np

To capture a video, we need to create a VideoCapture object. Its argument can be either the device index or the name of a video file. The device index is just the number to specify which camera. 

cap = cv2.VideoCapture(0)

Now, we will write a while and capture image frames continuously. The following line of code is used to capture the frames.

ret, frame = cap.read()
frame = cv2.flip(frame, +1)

The second line is used to mirror the image frame so that it becomes easy for writing. The image we read is in RGB format. We need to convert this image into  HSV(Hue Saturation Value) format. Click here to read about the HSV image.

frame2 = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)         ##BGR to HSVlb = np.array([153, 119, 212])
ub = np.array([255, 255, 255])

What are these lb and ub? These are the lower and upper bound thresholds. The thing is we want the background to be completely eliminated and only our object should be present in the image frame. For this purpose, we decide these threshold values. lb and ub are numpy arrays of 3 elements. Each element represents the value of Hue, Saturation, and Value. The HSV values of image which lie between lb and ub will be only kept in the image frame and rest will be discarded. The above values work fine for me. Your values might be different. The best way to define these values is by using trackbars.

Now we create a mask for our image which will only keep values lying between lb and ub and discard the rest of the values. 

mask = cv2.inRange(frame2, lb, ub)

The masked image looks something like this.

The white circle in the above image is my object. The rest of the background is eliminated. The above image looks but when you set the upper and lower bounds there are a lot of chances that noise will be added to the mask. That will look like a lot of small white dots in the image. To remove that noise we can use morphology. Opencv has a method called opening morphology which removes the small patches in the image. 

opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, Kernal)

The first argument here is the image on which you want to perform morphology. The second is the type of morphology you want to perform and the third is the kernal. Kernal is nothing but a matrix of any size defined by you with all the elements as '1'. After performing morphology the image looks like this.

You will notice the small patches at the border of the object are removed after applying morphology. Let us apply this mask to our original image. To do that we perform a bitwise AND operation between the original image and the masked image. This process will generate the image with the original color of the object. 

res = cv2.bitwise_and(frame, frame, mask = opening)

Here, the first argument is the input image. The second argument is the destination image and the third argument is the mask that we are using for ANDing operation. The resultant image is shown below.

Now we need to find the contours in this image. The contours method in Opencv library works only on the binary image(Of course to optimize the process). Therefore we will use the opening image(which is already binary) and use find contours method on it.

contours, hierarchy = cv2.findContours(opening, cv2.RETR_TREE,      
                                       cv2.CHAIN_APPROX_NONE)

The first argument in the find contours method is the source image. The second argument is the mode for finding contours and the third argument is the method to approximate the contours. The third argument is set to CHAIN_APPROX_NONE because we want to find all the points on the contour. If you set this to CHAIN_APPROX_SIMPLE, only the most relevant coordinates will be stored. Find contours returns a list of all the contours found in the image. This array is stored in the contours variable.

And now we draw the contours around the object.

if len(contours) != 0:
    cnt = contours[0]
    M = cv2.moments(cnt)
    Cx = int(M['m10']/M['m00'])
    Cy = int(M['m01'] / M['m00'])
    S = 'Location of object:' + '(' + str(Cx) + ',' + str(Cy) + ')'    cv2.putText(frame, S, (5, 50), font, 2, (0, 0, 255), 2,

                cv2.LINE_AA)
    cv2.drawContours(frame, cnt, -1, (0, 255, 0), 3)

The first line checks whether you found contour or not. After the filtering process, only one contour will be left. The second line stores the value of contour in cnt variable. We now find out the moments in contour. Moments returns the features of the contours using which you can calculate the center of mass of the object, area of the object, etc. We use it to find the centroid of the object (Cx, Cy). 

To display the location of the object, first, we create a string S and embed Cx and Cy into it. To put text on the video frame we use the method called cv2.putText. The first argument is the image on which you want to put the text. The second argument is the text that you want to put. The third argument is the starting coordinates on the video frame where the text will be displayed. The fourth argument is the type of font, the fifth argument is the size of the text, the sixth argument is the color of the text, seventh is the width, and the eighth is the line type.

To draw the contours around the object we use cv2.drawcontours method. The first argument here is the image on which you want to draw the contours, the second argument is the contour around which you want to draw. The third argument draws contour outlines or fills the contour with color. If the value is positive it draws only the outline and if the value is negative, it fills the contour. The fourth argument is the color of the contour.  And the last argument is the thickness of the contour. 

Now, display the image using imshow method. The final image is shown below. 

cv2.imshow('Original Image', frame)

Don't forget to release the memory at the end of the while loop.

cap.release()
cv2.destroyAllWindows()

And the project is done.

Check out the full code here:- https://github.com/vibhor69meshram/Object-Detection-and-Tracking

Check out code for selecting HSV values using trackbar:- https://github.com/vibhor69meshram/Set-HSV-values-using-Trackbar