Object Tracking and Following with OpenCV Python

Object tracking and the concepts learnt from developing an object tracking algorithm are necessary for computer vision implementation in robotics. By the end of this tutorial, you will have learnt to accurately track an object across the screen.

UPDATE: WordPress is changing some of my code blocks to ‘amp’ and I haven’t yet found a way to fix this. For further guidance (although it would be a good exercise to infer), head over to my github repository.

Prerequisites

This tutorial assumes you have some degree of proficiency with Python and can reasonably understand the OpenCV code here.

Determine HSV Range

Before you continue writing the code you’ll need to use this HSV Trackbar to determine the Hue Low/High, Saturation Low/High and Value Low/High for the object you want to track. Mess around with the trackbars until you can only see the color of the object you are looking for. Note these values down, you will need them for later.

Filter for HSV Color

#import necessary libraries
import cv2
import numpy as np
import time
#initialize the video stream
cap = cv2.VideoCapture(0)

#make two arrays, one for the points, and another for the timings
points = []
timer = []
while True:
    #start the timing
    startime = time.time()

    #append the start time to the array named 'timer'
    timer.append(g)

    #you only want to use the start time, so delete any other elements in the array
    del timer[1:]
    _, frame = cap.read()

    #resize and blur the frame (improves performance)
    sized = cv2.resize(frame, (600, 600))
    frame = cv2.GaussianBlur(sized, (7, 7), 0)

    #convert the frame to HSV and mask it
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
#fill in the values you obtained previously over here
    hlow = 17
    slow = 150
    vlow = 24
    hhigh = 78
    shigh = 255
    vhigh = 255
    HSVLOW  = np.array([hlow, slow, vlow])
    HSVHIGH = np.array([hhigh, shigh, vhigh])
    mask = cv2.inRange(hsv,HSVLOW, HSVHIGH)
    res = cv2.bitwise_and(frame,frame, mask =mask)

All of this stuff should be pretty straightforward after a few read-through’s. The only new function here is cv2.resize() and that itself is quite self explanatory (it resizes the frame). At this point, we have our new, ‘thresholded’ frame.

As a word of advice, make sure there isn’t a huge concentration of the color you’re looking for on the screen. For this basic object tracker, we’re only relying on color so if you have a lot of the color you want to track in the background, your best bet is to find a different colored object.

Find Maximum Contour

A lot of the time, you can simply visualize the algorithm necessary for solving most computer vision problem for robots if you understand what contours are. Since it is such a powerful tool, I suggest you build your foundation at this link (do the exercises, don’t just read), and come back when you kind of understand what contours are.

Once your done, try understanding this code.

    #create an edged frame of the thresholded frame
    edged = cv2.Canny(res, 50, 150)

    #find contours in the edged frame and append to the 'cnts' array
    cnts = cv2.findContours(edged, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[0]

    # if contours are present
    if len(cnts)> 0:

        #find the largest contour according to their enclosed area
        c = max(cnts, key=cv2.contourArea)

        #get the center and radius values of the circle enclosing the contour
        (x, y), radius = cv2.minEnclosingCircle(c)

 

We started with the cv2.Canny()  function (documentation) with a min and max  threshold. Ignore the technicalities of the numbers, that essentially finds all of the edges in the frame.

Since cv2.findContours() returns a list, we need to find the largest contour (which we assume is our object) in this list. We use the max(cnts, key=cv2.contourArea). Thus, this function finds the area of all of the contours in the list, and then returns it’s maximum.

Following that we use the cv2.minEnclosingCircle(c) function to find the (x,y) coordinates of the center of the circle, and it’s radius.

At this point, we have tracked our object in the frame. All that’s left is to draw the circle and the trailing line. 

        centercircle = (int(x), int(y))
        radius = int(radius)
        cv2.circle(sized, centercircle, radius, (255, 30,255), 2) #this circle is the object
        cv2.circle(sized, centercircle, 5, (0, 0, 255), -1) #this circle is the moving red dot
        points.append(centercircle) #append this dot to the 'points' list
        if points is not None:
            for centers in points:
                cv2.circle(sized, centers, 5, (0, 0, 255), -1) #make a dot for each of the points

    #show all the frames and cleanup
    cv2.imshow('frame', sized)
    cv2.imshow('mask', res)
    k = cv2.waitKey(5) & 0xFF
    g = time.time()
    timer.append(g)

    #if 10 seconds have passed, erase all the points
    delta_t = timer[1] - timer[0]
    if delta_t >= 10:
        del timer[:]
        del points[:]
    if k == 27:
        break

Lines 1-9 essentially draw the circle around the object, and draw another small red circle at it’s center. This dot constitutes a point in the trail.  This point is then appended to the existing array ‘points’.

The for loop that follows cycles through each of the centers’ (x,y) coordinates (from the ‘points’ array) and draws another red dot in each position, effectively creating the trail.

Finally, another value is appended to the timer array. Delta_t computes the difference between the start and final times. If this value is greater than 10, all points are erased and a new trail is begun.