Object Detection

A Step-by-Step Introduction to the Basic Object Detection Algorithms (Part 1)

A Simple Way of Solving an Object Detection Task (using Deep Learning)

A general object detection problem using a CNN:

• take an image as input

• divide the image into various regions

• consider each region as a separate image and pass all these regions (images) to the CNN and classify them into various classes.
• combine all these regions to get the original image with the detected objects

The problem with using this approach is that the objects in the image can have different aspect ratios and spatial locations.
We would require a very large number of regions resulting in a huge amount of computational time.
we can use region-based CNN to solve this problem and reduce the number of regions.

Understanding Region-Based Convolutional Neural Network

The RCNN algorithm proposes a bunch of boxes in the image and checks if any of these boxes contain any object. RCNN uses selective search to extract these boxes from an image (these boxes are called regions).
A brief overview of how selective search works:

• takes an image as input

• generate initial sub-segmentations so that we have multiple regions from this image

• combines the similar regions to form a larger region (based on color similarity, texture similarity, size similarity, and shape compatibility)

• these regions then produce the final object locations (Region of Interest).

Steps in RCNN to detect objects:

• take a pre-trained convolutional neural network
• train the last layer of the network based on the number of classes that need to be detected.
• get the Region of Interest for each image
reshape all these regions so that they can match the CNN input size.
• train SVM to classify objects and background.
For each class, we train one binary SVM. CNN then extracts features for each region and SVMs are used to divide these regions into different classes.
• we train a linear regression model to generate tighter bounding boxes for each identified object in the image.
A bounding box regression (Bbox reg) is used to predict the bounding boxes for each identified region.

Training an RCNN model is expensive and slow. Problems with RCNN:

• Extracting 2,000 regions for each image based on selective search
• Suppose we have N images, then the number of CNN features will be N*2,000
• The entire process of object detection using three models
• CNN for feature extraction
• Linear SVM classifier for identifying objects
• Regression model for tightening the bounding boxes.

Understanding Fast RCNN

Instead of running a CNN 2,000 times per image, we can run it just once per image and get all the regions of interest (regions containing some object).

Steps in Fast RCNN:

• aking an image as input

• This image is passed to a ConvNet which returns the region of interests

• apply the RoI pooling layer on the extracted regions of interest to make sure all the regions are of the same size

• these regions are passed on to a fully connected network which classifies them, as well as returns the bounding boxes

Problems with Fast RCNN is that it also uses selective search as a proposal method to find the Regions of Interest, which is a slow and time consuming process.

Understanding Faster RCNN

Faster RCNN uses “Region Proposal Network”(RPN).
RPN takes image feature maps as an input and generates a set of object proposals, each with an objectness score as output.
Steps in a Faster RCNN approach:

• Take an image as input and pass it to the ConvNet which returns the feature map for that image.
• Region proposal network is applied on these feature maps. This returns the object proposals along with their objectness score.
• A RoI pooling layer is applied on these proposals to bring down all the proposals to the same size.
• the proposals are passed to a fully connected layer which has a softmax layer and a linear regression layer at its top, to classify and output the bounding boxes for objects.

Summary of the Algorithms