You Only Look Once: Unified, Real-Time Object Detection

Idea

Use a single neural network to predict bounding boxes and class probabilities directly from full images in one evaluation.

Method

• Divide the input image into an $S\times S$ grid. If the center of an object falls into a grid cell, that grid cell is responsible for detecting that object
• Each grid cell predicts $B$ bounding boxes and confidence scores for those boxes. These confidence score reflect how confident the model is that the box contains an object and also how accurate it thinks the box is that it predicts.
• Each gird cell also predicts $C$ conditional class probabilities. These probabilities are conditioned on the grid cell containing an object. The model only predict one set of class probabilities per gird cell, regardless of the number of boxes $B$
• Therefore, the predictions are encoded as an $S\times S\times (5B+C)$ tensor
• After that, we have each grids having scores and box positions for every class. Then we could obtain the boxes for each class by combing all the gird box predictions together whose class scores for this certain class if relatively high.

Architecture

Apply $20$ convolutional layers pre-trained on ImageNet

Loss Function

$$\begin{array}{rl}\mathcal{L}& ={\lambda }_{\text{coord}}\sum _{i=0}^{S^2}\sum _{j=0}^B{1}_{ij}^{\text{obj}}\left[(x_i-{\hat{x}}_i{)}^2+(y_i-{\hat{y}}_i{)}^2\right]\\ & +{\lambda }_{\text{coord}}\sum _{i=0}^{S^2}\sum _{j=0}^B{1}_{ij}^{\text{obj}}\left[(\sqrt{w_i}-\sqrt{{\hat{w}}_i}{)}^2+(\sqrt{h_i}-\sqrt{{\hat{h}}_i}{)}^2\right]\\ & +\sum _{i=0}^{S^2}\sum _{j=0}^B{1}_{ij}^{\text{obj}}{\left(C_i-{\hat{C}}_i\right)}^2+{\lambda }_{\text{noobj}}\sum _{i=0}^{S^2}\sum _{j=0}^B{1}_{ij}^{\text{noobj}}{\left(C_i-{\hat{C}}_i\right)}^2\\ & +\sum _{i=0}^{S^2}{1}_i^{\text{obj}}{\left(p_i(c)-{\hat{p}}_i(c)\right)}^2\end{array}$$

where the $x,y$ coordinates represent the center of the box relative the bounds of the grid cell. and the width $w$ and height $h$ are predicted relative to the whole image