Dual Attention Network for Scene Segmentation

Idea

Append two types of attention modules on top of dilated FCN, which model the sematic interdependencies in spatial and channel dimensions respectively.

Architecture

Dual Attention Module

Positional Attention Module

Just like the attention mechanism in Transformer, here we may consider $\text{B}$ as key, $\text{C}$ as query and $\text{D}$ as value. The difference is that here we apply convolution layers instead of a linear layer to convert $\text{A}$ to $\text{B}$, $\text{C}$ and $\text{D}$. The final self-attention score is

$$\text{S}=\mathrm{Softmax}\left({\text{B}}^T\text{C}\right)\text{D}$$

Channel Attention Module

Apply channel-wise self-attention