Abstract:

In the scene recognition task, there are cases where heterogeneous scenes contain items with high similarity or the spatial layout of similar scenes is too different, that is, the inter-class similarity and intra-class difference of scenes.Existing methods improve the discriminant ability of classifiers by enhancing data sets or using multi-level information complementation. Although some improvements have been made, there are still limitations.In this study, the DPE(Discriminative Patch Extraction) and TSC(Two-Stage Classification) network method are proposed to overcome the inter-class similarity and intra-class difference of scenes. DPE avoids the impact of intra-class differences on scene recognition by preserving the key information regions in images, while the TSC network avoids the impact of inter-class similarities on scene recognition by the coarse-fine two-stage training.After combining the proposed method with baseline networks such as ViT(Vision Transformer), the classification accuracy of classical scene recognition data sets Scene15, MITindoor67 and SUN397 reaches 96.9%, 88.4% and 76.0%, respectively. The proposed method achieves the highest classification accuracy of 60.5% on the largest scene recognition dataset Places365.

Key words: scene recognition, deep neural networks, inter-class similarity, intra-class variability, data augmentation, discriminative patch extraction, two-stage classification, ViT