DBIF: Dual-Branch Feature Extraction Network for Infrared and Visible Image Fusion

Abstract

In image fusion, combining infrared and visible images from different sensors is crucial to create a complete representation that merges complementary information. However, current deep learning approaches, mainly using Convolutional Neural Networks (CNNs) or Transformer architectures, do not fully capitalize on the distinct features of infrared and visible images. To overcome this limitation, we introduce a novel Dual-Branch feature extraction network for infrared and visible image fusion (DBIF). DBIF optimally leverages the advantages of CNN and Transformer for feature extraction from different types of images. Specifically, the Transformer’s proficiency in extracting global features renders it more suitable for extracting target information from infrared images, while the CNN’s superior sensitivity to capturing local information makes it more adept at extracting background texture information from visible images. Consequently, our DBIF architecture incorporates two distinct branches, content and detail, for feature extraction from infrared and visible images, respectively. Additionally, we introduce a Detailed Feature Enhancement Module (DFEM) to consolidate and amplify the prominent features extracted by the detailed branch. Through extensive experimentation across multiple datasets, we validate the effectiveness of our proposed approach, showcasing its superiority over existing fusion algorithms. Furthermore, our method shows substantial performance improvements, especially in object detection tasks. This underscores its practical relevance in various real-world applications that require accurate and efficient fusion of diverse image data types.