Abstract
Automatic defect detection is a challenging task owing to the complex textured background with non-uniform intensity distribution, weak differences between defects and background, diversity of defect types, and high cost of annotated samples. In order to solve these challenges, this paper proposes a novel end-to-end defect classification and segmentation framework based on weakly supervised learning of a convolutional neural network (CNN) with attention architecture. Firstly, a novel end-to-end CNN architecture integrating the robust classifier and spatial attention module is proposed to enhance defect feature representation ability, which significantly improves the classification accuracy. Secondly, a new spatial attention class activation map (SA-CAM) is proposed to improve segmentation adaptability by generating more accurate heatmap. Moreover, for different surface texture, SA-CAM can significantly suppress the background’s inference and highlight defect area. Finally, the proposed weakly supervised learning framework is trained using only global image labels and devoted to two main visual recognition tasks: defect samples classification and area segmentation. At the same time, it is robust to complex backgrounds. Results of the experiments verify the generalization of the proposed method on three distinct datasets with different kinds of textures and backgrounds. In the classification tasks, the proposed method improves accuracy by 0.66–25.50%. In the segmentation tasks, the proposed method improves accuracy by 5.49–7.07%.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Xie X (2008) Review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electron Lett Comput Vis Image Anal 7(3):1
Luo Q, Sun Y, Li P, Simpson O, Tian L, He Y (2018) Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Trans Instrum Meas 68(3):667
Binyi S et al (2019) Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor. IEEE Trans Instrum Meas 68(12):4675–4688
Yapi D, Allili MS, Baaziz N (2017) Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Trans Autom Sci Eng 15(3):1014
Wang H, Qi H, Wang XF (2013) A new Gabor based approach for wood recognition. Neurocomputing 116:192
Zhang Z, Zou Y, Gan C (2018) Textual sentiment analysis via three different attention convolutional neural networks and cross-modality consistent regression. Neurocomputing 275:1407
Xie L, Huang R, Gu N, Cao Z (2014) A novel defect detection and identification method in optical inspection. Neural Comput Appl 24(7):1953
Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE (2017) Survey of deep neural network architectures and their applications. Neurocomputing 234:11
Mirjalili SM, Mirjalili SZ (2017) Single-objective optimization framework for designing photonic crystal filters. Neural Comput Appl 28(6):1463
Jung S, Tsai Y, Chiu W, Hu J, Sun C (2018) Defect detection on randomly textured surfaces by convolutional neural networks. In: 2018 IEEE/ASME international conference on advanced intelligent mechatronics (AIM) (IEEE, 2018), pp 1456–1461
Chen H, Pang Y, Hu Q, Liu K (2020) Solar cell surface defect inspection based on multispectral convolutional neural network. J Intell Manuf 31:453–468
Zhou S, Chen Y, Zhang D, Xie J, Zhou Y (2017) Classification of surface defects on steel sheet using convolutional neural networks. Mater Technol 51(1):123
Tang Y (2013) Deep learning using linear support vector machines. arXiv:1306.0239
Merentitis A, Debes C (2015) Automatic fusion and classification using random forests and features extracted with deep learning. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS) (IEEE, 2015), pp 2943–2946
Zhang H, Zhang L, Li P, Gu D (2018) Yarn-dyed fabric defect detection with yolov2 based on deep convolution neural networks. In: 2018 IEEE 7th data driven control and learning systems conference (DDCLS) (IEEE, 2018), pp 170–174
Singh J, Shekhar S (2018) Road damage detection and classification in smartphone captured images using mask r-cnn. arXiv:1811.04535
Yuille AL, Liu C (2018) Deep nets: What have they ever done for vision? arXiv:1805.04025
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929
Ren R, Hung T, Tan KC (2017) Generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929
Lin H, Li B, Wang X, Shu Y, Niu S (2019) Automated defect inspection of led chip using deep convolutional neural network. J Intell Manuf 30(6):2525
Li W, Leonardis A, Fritz M (2017) Visual stability prediction and its application to manipulation. In: 2017 AAAI Spring symposium series
Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025
Ji Y, Zhang H, Wu QMJ (2018) Salient object detection via multi-scale attention CNN. Neurocomputing 322:130–140
Breiman L (2001) Random forests. Mach Learn 45(1):5
Kairanbay M, See J, Wong LK, Hii YL (2017) Filling the gaps: reducing the complexity of networks for multi-attribute image aesthetic prediction. In: 2017 IEEE international conference on image processing (ICIP) (IEEE, 2017), pp 3051–3055
Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern 9(1):62
Silven O, Niskanen M, Kauppinen H (2003) Wood inspection with non-supervised clustering. Mach Vis Appl 13(5–6):275
Wang T, Chen Y, Qiao M, Snoussi H (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 94(9–12):3465
Zhang H et al (2018) Tree2Vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn Syst 99:1–15
Zhai W, Zhu J, Cao Y, Wang Z (2018) A generative adversarial network-based framework for unsupervised visual surface inspection. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP) (IEEE, 2018), pp 1283–1287
Zhang J, Sclaroff S (2015) Exploiting surroundedness for saliency detection: a Boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889
Donoser M, Bischof H (2008) Using covariance matrices for unsupervised texture segmentation. In: 2008 19th international conference on pattern recognition (IEEE, 2008), pp 1–4
Acknowledgements
This work is supported in part by National Natural Science Foundation (NNSF) of China under Grant 61873315, Natural Science Foundation of Hebei Province under Grant F2018202078, Science and Technology Program of Hebei Province under Grant 17211804D, Hebei Province Outstanding Youth Science Foundation F2017202062 and Young Talents Project in Hebei Province under Grant 210003.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service, and company that could be construed as influencing the position presented in or the review of the manuscript entitled.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, H., Hu, Q., Zhai, B. et al. A robust weakly supervised learning of deep Conv-Nets for surface defect inspection. Neural Comput & Applic 32, 11229–11244 (2020). https://doi.org/10.1007/s00521-020-04819-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-04819-5