Abstract
In this paper, a high-speed online neural network classifier based on extreme learning machines for multi-label classification is proposed. In multi-label classification, each of the input data sample belongs to one or more than one of the target labels. The traditional binary and multi-class classification where each sample belongs to only one target class forms the subset of multi-label classification. Multi-label classification problems are far more complex than binary and multi-class classification problems, as both the number of target labels and each of the target labels corresponding to each of the input samples are to be identified. The proposed work exploits the high-speed nature of the extreme learning machines to achieve real-time multi-label classification of streaming data. A new threshold-based online sequential learning algorithm is proposed for high speed and streaming data classification of multi-label problems. The proposed method is experimented with six different datasets from different application domains such as multimedia, text, and biology. The hamming loss, accuracy, training time and testing time of the proposed technique is compared with nine different state-of-the-art methods. Experimental studies shows that the proposed technique outperforms the existing multi-label classifiers in terms of performance and speed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Angelov P (2012) Autonomous learning systems: from data streams to knowledge in real-time. Wiley, New York
Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159:3160–3182
Bernardini FC, da Silva RB, Rodovalho RM, Meza EBM (2014) Cardinality and density measures and their influence to multi-label learning methods (Submitted to Learning and nonlinear models)
Bouchachia A (2010) An evolving classification cascade with self-learning. Evol Syst 1:143–160
Boutell M, Shen X, Luo J, Brown C (2003) Multi-label semantic scene classification. Technical report, Dept. Comp. Sci. U. Rochester
Crammer YS (2004) Online learning of complex categorical problems. Hebrew University of Jerusalem
de Carvalho ACPLF, Freitas AA (2009) A tutorial on multi-label classification techniques. In: Abraham A, Hassanien A-E, Snášel V (eds) Foundations of computational intelligence volume 5. Studies in computational intelligence, vol 205. Springer, Berlin, Heidelberg, pp 177–195
Ding S, Zhao H, Zhang Y, Xu X, Nie R (2015) Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 44:103–115
Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Massachusetts, USA, pp 681–687
Gama J (2010) Knowledge discovery from data streams. CRC Press, Boca Raton
Gonçalves T, Quaresma P (2003) A preliminary approach to the multilabel classification problem of Portuguese juridical documents. In: Pires F, Abreu S (eds) Progress in artificial intelligence, vol 2902. Lecture notes in computer science. Springer, Berlin, pp 435–444
Hua X-S, Qi G-J (2008) Online multi-label active learning for large-scale multimedia annotation. TechReport MSR-TR-2008-103
Huang G-B (2015) What are extreme learning machines? Filling the gap between Frank Rosenblatt’s dream and John von Neumann’s puzzle. Cogn Comput 7:263–278
Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501
Huang G-B, Wang D, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122
Iglesias JA, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agents’ behaviors: a general approach. Evol Syst 1:161–171
Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: Nédellec C, Rouveirol C (eds) Machine learning: ECML-98, vol 1398. Lecture notes in computer science. Springer, Berlin, pp 137–142
Karali A, Pirnat V (1991) Significance level based multiple tree classification. Informatica 15(5)
Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach. Springer Science & Business Media, Berlin
Lemos A, Caminhas W, Gomide F (2013) Adaptive fault detection and diagnosis using an evolving fuzzy classifier. Inf Sci 220:64–85
Li B, Wang J, Li Y, Song Y (2007) An improved on-line sequential learning algorithm for extreme learning machine. In: Liu D, Fei S, Hou Z-G, Zhang H, Sun C (eds) Advances in neural networks—ISNN 2007: 4th international symposium on neural networks, ISNN 2007, Nanjing, China, June 3–7, 2007, Proceedings, Part I. Springer, Berlin, Heidelberg, pp 1087–1093
Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423
Lughofer E, Buchtala O (2013) Reliable all-pairs evolving fuzzy classifiers. IEEE Trans Fuzzy Syst 21:625–641
Luo X, Zincir-Heywood AN (2005) Evaluation of two systems on multi-class multi-label document classification. In: Hacid M-S, Murray N, Raś Z, Tsumoto S (eds) Foundations of intelligent systems, vol 3488. Lecture notes in computer science. Springer, Berlin, pp 161–169
Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recognit 45:3084–3104
Mohiuddin K, Mao J (2014) A comparative study of different classifiers for handprinted character recognition. In: Pattern recognition in practice IV. Elsevier, Amsterdam, pp 437–448
Polikar R, Upda L, Upda SS, Honavar V (2001) Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans Syst Man Cybern Part C 31:497–508
Pratama M, Anavatti S, Lu J (2015a) Recurrent classifier based on an incremental meta-cognitive-based scaffolding algorithm. IEEE Trans Fuzzy Syst 23(6):2048–2066
Pratama M, Anavatti SG, Meng J, Lughofer ED (2015b) pClass: an effective classifier for streaming examples. IEEE Trans Fuzzy Syst 23:369–386
Pratama M, Lu J, Anavatti S, Lughofer E, Lim C-P (2015c) An incremental meta-cognitive-based scaffolding fuzzy neural network. Neurocomputing 171:89–105
Pratama M, Lu J, Zhang G (2015d) Evolving type-2 fuzzy classifier. IEEE Trans Fuzzy Syst 24(3):574–589
Read J, Bifet A, Holmes G, Pfahringer B (2011) Streaming multi-label classification. In: WAPA, pp 19–25
Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments: methods and applications. Springer Science & Business Media, Berlin
Shen X, Boutell M, Luo J, Brown C (2003) Multilabel machine learning and its application to semantic scene classification. In: Electronic imaging, pp 188–199
Song Y, Ben Salem M, Hershkop S, Stolfo SJ (2013) System level user behavior biometrics using Fisher features and Gaussian mixture models. In: Security and privacy workshops (SPW), 2013 IEEE. IEEE, pp 52–59
Sorower MS (2010) A literature survey on algorithms for multi-label learning. Oregon State University, Corvallis
Srivastava N, Agrawal U, Roy SK, Tiwary U (2015) Human identification using linear multiclass SVM and eye movement biometrics. In: International conference on contemporary computing (IC3). IEEE, Noida, pp 365–369
Tikk D, Biró G (2003) Experiments with multi-label text classifier on the Reuters collection. In: Proceedings of the international conference on computational cybernetics (ICCC 03), pp 33–38
Tsoumakas G, Katakis I (2006) Multi-label classification: an overview. Dept of Informatics, Aristotle University of Thessaloniki, Greece
Tsoumakas G, Katakis I, Vlahavas I (2010) Mining multi-label data. In: Maimon O, Rokach L (eds) Data mining and knowledge discovery handbook. Springer, US, pp 667–685
Wang Y, Cao F, Yuan Y (2011) A study on effectiveness of extreme learning machine. Neurocomputing 74:2483–2490
Wang N, Han M, Dong N, Er MJ (2014a) Constructive multi-output extreme learning machine with application to large tanker motion dynamics identification. Neurocomputing 128:59–72
Wang N, Er MJ, Han M (2014b) Parsimonious extreme learning machine using recursive orthogonal least squares. IEEE Trans Neural Netw Learn Syst 25:1828–1841
Wang N, Sun JC, Er MJ, Liu YC (2015a) A novel extreme learning control framework of unmanned surface vehicles. IEEE Transa Cybern 46(5):1106–1117
Wang N, Er MJ, Han M (2015b) Generalized single-hidden layer feedforward networks for regression problems. IEEE Trans Neural Netw Learn Syst 26:1161–1176
Xydeas C, Angelov P, Chiao S-Y, Reoullas M (2006) Advances in classification of EEG signals via evolving fuzzy classifiers and dependant multiple HMMs. Comput Biol Med 36:1064–1083
Yu K, Yu S, Tresp V (2005) Multi-label informed latent semantic indexing. Paper presented at the Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval, Salvador, Brazil
Zhang M-L, Zhou Z-H (2005) A k-nearest neighbor based algorithm for multi-label classification. In: 2005 IEEE international conference on granular computing, vol 2. IEEE, pp 718–721, 25–27 July 2005
Zhang M-L, Zhou Z-H (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40:2038–2048
Zhang X, Graepel T, Herbrich R (2010) Bayesian online learning for multi-label and multi-variate performance measures. In: International conference on artificial intelligence and statistics, pp 956–963
Zhu B, Poon CK (1999) Efficient approximation algorithms for multi-label map labeling. In: International symposium on algorithms and computation. Springer, Berlin, Heidelberg, pp 143–152
Acknowledgments
The authors would like to acknowledge the funding support from the Ministry of Education, Singapore (Tier 1 AcRF, RG29/15) and the Research student scholarship awarded to the first author.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Venkatesan, R., Er, M.J., Dave, M. et al. A novel online multi-label classifier for high-speed streaming data applications. Evolving Systems 8, 303–315 (2017). https://doi.org/10.1007/s12530-016-9162-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-016-9162-8