Skip to main content
Springer Nature Link
Log in

An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Nonnegative matrix factorization (NMF) for image clustering attains impressive machine learning performances. However, the current iterative methods for optimizing NMF problems involve numerous matrix calculations and suffer from high computational costs in large-scale images. To address this issue, this paper presents an ordered subsets orthogonal NMF framework (OS-ONMF) that divides the data matrix in an orderly manner into several subsets and performs NMF on each subset. It balances clustering performance and computational efficiency. After decomposition, each ordered subset still contains the core information of the original data. That is, blocking does not reduce image resolutions but can greatly shorten running time. This framework is a general model that can be applied to various existing iterative update algorithms. We also provide a subset selection method and a convergence analysis of the algorithm. Finally, we conducted clustering experiments on seven real-world image datasets. The experimental results showed that the proposed method can greatly shorten the running time without reducing clustering accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability statement

The original datasets are available in the article.

References

  1. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791. https://doi.org/10.1038/44565

    Article  MATH  Google Scholar 

  2. Lee DD, Seung HS (2000) Algorithms for non-negative matrix factorization. Advances in neural information processing systems. MIT Press, Cambridge, pp 556–562

    MATH  Google Scholar 

  3. Ding C, Li T, Jordan MI (2008) Nonnegative matrix factorization for combinatorial optimization: spectral clustering, graph matching, and clique finding. 2008 Eighth IEEE International Conference on Data Mining. IEEE, Pisa Italy, pp 183–192

  4. Cai D, He X, Han J, Han JW, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560. https://doi.org/10.1109/TPAMI.2010.231

    Article  MATH  Google Scholar 

  5. Zhou G, Cichocki A, Zhao Q, Xie S (2014) Nonnegative matrix and tensor factorizations: an algorithmic perspective. IEEE Signal Proc Mag 31(3):54–65. https://doi.org/10.1109/MSP.2014.2298891

    Article  MATH  Google Scholar 

  6. Liu H, Wu Z, Li X, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311. https://doi.org/10.1109/TPAMI.2011.217

    Article  MATH  Google Scholar 

  7. Meng Y, Shang R, Jiao L, Zhang W, Yang S (2018) Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Eng Appl Artif Intel 69:24–35. https://doi.org/10.1016/j.engappai.2017.11.008

    Article  MATH  Google Scholar 

  8. Wang X, Zhong Y, Zhang L, Xu Y (2017) Spatial group sparsity regularized nonnegative matrix factorization for hyperspectral unmixing. IEEE Trans Geosci Remote 55(11):6287–6304. https://doi.org/10.1109/TGRS.2017.2724944

    Article  MATH  Google Scholar 

  9. Thamer MK, Algamal ZY, Zine R (2023) Enhancement of kernel clustering based on pigeon optimization algorithm. Int J Uncertain Fuzz 31(Supp01):121–133. https://doi.org/10.1142/S021848852340007X

    Article  MathSciNet  MATH  Google Scholar 

  10. Al-Kababchee SGM, Algamal ZY, Qasim OS (2023) Improving penalized-based clustering model in big fusion data by hybrid black hole algorithm. Fusion 11(1):70–76. https://doi.org/10.54216/fpa.110105

    Article  MATH  Google Scholar 

  11. Al-Kababchee SGM, Algamal ZY, Qasim OS (2023) Enhancement of K-means clustering in big data based on equilibrium optimizer algorithm. J Intell Syst 32(1):20220230. https://doi.org/10.1515/jisys-2022-0230

    Article  MATH  Google Scholar 

  12. Al-Kababchee SGM, Algamal ZY, Qasim OS (2021) Improving penalized regression-based clustering model in big data. J Phys Conf Ser 1897(1):012036. https://doi.org/10.1088/1742-6596/1897/1/012036

    Article  MATH  Google Scholar 

  13. Al-Radhwani AMN, Algamal ZY (2021) Improving K-means clustering based on firefly algorithm. J Phys Conf Ser 1897(1):012004. https://doi.org/10.1088/1742-6596/1897/1/012004

    Article  MATH  Google Scholar 

  14. Movassagh AA, Alzubi JA, Gheisari M et al (2023) Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model. J Amb Intel Hum Comp. https://doi.org/10.1007/s12652-020-02623-6

    Article  MATH  Google Scholar 

  15. Li Y, Ngom A (2013) The non-negative matrix factorization toolbox for biological data mining. Source Code Biol Med 8(10):1–15. https://doi.org/10.1186/1751-0473-8-10

    Article  MATH  Google Scholar 

  16. Liu W, Zheng N, You Q (2006) Nonnegative matrix factorization and its applications in pattern recognition. Chinese Sci Bull 51(1):7–18. https://doi.org/10.1007/s11434-005-1109-6

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhou J, Zhang S, Mei H et al (2016) A method of facial expression recognition based on Gabor and NMF. Pattern Recognit Image Anal 26(1):119–124. https://doi.org/10.1134/S1054661815040070

    Article  MATH  Google Scholar 

  18. Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval. Association for Computing Machinery, New York, pp 267–273. https://doi.org/10.1145/860435.860485

  19. Wang S, Chang TH, Cui Y et al (2021) Clustering by orthogonal NMF model and non-convex penalty optimization. IEEE T Signal Proces 69:5273–5288. https://doi.org/10.1109/TSP.2021.3102106

    Article  MathSciNet  MATH  Google Scholar 

  20. Kim J, Park H (2008) Sparse nonnegative matrix factorization for clustering. Georgia Institute of Technology

    MATH  Google Scholar 

  21. Ding C, He X, Simon H D (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM international conference on data mining, SIAM, pp 606-610

  22. Li T, Ding C (2018) Nonnegative matrix factorizations for clustering: a survey. Data Clustering 3:149–176. https://doi.org/10.1201/9781315373515-7

    Article  MATH  Google Scholar 

  23. Ding C, He X. K-means clustering via principal component analysis (2004). In: Proceedings of the twenty-first international conference on Machine learning. Association for Computing Machinery, New York, pp 29. https://doi.org/10.1145/1015330.1015408

  24. Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, New York, pp 126–135. https://doi.org/10.1145/1150402.1150420

  25. Tong C, Wei J, Qi SL, Yao YD, Zhang T, Teng YY (2023) A majorization-minimization based solution to penalized nonnegative matrix factorization with orthogonal regularization. J Comput Appl Math 421:114877. https://doi.org/10.1016/j.cam.2022.114877

    Article  MathSciNet  MATH  Google Scholar 

  26. Li Z, Wu X, Peng H (2010) Nonnegative matrix factorization on orthogonal subspace. Pattern Recogn Lett 31(9):905–911. https://doi.org/10.1016/j.patrec.2009.12.023

    Article  MATH  Google Scholar 

  27. Choi s (2008) Algorithms for orthogonal nonnegative matrix factorization. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). IEEE, Hong Kong, China, pp 1828–1832. https://doi.org/10.1109/IJCNN.2008.4634046

  28. Yoo JH, Choi SJ (2010) Nonnegative matrix factorization with orthogonality constraints. J Comput Sci Eng 4(2):97–109. https://doi.org/10.5626/JCSE.2010.4.2.097

    Article  MATH  Google Scholar 

  29. Yoo JH, Choi SJ (2010) Orthogonal nonnegative matrix tri-factorization for co-clustering: multiplicative updates on Stiefel manifolds. Inform Process Manag 46(5):559–570. https://doi.org/10.1016/j.ipm.2009.12.007

    Article  MATH  Google Scholar 

  30. Wu B, Wang E, Zhu Z, Chen W, Xiao P (2018) Manifold NMF with \(L_{21}\) norm for clustering. Neurocomputing 273:78–88. https://doi.org/10.1016/j.neucom.2017.08.025

    Article  MATH  Google Scholar 

  31. Yu N, Wu MJ, Liu JX et al (2021) Correntropy-based hypergraph regularized NMF for clustering and feature selection on multi-cancer integrated data. IEEE Trans Cybern 51(8):3952–3963. https://doi.org/10.1109/TCYB.2020.3000799

    Article  MATH  Google Scholar 

  32. Hedjam R, Abdesselam A, Melgani F (2001) NMF with feature relationship preservation penalty term for clustering problems low-rank matrix factorization. Pattern Recog 112:107814. https://doi.org/10.1016/j.patcog.2021.107814

    Article  Google Scholar 

  33. Movassagh AA, Alzubi JA, Gheisari M et al (2014) Forward error correction based on algebraic-geometric theory. Springer International Publishing, pp 31–39

    MATH  Google Scholar 

  34. Hudson HM, Larkin RS (2002) Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans Med Imaging 13(4):601–609. https://doi.org/10.1109/42.363108

    Article  MATH  Google Scholar 

  35. Erdogan H, Fessler JA (1999) Ordered subsets algorithms for transmission tomography. Phys Med Biol 44(11):2835. https://doi.org/10.1088/0031-9155/44/11/311

    Article  MATH  Google Scholar 

  36. He AJ, Tuo XG, Shi R, Zheng HL (2018) An improved OSEM iterative reconstruction algorithm for transmission tomographic gamma scannin. Appl Radiat Isotopes 142:51–55. https://doi.org/10.1016/j.apradiso.2018.09.001

    Article  MATH  Google Scholar 

  37. Gillis N, Glineur F (2012) Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization. Neural Comput 24(4):1085–1105. https://doi.org/10.1162/NECO_a_00256

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Liaoning Province (2002-MS-114).

Author information

Authors and Affiliations

  1. College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, 110169, China

    Limin Ma, Shouliang Qi & Yueyang Teng

  2. Changshu Institute of Technology, School of Computer Science and Engineering, Suzhou, 215500, China

    Can Tong

  3. Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ, 07030, USA

    Yudong Yao

  4. Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Shenyang, 110169, China

    Yueyang Teng

Authors
  1. Limin Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Can Tong
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shouliang Qi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yudong Yao
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yueyang Teng
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yueyang Teng.

Ethics declarations

Conflict of interest

The authors have no relevant financial or nonfinancial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, L., Tong, C., Qi, S. et al. An ordered subsets orthogonal nonnegative matrix factorization framework with application to image clustering. Int. J. Mach. Learn. & Cyber. 16, 1531–1543 (2025). https://doi.org/10.1007/s13042-024-02350-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02350-w

Keywords

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy