Skip to main content
Springer Nature Link
Log in

Multiple instance classification via quadratic programming

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Multiple instance learning (MIL) is a variation of supervised learning, where data consists of labeled bags and each bag contains a set of instances. Unlike traditional supervised learning, labels are not known for the instances in MIL. Existing approaches in the literature make use of certain assumptions regarding the instance labels and propose mixed integer quadratic programs, which introduce computational difficulties. In this study, we present a novel quadratic programming (QP)-based approach to classify bags. Solution of our QP formulation links the instance-level contributions to the bag label estimates, and provides a linear bag classifier along with a decision threshold. Our approach imposes no additional constraints on relating instance labels to bag labels and can be adapted to learning applications with different MIL assumptions. Unlike existing specialized heuristic approaches to solve previous MIL formulations, our QP models can be directly solved to optimality using any commercial QP solver. Also, kindly confirm Our computational experiments show that proposed QP formulation is efficient in terms of solution time, overcoming a main drawback of previous optimization algorithms for MIL. We demonstrate the classification success of our approach compared to the state-of-the-art methods on a wide range of real world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Amores, J.: Multiple instance classification: review, taxonomy and comparative study. Artif. Intell. 201, 81–105 (2013)

    Article  MathSciNet  Google Scholar 

  2. Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems 15, pp. 561–568. MIT Press (2003)

  3. Briggs, F., Lakshminarayanan, B., Neal, L., Fern, X.Z., Raich, R., Hadley, S.J., Hadley, A.S., Betts, M.G.: Acoustic classification of multiple simultaneous bird species: a multi-instance multi-label approach. J Acoust. Soc. Am. 131(6), 4640–4650 (2012)

    Article  Google Scholar 

  4. Bunescu, R.C., Mooney, R.J.: Multiple instance learning for sparse positive bags. In: Proceedings of the 24th International Conference on Machine Learning, pp. 105–112. ACM (2007)

  5. Carbonneau, M.A., Cheplygina, V., Granger, E., Gagnon, G.: Multiple instance learning: a survey of problem characteristics and applications. Pattern Recognit 77, 329–353 (2018)

    Article  Google Scholar 

  6. Chen, Y., Bi, J., Wang, J.Z.: MILES: multiple-instance learning via embedded instance selection. Pattern Anal. Mach. Intell., IEEE Trans. 28(12), 1931–1947 (2006)

    Article  Google Scholar 

  7. Cheplygina, V., Tax, D.M., Loog, M.: Multiple instance learning with bag dissimilarities. Pattern Recognit. 48(1), 264–275 (2015)

    Article  Google Scholar 

  8. Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. intell. 89(1), 31–71 (1997)

    Article  Google Scholar 

  9. Doran, G., Ray, S.: A theoretical and empirical analysis of support vector machine methods for multiple-instance classification. Mach. Learn. 97(1–2), 79–102 (2014)

    Article  MathSciNet  Google Scholar 

  10. Erdem, A., Erdem, E.: Multiple-instance learning with instance selection via dominant sets. In: Similarity-Based Pattern Recognition, pp. 177–191. Springer (2011)

  11. Fischetti, M.: Fast training of support vector machines with gaussian kernel. Discr Optim. 22, 183–194 (2016)

    Article  MathSciNet  Google Scholar 

  12. Foulds, J., Frank, E.: A review of multi-instance learning assumptions. Knowl. Eng. Rev. 25(01), 1–25 (2010)

    Article  Google Scholar 

  13. Fu, Z., Lu, G., Ting, K.M., Zhang, D.: Learning sparse kernel classifiers for multi-instance classification. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1377–1389 (2013)

    Article  Google Scholar 

  14. Fu, Z., Robles-Kelly, A., Zhou, J.: MILIS: multiple instance learning with instance selection. Pattern Anal. Mach. Intell., IEEE Trans. 33(5), 958–977 (2011)

    Article  Google Scholar 

  15. Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2018). http://www.gurobi.com

  16. Huang, J., Ling, C.X.: Using AUC and accuracy in evaluating learning algorithms. IEEE Trans. Knowl. Data Eng. 17(3), 299–310 (2005)

    Article  Google Scholar 

  17. Kandemir, M., Zhang, C., Hamprecht, F.A.: Empowering multiple instance histopathology cancer diagnosis by cell graphs. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014, pp. 228–235. Springer (2014)

  18. Ketchen, D.J., Jr., Shook, C.L.: The application of cluster analysis in strategic management research: an analysis and critique. Strateg. Manage. J. 17(6), 441–458 (1996)

    Article  Google Scholar 

  19. Kucukasci, E.S., Baydogan, M.G.: Bag-level representations for multiple instance learning (2018). http://ww3.ticaret.edu.tr/eskucukasci/multiple-instance-learning/

  20. Kundakcioglu, O.E., Seref, O., Pardalos, P.M.: Multiple instance learning via margin maximization. Appl. Numer. Math. 60(4), 358–369 (2010)

    Article  MathSciNet  Google Scholar 

  21. Li, W.J., Yeung, D.Y.: MILD: multiple-instance learning via disambiguation. Knowl. Data Eng., IEEE Trans. 22(1), 76–89 (2010)

    Article  Google Scholar 

  22. Li, Y.F., Kwok, J., Tsang, I., Zhou, Z.H.: A convex method for locating regions of interest with multi-instance learning. In: Machine Learning and Knowledge Discovery in Databases pp. 15–30 (2009)

  23. Mangasarian, O.L., Wild, E.W.: Multiple instance classification via successive linear programming. J. Optim. Theor. Appl. 137(3), 555–568 (2008)

    Article  MathSciNet  Google Scholar 

  24. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  25. Poursaeidi, M.H., Kundakcioglu, O.E.: Robust support vector machines for multiple instance learning. Ann. Oper. Res. 216(1), 205–227 (2014)

    Article  MathSciNet  Google Scholar 

  26. Şeref, O., Chaovalitwongse, W.A., Brooks, J.P.: Relaxing support vectors for classification. Ann. Oper. Res. 216(1), 229–255 (2014)

    Article  MathSciNet  Google Scholar 

  27. Srinivasan, A., Muggleton, S., King, R.: Comparing the use of background knowledge by inductive logic programming systems. In: Proceedings of the 5th International Workshop on Inductive Logic Programming, pp. 199–230. Department of Computer Science, Katholieke Universiteit Leuven (1995)

  28. Tao, Q., Scott, S., Vinodchandran, N., Osugi, T.T.: SVM-based generalized multiple-instance learning via approximate box counting. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 101. ACM (2004)

  29. Tax David M. J., C.V.: MIL, A Matlab toolbox for multiple instance learning (2015). http://prlab.tudelft.nl/david-tax/mil.html. Version 1.1.0

  30. Tax, D.M., Duin, R.P.: Learning curves for the analysis of multiple instance classifiers. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 724–733. Springer (2008)

  31. Tax, D.M., Hendriks, E., Valstar, M.F., Pantic, M.: The detection of concept frames using clustering multi-instance learning. In: Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 2917–2920. IEEE (2010)

  32. The Mathworks, I.: MATLAB version 8.5.0.197613 (R2015a). Natick, Massachusetts (2015)

  33. Wei, X.S., Wu, J., Zhou, Z.H.: Scalable algorithms for multi-instance learning. IEEE Trans. Neural Netw. Learn. Syst. 28(4), 975–987 (2017)

    Article  Google Scholar 

  34. Weidmann, N., Frank, E., Pfahringer, B.: A two-level learning method for generalized multi-instance problems. In: Machine Learning: ECML 2003, pp. 468–479. Springer (2003)

  35. Zhou, Z.H., Jiang, K., Li, M.: Multi-instance learning based web mining. Appl. Intell. 22(2), 135–147 (2005)

    Article  Google Scholar 

  36. Zhou, Z.H., Sun, Y.Y., Li, Y.F.: Multi-instance learning by treating instances as non-iid samples. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1249–1256. ACM (2009)

  37. Zhou, Z.H., Xu, J.M.: On the relation between multi-instance learning and semi-supervised learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1167–1174. ACM (2007)

  38. Zhou, Z.H., Zhang, M.L.: Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowl. Inf. Syst. 11(2), 155–170 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

Z. Caner Taşkın’s research was partially supported by Turkish Science Academy BAGEP award.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Istanbul Commerce University, Istanbul, Turkey

    Emel Şeyma Küçükaşcı

  2. Department of Industrial Engineering, Boğaziçi University, Istanbul, Turkey

    Mustafa Gökçe Baydoğan & Z. Caner Taşkın

Authors
  1. Emel Şeyma Küçükaşcı
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mustafa Gökçe Baydoğan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Z. Caner Taşkın
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emel Şeyma Küçükaşcı.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Küçükaşcı, E.Ş., Baydoğan, M.G. & Taşkın, Z.C. Multiple instance classification via quadratic programming. J Glob Optim 83, 639–670 (2022). https://doi.org/10.1007/s10898-021-01120-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-021-01120-0

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy