Skip to main content

Advertisement

Springer Nature Link
Log in

Unsupervised domain adaptation via transferred local Fisher discriminant analysis

  • Research
  • Published:
Iran Journal of Computer Science Aims and scope Submit manuscript

Abstract

Domain adaptation in machine learning and image processing aims to benefit from gained knowledge of the multiple labeled training sets (i.e. source domain) to classify the unseen test set (i.e. target domain). Therefore, the major issue emerges from dataset bias where the source and target domains have different distributions. In this paper, we introduce a novel unsupervised domain adaptation method for cross-domain visual classification. We suggest a unified framework that reduces both statistical and geometrical shifts across domains, referred to as unsupervised domain adaptation via transferred local Fisher discriminant analysis (TLFDA). Specifically, TLFDA projects data into a shared subspace to minimize the distribution shift between domains and simultaneously preserves the discrimination across different classes. TLFDA maximizes the between-class separability and preserves the within-class local structure in form of an objective function metric. The objective function is solved effectively in closed form. Broad experiments demonstrate that TLFDA significantly outperforms many state-of-the-art domain adaptation methods on different cross-domain visual classification tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Datasets are available and from previous works [41,42,43,44,45,46].

References

  1. Ahmadvand, M., Tahmoresnezhad, J.: Metric transfer learning via geometric knowledge embedding. Appl. Intell. 51(2), 921–934 (2021)

    Article  Google Scholar 

  2. Shao, L., Zhu, F., Li, X.: Transfer learning for visual categorization: a survey. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1019–1034 (2014)

    Article  MathSciNet  Google Scholar 

  3. Li, J., Yue, W., Ke, L.: Structured domain adaptation. IEEE Trans. Circ. Syst. Video Technol. 27(8), 1700–1713 (2016)

    Article  Google Scholar 

  4. Tahmoresnezhad, J., Hashemi, S.: Diret: an effective discriminative dimensionality reduction approach for multi-source transfer learning. Sci. Iran. 24(3), 1303–1311 (2017)

    Google Scholar 

  5. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)

    Article  Google Scholar 

  6. He, X., Yan, S., Yuxiao, H., Niyogi, P., Zhang, H.-J.: Face recognition using laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 328–340 (2005)

    Article  Google Scholar 

  7. Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7(3), 200–217 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  8. Wand, M.P., Jones, M.C.: Kernel Smoothing. CRC Press, London (1994)

    Book  MATH  Google Scholar 

  9. Sugiyama, M.: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J. Mach. Learn. Res. 8(5), 1 (2007)

    MathSciNet  MATH  Google Scholar 

  10. Tian, L., Tang, Y., Hu, L., Ren, Z., Zhang, W.: Domain adaptation by class centroid matching and local manifold self-learning. IEEE Trans. Image Process. 29, 9703–9718 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  11. Noori Saray, S., Tahmoresnezhad, J.: Joint distinct subspace learning and unsupervised transfer classification for visual domain adaptation. Signal Image Video Process. 15(2), 279–287 (2021)

    Article  Google Scholar 

  12. Sun, Q., Chattopadhyay, R., Panchanathan, S., Ye, J.: A two-stage weighting framework for multi-source domain adaptation. Adv. Neural Inf. Process. Syst. 24, 1 (2011)

    Google Scholar 

  13. Ishii, M., Sato, A.: Joint optimization of feature transform and instance weighting for domain adaptation. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 3793–3799. IEEE, New York(2017)

  14. Gong, B., Grauman, K., Sha, F.: Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 222–230. PMLR (2013)

  15. Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A.: A kernel method for the two-sample-problem. Adv. Neural Inf. Process. Syst. 19, 1 (2006)

    MATH  Google Scholar 

  16. Aljundi, R., Emonet, R., Muselet, D., Sebban, M.: Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 56–63 (2015)

  17. Aytar, Y., Zisserman, A.: Tabula rasa: model transfer for object category detection. In: 2011 International Conference on Computer Vision, pp. 2252–2259. IEEE (2011)

  18. Jiang, W., Zavesky, E., Chang, S.-F., Loui, A.: Cross-domain learning methods for high-level visual concept classification. In: 2008 15th IEEE International Conference on Image Processing, pp. 161–164. IEEE (2008)

  19. Duan, L., Tsang, I.W., Xu, D., Chua, T.-S.: Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 289–296 (2009)

  20. Long, M., Wang, J., Ding, G., Pan, S.J., Philip, S.Y.: Adaptation regularization: a general framework for transfer learning. IEEE Trans. Knowl. Data Eng. 26(5), 1076–1089 (2013)

    Article  Google Scholar 

  21. Long, M., Wang, J., Ding, G., Sun, J., Philip, S.Y.: Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2200–2207 (2013)

  22. Yong, X., Fang, X., Jian, W., Li, X., Zhang, D.: Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans. Image Process. 25(2), 850–863 (2015)

    MathSciNet  MATH  Google Scholar 

  23. Tahmoresnezhad, J., Hashemi, S.: Visual domain adaptation via transfer feature learning. Knowl. Inf. Syst. 50(2), 585–605 (2017)

    Article  Google Scholar 

  24. Liu, J., Li, J., Ke, L.: Coupled local-global adaptation for multi-source transfer learning. Neurocomputing 275, 247–254 (2018)

    Article  Google Scholar 

  25. Li, S., Song, S., Huang, G., Ding, Z., Cheng, W.: Domain invariant and class discriminative feature learning for visual domain adaptation. IEEE Trans. Image Process. 27(9), 4260–4273 (2018)

    Article  MathSciNet  Google Scholar 

  26. Luo, L., Chen, L., Shiqiang, H., Ying, L., Wang, X.: Discriminative and geometry-aware unsupervised domain adaptation. IEEE Trans. Cybern. 50(9), 3914–3927 (2020)

    Article  Google Scholar 

  27. Rezaei, S., Tahmoresnezhad, J., Solouk, V.: A transductive transfer learning approach for image classification. Int. J. Mach. Learn. Cybern. 12(3), 747–762 (2021)

    Article  Google Scholar 

  28. Sun, J., Wang, Z., Wang, W., Li, H., Sun, F.: Domain adaptation with geometrical preservation and distribution alignment. Neurocomputing 454, 152–167 (2021)

    Article  Google Scholar 

  29. Liu, W., Li, J., Liu, B., Guan, W., Zhou, Y., Changsheng, X.: Unified cross-domain classification via geometric and statistical adaptations. Pattern Recogn. 110, 107658 (2021)

    Article  Google Scholar 

  30. Sanodiya, R.K., Paul, D., Yao, L., Mathew, J., Juhi, A.: A feature selection approach to visual domain adaptation in classification. In: International Conference on Neural Information Processing, pp 77–89. Springer, London (2020)

  31. Luo, L., Wang, X., Hu, S., Wang, C., Tang, Y., Chen, L.: Close yet distinctive domain adaptation. Preprint arXiv:1704.04235 (2017)

  32. Lee, D., Seung, H.S.: Algorithms for non-negative matrix factorization. Adv. Neural Inf. Process. Syst. 13, 1 (2000)

    Google Scholar 

  33. Kennedy, J.: Particle swarm optimization. In: Encyclopedia of Machine Learning, pp. 760–766. Springer, Berlin (2011)

  34. Wang, J., Feng, W., Chen, Y., Yu, H., Huang, M., Philip, S.Y.: Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 402–410 (2018)

  35. Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5018–5027 (2017)

  36. Minnehan, B., Savakis, A.: Deep domain adaptation with manifold aligned label transfer. Mach. Vis. Appl. 30(3), 473–485 (2019)

    Article  Google Scholar 

  37. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  38. Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting batch normalization for practical domain adaptation. Preprint arXiv:1603.04779 (2016)

  39. De Maesschalck, R., Jouan-Rimbaud, D., Massart, D.L.: The mahalanobis distance. Chemomet. Intell. Lab. Syst. 50(1), 1–18 (2000)

    Article  Google Scholar 

  40. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  41. Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European Conference on Computer Vision, pp. 213–226. Springer, London (2010)

  42. Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset (2007)

  43. Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655. PMLR (2014)

  44. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  45. Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)

    Article  Google Scholar 

  46. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: Proceedings of fifth IEEE International Conference on Automatic Face Gesture Recognition, pp. 53–58. IEEE, New York (2002)

  47. Gheisari, M., Baghshah, M.S.: Joint predictive model and representation learning for visual domain adaptation. Eng. Appl. Artif. Intell. 58, 157–170 (2017)

    Article  Google Scholar 

  48. Ding, Z., Yun, F.: Robust transfer metric learning for image classification. IEEE Trans. Image Process. 26(2), 660–670 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  49. Zhang, J., Li, W., Ogunbona, P.: Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1859–1867 (2017)

  50. Zang, S., Cheng, Y., Wang, X., Qiang, Yu., Xie, G.-S.: Cross domain mean approximation for unsupervised domain adaptation. IEEE Access 8, 139052–139069 (2020)

    Article  Google Scholar 

  51. Luo, L., Chen, L., Hu, S.: Discriminative noise robust sparse orthogonal label regression-based domain adaptation. Preprint arXiv:2101.04563 (2021)

  52. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1 (2012)

    Google Scholar 

  53. Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep domain confusion: maximizing for domain invariance. Preprint arXiv:1412.3474 (2014)

  54. Uzair, M., Mian, A.: Blind domain adaptation with augmented extreme learning machine features. IEEE Trans. Cybern. 47(3), 651–660 (2016)

    Article  Google Scholar 

  55. Gholami, B., Pavlovic, V., et al.: Punda: probabilistic unsupervised domain adaptation for knowledge transfer across visual categories. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3581–3590 (2017)

  56. Lu, H., Zhang, L., Cao, Z., Wei, W., Xian, K., Shen, C., van den Hengel, A.: When unsupervised domain adaptation meets tensor representations. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 599–608 (2017)

  57. Ghifary, M., Balduzzi, D., Kleijn, W.B., Zhang, M.: Scatter component analysis: a unified framework for domain adaptation and domain generalization. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1414–1430 (2016)

  58. Li, J., Lu, K., Huang, Z., Zhu, L., Shen, H.T.: Transfer independently together: a generalized framework for domain adaptation. IEEE Trans. Cybern. 49(6), 2144–2155 (2018)

Download references

Funding

No funding was received for the submitted work.

Author information

Authors and Affiliations

  1. Faculty of IT and Computer Engineering, Urmia University of Technology, Urmia, Iran

    Mozhdeh Zandifar

  2. Department of IT and Computer Engineering, Urmia University of Technology, Urmia, Iran

    Samaneh Rezaei & Jafar Tahmoresnezhad

Authors
  1. Mozhdeh Zandifar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samaneh Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jafar Tahmoresnezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The methodology, writing and theoretical analysis: MZ. Supervision: JT. Writing and analysis: SR.

Corresponding author

Correspondence to Jafar Tahmoresnezhad.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zandifar, M., Rezaei, S. & Tahmoresnezhad, J. Unsupervised domain adaptation via transferred local Fisher discriminant analysis. Iran J Comput Sci 6, 345–364 (2023). https://doi.org/10.1007/s42044-023-00144-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42044-023-00144-x

Keywords

Search

Navigation

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy