Abstract
We study estimation of mixture models for problems in which multiple views of the instances are available. Examples of this setting include clustering web pages or research papers that have intrinsic (text) and extrinsic (references) attributes. Our optimization criterion quantifies the likelihood and the consensus among models in the individual views; maximizing this consensus minimizes a bound on the risk of assigning an instance to an incorrect mixture component. We derive an algorithm that maximizes this criterion. Empirically, we observe that the resulting clustering method incurs a lower cluster entropy than regular EM for web pages, research papers, and many text collections.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Abney, S.: Bootstrapping. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (2002)
Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D., Jordan, M.: Matching words and pictures. Journal of Machine Learning Research 3, 1107–1135 (2002)
Becker, S., Hinton, G.: A self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355, 161–163 (1992)
Bickel, S., Scheffer, T.: Multi-view clustering. In: Proceedings of the IEEE International Conference on Data Mining (2004)
Bickel, S., Scheffer, T.: Estimation of mixture models using Co-EM. In: Proceedings of the ICML Workshop on Learning with Multiple Views (2005)
Blei, D., Jordan, M.: Modeling annotated data. In: Proceedings of the ACM SIGIR Conference on Information Retrieval (2003)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the Conference on Computational Learning Theory (1998)
Brefeld, U., Scheffer, T.: Co-EM support vector learning. In: Proceedings of the International Conference on Machine Learning (2004)
Collins, M., Singer, Y.: Unsupervised models for named entity classification. In: Proc. of the Conf. on Empirical Methods in Natural Language Processing (1999)
Dasgupta, S., Littman, M., McAllester, D.: PAC generalization bounds for co-training. In: Proceedings of Neural Information Processing Systems (2001)
de Sa, V.: Learning classification with unlabeled data. In: Proceedings of Neural Information Processing Systems (1994)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39 (1977)
Ghani, R.: Combining labeled and unlabeled data for multiclass text categorization. In: Proceedings of the International Conference on Machine Learning (2002)
Kailing, K., Kriegel, H., Pryakhin, A., Schubert, M.: Clustering multi-represented objects with noise. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 394–403. Springer, Heidelberg (2004)
McCallum, A., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Proc. of the International Conference on Machine Learning (1998)
Muslea, I., Kloblock, C., Minton, S.: Active + semi-supervised learning = robust multi-view learning. In: Proc. of the International Conf. on Machine Learning (2002)
Nigam, K., Ghani, R.: Analyzing the effectiveness and applicability of co-training. In: Proceedings of the Workshop on Information and Knowledge Management (2000)
Sinkkonen, J., Nikkilä, J., Lahti, L., Kaski, S.: Associative clustering. In: Proceedings of the European Conference on Machine Learning (2004)
Wu, J.: On the convergence properties of the EM algorithm. The Annals of Statistics 11, 95–103 (1983)
Yarowsky, D.: Unsupervised word sense disambiguation rivaling supervised methods. In: Proc. of the Annual Meeting of the Association for Comp. Ling. (1995)
Zhao, Y., Karypis, G.: Criterion functions for document clustering: Experiments and analysis. Technical Report TR 01-40, Department of Computer Science, University of Minnesota, Minneapolis, MN, 2001 (2001)
Zhou, Z., Li, M.: Semi-supervised regression with co-training. In: Proceedings of the International Joint Conference on Artificial Intelligence (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bickel, S., Scheffer, T. (2005). Estimation of Mixture Models Using Co-EM. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_9
Download citation
DOI: https://doi.org/10.1007/11564096_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)