Abstract
In order to conquer the major challenges of current web document clustering, i.e. huge volume of documents, high dimensional process and understandability of the cluster, we propose a simple hybrid algorithm (SHDC) based on top-k frequent term sets and k-means. Top-k frequent term sets are used to produce k initial means, which are regarded as initial clusters and further refined by k-means. The final optimal clustering is returned by k-means while the understandable description of clustering is provided by k frequent term sets. Experimental results on two public datasets indicate that SHDC outperforms other two representative clustering algorithms (the farthest first k-means and random initial k-means) both on efficiency and effectiveness.
This project is sponsored by national 863 high technology development foundation (No. 2004AA112020) and the National Grand Fundamental Research 973 Program of China (No. 2005CB321804).
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Zhuang, L., Dai, H.: A Maximal Frequent Itemset Approach for Web Document Clustering. In: Proceedings of the Fourth International Conference on Computer and Information Technology (CIT 2004) (2004)
Yongheng, W., Yan J., Shuqiang, Y.: Parallel Mining of Top-K Frequent Items in Very Large Text Database. WAIM (2005)
Han, J., Kamber, M.: Data Mining. Concepts and Techniques, 2nd edn. Morgan Kaufmann Press, Seattle, Washington, USA (2006)
Hotho, A., Nürnberger, A., Paaß, G.: A Brief Survey of Text Mining. LDV FORUM 20, 19–62 (2005)
Beil, F., Ester, M., Xu, X.: Frequent Term-Based Text Clustering. In: Proc. 8th Int. Conf. on Knowledge Discovery and Data Mining (KDD 2002), Edmonton, Alberta, Canada (2002)
Fung, B.C.M., Wang, K., Ester, M.: Hierarchical Document Clustering using Frequent Itemsets. SDM (2003)
MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceeding of the 5th Berkeley symposium in mathematics and probability (1967)
Steinbach, M., Karypis, G., Kumar, V.: A Comparison of Document Clustering Techniques. In: Proc. TextMining Workshop, KDD 2000 (2000)
HE, J., LAN, M., et al.: Initialization of cluster refinement algorithms: a review and comparative study. International Joint Conference on Neural Networks (2004)
Hotho, A., Maedche, A., Staab, S.: Ontology-based Text Document Clustering. KI 16(4), 48–54 (2002)
Strehl, A., Ghosh, J.: Cluster ensembles - a knowledge reuse framework for combining partitions. Journal of Machine Learning Research 3, 583–617 (2002)
Shi, Z., Ester, M.: Performance Improvement for Frequent Term-based Text Clustering Algorithm. Technique Report in Computing Science, Simon Fraser University (April 2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, L., Tian, L., Jia, Y., Han, W. (2007). A Hybrid Algorithm for Web Document Clustering Based on Frequent Term Sets and k-Means. In: Chang, K.CC., et al. Advances in Web and Network Technologies, and Information Management. APWeb WAIM 2007 2007. Lecture Notes in Computer Science, vol 4537. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72909-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-72909-9_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72908-2
Online ISBN: 978-3-540-72909-9
eBook Packages: Computer ScienceComputer Science (R0)