Abstract
In this paper, we evaluate 7 machine learning algorithms for image classification including our recent approach that combines building of ensembles of extremely randomized trees and extraction of sub-windows from the original images. For the approach to be generic, all these methods are applied directly on pixel values without any feature extraction. We compared them on four publicly available datasets corresponding to representative applications of image classification problems: handwritten digits (MNIST), faces (ORL), 3D objects (COIL-IOO), and textures (OUTEX). A comparison with studies from the computer vision literature shows that generic methods can come remarkably close to specialized methods. In particular, our sub-window algorithm is competitive with the state of the art, a remarkable result considering its generality and conceptual simplicity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.
L. Breiman. Random forests. Machine learning, 45:5–32, 2001.
L. Breiman, J.H. Friedman, R.A. Olsen, and C.J. Stone. Classification and Regression Trees. Wadsworth International (California), 1984.
C.J.C. Burges and B. Schölkopf, Improving the accuracy and speed of support vector machines. In M. C. Mozer, M. I. Jordan, and Thomas Petsche, editors, Advances in Neural Information Processing Systems, volume 9, page 375. The MIT Press, 1997.
Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. Technical report, Computer Science and Information Engineering, National Taiwan University, 2003.
J. Dahmen, D. Keysers, and H. Ney. Combined classification of handwritten digits using the ‘virtual test sample method’. In Proc. Second International Workshop, MCS 2001 Cambridge, UK, pages 99–108, July 2001.
H. Drucker. Fast decision tree ensembles for optical character recognition. In Proc. Fifth Annual Symposium on Document Analysis and Information Retrieval, pages 137–147, 1996.
Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Proc. Thirteenth International Conference on Machine Learning, pages 148–156, 1996.
P. Geurts. Contributions to decision tree induction: bias/variance tradeoff and time series classification. Phd. thesis, Department of Electrical Engineering and Computer Science, University of Liège, May 2002.
P. Geurts. Extremely randomized trees. Technical report, Department of Electrical Engineering and Computer Science, University of Liège, 2003.
G.-D. Guo, S. Li, and K. Chan. Face recognition by support vector machines. In Proc. International Conference on Automatic Face and Gesture Recognition, 196–201., 2000.
G.-D. Guo and H.-J. Zhang. Boosting for fast face recognition. In Proc. IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pages 96–100, 200l.
M.S. Hoque and M. C. Fairhurst. A moving window classifier for offline character recognition. In Proc. of the Seventh International Workshop on Frontiers in Handwriting Recognition, Amsterdam, pages 595–600, September 2000.
T.O. Kvalseth. Entropy and correlation: Some comments. IEEE Trans. on Systems, Man and Cybernetics, SMC-17(3):517–519, 1987.
S. Lawrence, C. Lee Giles, A. C. Tsoi, and A. D. Back. Face recognition: A convolutional neural network approach. IEEE Transactions on Neural Networks, 8(1):98–113, 1997.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc. of the IEEE, 86(11):2278–2324, 1998.
R. Maree, Geurts P., Piater J., and Wehenkel L. A generic approach for image classification based on decision tree ensembles. Submitted.
T. Menp, M. Pietikinen, and J. Viertola. Separating color and pattern information for color texture discrimination. In Proc. 16th International Conference on Pattern Recognition, 2002.
A. Nefian and M. Hayes. Face recognition using an embedded HMM. In Proc. IEEE Conference on Audio and Video-based Biometric Person Authentication, pages 19–24, March 1999.
S. Obrzalek and J. Matas. Object recognition using local affine frames on distinguished regions. In Electronic Proceedings of the 13th British Machine Vision Conference, University of Cardiff, 2002.
T. Ojala, T. Menp, M. Pietikinen, J. Viertola, J. Kyllnen, and S. Huovinen. Outex-new framework for empirical evaluation of texture analysis algorithms. Proc. 16th International Conference on Pattern Recognition, Quebec, Canada, 1:701–706, 2002.
R. Paredes and A. Perez-Cortes. Local representations and a direct voting scheme for face recognition. In Pattern Recognition in Information Systems, Proc. 1st International Workshop on Pattern Recognition in Information Systems, pages 71–79, July 2001.
M. Pontil and A. Verri. Support vector machines for 3d object recognition. IEEE 1hmsactions on Pattern Analysis and Machine Intelligence, 20(6):637–646, 1998.
V.N. Vapnik. The nature of statistical learning theory. Springer Verlag, 1995.
L. Wehenkel. Automatic learning techniques in power systems. Kluwer Academic, Boston, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag London
About this paper
Cite this paper
Marée, R., Geurts, P., Visimberga, G., Piater, J., Wehenkel, L. (2004). A Comparison of Generic Machine Learning Algorithms for Image Classification. In: Coenen, F., Preece, A., Macintosh, A. (eds) Research and Development in Intelligent Systems XX. SGAI 2003. Springer, London. https://doi.org/10.1007/978-0-85729-412-8_13
Download citation
DOI: https://doi.org/10.1007/978-0-85729-412-8_13
Publisher Name: Springer, London
Print ISBN: 978-1-85233-780-3
Online ISBN: 978-0-85729-412-8
eBook Packages: Springer Book Archive