Abstract
In this paper, an experimental study of different ordinal regression methods and measures is presented. The first objective is to gather the results of a considerably high number of methods, datasets and measures, since there are not many previous comparative studies of this kind in the literature. The second objective is to detect the redundancy between the evaluation measures used for ordinal regression. The results obtained present the maximum MAE (maximum of the mean absolute error of the difference between the true and the predicted ranks of the worst classified class) as a very interesting alternative for ordinal regression, being the less uncorrelated with respect to the rest of measures. Additionally, SVOREX and SVORIM are found to yield very good performance when the objective is to minimize this maximum MAE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Agresti, A.: Categorical Data Analysis, 2nd edn. John Wiley and Sons (2002)
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 2009), Pisa, Italy (December 2009)
Campbell, M., Donner, A.: Classification efficiency of multinomial logistic regression relative to ordinal logistic regression. Journal of the American Statistical Association 84, 587–591 (1989)
Cardoso, J.S., da Costa, J.F.P.: Learning to classify ordinal data: The data replication method. Journal of Machine Learning Research 8, 1393–1429 (2007)
Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. Journal of Machine Learning Research 6, 1019–1041 (2005)
Chu, W., Keerthi, S.S.: Support vector ordinal regression. Neural Computation 19(3), 792–815 (2007)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)
Crammer, K., Singer, Y.: Online ranking by projecting. Neural Computation 17(1), 145–175 (2005)
Cruz-Ramírez M., Hervás-Martínez, C., Sánchez-Monedero, J., Gutiérrez, P.A.: A preliminary study of ordinal metrics to guide a multi-objective evolutionary algorithm. In: Proceedings of the 11th International Conference on Intelligent Systems Design and Applications (ISDA 2011), Córdoba, Spain, pp. 1176–1181 (2011)
Frank, E., Hall, M.: A Simple Approach to Ordinal Classification. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, p. 145. Springer, Heidelberg (2001)
Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machines. IEEE Transaction on Neural Networks 13(2), 415–425 (2002)
Kendall, M.G.: Rank Correlation Methods. Hafner Press, New York (1962)
Kotsiantis, S.B., Pintelas, P.E.: A Cost Sensitive Technique for Ordinal Classification Problems. In: Vouros, G.A., Panayiotopoulos, T. (eds.) SETN 2004. LNCS (LNAI), vol. 3025, pp. 220–229. Springer, Heidelberg (2004)
Kramer, S., Widmer, G., Pfahringer, B., de Groeve, M.: Prediction of Ordinal Classes using Regression Trees. In: Ohsuga, S., Raś, Z.W. (eds.) ISMIS 2000. LNCS (LNAI), vol. 1932, pp. 426–434. Springer, Heidelberg (2000)
Li, L., Lin, H.T.: Ordinal Regression by Extended Binary Classification. In: Advances in Neural Information Processing Systems, vol. 19, pp. 865–872 (2007)
McCullagh, P.: Regression models for ordinal data (with discussion). Journal of the Royal Statistical Society 42(2), 109–142 (1980)
McCullagh, P., Nelder, J.A.: Generalized Linear Models. Monographs on Statistics and Applied Probability, 2nd edn. Chapman & Hall,CRC (1989)
PASCAL: Pascal (Pattern Analysis, Statistical Modelling and Computational Learning) machine learning benchmarks repository (2011), http://mldata.org/
Shashua, A., Levin, A.: Ranking with large margin principle: Two approaches. In: Advances in Neural Information Processing Systems, vol. 15, pp. 937–944. MIT Press, Cambridge (2003)
Verwaeren, J., Waegeman, W., De Baets, B.: Learning partial ordinal class memberships with kernel-based proportional odds models. Computational Statistics & Data Analysis (2011) (in press) doi:10.1016/j.csda.2010.12.007
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gutiérrez, P.A., Pérez-Ortiz, M., Fernández-Navarro, F., Sánchez-Monedero, J., Hervás-Martínez, C. (2012). An Experimental Study of Different Ordinal Regression Methods and Measures. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, SB. (eds) Hybrid Artificial Intelligent Systems. HAIS 2012. Lecture Notes in Computer Science(), vol 7209. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28931-6_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-28931-6_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28930-9
Online ISBN: 978-3-642-28931-6
eBook Packages: Computer ScienceComputer Science (R0)