Abstract
In order to make a support vector machine applicable to time-varying problems, a forgetting factor is introduced to its cost function, in the same way as the RLS algorithm for adaptive filters. The idea of the forgetting factor is simple but it is shown to drastically change the performance of SVMs by deriving the average generalization error in a simple case where input space is one-dimensional. The average generalization error does not converge to zero, differently from the SVM in batch or online. We confirmed our results by computer simulations.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Schölkopf, B., Burges, C., Smola, A.J.: Advances in Kernel Methods: Support Vector Learning. Cambridge Univ. Press, Cambridge (1998)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge Univ. Press, Cambridge (2000)
Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmans, D. (eds.): Advances in Large Margin Classifiers. MIT Press, Cambridge (2000)
Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Proc. ICML, pp. 487–494 (2000)
Liu, F., Zhang, T., Zhang, R.: Modified kernel RLS-SVM based multiuser detection over multipath channels. IEICE Trans. Fundamentals E86-A(8), 1979–1984 (2003)
Haykin, S.: Adaptive Filter Theory, 3rd edn. Prentice-Hall, Englewood Cliffs (1996)
Schölkopf, B., Smola, A.J., Williamson, R.C., Bartlett, P.L.: New support vector algorithms. Neural Computation 12(5), 1207–1245 (2000)
Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)
Ikeda, K., Aoishi, T.: An asymptotic statistical analysis of support vector machines with soft margins. Neural Networks 18(3), 251–259 (2005)
Bennett, K.P., Bredensteiner, E.J.: Duality and geometry in SVM classifiers. In: Proc. ICML, pp. 57–64 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Funaya, H., Nomura, Y., Ikeda, K. (2009). A Support Vector Machine with Forgetting Factor and Its Statistical Properties. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5506. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02490-0_113
Download citation
DOI: https://doi.org/10.1007/978-3-642-02490-0_113
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02489-4
Online ISBN: 978-3-642-02490-0
eBook Packages: Computer ScienceComputer Science (R0)