Abstract
For the recognition of continuous sign language we analyse whether we can improve the results by explicitly incorporating depth information. Accurate hand tracking for sign language recognition is made difficult by abrupt and fast changes in hand position and configuration, overlapping hands, or a hand signing in front of the face. In our system depth information is extracted using a stereo-vision method that considers the time axis by using pre- and succeeding frames. We demonstrate that depth information helps to disambiguate overlapping hands and thus to improve the tracking of the hands. However, the improved tracking has little influence on the final recognition results.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Dreuw, P., Deselaers, T., Rybach, D., Keysers, D., Ney, H.: Tracking using dynamic programming for appearance-based sign language recognition. In: FG, pp. 293–298. IEEE, Los Alamitos (2006)
Dreuw, P., Rybach, D., Deselaers, T., Zahedi, M., Ney, H.: Speech recognition techniques for a sign language recognition system. In: Interspeech, Antwerp, Belgium, August 2007, pp. 2513–2516 (2007)
Fang, G., Gao, W., Zhao, D.: Large-vocabulary continuous sign language recognition based on transition-movement models. IEEE Trans. on Systems, Man, and Cybernetics 37(1) (January 2007)
Fujimara, K., Liu, X.: Sign recognition using depth image streams. In: FG, Southampton, UK, April 2006, pp. 381–386 (2006)
Kolmogorov, V., Criminisi, A., Crogs, G., Blake, A., Rother, C.: Probabilistic fusion of stereo with color and contrast for bi-layer segmentation. PAMI (2006)
Lichtenauer, J., ten Holt, G., Hendriks, E., Reinders, M.: 3d visual detection of correct ngt sign production. In: Annual Conf. of the Advanced School for Computing and Imaging (2007)
Neidle, C.: SignstreamTMAnnotation: Conventions used for the American Sign Language Linguistic Research Project and addendum. Technical Report 11 and 13, American Sign Language Linguistic Research Project, Boston University (2002) (2007)
Ohta, Y., Kanade, T.: Stereo by intra- and inter-scanline search using dynamic programming. PAMI 7(2), 139–154 (1985)
Ong, S., Ranganath, S.: Automatic sign language analysis: A survey and the future beyond lexical meaning. PAMI 27(6), 873–891 (2005)
Robertson, D., Ramalingam, S., Fitzgibbon, A., Criminisi, A., Blake, A.: Learning priors for calibrating families of stereo cameras. In: ICCV, Rio de Janeiro, Brazil (October 2007)
Ruiduo Yang, S.S., Loeding, B.: Enhanced level building algorithm to the movement epenthesis problem in sign language. In: CVPR, Minneapolis, MN, USA (June 2007)
Viola, P., Jones, M.J.: Robust real-time face detection. IJCV 57(2), 137–154 (2004)
Vogler, C., Metaxas, D.: A framework for recognizing the simultaneous aspects of american sign language. CVIU 81(3), 358–384 (2001)
Wang, S.B., Quattoni, A., Morency, L.-P., Demirdjian, D., Darrell, T.: Hidden conditional random fields for gesture recognition. In: CVPR, New York, USA, June 2006, vol. 2, pp. 1521–1527 (2006)
Wrobel, U.R.: Referenz in Gebärdensprachen: Raum und Person. Institut für Phonetik und Sprachliche Kommunikation, Universität München 37, 25–50 (2001)
Yao, G., Yao, H., Liu, X., Jiang, F.: Real time large vocabulary continuous sign language recognition based on op/viterbi algorithm. In: ICPR, Hong Kong, vol. 3, pp. 312–315 (August 2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dreuw, P., Steingrube, P., Deselaers, T., Ney, H. (2009). Smoothed Disparity Maps for Continuous American Sign Language Recognition. In: Araujo, H., Mendonça, A.M., Pinho, A.J., Torres, M.I. (eds) Pattern Recognition and Image Analysis. IbPRIA 2009. Lecture Notes in Computer Science, vol 5524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02172-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-02172-5_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02171-8
Online ISBN: 978-3-642-02172-5
eBook Packages: Computer ScienceComputer Science (R0)