基于灰度共生矩阵和精度高斯支持向量机的中国手语手指语识别

摘要/Abstract

摘要： 手语识别是打破聋人和健听人之间交流障碍的有效途径。中国手语一般可以分为手势语和手指语,手势语因为地区性和个体差异性导致种类和变化繁多,识别相对困难,所以需要不断学习和训练;手指语通过拼音字母的表现形式给出结果,表达具有确定性,尤其在姓名、特殊含义、抽象表达方面效果明显。手语识别中,大部分的研究主要聚焦于某种手势,围绕手形、方向、位置和运动轨迹等关键特征,并结合某些学习算法来提升识别的准确率,然而最基本可靠的手指语识别却往往被忽略。为此,文中提出了一种基于灰度共生矩阵(GLCM)和精度高斯支持向量机(FGSVM)的方法来更准确有效地识别中国手语手指语。首先构建手指语数据集,即通过数码相机直接获取手指语图像或者从视频中选取关键帧作为手语图像素材,然后将手形从图像背景中分割出来,把每个图像调整为N×N的特定尺寸并转换为灰度图像;其次是提取特征,即对灰度图像中强度值的数量进行降维,同时创建对应的灰度共生矩阵,通过调整像素间的距离和角度等参数来获取增强的数据特征;最后,将提取的图像的特征数据提交到精度高斯支持向量机分类器中,进行10倍交叉验证和分类测试。对30种类别的510个中国手语手指语图像样本的实验结果表明,基于GLCM-FGSVM的分类准确率最高可达到92.7％,可以认为该方法在中国手语手指语分类方面卓有成效。

关键词: 灰度共生矩阵, 精度高斯支持向量机, 手语识别, 手指语, 中国手语

Abstract: Sign language recognition is an effective way to break the barriers between communication between deaf and hearing people.Generally,Chinese sign language can be divided into gesture language and finger language.Regional and individual differences lead to a wide variety,therefore gesture language recognition is relatively difficult,which requires constant learning and training.The finger language gives the result through the expression of the Chinese pinyin letters,which is deterministic,especially in terms of name,special meaning,and abstract expression.Most of the researches in sign language recognition concentrate on a certain gesture,focusing on key features such as hand shape,direction,position and motion trajectory,and combine some learning algorithms to improve the recognition accuracy,but neglect the most basic and reliable finger recognition.To this end,an effective method using gray level co-occurrence matrix (GLCM) and fine Gaussian support vector machine (FGSVM) was proposed to solve the problem of identifying Chinese finger sign language more accurately and effectively.The research method is as follows.Firstly,the finger sign language data set was constructed.The finger language image was directly obtained by the digital camera or got from the key frame of the video,meanwhile the hand shape was segmented from the image,and each image was adjusted to N×N specific size and converted to grayscale images.Secondly,feature extraction was performed to reduce the dimension of the intensity values in the grayscale image,and at the same time,the corresponding gray level co-occurrence matrix was created,and the enhanced data features were obtained by adjusting the parameters of inter-pixel distance and angle.Finally,the extracted image feature data were submitted to the fine Gaussian support vector machine classifier based on the 10-fold cross-validation classification.Experiments on 510 Chinese finger sign language image samples from 30 categories show that the classification accuracy based on GLCM-FGSVM is up to 92.7%,and this method can be considered as effective approach in Chinese finger sign language classification.

Key words: Chinese sign language, Fine Gaussian support vector machine, Finger sign language, Gray level co-occurrence matrix, Sign language recognition

中图分类号:

TP391.41

蒋贤维, 张妙娴, 朱兆松. 基于灰度共生矩阵和精度高斯支持向量机的中国手语手指语识别[J]. 计算机科学, 2019, 46(11A): 303-308. https://doi.org/

JIANG Xian-wei, ZHANG Miao-xian, ZHU Zhao-song. Recognition of Chinese Finger Sign Language Based on Gray Level Co-occurrence Matrix and Fine Gaussian Support Vector Machine[J]. Computer Science, 2019, 46(11A): 303-308. https://doi.org/

参考文献

[1]ZHAN T.Pathological brain detection by artificial intelligence in magnetic resonance imaging scanning[J].Progress in ElectromagneticsResearch,2016,156:105-133.
[2]杨军辉.中国手语和汉语双语教育初探[J].中国特殊教育,2002(1):33.
[3]DIMITRIS METAXAS M D,CAROL N.Scalable ASL sign recognition using model-based machine learning and linguistically annotated corpora [C]∥8th Workshop on the Representation &Processing of Sign Languages:Involving the Language Community,Language Resources and Evaluation Conference 2018.Miyazaki,Japan,2018:1-5.
[4]PAN T Y,YL L,YEH C W,et al.Sign language recognition in complex background scene based on adaptive skin colourmodelling and support vector machine [J].International Journal of Big Data Intelligence,2018(5):1-2.
[5]司阳,任松,肖秦琨,等.基于彩色-深度图像的手语识别算法[J].科学技术与工程,2018,18(11):104-109.
[6]许天然,吴垚,苏红旗.基于移动终端的汉语手语识别技术研究[J].科技资讯,2012,19(1):24-26.
[7]林水强,吴亚东,陈永辉.基于几何特征的手势识别方法[J].计算机工程与设计,2014,35(2):637-639.
[8]郑津津,徐士海.基于结构特征提取的手语识别系统研究[D].合肥:中国科学技术大学,2015.
[9]SU R L,X C,CAO S,et al.Random Forest-Based Recognition of Isolated Sign Language Subwords Using Data from Accelerometers and Surface Electromyographic Sensors [J].Journals Sensors,2016,16(1):100.
[10]WASHEF AHMED K C,SOMA M.Vision based Hand Gesture Recognition using Dynamic Time Warping for Indian Sign Language[C]∥International Conference on Information Science (ICIS).2016:1-6.
[11]BISWAS N,CHAKRABORTY S,MULLICK S S.A parameter independent fuzzy weighted k-Nearest neighbor classifier [J].Pattern Recognition Letters,2018,101:80-87.
[12]HAO Z Y,ALIFU K,LI X H,et al.Chinese finger language re-cognition use CapsNet [J].Application Research of Computers,2018,36:216-228.
[13]PAN C.Multiple sclerosis identification by convolutional neural network with dropout and parametric ReLU[J].Journal of Computational Science,2018,28:1-10.
[14]YANG W B,YANG H C.Gesture recognition method based on convolutional neural network[J].Journal of Anhui Polytechnic University,2018,33:41-46.
[15]WU J.Fruit classification by biogeography-based optimizationand feedforward neural network [J].Expert Systems,2016,33(3):239-253.
[16]LU S.Pathological Brain Detection in Magnetic Resonance Imaging Using Combined Features and Improved Extreme Learning Machines [J].Journal of Medical Imaging and Health Informa-tics,2018,8:1486-1490.
[17]刘艳虹,顾定倩,程黎,等.我国手语使用状况的调查研究[J].语言文字应用,2013,5(2):35-41.
[19]徐鑫鑫,黄元元,胡作进.连续复杂手语中关键动作的提取算法[J].计算机科学,2018,45(S2):189-193.
[20]MELLISA P A,JEKLIN H,SAKKA N.Mammograms Classification Using Gray-level Co-occurrence Matrix and Radial Basis Function Neural Network [J].Procedia Computer Science,2015,59:83-91.
[21]LU H M.Facial Emotion Recognition Based on BiorthogonalWavelet Entropy,Fuzzy Support Vector Machine,and Stratified Cross Validation [J].IEEE Access,2016,4:8375-8385.
[22]毛思晨.基于卷积网络和长短时记忆网络的中国手语词识别方法研究[D].合肥:中国科技大学,2018.
[23]ZHOU X X,SHENG H.Combination of stationary wavelettransform and kernel support vector machines for pathological brain detection [J].Simulation,2016,92(9):827-837.
[24]ZHANG H Y,YUAN J Z.Survey on New Methods of Vision-based Hand Gesture Recognition[J].Journal of Computational Science,2017,44:1-6.

相关文章 15

[1]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2]	郭丹, 唐申庚, 洪日昌, 汪萌. 手语识别、翻译与生成综述 Review of Sign Language Recognition, Translation and Generation 计算机科学, 2021, 48(3): 60-70. https://doi.org/10.11896/jsjkx.210100227
[3]	冉孟元, 刘礼, 李艳德, 王珊珊. 基于惯性传感器融合控制算法的聋哑手语识别 Deaf Sign Language Recognition Based on Inertial Sensor Fusion Control Algorithm 计算机科学, 2021, 48(2): 231-237. https://doi.org/10.11896/jsjkx.191200143
[4]	郑波, 马昕. 基于双变异粒子群优化算法优化的支持向量机及其在民航发动机损伤类型识别中的应用 Application on Damage Types Recognition in Civil Aeroengine Based on SVM Optimized by DMPSO 计算机科学, 2020, 47(11A): 132-138. https://doi.org/10.11896/jsjkx.200600101
[5]	叶鹏, 王永芳, 夏雨蒙, 安平. 一种融合深度基于灰度共生矩阵的感知模型 Perceptual Model Based on GLCM Combined with Depth 计算机科学, 2019, 46(3): 92-96. https://doi.org/10.11896/j.issn.1002-137X.2019.03.012
[6]	杜秀丽, 张薇, 顾斌斌, 陈波, 邱少明. 基于灰度共生矩阵的图像自适应分块压缩感知方法 GLCM-based Adaptive Block Compressed Sensing Method for Image 计算机科学, 2018, 45(8): 277-282. https://doi.org/10.11896/j.issn.1002-137X.2018.08.050
[7]	邹娜, 田金文. 多特征融合红外舰船尾流检测方法研究 Research on Multi Feature Fusion Infrared Ship Wake Detection 计算机科学, 2018, 45(11A): 172-175.
[8]	徐鑫鑫, 黄元元, 胡作进. 连续复杂手语中关键动作的提取算法 Extraction Algorithm of Key Actions in Continuous and Complex Sign Language 计算机科学, 2018, 45(11A): 189-193.
[9]	梁文乐,黄元元,胡作进. 基于二级匹配策略的实时动态手语识别 Real-time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy 计算机科学, 2017, 44(7): 299-303. https://doi.org/10.11896/j.issn.1002-137X.2017.07.054
[10]	苏慧嘉,郑继明. 结合游程长度与共生矩阵的图像拼接篡改检测方法 Image Splicing Blind Detection Method Combined RLRN with GLCM 计算机科学, 2017, 44(6): 150-154. https://doi.org/10.11896/j.issn.1002-137X.2017.06.025
[11]	李文莉,高宏伟,冀大雄,李岩. 基于遗传算法的海底沉积物纹理特征优化方法 Optimization Method of Seabed Sediment Texture Feature Based on Genetic Algorithm 计算机科学, 2016, 43(Z6): 130-133. https://doi.org/10.11896/j.issn.1002-137X.2016.6A.031
[12]	杨全,彭进业. 采用SIFT-BoW和深度图像信息的中国手语识别研究 Chinese Sign Language Recognition Research Using SIFT-BoW and Depth Image Information 计算机科学, 2014, 41(2): 302-307.
[13]	张永良,刘超凡,肖刚,方珊珊. 基于曲波纹理分析和SVM-KNN分类的假指纹检测算法 Fake Fingerprint Detection Algorithm Based on Curvelet Texture Analysis and SVM-KNN Classification 计算机科学, 2014, 41(12): 303-308. https://doi.org/10.11896/j.issn.1002-137X.2014.12.065
[14]	于美娟许力刘岩恺马希荣. 基于索引结构的手语词库的设计 Design of the Sign Language Template Library Based on Index Structure 计算机科学, 2012, 39(12): 195-197.
[15]	王娴,周宇,云挺,邓玉和. 基于Curvelet变换的荻草细胞图像分割 Miscanthus Sacchariflorus Cells Image Segmentation Based on Curvelet Transformation 计算机科学, 2012, 39(11): 277-279.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed