Научная визуализация, 2018, том 10, номер 5, страницы 140 - 159, DOI: 10.26583/sv.10.5.09
Arabic Dynamic Gestures Recognition Using Microsoft Kinect
Авторы: B. Hisham1, A. Hamouda2
Computer engineering department, Al-Azhar University, Cairo, Egypt
1 ORCID: 0000-0002-1468-0145, basmahisham.2015@gmail.com
2 ORCID: 0000-0001-9041-1978
Аннотация
Sign language is an expressive way for deaf persons and hearing impaired to communicate with their societies, it is the basic alternative communication method between them and others. There are several studies have been done on sign language recognition systems, however, practically deployable system for real-time use is still a challenge also the researches in Arabic Sign Language Recognition (ArSLR) is very limited. This paper proposes Arabic Sign Language (ArSL) recognition system using Microsoft Kinect. The proposed system normalizes user's position and size captured by Microsoft Kinect then applies machine learning algorithms such as Support Vector Machine (SVM), K- Nearest Neighbors (KNN) and Artificial Neural Network (ANN) in order to provide a comparison on recognition accuracy. Also, we used Dynamic Time Wrapping (DTW) in order to match the sequence that represents the captured sign with the stored reference sequences, this is based on that all signs are dynamic. Recognized continuous signs are segmented using motion speed that segment a sequence of words with an accurate manner. We use a dataset for ArSL words from collected signs; it is composed of 42 Arabic signs in medical field to aid communication between a deaf or hard-of-hearing patient with the doctor. The experimental results showed that the proposed system recognition rate reached 89 % for KNN classifier with majority voting and the segmentation accuracy reached 91%. The system was trained on 840 samples and tested on 420 samples.
Ключевые слова: Sign Language, Static Gestures, Microsoft Kinect, KNN, ANN, SVM, DTW.
