Abstract

This paper presents a novel feature extraction framework for sign language recognition application to assist the deaf and speech-impaired in daily communication. Our method extracts visual features from the hand gesture using convexity defects, K-curvature and Hough line which can differentiate visually similar signs. In segmentation stage, Canny edge detection and skin color segmentation with histogram backprojection method are used to extract the hand region from the background. The features to be extracted are categorized into shape, orientation and motion. The shape features of the sign language will then be extracted using convexity defect, K-curvature and Hough line techniques. The orientation feature will be extracted by calculating the palm center to wrist angle. The trajectory motion of the dynamic gesture is extracted using the chain code method. Lastly, Decision Tree classification is employed in classification of both static and dynamic gestures. The proposed framework is carried out in smartphone platform to recognize 26 alphabets, 10 numbers and eight dynamic American Sign Language (ASL). The average accuracy of 29 static ASL achieved is 85.72% and average accuracy of 10 dynamic ASL achieved is 77%. Through this research, it is found that the proposed framework can recognize larger sign languages database as compared with previous convexity defect-based sign language recognition research.