Viseme recognition using lip curvature and neural networks to detect Bangla vowels

Automatic speech recognition plays an important role in human-computer interaction,which can be applied in various vital applications like crime-fighting and helping the hearing-impaired. this paper provides a new method for recognition of bengali visemes based on a combination of image-based lip segmentation techniques,use of curvature of the both inner and outer lips as well as neural networks. the method is divided into three steps. first step is a lip segmentation step that uses a combination of red exclusion method,hsv space and cie spaces to produce illumination invariant images. next,inner and outer lips are extracted separately using a new technique for curve-fitting. second step is the feature extraction step,which makes use of quadratic curve-coefficients of the inner and outer lip contours. finally,viseme recognition is done using a neural network. a dataset was created with 171 lip images of bangla visemes being spoken by different speakers and under different lighting conditions. the proposed method gave a viseme recognition result of 87.3%. due to the use of non-iterative method as opposed to conventional methods,the algorithm was found to be faster in detecting lip contours.