utilizing surf features and klt tracking algorithm in augmented reality (ar), using kinect v.2 with the aim of autism therapy

Using augmented reality (ar) played important role in learning, education, therapy and especially autism therapy in last decade. rehabilitating people with autistic spectrum disorder (asd) using ar was so mentionable in the therapy process. this paper presents a new method for rehabilitation and therapy for people with asd using microsoft kinect v.2. system starts with human detection using depth sensor. in the second step system detects the subject’s face using viola and jones algorithm and recognize subject’s face using convolutional neural network (cnn) using color sensor. third step is consisting of feature extraction process using bag of features (bof) for content based image retrieval (cbir) purpose and tracing these points using kanade–lucas–tomasi (klt) algorithm for finalizing ar process. in the meanwhile, morphing subject’s face to a new avatar takes place (randomly), and also emotion recognition using cnn helps the human therapist. experiments is done on 4 children with asd. therapy estimation after a weak showed experiment’s improvement during 7 days or meetings.