ARTIFICIAL INTELLIGENCE-BASED SOFTWARE FOR RECOGNIZING DYNAMIC GESTURES OF THE KARAKALPAK SIGN LANGUAGE IN INCLUSIVE EDUCATION
Keywords:
inclusive education, artificial intelligence, dynamic gesture recognition, Karakalpak Sign Language, computer vision, neural networks, digital pedagogy, pedagogical feedback.Abstract
The development of inclusive education requires the introduction of modern digital technologies that provide equal access to learning for students with special educational needs. In particular, learners with hearing impairments need effective methodological and technological support for mastering sign language as a major means of communication, social interaction, and educational participation. This thesis examines the design and pedagogical application of artificial intelligence-based software for recognizing dynamic gestures of the Karakalpak Sign Language in an inclusive educational environment.
The relevance of the study is determined by the fact that dynamic gestures have a complex spatio-temporal structure. They include hand trajectory, palm orientation, movement speed, gesture rhythm, and a sequence of motion phases. Traditional methods of teaching and assessing gestures often depend on the subjective observation of the teacher and do not always provide immediate, objective, and individualized feedback. Therefore, the use of artificial intelligence, computer vision, and neural network technologies creates new opportunities for improving the quality of inclusive education.
The proposed software system is designed to analyze video sequences of learners’ hand movements, extract spatial and temporal features, classify dynamic gestures, and generate pedagogical feedback. The system includes video capture, preprocessing, feature extraction, gesture classification, and feedback modules. Convolutional neural networks are used to identify spatial features of hand movements, while recurrent neural network-based approaches help analyze the temporal dynamics of gesture execution. The software supports repeated practice, self-control, correction of errors, and objective assessment of learning outcomes.
The results of the study show that the integration of artificial intelligence-based gesture recognition software into the learning process contributes to the formation of sign language communicative competence, increases learner independence, and improves the objectivity of assessment. The proposed approach can be considered an effective digital pedagogical tool for teaching regional sign languages, including the Karakalpak Sign Language, in inclusive education.
Downloads
References
1. Bekarystankyzy, A., Mamyrbayev, O., Mendes, M., Zhumazhanov, B., & Fazylzhanova, A. Automatic Speech Recognition Improvement for Kazakh Language with Enhanced Language Model. Communications in Computer and Information Science, 2023, Vol. 1863, pp. 538–545.
2. Seitnazarov, K. K., & Turemurаtova, B. K. Application of artificial intelligence technology in the distance education system. News of Education: Research in the XXI Century, 2022.
3. Kudryavtsev, V. B., Gasanov, E. E., & Podkolzin, A. S. Pattern Recognition. Moscow: Yurait Publishing House, 2023.
4. Seitnazarov, K. K., & Turemurаtova, B. K. Difference between deep learning and machine learning. Periodica Journal of Modern Philosophy, Social Sciences and Humanities, 2022.
5. Nagoev, Z. V., & Gurtaeva, I. A. Basic elements of the cognitive model and speech perception mechanism based on multi-agent recursive intelligence. Izvestiya Kabardino-Balkarian Scientific Center of the Russian Academy of Sciences, 2019, No. 3, pp. 30–38.
6. Aitulen, A. D., Mukhanov, S. B., & Khassenova, G. I. Data Science and Machine Learning: Face Recognition Through Various Facial Expressions. Almaty, 2019.
7. Aitulen, A. D., & Mukhanov, S. B. Processing, identification and face recognition using the Viola-Jones method. Bulletin of KazNRTU, 2019, No. 6, pp. 16–24.
8. Kenesbaevich, S. K., Muxambetmustapayevich, D. A., & Arzubaevich, N. A. Development of software for calculating the forecast of groundwater regime based on probabilistic and statistical methods. ACADEMICIA: An International Multidisciplinary Research Journal, 2020, Vol. 10, No. 7, pp. 526–530.
9. Uskenbayeva, R. K., & Mukhanov, S. B. Contour analysis of external images. ACM International Conference Proceeding Series, Almaty, 2020.
10. Mukhanov, S. B., & Uskenbayeva, R. K. Pattern Recognition with Using Effective Algorithms and Methods of Computer Vision Library. Advances in Intelligent Systems and Computing, 2020, No. 1, pp. 31–37.
