DOI: 10.18413/2518-1092-2023-8-4-0-6

DEVELOPMENT OF MACHINE LEARNING METHODS AND A LIBRARY OF INTERPRETABLE PREDICTIVE MODELING OF HUMAN BEHAVIOR DURING HIS ONLINE PROFILING

Ivan Zakharovich Smirnov
Anatoly Andreevich Medvedev
Timur Ruslanovich Samigulin
Alena Alekseevna Komarova
Artyom Igorevich Timoshchuk-Bondar
Mikhail Vitalievich Sinko
Anastasia Alexandrov Laushkina
Olga Olegovna Goffman
Oleg Olegovich Basov

The study of individual psychological characteristics of people is important in the areas of education, management and administration, ensuring the safety of individuals and communities. There are various tools for solving the problem of determining and analyzing personal characteristics, but they have a number of limitations. We present a solution that extracts and uses machine learning to analyze human facial and speech features from video footage, applicable to the study of eight different individual psychological characteristics in an online digital profiling task. The user is invited to use the developed Expert library to obtain new characteristics by applying and combining existing ML modules to solve a wide class of problems.

Keywords: machine learning, open source, multimodal analysis, verbal and non-verbal signs.

Number of views: 234 (view statistics)

Количество скачиваний: 645

Full text (PDF)To articles list

Information for citation:

Smirnov I.Z., Medvedev A.A., Samigulin T.R., Komarova A.A., Timoshchuk-Bondar A.I., Sinko M.V., Laushkina A.A., Goffman O.O., Basov O.O. Development of machine learning methods and a library of interpretable predictive modeling of human behavior in the process of online profiling // Research result. Information technologies. – Т. 8, №4, 2023. – P. 59-69. DOI: 10.18413/2518-1092-2023-8-4-0-6

User comments
Reference lists
Thanks

While nobody left any comments to this publication.
You can be first.

Goupil L., Ponsot E., Richardson D. et al. Listeners’ perceptions of the certainty and honesty of a speaker are associated with a common prosodic signature // Nat Commun. 2021. – №12.
Teixeira J. P., Oliveira C., Lopes C. Vocal Acoustic Analysis – Jitter, Shimmer and HNR Parameters // Procedia Technology. – 2013. – V. 9. – P. 1112-1122.
Kirillov S., Lukyanov D. Evaluation of psycho-emotional status of robotic system operator in the Arctic // IOP Conference Series: Earth and Environmental Science. – 2019. – № 302.
Rammstedt B., Danner D., Lechner C. Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study // Large-scale Assess Educ 5. – 2017. – №2.
Anbesaw T., Zenebe Y., Asmamaw A., et. al. Post-traumatic stress disorder and associated factors among people who experienced traumatic events in Dessie town, Ethiopia, 2022: A community based study // Frontiers in Psychiatry. 2022. №13.
Reeve D. Psycho-Emotional Disablism: The Missing Link? // Routledge Handbook of Disability StudiesEdition: 1stChapter: 7. 2012.
Le Duc T., Huynh S., Vu T., et. al. Personality Traits and Aggressive Behavior in Vietnamese Adolescents // Psychology Research and Behavior Management. – 2023. – №16. – Р. 1987-2003.
Cheng S., Dawson J., Thamby J., et al. How do aggression source, employee characteristics and organisational response impact the relationship between workplace aggression and work and health outcomes in healthcare employees? A cross-sectional analysis of the National Health Service staff survey in England // BMJ Open. 2020. – №10(8).
Sokolova M.S. Adaptation to the interlocutor as a component of positive communication: constitutive features // Current problems of philology and pedagogical linguistics. 2017. No. 2 (26).
Danilin M.V. Methods of teaching listening in conditions of multimodal communication using authentic audio-video materials (English, secondary general education): dis. Ph.D. ped. Sciences: 5.8.2. - M., 2021. – 173 p.
Zobkov V. A. A person’s self-confidence in decision-making situations // Bulletin of Kostroma State University. Series: Pedagogy. Psychology. Sociokinetics. – 2018. – No. 2.
Romek V. G. Self-confidence as a socio-psychological characteristic of personality: abstract. dis. Ph.D. social psycho. Sciences: 19.00.05. - Rostov-on-Don, 1997. – 12 p.
Kashapova E. R., Ryzhkova M. V. Cognitive distortions and their influence on individual behavior // Vestn. Volume. state un-ta. Economy. – 2015. – No. 2 (30).
Aneri R., Sonali J. Emotion Based Hate Speech Detection using Multimodal Learning // arXiv Computation and Language. 2022.
Jianyuan G., Kai H., Han W., et al. CMT: Convolutional Neural Networks Meet Vision Transformers // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. – Р. 12175-12185.
EmoPy: a machine learning toolkit for emotional expression // thoughtworks URL: https://www.thoughtworks.com/ (data access: 13.07.2023).
Camillo L., Jiuqiang T., Hadon N., et al. EmoPy: a machine learning toolkit for emotional expression // arXiv Distributed, Parallel, and Cluster Computing. 2019.
TextBlob: Simplified Text Processing // TextBlob URL: https://textblob.readthedocs.io/en/dev/ (data access: 13.07.2023).
Razzaq M.A., Hussain J., Bang J., et. al. A Hybrid Multimodal Emotion Recognition Framework for UX Evaluation Using Generalized Mixture Functions. Sensors 23. 2023. №23(9).
Detoxify // github URL: https://github.com/unitaryai/detoxify (data access: 13.07.2023).
Boersma P., Van Heuven V. Speak and unSpeak with PRAAT // Glot International. – V. 5. – №9/10. – Р. 341-347.
Gedas B., Heng W., Lorenzo T. Is Space-Time Attention All You Need for Video Understanding? // arXiv Computer Vision and Pattern Recognition. 2021.
Grishchenko I., Ablavatski A., Kartynnik Y., et. al. Attention Mesh: High-fidelity Face Mesh Prediction in Real-time // arXiv Computer Vision and Pattern Recognition. 2022.
Samigulin T.R., Smirnov I.Z., Laushkina A.A. Determination of markers of aggressive human behavior based on analysis of audio and text channels // Scientific result. Information Technology. – 2022. – T.7. – No. 2. – pp. 56-61.
Jacob D., Ming-Wei C., Kenton L., Kristina T. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // arXiv preprint arXiv:1810.04805. 2019.
Wen Z., Lin W., Wang T., Xu G. Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition // Biomimetics 8. 2023. – №2. – P. 199.
Peng Z., Lu Y., Pan S., Liu Y. Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention // ICASSP 2021 – 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). – 2021. – P. 3020-3024.
Amit G., Noura Al M., Steven B. ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference // arXiv Computation and Language. 2021.
Shickel B., Scott S., Martin H., at. al. Automatic Detection and Classification of Cognitive Distortions in Mental Health Text // IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). 2019.
Xuejiao Z., Chunyan M., Zhenchang X. Identifying Cognitive Distortion by Convolutional Neural Network based Text Classification // International Journal of Information Technology. 2017. – №23.
Simms T., Ramstedt C., Rich M., et.al. Detecting Cognitive Distortions Through Machine Learning Text Analytics // 2017 IEEE International Conference on Healthcare Informatics (ICHI). – 2017. – P. 508-512.
Beck A. Cognitive therapy and the emotional disorders // New York: New American Library. 1979. – P. 374.
Breiman L. Random Forests // Machine Learning. 2001. – №45. – P. 5–32.
Ferracane E., Durrett G., Li J., et. al. Did they answer? Subjective acts and intents in conversational discourse // Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. – P. 1626–1644.

All journals

Send article

Research result. Information technologies is included in the scientific database of the RINTs (license agreement No. 765-12/2014 dated 08.12.2014).

The journal is indexed by the following scientific databases and platforms

Research Result. Research result. Information technologies (ISSN 2518-1092)

The journal materials and website are licensed under Creative Commons «Attribution» 4.0 International.

The Founder: Federal State Autonomous Educational Institution of Higher Education "Belgorod National Research University"The Founder’s address: 85 Pobedy Street, Belgorod, the Belgorod region, 308015, Russia

The Publisher: Federal State Autonomous Educational Institution of HigherEducation "Belgorod National Research University" The Founder’s address:85 Pobedy Street, Belgorod, the Belgorod region, 308015, Russia

Editors Office: chief editor Chernomorets Andrey Alekseevich, e-mail: RR_IT@bsu.edu.ru, phone: +7 (4722) 30-13-92.

Registered by the Federal Service for Supervision of Communications, Information Technology and Mass Media (Roskomnadzor)

Certificate

Info letter (Russian)

Have questions?
You can write to us:

✉ Content manager

✉ Site administration

✉ Executive Secretary