Список литературы

2518-1092

Research result. Information technologies

2518-1092

10.18413/2518-1092-2023-8-4-0-6

3302

ARTIFICIAL INTELLIGENCE AND DECISION MAKING

<strong>DEVELOPMENT OF MACHINE LEARNING METHODS AND A LIBRARY OF INTERPRETABLE PREDICTIVE MODELING OF HUMAN BEHAVIOR DURING HIS ONLINE PROFILING</strong> <div> </div>

Smirnov

Ivan Zakharovich

Smirnov

Ivan Zakharovich

Medvedev

Anatoly Andreevich

Medvedev

Anatoly Andreevich

Samigulin

Timur Ruslanovich

Samigulin

Timur Ruslanovich

Komarova

Alena Alekseevna

Komarova

Alena Alekseevna

Timoshchuk-Bondar

Artyom Igorevich

Timoshchuk-Bondar

Artyom Igorevich

Sinko

Mikhail Vitalievich

Sinko

Mikhail Vitalievich

Laushkina

Anastasia Alexandrov

Laushkina

Anastasia Alexandrov

Goffman

Olga Olegovna

Goffman

Olga Olegovna

Basov

Oleg Olegovich

Basov

Oleg Olegovich

oobasov@mail.ru

2023

8400

The study of individual psychological characteristics of people is important in the areas of education, management and administration, ensuring the safety of individuals and communities. There are various tools for solving the problem of determining and analyzing personal characteristics, but they have a number of limitations. We present a solution that extracts and uses machine learning to analyze human facial and speech features from video footage, applicable to the study of eight different individual psychological characteristics in an online digital profiling task. The user is invited to use the developed Expert library to obtain new characteristics by applying and combining existing ML modules to solve a wide class of problems.

machine learningopen sourcemultimodal analysisverbal and non-verbal signs

The research was carried out with the financial support of the Russian Science Foundation, Agreement No. 22-21-00604.

Список литературы

Goupil L., Ponsot E., Richardson D. et al. Listeners’ perceptions of the certainty and honesty of a speaker are associated with a common prosodic signature // Nat Commun. 2021. – №12.

Teixeira J. P., Oliveira C., Lopes C. Vocal Acoustic Analysis – Jitter, Shimmer and HNR Parameters // Procedia Technology. – 2013. – V. 9. – P. 1112-1122.

Kirillov S., Lukyanov D. Evaluation of psycho-emotional status of robotic system operator in the Arctic // IOP Conference Series: Earth and Environmental Science. – 2019. – № 302.

Rammstedt B., Danner D., Lechner C. Personality, competencies, and life outcomes: results from the German PIAAC longitudinal study // Large-scale Assess Educ 5. – 2017. – №2.

Anbesaw T., Zenebe Y., Asmamaw A., et. al. Post-traumatic stress disorder and associated factors among people who experienced traumatic events in Dessie town, Ethiopia, 2022: A community based study // Frontiers in Psychiatry. 2022. №13.

Reeve D. Psycho-Emotional Disablism: The Missing Link? // Routledge Handbook of Disability StudiesEdition: 1stChapter: 7. 2012.

Le Duc T., Huynh S., Vu T., et. al. Personality Traits and Aggressive Behavior in Vietnamese Adolescents // Psychology Research and Behavior Management. – 2023. – №16. – Р. 1987-2003.

Cheng S., Dawson J., Thamby J., et al. How do aggression source, employee characteristics and organisational response impact the relationship between workplace aggression and work and health outcomes in healthcare employees? A cross-sectional analysis of the National Health Service staff survey in England // BMJ Open. 2020. – №10(8).

Sokolova M.S. Adaptation to the interlocutor as a component of positive communication: constitutive features // Current problems of philology and pedagogical linguistics. 2017. No. 2 (26).

Danilin M.V. Methods of teaching listening in conditions of multimodal communication using authentic audio-video materials (English, secondary general education): dis. Ph.D. ped. Sciences: 5.8.2. - M., 2021. – 173 p.

Zobkov V. A. A person’s self-confidence in decision-making situations // Bulletin of Kostroma State University. Series: Pedagogy. Psychology. Sociokinetics. – 2018. – No. 2.

Romek V. G. Self-confidence as a socio-psychological characteristic of personality: abstract. dis. Ph.D. social psycho. Sciences: 19.00.05. - Rostov-on-Don, 1997. – 12 p.

Kashapova E. R., Ryzhkova M. V. Cognitive distortions and their influence on individual behavior // Vestn. Volume. state un-ta. Economy. – 2015. – No. 2 (30).

Aneri R., Sonali J. Emotion Based Hate Speech Detection using Multimodal Learning // arXiv Computation and Language. 2022.

Jianyuan G., Kai H., Han W., et al. CMT: Convolutional Neural Networks Meet Vision Transformers // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022. – Р. 12175-12185.

EmoPy: a machine learning toolkit for emotional expression // thoughtworks URL: https://www.thoughtworks.com/ (data access: 13.07.2023).

Camillo L., Jiuqiang T., Hadon N., et al. EmoPy: a machine learning toolkit for emotional expression // arXiv Distributed, Parallel, and Cluster Computing. 2019.

TextBlob: Simplified Text Processing // TextBlob URL: https://textblob.readthedocs.io/en/dev/ (data access: 13.07.2023).

Razzaq M.A., Hussain J., Bang J., et. al. A Hybrid Multimodal Emotion Recognition Framework for UX Evaluation Using Generalized Mixture Functions. Sensors 23. 2023. №23(9).

Detoxify // github URL: https://github.com/unitaryai/detoxify (data access: 13.07.2023).

Boersma P., Van Heuven V. Speak and unSpeak with PRAAT // Glot International. – V. 5. – №9/10. – Р. 341-347.

Gedas B., Heng W., Lorenzo T. Is Space-Time Attention All You Need for Video Understanding? // arXiv Computer Vision and Pattern Recognition. 2021.

Grishchenko I., Ablavatski A., Kartynnik Y., et. al. Attention Mesh: High-fidelity Face Mesh Prediction in Real-time // arXiv Computer Vision and Pattern Recognition. 2022.

Samigulin T.R., Smirnov I.Z., Laushkina A.A. Determination of markers of aggressive human behavior based on analysis of audio and text channels // Scientific result. Information Technology. – 2022. – T.7. – No. 2. – pp. 56-61.

Jacob D., Ming-Wei C., Kenton L., Kristina T. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // arXiv preprint arXiv:1810.04805. 2019.

Wen Z., Lin W., Wang T., Xu G. Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition // Biomimetics 8. 2023. – №2. – P. 199.

Peng Z., Lu Y., Pan S., Liu Y. Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention // ICASSP 2021 – 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). – 2021. – P. 3020-3024.

Amit G., Noura Al M., Steven B. ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference // arXiv Computation and Language. 2021.

Shickel B., Scott S., Martin H., at. al. Automatic Detection and Classification of Cognitive Distortions in Mental Health Text // IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). 2019.

Xuejiao Z., Chunyan M., Zhenchang X. Identifying Cognitive Distortion by Convolutional Neural Network based Text Classification // International Journal of Information Technology. 2017. – №23.

Simms T., Ramstedt C., Rich M., et.al. Detecting Cognitive Distortions Through Machine Learning Text Analytics // 2017 IEEE International Conference on Healthcare Informatics (ICHI). – 2017. – P. 508-512.

Beck A. Cognitive therapy and the emotional disorders // New York: New American Library. 1979. – P. 374.

Breiman L. Random Forests // Machine Learning. 2001. – №45. – P. 5–32.

Ferracane E., Durrett G., Li J., et. al. Did they answer? Subjective acts and intents in conversational discourse // Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. – P. 1626–1644.