<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<article article-type="research-article" dtd-version="1.2" xml:lang="ru" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><front><journal-meta><journal-id journal-id-type="issn">2518-1092</journal-id><journal-title-group><journal-title>Научный результат. Информационные технологии</journal-title></journal-title-group><issn pub-type="epub">2518-1092</issn></journal-meta><article-meta><article-id pub-id-type="doi">10.18413/2518-1092-2024-9-3-0-7</article-id><article-id pub-id-type="publisher-id">3561</article-id><article-categories><subj-group subj-group-type="heading"><subject>ИСКУССТВЕННЫЙ ИНТЕЛЛЕКТ И ПРИНЯТИЕ РЕШЕНИЙ</subject></subj-group></article-categories><title-group><article-title>&lt;strong&gt;МЕТОД ОБУЧЕНИЯ ИНТЕЛЛЕКТУАЛЬНОГО АГЕНТА С ПОМОЩЬЮ СЕТЕЙ DOUBLE DQN, ПУТЕВЫХ ТОЧЕК И ФУНКЦИИ ВОЗНАГРАЖДЕНИЯ&lt;/strong&gt;</article-title><trans-title-group xml:lang="en"><trans-title>&lt;strong&gt;A METHOD FOR TRAINING AN INTELLIGENT AGENT USING DOUBLE DQN NETWORKS, WAYPOINTS AND REWARD FUNCTION&lt;/strong&gt;</trans-title></trans-title-group></title-group><contrib-group><contrib contrib-type="author"><name-alternatives><name xml:lang="ru"><surname>Тихонов</surname><given-names>Максим Константинович</given-names></name><name xml:lang="en"><surname>Tikhonov</surname><given-names>Maksim Konstantinovich</given-names></name></name-alternatives><email>samualgame@gmail.com</email></contrib><contrib contrib-type="author"><name-alternatives><name xml:lang="ru"><surname>Непомнящий</surname><given-names>Дмитрий Олегович</given-names></name><name xml:lang="en"><surname>Nepomnyashchiy</surname><given-names>Dmitry Olegovich</given-names></name></name-alternatives></contrib><contrib contrib-type="author"><name-alternatives><name xml:lang="ru"><surname>Хайдукова</surname><given-names>Валерия Николаевна</given-names></name><name xml:lang="en"><surname>Khaidukova</surname><given-names>Valeria Nikolaevna</given-names></name></name-alternatives></contrib></contrib-group><pub-date pub-type="epub"><year>2024</year></pub-date><volume>9</volume><issue>3</issue><fpage>0</fpage><lpage>0</lpage><self-uri content-type="pdf" xlink:href="/media/information/2024/3/НР_ИТ_9_3_7.pdf" /><abstract xml:lang="ru"><p>Рассмотрены задачи повышения эффективности управления автономных транспортных средств. Выделена проблема снижения требуемых вычислительных ресурсов для интеллектуального модуля управления автомобилем. Предложен алгоритм обучения нейронной сети для архитектуры Double DQN с модифицированной функций вознаграждения. Основой предлагаемого решения является использование сегментации полосы движения, функции вознаграждения и использования дополнительных путевых точек при обучении. Разработана программная модель и выполнено моделирование процесса обучения. Полученные результаты сравнительного анализа с известными решениями показывают стабильное повышение длительности эпизода, и эффективное обучение в реалистичной городской симуляции. Исследование указывает на возможность уменьшения необходимости в высокой вычислительной мощности, что даст возможность использовать центральные процессоры (CPU) для основных функций беспилотных автомобилей вместо графических процессоров (GPU).</p></abstract><trans-abstract xml:lang="en"><p>The problems of increasing the control efficiency of autonomous vehicles are considered. The problem of reducing the required computational resources for the intelligent vehicle control module is highlighted. A neural network training algorithm for Double DQN architecture with modified reward functions is proposed. The basis of the proposed solution is the use of lane segmentation, reward function and the use of additional waypoints in training. A software model has been developed and simulation of the learning process has been performed. The results obtained from a comparative analysis with known solutions show a stable increase in episode duration, and effective training in a realistic urban simulation. The study points to the possibility of reducing the need for high computing power, which will enable the use of central processing units (CPUs) for basic functions of unmanned vehicles instead of graphics processing units (GPUs).</p></trans-abstract><kwd-group xml:lang="ru"><kwd>беспилотный автомобиль</kwd><kwd>полоса движения</kwd><kwd>интеллектуальное управление</kwd><kwd>агент</kwd><kwd>нейронная сеть</kwd><kwd>глубокое Q-обучение</kwd><kwd>симуляция</kwd></kwd-group><kwd-group xml:lang="en"><kwd>self-driving car</kwd><kwd>lane following</kwd><kwd>intelligent control</kwd><kwd>agent</kwd><kwd>neural network</kwd><kwd>deep Q-learning</kwd><kwd>simulation</kwd></kwd-group></article-meta></front><back><ref-list><title>Список литературы</title><ref id="B1"><mixed-citation>Final Rule Occupant Protection Amendment Automated Vehicles [Электронный ресурс]. URL: https://www.nhtsa.gov/document/final-rule-occupant-protection-amendment-automated-vehicles (дата обращения 10.09.2023).</mixed-citation></ref><ref id="B2"><mixed-citation>RAND Corporation [Электронный ресурс]. URL: https://www.rand.org/ (дата обращения 23.09.2023).</mixed-citation></ref><ref id="B3"><mixed-citation>Malayjerdi M. et al. Virtual simulations environment development for autonomous vehicles interaction // ASME International Mechanical Engineering Congress and Exposition. &amp;ndash; American Society of Mechanical Engineers, 2020. &amp;ndash; Т. 84492. &amp;ndash; С. V02BT02A009.</mixed-citation></ref><ref id="B4"><mixed-citation>Young R. A. Automated Driving System Safety: Miles for 95% Confidence in &amp;ldquo;Vision Zero&amp;rdquo; // SAE International Journal of Advances and Current Practices in Mobility. &amp;ndash; 2020. &amp;ndash; Т. 2. &amp;ndash; №. 2020-01-1205. &amp;ndash; С. 3454-3480.</mixed-citation></ref><ref id="B5"><mixed-citation>Jo K. et al. Development of autonomous car&amp;mdash;Part II: A case study on the implementation of an autonomous driving system based on distributed architecture // IEEE Transactions on Industrial Electronics. &amp;ndash; 2015.&amp;nbsp;&amp;ndash; Т. 62. &amp;ndash; №. 8. &amp;ndash; С. 5119-5132.</mixed-citation></ref><ref id="B6"><mixed-citation>Rodrigues P., Vieira S. Optimizing agent training with deep q-learning on a self-driving reinforcement learning environment //2020 IEEE Symposium Series on Computational Intelligence (SSCI). &amp;ndash; IEEE, 2020. &amp;ndash; С.&amp;nbsp;745-752.</mixed-citation></ref><ref id="B7"><mixed-citation>Chishti S. O. A. et al. Self-driving cars using CNN and Q-learning // 2018 IEEE 21st International Multi-Topic Conference (INMIC). &amp;ndash; IEEE, 2018. &amp;ndash; С. 1-7.</mixed-citation></ref><ref id="B8"><mixed-citation>Chhikara P. et al. Data dimensionality reduction techniques for Industry 4.0: Research results, challenges, and future research directions //Software: Practice and Experience. &amp;ndash; 2022. &amp;ndash; Т. 52. &amp;ndash; №. 3. &amp;ndash; С. 658-688.</mixed-citation></ref><ref id="B9"><mixed-citation>Malekloo A. et al. Machine learning and structural health monitoring overview with emerging technology and high-dimensional data source highlights //Structural Health Monitoring. &amp;ndash; 2022. &amp;ndash; Т. 21. &amp;ndash; №. 4. &amp;ndash; С. 1906-1955.</mixed-citation></ref><ref id="B10"><mixed-citation>Peng B. et al. End-to-end autonomous driving through dueling double deep Q-network // Automotive Innovation. &amp;ndash; 2021. &amp;ndash; Т. 4. &amp;ndash; С. 328-337.</mixed-citation></ref><ref id="B11"><mixed-citation>Riboni A., Candelieri A., Borrotti M. Deep Autonomous Agents Comparison for Self-driving Cars // International Conference on Machine Learning, Optimization, and Data Science. &amp;ndash; Cham: Springer International Publishing, 2021. &amp;ndash; С. 201-213.</mixed-citation></ref><ref id="B12"><mixed-citation>Kiran B. R. et al. Deep reinforcement learning for autonomous driving: A survey // IEEE Transactions on Intelligent Transportation Systems. &amp;ndash; 2021. &amp;ndash; Т. 23. &amp;ndash; №. 6. &amp;ndash; С. 4909-4926.</mixed-citation></ref><ref id="B13"><mixed-citation>Lin Q. et al. Reachflow: An online safety assurance framework for waypoint-following of self-driving cars // 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). &amp;ndash; IEEE, 2020. &amp;ndash; С. 6627-6632.</mixed-citation></ref><ref id="B14"><mixed-citation>Szoke L. et al. Skills to drive: Successor features for autonomous highway pilot // IEEE Transactions on Intelligent Transportation Systems. &amp;ndash; 2022. &amp;ndash; Т. 23. &amp;ndash; №. 10. &amp;ndash; С. 18707-18718.</mixed-citation></ref><ref id="B15"><mixed-citation>Dosovitskiy A. et al. CARLA: An open urban driving simulator // Conference on robot learning. &amp;ndash; PMLR, 2017. &amp;ndash; С. 1-16.</mixed-citation></ref><ref id="B16"><mixed-citation>Barbour E., McFall K. Autonomous vehicle simulation using open source software Carla // Journal of UAB ECTC. &amp;ndash; 2019. &amp;ndash; Т. 18.</mixed-citation></ref><ref id="B17"><mixed-citation>GitHub [Электронный ресурс]. &amp;ndash; Режим доступа: URL: https://github.com/rzambrano1/rl-agent-self-driving-carla (дата обращения 22.02.2024).</mixed-citation></ref></ref-list></back></article>