Road surface diversion analysis while driving an autonomous vehicle

DOI: https://doi.org/10.30686/1609-9192-2024-5S-54-58

Читать на русскоя языкеD.Yu. Khudonogov, M.S. Nikitenko , S.A. Kizilov
Federal Research Center of Coal and Coal Chemistry of Siberian Branch of the Russian Academy of Sciences, Kemerovo, Russian Federation
Russian Mining Industry №5 / 2024 p.54-58

Abstract: The main purpose of the study was to solve the problem of analyzing the road surface diversion in real time mode while driving an autonomous vehicle using machine vision combined with light line projection. The objects of the study were the machine vision video images and the subjects were the parameters and its processing algorithms. The results of assessing the deviation of the road surface from the horizontal level are shown based on processing of video images of the specific segments of a scale open-pit road model in laboratory conditions. For each specific segment, such as the beginnings and endings of longitudinal and transverse slopes, specific conditions by which the machine vision system processes changes in the geometry of the light line projections. The values of the perimeter and geometric shapes of the light line projection, as well as the values of their curvature and the degree of their pattern matching were used as the main parameters. The Shi-Tomasi angle detection algorithm was used to calculate the perimeter. A block diagram and a description of the software module functions are presented for analyzing the road surface diversion. The obtained results in the form of software and the underlying algorithms can be used to solve industrial problems concerned with controlling autonomous vehicles.

Keywords: autonomous vehicle, control system, machine vision, light line projection, image recognition, control algorithm, software module, road surface diversion, Shi-Tomasi

Acknowledgments: The study was carried out with the financial support of the Ministry of Science and Higher Education of the Russian Federation under the event “Development of a control system for autonomous vehicles based on the projected path of movement” (Agreement No. 075-15-2022-1199 dated September 28, 2022), which is conducted as part of the comprehensive scientific and technical program of a complete innovative cycle "Development and implementation of complex technologies in the fields of exploration and extraction of minerals, ensuring of industrial safety, bioremediation, creation of new products of deep processing of coal raw materials with consecutive amelioration of ecological impact on the environment and risks to human life", approved by the Decree of the Government of the Russian Federation No. 1144-r as of 11.05.2022.

For citation: Khudonogov D.Yu., Nikitenko M.S., Kizilov S.A. Road surface diversion analysis while driving an autonomous vehicle. Russian Mining Industry. 2024;(5S):54–58. (In Russ.) https://doi.org/10.30686/1609-9192-2024-5S-54-58

Article info

Received: 04.08.2024

Revised: 02.10.2024

Accepted: 08.10.2024

Information about the authors

Danila Yu. Khudonogov – Scientific Researcher, Federal Research Center of Coal and Coal-Chemistry of Siberian Branch of the Russian Academy of Sciences, Kemerovo, Russian Federation; e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Mikhail S. Nikitenko – Cand. Sci. (Eng.), Head of Laboratory, Federal Research Center of Coal and Coal-Chemistry of Siberian Branch of the Russian Academy of Sciences, Kemerovo, Russian Federation; https://orcid.org/0000-0001-8752-1332; e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Sergey A. Kizilov – Cand. Sci. (Eng.), Scientific Researcher, Federal Research Center of Coal and Coal-Chemistry of Siberian Branch of the Russian Academy of Sciences, Kemerovo, Russian Federation; https://orcid.org/0000-0003-2554-1383; e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

References

1. Никитенко М.С., Кизилов С.А., Худоногов Д.Ю. Анализ подходов к управлению автономными транспортными средствами. Современные наукоемкие технологии. 2022;(12-2):278–283. https://doi.org/10.17513/snt.39472 Nikitenko M.S., Kizilov S.A., Khudonogov D.Yu. Analysis of approaches to autonomous vehicle control. Modern High Technologies. 2022;(12-2):278–283. (In Russ.) https://doi.org/10.17513/snt.39472

2. Турлаев Р.С. Перспективы развития беспилотной логистики в автомобильных грузоперевозках. Вестник Алтайской академии экономики и права. 2019;(11-2):187–192. https://doi.org/10.17513/vaael.839 Turlaev R.S. Prospects for the development of unmanned logistics in road cargo transport. Vestnik Altaiskoi Akademii Ekonomiki i Prava. 2019;(11-2):187–192. (In Russ.) https://doi.org/10.17513/vaael.839

3. Xiaoyan Y., Marin M. A study on recent developments and issues with obstacle detection systems for automated vehicles. Sustainability. 2020;12(8):3281. https://doi.org/10.3390/su12083281

4. Biletska O., Kurtz G.L., Zadek H. Operation control center for automated vehicles: Conceptual design. Proceedings of the Hamburg International Conference of Logistics. 2022;33:731–752. https://doi.org/10.15480/882.4702

5. Журавлев А.Г., Черных В.В. Компьютерное моделирование при оценке влияния дорожных факторов на производительность карьерного автотранспорта. Известия Томского политехнического университета. Инжиниринг георесурсов. 2023;334(12):20–31. https://doi.org/10.18799/24131830/2023/12/4308 Zhuravlev A.G., Chernykh V.V. Computer modeling when assessing the impact of road factors on performance of quarry vehicles. Bulletin of the Tomsk Polytechnic University. Geo Assets Engineering. 2023;334(12):20–31. (In Russ.) https://doi.org/10.18799/24131830/2023/12/4308

6. Журавлев А.Г., Черепанов В.А., Карпов В.А., Невежин А.Ю. Рекомендации по эксплуатации и улучшению качества содержания карьерных автодорог Олимпиадинского и Благодатного горно-обогатительных комбинатов. Горная промышленность. 2023;(5):88–95. https://doi.org/10.30686/1609-9192-2023-5-88-95 Zhuravlev A.G., Cherepanov V.A., Karpov V.A., Nevezhin A.Yu. Recommendations on operation and improvement in the quality of in-pit road maintenance at the Olympiadinsky and Blagodatny mining and processing plants. Russian Mining Industry. 2023;(5):88–95. (In Russ.) https://doi.org/10.30686/1609-9192-2023-5-88-95

7. Клебанов А.Ф. Автоматизация и роботизация открытых горных работ: опыт цифровой трансформации. Горная промышленность. 2020;(1):8–11. Режим доступа: https://mining-media.ru/ru/article/ogr/15630-avtomatizatsiya-irobotizatsiya-otkrytykh-gornykh-rabot-opyt-tsifrovoj-transformatsii (дата обращения: 20.08.2024). Klebanov A.F. Automation and robotization in opencast mining: experience in digital transformation. Russian Mining Industry. 2020;(1):8–11. (In Russ.) Available at: https://mining-media.ru/ru/article/ogr/15630-avtomatizatsiya-i-robotizatsiyaotkrytykh-gornykh-rabot-opyt-tsifrovoj-transformatsii (accessed: 20.08.2024).

8. Hossain I., Al Saif M.S., Biswas R.I., Mia S., Ahmed A., Zishan S.R. Design and development of road surface condition monitoring system. In: Hossain M.S., Majumder S.P., Siddique N., Hossain M.S. (eds) The Fourth Industrial Revolution and Beyond. Lecture Notes in Electrical Engineering, vol. 980. Springer Singapore; 2023, pp. 413–425. https://doi.org/10.1007/978-981-19-8032-9_29

9. Крылов М.А., Курганов К.И., Чащин Е.А. Аппаратно-программный комплекс мониторинга, моделирования и прогнозирования состояния и износа дорожного покрытия. Транспортные сооружения. 2018;5(1):04SATS118. https://doi.org/10.15862/04SATS118 Krylov M.A., Kurganov K.I., Chaschin Y.A. Monitoring and modeling hardware-software complex for forecasting the condition and deterioration of the road surface. Russian Journal of Transport Engineering. 2018;5(1):04SATS118. (In Russ.) https://doi.org/10.15862/04SATS118

10. Смирнов К.Д. Разработка системы обнаружения препятствий и формирования траектории отклонения на базе сигнала изображения фронтальной видеокамеры БПЛА. REDS: Телекоммуникационные устройства и системы. 2020;10(2):40–48. Smirnov K.D. Obstacle detections system and diversion route development based on drone front camera images. REDS: Telekommunikatsionnye Ustroistva i Sistemy. 2020;10(2):40–48. (In Russ.)

11. Nikitenko M.S., Khudonogov D.Yu., Popinako Ya.V., Kizilov S.A. Determining the route and roadway condition in front of autonomous vehicle. Proceedings of the Third International Conference on Digital Technologies, Optics, and Materials Science (DTIEE 2024). Vol. 13217. 2024, 1321713. https://doi.org/10.1117/12.3036935

12. Попинако Я.В., Ефременкова М.В., Никитенко М.С., Худоногов Д.Ю. Обоснование параметров масштабного материального макета автодороги для отладки и тестирования автономного роботизированного шасси. Горное оборудование и электромеханика. 2023;(6):41–49. https://doi.org/10.26730/1816-4528-2023-6-41-49 Popinako Y.V., Efremenkova M.V., Nikitenko M.S., Khudonogov D.Yu. An opencast automobile road model parameters determining for testing and tunning automated robotic chassis. Mining Equipment and Electromechanics. 2023;(6):41–49. (In Russ.) https://doi.org/10.26730/1816-4528-2023-6-41-49

13. Burghardt T.E., Popp R., Helmreich B., Reiter T., Bohm G., Pitterle G., Artmann M. Visibility of various road markings for machine vision. Case Studies in Construction Materials. 2021;15:e00579. https://doi.org/10.1016/j.cscm.2021.e00579

14. Relf C. Image acquisition and processing with LabVIEW. CRC Press; 2004. 201 p.

15. Klinger T. Image Processing with LabVIEW and IMAQ Vision. Prentice Hall PTR; 2020. 319 p.