Abstract
This paper explores cooperative trajectory planning approaches within the context of human-machine shared control. In shared control research, it is typically assumed that the human and the automation use the same reference trajectory to stabilize the coupled system. However, this assumption is often incorrect, as they usually follow different trajectories, causing control conflicts at the action level that have not been widely researched. To address this, it is logical to extend shared control concepts to include human-machine interaction at the trajectory-level before action execution, resulting in a unified reference trajectory for both human and automation. This paper begins with a literature overview on approaches of cooperative trajectory planning. It then presents an approach of finding a joint trajectory by modelling cooperative trajectory planning as an agreement process. A generally valid system structure is proposed for this purpose. Finally, it proposes concepts to implement cooperative trajectory planning as an agreement process.
Zusammenfassung
In diesem Beitrag werden Ansätze zur kooperativen Trajektorienplanung im Kontext des Shared Control zwischen Mensch und Maschine untersucht. In der Shared Control Forschung wird typischerweise davon ausgegangen, dass der Mensch und die Automatisierung dieselbe Referenztrajektorie verwenden, um das gemeinsame System zu regeln. Diese Annahme ist jedoch häufig nicht korrekt, da sie in der Regel unterschiedlichen Trajektorien folgen, was zu Stellgrößenkonflikten auf der Ausführungsebene führt, die bisher noch nicht umfassend erforscht wurden. Um dieses Problem zu lösen, macht es Sinn, das Shared Control so zu erweitern, dass die Interaktion zwischen Mensch und Maschine auf der Trajektorebene vor der Ausführungsebene berücksichtigt wird, was zu einer einheitlichen Referenztrajektorie für Mensch und Automatisierung führt. Der Beitrag beginnt mit einem Literaturüberblick über Ansätze der kooperativen Trajektorienplanung. Anschließend wird ein Ansatz zur Ermittlung einer gemeinsamen Trajektorie vorgestellt, bei dem die kooperative Trajektorienplanung als ein Einigungsprozess modelliert wird. Zu diesem Zweck wird eine allgemein gültige Systemstruktur vorgeschlagen. Schließlich werden Lösungskonzepte zur Implementierung der kooperativen Trajektorienplanung als Einigungsprozess vorgeschlagen.
About the authors
Julian Schneider received the B.Sc. in 2017 and the M.Sc. in 2019 both in mechatronics and information technology from the Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany in 2019, where he is pursuing the Ph.D. degree in control engineering with the Institute of Control Systems (IRS). His research interests include cooperative trajectory planning in physically coupled human-machine system.
Balint Varga received his B.Sc. in Mechatronics from the Technical University of Budapest, Hungary, in 2016. He earned an M.Sc. in Mechanical Engineering in 2017 and a Dr.-Ing. in Electrical Engineering and Information Technology in 2023, both from the Karlsruhe Institute of Technology (KIT), Germany. From 2017 to 2020, he worked as a research scientist at the FZI Research Center for Information Technology. Since 2020, he has been a research assistant at KIT. In 2023, he became the head of the research group on Cooperative Systems. His research interests include modeling human-machine interaction and designing shared-control concepts for various applications, such as autonomous vehicles, mobile manipulators, and healthcare systems. He is a member of the IEEE SMC Society and leads the Technical Committee on Shared Control. He serves as an Associate Editor for the IEEE Transactions on Human-Machine Systems and the International Journal of Robotics and Automation.
Sören Hohmann (Member, IEEE) received the Diploma and Ph.D. degrees from the University of Karlsruhe in 1997 and 2002, respectively, after studying electrical engineering jointly at the Technische Universität Braunschweig, Brunswick, Germany; the University of Karlsruhe, Karlsruhe, Germany; and the École Nationale Supérieure d'Électricité et de Mécanique, Nancy, France. Afterwards, until 2010, he worked in the industry for BMW, Munich, Germany, where his last position was the head of the predevelopment and series development of active safety systems. He is currently the Head of the Institute of Control Systems (IRS), Karlsruhe Institute of Technology (KIT), Karlsruhe, and a Director's Board Member of the Research Center for Information Technology (FZI), Karlsruhe. His research interests are cooperative control, networked multi-energy systems, and system guarantees by design.
References
[1] J. Inga, et al.., “Human-machine symbiosis: a multivariate perspective for physically coupled human-machine systems,” Int. J. Hum. Comput. Stud., vol. 170, p. 102926, 2022, https://doi.org/10.1016/j.ijhcs.2022.102926.Search in Google Scholar
[2] M. Marcano, S. Díaz, J. Pérez, and E. Irigoyen, “A review of shared control for automated vehicles: theory and applications,” IEEE Trans. Hum.-Mach. Syst., vol. 50, no. 6, pp. 475–491, 2020. https://doi.org/10.1109/thms.2020.3017748.Search in Google Scholar
[3] G. Li, Q. Li, C. Yang, Y. Su, Z. Yuan, and X. Wu, “The classification and new trends of shared control strategies in telerobotic systems: a survey,” IEEE Trans. Haptics, vol. 16, no. 2, pp. 118–133, 2023. https://doi.org/10.1109/toh.2023.3253856.Search in Google Scholar PubMed
[4] J. Philips, et al.., “Adaptive shared control of a brain-actuated simulated wheelchair,” in 2007 IEEE 10th International Conference on Rehabilitation Robotics, 2007, pp. 408–414.10.1109/ICORR.2007.4428457Search in Google Scholar
[5] T. Carlson and Y. Demiris, “Human-wheelchair collaboration through prediction of intention and adaptive assistance,” in 2008 IEEE International Conference on Robotics and Automation, 2008, pp. 3926–3931.10.1109/ROBOT.2008.4543814Search in Google Scholar
[6] A. Kucukyilmaz and Y. Demiris, “Learning shared control by demonstration for personalized wheelchair assistance,” IEEE Trans. Haptics, vol. 11, no. 3, pp. 431–442, 2018. https://doi.org/10.1109/toh.2018.2804911.Search in Google Scholar PubMed
[7] M. Flad, J. Otten, S. Schwab, and S. Hohmann, “Steering driver assistance system: a systematic cooperative shared control design approach,” in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2014, pp. 3585–3592.10.1109/SMC.2014.6974486Search in Google Scholar
[8] L. Claussmann, M. Revilloud, D. Gruyer, and S. Glaser, “A review of motion planning for highway autonomous driving,” IEEE Trans. Intell. Transport. Syst., vol. 21, no. 5, pp. 1826–1848, 2020. https://doi.org/10.1109/tits.2019.2913998.Search in Google Scholar
[9] B. Gunay, “Car following theory with lateral discomfort,” Transp. Res. Part B Methodol., vol. 41, no. 7, pp. 722–735, 2007. https://doi.org/10.1016/j.trb.2007.02.002.Search in Google Scholar
[10] A. Benloucif, A.-T. Nguyen, C. Sentouh, and J.-C. Popieul, “Cooperative trajectory planning for haptic shared control between driver and automation in highway driving,” IEEE Trans. Ind. Electron., vol. 66, no. 12, pp. 9846–9857, 2019. https://doi.org/10.1109/tie.2019.2893864.Search in Google Scholar
[11] R. Delpiano, J. Herrera M, and J. Coeymans A, “Characteristics of lateral vehicle interaction,” Transportmetrica A: Transport Sci., vol. 11, no. 7, pp. 636–647, 2015. https://doi.org/10.1080/23249935.2015.1059377.Search in Google Scholar
[12] B. Varga, A. Shahirpour, Y. Burkhardt, S. Schwab, and S. Hohmann, “Validation of cooperative shared-control concepts for large vehicle-manipulators,” in 2020 IEEE Conference on Control Technology and Applications (CCTA), 2020, pp. 542–548.10.1109/CCTA41146.2020.9206372Search in Google Scholar
[13] B. Varga, J. Inga, and S. Hohmann, “Limited information shared control: a potential game approach,” IEEE Trans. Hum.-Mach. Syst., vol. 53, no. 2, pp. 282–292, 2023. https://doi.org/10.1109/thms.2022.3216789.Search in Google Scholar
[14] D. A. Abbink, M. Mulder, and E. R. Boer, “Haptic shared control: smoothly shifting control authority?” Cognit. Technol. Work, vol. 14, no. 1, pp. 19–28, 2012. https://doi.org/10.1007/s10111-011-0192-5.Search in Google Scholar
[15] B. Varga, A. Shahirpour, S. Schwab, and S. Hohmann, “Control of large vehicle-manipulators with human operator,” IFAC-PapersOnLine, vol. 52, no. 30, pp. 373–378, 2019. https://doi.org/10.1016/j.ifacol.2019.12.571.Search in Google Scholar
[16] D. A. Abbink, et al.., “A topology of shared control systems–finding common ground in diversity,” IEEE Trans. Hum.-Mach. Syst., vol. 48, no. 5, pp. 509–525, 2018. https://doi.org/10.1109/thms.2018.2791570.Search in Google Scholar
[17] S. Rothfuß, R. Schmidt, M. Flad, and S. Hohmann, “A concept for human-machine negotiation in advanced driving assistance systems,” in 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019, pp. 3116–3123.10.1109/SMC.2019.8914282Search in Google Scholar
[18] F. Flemisch, D. A. Abbink, M. Itoh, M.-P. Pacaux-Lemoine, and G. Weßel, “Joining the blunt and the pointy end of the spear: towards a common framework of joint action, human–machine cooperation, cooperative guidance and control, shared, traded and supervisory control,” Cognit. Technol. Work, vol. 4, no. 21, pp. 555–568, 2019. https://doi.org/10.1007/s10111-019-00576-1.Search in Google Scholar
[19] B. Corteville, E. Aertbelien, H. Bruyninckx, J. De Schutter, and H. Van Brussel, “Human-inspired robot assistant for fast point-to-point movements,” in Proceedings 2007 IEEE International Conference on Robotics and Automation, 2007, pp. 3639–3644.10.1109/ROBOT.2007.364036Search in Google Scholar
[20] M. S. Erden and T. Tomiyama, “Human-intent detection and physically interactive control of a robot without force sensors,” IEEE Trans. Robot., vol. 26, no. 2, pp. 370–382, 2010. https://doi.org/10.1109/tro.2010.2040202.Search in Google Scholar
[21] Y. Li and S. S. Ge, “Human–robot collaboration based on motion intention estimation,” IEEE/ASME Trans. Mechatron., vol. 19, no. 3, pp. 1007–1014, 2014. https://doi.org/10.1109/tmech.2013.2264533.Search in Google Scholar
[22] J. Schneider, S. Rothfuß, and S. Hohmann, “Negotiation-based cooperative planning of local trajectories,” Front. Control Eng., vol. 3, 2022, https://doi.org/10.3389/fcteg.2022.1058980.Search in Google Scholar
[23] J.-M. Hoc, “Towards a cognitive approach to human-machine cooperation in dynamic situations,” Int. J. Hum. Comput. Stud., vol. 54, no. 4, pp. 509–540, 2001. https://doi.org/10.1006/ijhc.2000.0454.Search in Google Scholar
[24] S. Gnatzig, F. Schuller, and M. Lienkamp, “Human-machine interaction as key technology for driverless driving – a trajectory-based shared autonomy control approach,” in 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012, pp. 913–918.10.1109/ROMAN.2012.6343867Search in Google Scholar
[25] R. Boink, M. M. van Paassen, M. Mulder, and D. A. Abbink, “Understanding and reducing conflicts between driver and haptic shared control,” in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2014, pp. 1510–1515.10.1109/SMC.2014.6974130Search in Google Scholar
[26] C. Huang, H. Huang, J. Zhang, P. Hang, Z. Hu, and C. Lv, “Human-machine cooperative trajectory planning and tracking for safe automated driving,” IEEE Trans. Intell. Transport. Syst., vol. 23, no. 8, pp. 12050–12063, 2022. https://doi.org/10.1109/tits.2021.3109596.Search in Google Scholar
[27] B. Jiang, X. Li, Y. Zeng, and D. Liu, “Human-machine cooperative trajectory planning for semi-autonomous driving based on the understanding of behavioral semantics,” Electronics, vol. 10, no. 8, p. 946, 2021. https://doi.org/10.3390/electronics10080946.Search in Google Scholar
[28] P. Trautman, “Assistive planning in complex, dynamic environments: a probabilistic approach,” in 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 3072–3078.10.1109/SMC.2015.534Search in Google Scholar
[29] D. P. Losey and M. K. O’Malley, “Trajectory deformations from physical human–robot interaction,” IEEE Trans. Robot., vol. 34, no. 1, pp. 126–138, 2018. https://doi.org/10.1109/tro.2017.2765335.Search in Google Scholar
[30] E. Todorov and M. I. Jordan, “Optimal feedback control as a theory of motor coordination,” Nat. Neurosci., vol. 5, no. 11, pp. 1226–1235, 2002. https://doi.org/10.1038/nn963.Search in Google Scholar PubMed
[31] S. Rothfuß, Human-Machine Cooperative Decision Making, Ph.D. thesis, Karlsruhe, Karlsruher Institut für Technologie (KIT), 2022.Search in Google Scholar
© 2024 Walter de Gruyter GmbH, Berlin/Boston
