Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-27T05:01:32.739Z Has data issue: false hasContentIssue false
  • English
  • Français

CONTROL OF A DRONE WITH BODY GESTURES

Published online by Cambridge University Press:  27 July 2021

Nicolas Gio ,
Ross Brisco  and
Tijana Vuletic
Nicolas Gio*
Affiliation:
DMEM, University of Strathclyde Arts et Métiers ParisTech
Ross Brisco
Affiliation:
DMEM, University of Strathclyde
Tijana Vuletic
Affiliation:
DMEM, University of Strathclyde
*
Gio, Nicolas Clément, University of Strathclyde, DMEM, France, [email protected]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Drones are becoming more popular within military applications and civil aviation by hobbyists and business. Achieving a natural Human-Drone Interaction (HDI) would enable unskilled drone pilots to take part in the flying of these devices and more generally easy the use of drones. The research within this paper focuses on the design and development of a Natural User Interface (NUI) allowing a user to pilot a drone with body gestures. A Microsoft Kinect was used to capture the user’s body information which was processed by a motion recognition algorithm and converted into commands for the drone. The implementation of a Graphical User Interface (GUI) gives feedback to the user. Visual feedback from the drone’s onboard camera is provided on a screen and an interactive menu controlled by body gestures and allowing the choice of functionalities such as photo and video capture or take-off and landing has been implemented. This research resulted in an efficient and functional system, more instinctive, natural, immersive and fun than piloting using a physical controller, including innovative aspects such as the implementation of additional functionalities to the drone's piloting and control of the flight speed.

Keywords

Human-Drone InteractionNatural User InterfaceDesign engineeringDesign for interfacesUser centred design
Type
Article
Information
Proceedings of the Design Society , Volume 1: ICED21 , August 2021 , pp. 761 - 770
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2021. Published by Cambridge University Press

References

Abtahi, P., Y. Zhao, D., L.E, J. & Landay, J. A., (2017). “Drone Near Me: Exploring Touch-Based Human-Drone Interaction”, The Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, September, No 34, pp. 8.Google Scholar
Al-Shamayleh, A.S., Ahmad, R., Abushariah, M.A., Alam, K.A., Jomhari, N., (2018). “A systematic literature review on vision based gesture recognition techniques” in Multimedia Tools and Applications 77.10.1007/s11042-018-5971-zCrossRefGoogle Scholar
Andujar, M. & Tezza, D., (2019). “The State-of-the-Art of Human–Drone Interaction: A Survey,” in IEEE Access, vol. 7, pp. 167438–167454, 2019, doi: https://doi.org/10.1109/ACCESS.2019.2953900.Google Scholar
Ars Electronica, (2020). Humphrey II. [online], Available at: https://ars.electronica.art/futurelab/project/humphrey-ii/ (12 04 2020).Google Scholar
Baudouin-Lafon, T. B. a. M., (1993). “Remote Control of Objects Using Free-Hand Gestures”, Charade, 36(7), pp. 28–35.Google Scholar
Cauchard, J. R., E, Zhai, J. L., K. Y. & Landay, J. A., (2015). “Drone & me: an exploration into natural human-drone interaction”, Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing.10.1145/2750858.2805823CrossRefGoogle Scholar
Cherpillod, A., Mintchev, S. & Floreano, D., (2016). “Embodied Flight with a Drone”, [online], Available at: https://arxiv.org/ftp/arxiv/papers/1707/1707.01788.pdf (10/11/2020)Google Scholar
Dante Tezz, M. A., (2019). “The State-of-the-Art of Human–Drone Interaction: A Survey”, IEEE Access.10.1109/ACCESS.2019.2953900CrossRefGoogle Scholar
DJI, (2020), “Tello”. [online], Available at: https://store.dji.com/fr/shop/tello-series (15/11/2020).Google Scholar
Suarez-Fernandez, R., Sanchez-Lopez, J.L., Sampedro, C., Bavle, H., Molina, M., and Campoy, P. (2016), “Natural user interfaces for human-drone multi-modal interaction”, June 2016, International Conference on Unmanned Aircraft Systems (ICUAS), pp. 0.10.1109/ICUAS.2016.7502665CrossRefGoogle Scholar
Goodrich, M. A., & Schultz, A. C. (2008). Human-robot interaction: a survey. Now Publishers Inc.Google Scholar
Hanker-lu, (2019). Tello_Video Python Library. [online] Available at: https://github.com/dji-sdk/Tello-Python/tree/master/Tello_Video (22/07/2020).Google Scholar
Hasanuzzaman, M., Ampornaramveth, V., Zhang, T., Bhuiyan, M. A., Shirai, Y., & Ueno, H. (2004). “Real-time vision-based gesture recognition for human robot interaction”, IEEE International Conference on Robotics and Biomimetics, IEEE, pp. 413–418.Google Scholar
Mashood, A., Noura, H., Jawhar, I., & Mohamed, N. (2015). “A gesture based kinect for quadrotor control”, International Conference on Information and Communication Technology Research (ICTRC) (pp. 298–301). IEEE.Google Scholar
Microsoft, (2012). “Kinect Sensor” [online], Available at: https://docs.microsoft.com/en-us/previous-versions/microsoft-robotics/hh438998(v=msdn.10)?redirectedfrom=MSDN (10/11/2020).Google Scholar
Natarajan, K., Nguyen, T. H. D., & Mete, M. (2018). “Hand gesture controlled drones: An open source library”, 1st International Conference on Data Intelligence and Security (ICDIS), IEEE, pp. (168–175).Google Scholar
Pfeil, K., Koh, S. L., & LaViola, J. (2013). “Exploring 3d gesture metaphors for interaction with unmanned aerial vehicles”, International conference on Intelligent user interfaces, pp. (257–266).Google Scholar
Pisharady, P.K., Saerbeck, M., (2015). “Recent methods and databases in vision-based hand gesture recognition: A review”. Computer Vision and Image Understanding 141, 152–165.10.1016/j.cviu.2015.08.004CrossRefGoogle Scholar
Foundation, Python Software, (2020). “Drawnow library”. [online], Available at: https://pypi.org/project/drawnow/ (23/07/2020).Google Scholar
Rautaray, S.S., Agrawal, A., (2015). “Vision based hand gesture recognition for human computer interaction: a survey”, Artificial intelligence review 43, 1–54.10.1007/s10462-012-9356-9CrossRefGoogle Scholar
Rognon, C., Mintchev, S., Dell'Agnola, F., Cherpillod, A., Atienza, D., & Floreano, D. (2018). “Flyjacket: An upper body soft exoskeleton for immersive drone control”, IEEE Robotics and Automation Letters, 3(3).10.1109/LRA.2018.2810955CrossRefGoogle Scholar
Sanna, A., Lamberti, F., Paravati, G., & Manuri, F. (2013). “A Kinect-based natural interface for quadrotor control”, Entertainment Computing, 4(3), 179–186.10.1016/j.entcom.2013.01.001CrossRefGoogle Scholar
Sarkar, A., Patel, K. A., Ram, R. G., & Capoor, G. K. (2016). Gesture control of drone using a motion controller. International Conference on Industrial Informatics and Computer Systems (CIICS), IEEE, pp. (1–5).10.1109/ICCSII.2016.7462401CrossRefGoogle Scholar
Shotton, J., Fitzgibbon, A. & Sharp, T., (2010). “Real-Time Human Pose Recognition in Parts from Single Depth Images”, IEEE, CVPR 2011, pp. (1297–1304).Google Scholar
Stoica, A., Salvioli, F., & Flowers, C. (2014), “Remote control of quadrotor teams, using hand gestures”, Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pp. (296–297).10.1145/2559636.2559853CrossRefGoogle Scholar
Stone, Z., (2017). “Review: The Hand-Controlled Aura Drone”. [online], Available at: https://www.forbes.com/sites/zarastone/2017/11/20/review-the-hand-controlled-aura-drone-will-make-you-jealous-of-your-kids/#5fb9dfb85041 (03/05/2020).Google Scholar
Sun, T., Nie, S., Yeung, D. Y., & Shen, S. (2017), “Gesture-based piloting of an aerial robot using monocular vision”, IEEE International Conference on Robotics and Automation (ICRA), pp. (5913–5920).10.1109/ICRA.2017.7989696CrossRefGoogle Scholar
Toumi, T., & Zidani, A. (2014). “From human-computer interaction to human-robot social interaction”, arXiv.10.1109/ICBR.2013.6729281CrossRefGoogle Scholar
Ultraleap, (2019). “Leap Motion Controller”. [online], Available at: https://www.ultraleap.com/datasheets/Leap_Motion_Controller_Datasheet.pdf, (02/05/2020).Google Scholar