Hostname: page-component-586b7cd67f-2plfb Total loading time: 0 Render date: 2024-11-24T10:32:37.387Z Has data issue: false hasContentIssue false
  • English
  • Français

Ai Motion Control – A Generic Approach to Develop Control Policies for Robotic Manipulation Tasks

Part of: Systems Engineering

Published online by Cambridge University Press:  26 July 2019

Philip Kurrek ,
Mark Jocas ,
Firas Zoghlami ,
Martin Stoelen  and
Vahid Salehi
Philip Kurrek*
Affiliation:
University of Applied Sciences Munich;
Mark Jocas
Affiliation:
University of Applied Sciences Munich;
Firas Zoghlami
Affiliation:
University of Applied Sciences Munich;
Martin Stoelen
Affiliation:
University of Plymouth
Vahid Salehi
Affiliation:
University of Applied Sciences Munich;
*
Contact: Kurrek, Philip, Munich University of Applied Sciences, Department of Applied Sciences and Mechatronics, Germany, [email protected]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Current robotic solutions are able to manage specialized tasks, but they cannot perform intelligent actions which are based on experience. Autonomous robots that are able to succeed in complex environments like production plants need the ability to customize their capabilities. With the usage of artificial intelligence (AI) it is possible to train robot control policies without explicitly programming how to achieve desired goals. We introduce AI Motion Control (AIMC) a generic approach to develop control policies for diverse robots, environments and manipulation tasks. For safety reasons, but also to save investments and development time, motion control policies can first be trained in simulation and then transferred to real applications. This work uses the descriptive study I according to Blessing and Chakrabarti and is about the identification of this research gap. We combine latest motion control and reinforcement learning results and show the potential of AIMC for robotic technologies with industrial use cases.

Keywords

Design methodsArtificial intelligenceMotion controlRoboticsMachine learning
Type
Article
Information
Proceedings of the Design Society: International Conference on Engineering Design , Volume 1 , Issue 1 , July 2019 , pp. 3561 - 3570
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s) 2019

References

Arulkumaran, K., Deisenroth, M.P, Brundage, M. and Bharath, A.A. (2017), “Deep reinforcement learning: A brief survey”, IEEE Signal Processing Magazine, Vol. 34 No. 6, pp. 26–38. https://doi.org/10.1109/msp.2017.2743240Google Scholar
Bai, T., Yang, J., Chen, J., Guo, X., Huang, X. and Yao, Y. (2017), “Double-task deep q-learning with multiple views”, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 1050–1058. https://doi.org/10.1109/iccvw.2017.128.Google Scholar
Beer, M. (2018), “Artificial Intelligence, Robotics and ’Autonomous’ Systems”, European Group on Ethics in Science and New Technologies. https://doi.org/10.2777/786515Google Scholar
Blessing, L. and Chakrabarti, A. (2009), In Design Research Methodology, Springer-Verlag London, p. 39. https://doi.org/10.1007/978-1-84882-587-1Google Scholar
Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., Levine, S. and Vanhoucke, V. (2017), “Using simulation and domain adaptation to improve efficiency of deep robotic grasping”, CoRR, abs/1709.07857. https://doi.org/10.1109/icra.2018.8460875Google Scholar
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J. and Zaremba, W. (2016), “Openai gym”, CoRR, abs/1606.01540.Google Scholar
Colledanchise, M., Marzinotto, A. and Ögren, P. (2014), “Performance analysis of stochastic behavior trees”, 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3265–3272. https://doi.org/10.1109/icra.2014.6907328.Google Scholar
Clavera, I., Held, D. and Abbeel, P. (2017), “Policy transfer via modularity and reward guiding”, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1537–1544. https://doi.org/10.1109/iros.2017.8205959.Google Scholar
Ertel, W., Schneider, M., Cubek, R. and Tokic, M. (2009), “The teaching-box: A universal robot learning framework”, 2009 International Conference on Advanced Robotics, pp. 1–6.Google Scholar
European Group on Ethics in Science and New Technologies (2014), “Toward a Framework for Levels of Robot Autonomy in Human-Robot Interaction”, Journal of Human-Robot Interaction, Vol. 3 No. 2. https://doi.org/10.5898/jhri.3.2.beer.Google Scholar
Ghadirzadeh, A., Maki, A., Kragic, D. and Björkman, M. (2017), “Deep predictive policy training using reinforcement learning”, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2351–2358. https://doi.org/10.1109/iros.2017.8206046.Google Scholar
Goertzel, B. and Yu, G. (2014), “From here to agi: A roadmap to the realization of human-level artificial general intelligence”, 2014 International Joint Conference on Neural Networks (IJCNN), pp. 1525–1533. https://doi.org/10.1109/ijcnn.2014.6889801.Google Scholar
Gu, S., Holly, E., Lillicrap, T. and Levine, S. (2017), “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates”, 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389–3396. https://doi.org/10.1109/icra.2017.7989385.Google Scholar
Guerin, K.R., Lea, C., Paxton, C. and Hager, G.D. (2015), “A framework for end-user instruction of a robot assistant for manufacturing”, 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 6167–6174. https://doi.org/10.1109/icra.2015.7140065.Google Scholar
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Sendonaris, A., Dulac-Arnold, G., Osband, I., Agapiou, J., Leibo, J.Z. and Gruslys, A. (2017), “Learning from demonstrations for real world reinforcement learning”, CoRR, abs/1704.03732.Google Scholar
Hersch, M., Guenter, F., Calinon, S. and Billard, A. (2008), “Dynamical system modulation for robot learning via kinesthetic demonstrations”, IEEE Transactions on Robotics, Vol. 24 No. 6, pp. 1463–1467. https://doi.org/10.1109/tro.2008.2006703Google Scholar
International Federation of Robotics. (2018), “Executive Summary World Robotics 2018 Industrial Robots”. World Robotics 2018, pp. 13–22.Google Scholar
Johannsmeier, L. and Haddadin, S. (2017), “A hierarchical human-robot interaction-planning framework for task allocation in collaborative industrial assembly processes”, IEEE Robotics and Automation Letters, Vol. 2 No. 1, pp. 41–48. https://doi.org/10.1109/lra.2016.2535907.Google Scholar
Kemp, C.C, Edsinger, A. and Torres-Jara, E. (2007), “Challenges for robot manipulation in human environments [grand challenges of robotics]”, IEEE Robotics Automation Magazine, Vol. 14 No. 1, pp. 20–29. https://doi.org/10.1109/mra.2007.339604.Google Scholar
Kober, J. and Peters, J. (2012), “Reinforcement Learning in Robotics: A Survey”, pp. 579–610. Springer Berlin Heidelberg, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27645-318.Google Scholar
Kok, Z.K.J., Causo, A., Chong, Z.H. and Chen, I.M. (2017), “Designing modular robotic architecture for e-commerce bin picking task fulfillment”, TENCON 2017 - 2017 IEEE Region 10 Conference, pp. 1109–1114. https://doi.org/10.1109/tencon.2017.8228023.Google Scholar
Levine, S., Finn, C., Darrell, T. and Abbeel, P. (2015), “End-to-end training of deep visuomotor policies”, CoRR, abs/1504.00702.Google Scholar
Levine, S., Wagener, N. and Abbeel, P. (2015), “Learning contact-rich manipulation skills with guided policy search”, CoRR, abs/1501.05611. https://doi.org/10.1109/icra.2015.7138994Google Scholar
Li, Q., Haschke, R. and Ritter, H. (2015), “A visuo-tactile control framework for manipulation and exploration of unknown objects”, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 610–615. https://doi.org/10.1109/humanoids.2015.7363434.Google Scholar
Mar, T., Tikhanoff, V., Metta, G. and Natale, L. (2015), “Multi-model approach based on 3d functional features for tool affordance learning in robotics”, In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 482–489. https://doi.org/10.1109/humanoids.2015.7363593.Google Scholar
Marzinotto, A., Colledanchise, M., Smith, C. and Ögren, P. (2014), “Towards a unified behavior trees framework for robot control”, 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5420–5427. https://doi.org/10.1109/icra.2014.6907656.Google Scholar
Nguyen-Tuong, D. and Peters, J. (2011), “Model learning for robot control: a survey”, Cognitive processing, Vol. 12, pp. 319–340. https://doi.org/10.1007/s10339-011-0404-1Google Scholar
Peng, X.B., Andrychowicz, M., Zaremba, W. and Abbeel, P. (2017), “Sim-to-real transfer of robotic control with dynamics randomization”, CoRR, abs/1710.06537. https://doi.org/10.1109/icra.2018.8460528Google Scholar
Pfeiffer, S. (2016), “Robots, industry 4.0 and humans, or why assembly work is more than routine work”, Societies, Vol. 6 No. 2. https://doi.org/10.3390/soc6020016Google Scholar
Plappert, M., Andrychowicz, M., Ray, A., McGrew, B., Baker, B., Powell, G., Schneider, J.T, Chociej, M., Welinder, P., Kumar, V. and Zaremba, W. (2018), “Multi-goal reinforcement learning: Challenging robotics environments and request for research”, CoRR, abs/1802.09464.Google Scholar
Quigley, M., Gerkey, B.P., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R. and Ng, Andrew. (2009), “ROS: an open-source robot operating system”.Google Scholar
Ridge, B., Gaspar, T. and Ude, A. (2017), “Rapid state machine assembly for modular robot control using meta-scripting, templating and code generation”, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 661–668. https://doi.org/10.1109/humanoids.2017.8246943.Google Scholar
Russell, S. and Norvig, P. Artificial Intelligence - A Modern Approach, Pearson Higher Education, New Jersey.Google Scholar
Rusu, A.A, Vecerik, M., Rothörl, T., Heess, N., Pascanu, R. and Hadsell, R. (2016), “Sim-to-real robot learning from pixels with progressive nets”, CoRR, abs/1610.04286.Google Scholar
Sasaki, K., Noda, K. and Ogata, T. (2016), “Visual motor integration of robot's drawing behavior using recurrent neural network”, Robotics and Autonomous Systems, Vol. 86, pp. 184 – 195. https://doi.org/10.1016/j.robot.2016.08.022Google Scholar
Tai, L. and Liu, M. (2016), “A robot exploration strategy based on q-learning network”, 2016 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 57–62. https://doi.org/10.1109/rcar.2016.7784001.Google Scholar
Torrado, R.R, Bontrager, P. and Perez-Liebana, D. (2018), “Deep reinforcement learning for general video game ai”, CoRR, abs/1806.02448. https://doi.org/10.1109/cig.2018.8490422.Google Scholar
Wisskirchen, G. IBA Global Employment Institute. (2017), “Artificial intelligence and robotics and their impact on the workplace.Google Scholar
Yan, Z., Fabresse, L., Laval, J. and Bouraqadi, N. (2017), “Building a ros-based testbed for realistic multi-robot simulation: Taking the exploration as an example”, Robotics, Vol. 6 No. 3. https://doi.org/10.3390/robotics6030021Google Scholar
Zadeh, S.M.K. (2012), “A dynamical system-based approach to modeling stable robot control policies via imitation learning”.Google Scholar
Zamora, I., Lopez, N.G., Vilches, V.M. and Cordero, A.H. (2016), “Extending the openai gym for robotics: a toolkit for reinforcement learning using ROS and gazebo”. CoRR, abs/1608.05742.Google Scholar