Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-13T22:36:34.822Z Has data issue: false hasContentIssue false
  • English
  • Français

Automated robotic monitoring and inspection of steel structures and bridges

Published online by Cambridge University Press:  11 January 2018

Hung Manh La ,
Tran Hiep Dinh ,
Nhan Huu Pham ,
Quang Phuc Ha  and
Anh Quyen Pham
Hung Manh La*
Affiliation:
Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, USA. E-mails: [email protected], [email protected], [email protected]
Tran Hiep Dinh
Affiliation:
School of Electrical, Mechanical and Mechatronic Systems, University of Technology Sydney, Australia. E-mails: [email protected], [email protected]
Nhan Huu Pham
Affiliation:
Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, USA. E-mails: [email protected], [email protected], [email protected]
Quang Phuc Ha
Affiliation:
School of Electrical, Mechanical and Mechatronic Systems, University of Technology Sydney, Australia. E-mails: [email protected], [email protected]
Anh Quyen Pham
Affiliation:
Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, USA. E-mails: [email protected], [email protected], [email protected]
*
*Corresponding author. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Summary

This paper presents visual and 3D structure inspection for steel structures and bridges using a developed climbing robot. The robot can move freely on a steel surface, carry sensors, collect data and then send to the ground station in real-time for monitoring as well as further processing. Steel surface image stitching and 3D map building are conducted to provide a current condition of the structure. Also, a computer vision-based method is implemented to detect surface defects on stitched images. The effectiveness of the climbing robot's inspection is tested in multiple circumstances to ensure strong steel adhesion and successful data collection. The detection method was also successfully evaluated on various test images, where steel cracks could be automatically identified, without the requirement of some heuristic reasoning.

Keywords

Field roboticsClimbing robotsSteel bridge inspectionImage stitching3D map constructionSteel crackHistogram thresholdingImage segmentation
Type
Articles
Information
Robotica , Volume 37 , Issue 5 , May 2019 , pp. 947 - 967
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

U.S. Department of Transportation Federal Highway Administration, “National bridge inventory data,” Available at: http://www.fhwa.dot.gov/bridge/nbi.cfm, accessed January 30, 2016.Google Scholar
Minnesota Department of Transportation, “I-35W St. Anthony Falls Bridge collapse,” Available at: http://www.dot.state.mn.us/i35wbridge/collapse.html, accessed January 30, 2016.Google Scholar
La, H. M., Lim, R. S., Basily, B. B., Gucunski, N., Yi, J., Maher, A., Romero, F. A. and Parvardeh, H., “Mechatronic systems design for an autonomous robotic system for high-efficiency bridge deck inspection and evaluation,” IEEE/ASME Trans. Mech. 18 (6), 1655–1664 (2013).CrossRefGoogle Scholar
La, H. M., Gucunski, N., Kee, S. H. and Nguyen, L. V., “Data analysis and visualization for the bridge deck inspection and evaluation robotic system,” Vis. Eng. 3 (1), 1–16 (2015).CrossRefGoogle Scholar
La, H. M., Gucunski, N., Kee, S. H. and Nguyen, L. V., “Visual and Acoustic Data Analysis for the Bridge Deck Inspection Robotic System,” Proceedings of the 31st International Symposium on Automation and Robotics in Construction and Mining (ISARC) (2014) pp. 50–57.Google Scholar
La, H. M., Gucunski, N., Kee, S. H. Yi, J., Senlet, T. and Nguyen, L. V., “Autonomous Robotic System for Bridge Deck Data Collection and Analysis” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, USA (Sep. 14–18, 2014) pp. 1950–1955.Google Scholar
La, H. M., Gucunski, N., Dana, K., and Kee, S. H.. “Development of an Autonomous Bridge Deck Inspection Robotic System,” J. Field Robot. 34 (8), 1489–1504 (2017).CrossRefGoogle Scholar
Lim, R. S., La, H. M., Shan, Z. and Sheng, W., “Developing a Crack Inspection Robot for Bridge Maintenance,” Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), (2011) pp. 6288–6293.Google Scholar
Lim, R. S., La, H. M. and Sheng, W., “A robotic crack inspection and mapping system for bridge deck maintenance,” IEEE Trans. Autom. Sci. Eng. 11 (2), 367–378 (2014).CrossRefGoogle Scholar
La, H. M., Lim, R. S., Basily, B. B., Gucunski, N., Yi, J., Maher, A., Romero, F. A. and Parvardeh, H., “Autonomous Robotic System for High-Efficiency Non-Destructive Bridge Deck Inspection and Evaluation,” Proceedings of the IEEE International Conference on Automation Science and Engineering (CASE), Madison, WI, USA (August 17–21, 2013) pp. 1053–1058.Google Scholar
Li, B., Cao, J., Xiao, J., Zhang, X. and Wang, H., “Robotic Impact-Echo Non-Destructive Evaluation Based On FFT and SVM,” Proceedings of the 11th World Congress on Intelligent Control and Automation (WCICA) (2014) pp. 2854–2859.Google Scholar
Gucunski, N., Basily, B., Kee, S-H., La, H. M., Parvardeh, H., Maher, A. and Ghasemi, H., “Multi NDE Technology Condition Assessment of Concrete Bridge Decks by RABITTM Platform,” Proceedings of the NDE/NDT for Structural Materials Technology for Highway & Bridges (2014) pp. 161–168.Google Scholar
Gucunski, N., Kee, S.-H., La, H. M., Basily, B., Maher, A. and Ghasemi, H., “Implementation of a Fully Autonomous Platform for Assessment of Concrete Bridge Decks RABIT,” Proceedings of Structures Congress (2015) pp. 367–378.Google Scholar
Gucunski, N., Kee, S.-H., La, H. M., Basily, B. and Maher, A., “Delamination and concrete quality assessment of concrete bridge decks using a fully autonomous RABIT platform,” Int. J. Struct. Monit. Maint. 2 (1), 19–34 (2015).Google Scholar
Gibb, S., Le, T. D., La, H. M., Schmid, R. and Berendsen, T., “A Multi-Functional Inspection Robot for Civil Infrastructure Evaluation and Maintenance,” Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada (Sep. 24–28, 2017).Google Scholar
Le, T. D., Gibb, S., Pham, N. H., La, H. M., Falk, L. and Berendsen, T., “Autonomous Robotic System using Non-Destructive Evaluation methods for Bridge Deck Inspection,” Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), May 29–June 3, 2017, Singapore.Google Scholar
Gibb, S. and La, H. M., “Automated Rebar Detection for Ground-Penetrating Radar,” Proceedings of the 12th International Symposium on Visual Computing, Las Vegas, Nevada, USA (Dec. 12–14, 2016).Google Scholar
Dinh, K., Gucunski, N., Kim, J. Y., Duong, T. and La, H. M., “Attenuation-based Methodology for Condition Assessment of Concrete Bridge Decks using GPR,” Proceedings of the 32nd International Symposium on Automation and Robotics in Construction and Mining (ISARC), Oulu, Finland, Jun. 15–18, 2015.Google Scholar
Gucunski, N., Maher, A., Basily, B. B., La, H. M., Lim, R. S., Parvardeh, H. and Kee, S. H., “Robotic Platform RABIT for Condition Assessment of Concrete Bridge Decks Using Multiple NDE Technologies,” J. Croatian Soc. Non Destr. Testing 12, 5–12 (2013).Google Scholar
Prasanna, P., Dana, K. J., Gucunski, N., Basily, B. B., La, H. M., Lim, R. S. and Parvardeh, H., “Automated crack detection on concrete bridges,” IEEE Trans. Autom. Sci. Eng. 13 (2), 591–599 (Apr. 2016).CrossRefGoogle Scholar
Xu, F. and Wang, X., “Design and Experiments on A New Wheel-Based Cable Climbing Robot,” Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (2008) pp. 418–423.Google Scholar
Cho, K. H., Kim, H. M., Jin, Y. H., Liu, F., Moon, H., Koo, J. C. and Choi, H. R., “Inspection robot for hanger cable of suspension bridge,” IEEE/ASME Trans. Mech. 18 (6), 1665–1674 (2013).CrossRefGoogle Scholar
Mazumdar, A. and Asada, H. H., “Mag-Foot: A Steel Bridge Inspection Robot,” Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2009) pp. 1691–1696.Google Scholar
Wang, R. and Kawamura, Y., “A Magnetic Climbing Robot for Steel Bridge Inspection,” Proceedings of the 11th World Congress on Intelligent Control and Automation (WCICA) (2014) pp. 3303–3308.Google Scholar
Liu, Q. and Liu, Y., “An Approach for Auto Bridge Inspection Based On Climbing Robot,” Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO) (2013) pp. 2581–2586.Google Scholar
Liu, Y., Dai, Q. and Liu, Q., “Adhesion-Adaptive Control of a Novel Bridge-Climbing Robot,” Proceedings of the IEEE 3rd Annual International Conference on Cyber Technology in Automation, Control and Intelligent Systems (CYBER) (2013) pp. 102–107.Google Scholar
Leibbrandt, A., Caprari, G., Angst, U., Siegwart, R. Y., Flatt, R. J. and Elsener, B., “Climbing Robot for Corrosion Monitoring of Reinforced Concrete Structures,” Proceedings of the 2nd International Conference on Applied Robotics for the Power Industry (CARPI) (2012) pp. 10–15.Google Scholar
Leon-Rodriguez, H., Hussain, S. and Sattar, T., “A Compact Wall-Climbing and Surface Adaptation Robot for Non-Destructive Testing,” Proceedings of the 12th International Conference on Control, Automation and Systems (ICCAS) (2012) pp. 404–409.Google Scholar
San-Millan, A., “Design of a Teleoperated Wall Climbing Robot for Oil Tank Inspection,” Proceedings of the 23th Mediterranean Conference on Control and Automation (MED) (2015) pp. 255–261.Google Scholar
Zhu, D., Guo, J., Cho, C., Wang, Y. and Lee, K.M., “Wireless Mobile Sensor Network for the System Identification of a Space Frame Bridge,” IEEE/ASME Trans. Mech. 17 (3), 499–507 (2012).CrossRefGoogle Scholar
Koch, C., Georgieva, K., Kasireddy, V., Akinci, B. and Fieguth, P., “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure,” Adv. Eng. Inform. 29 (2), 196–210 (2015).CrossRefGoogle Scholar
Oh, J. K. et al., “Bridge inspection robot system with machine vision,” Autom. Constr. 18, 929–941 (2009).CrossRefGoogle Scholar
Adhikari, R. S., Moselhi, O. and Bagchi, A., “Image-based retrieval of concrete crack properties for bridge inspection,” Autom. Constr. 39, 180–194 (2014).CrossRefGoogle Scholar
Lins, R. G. and Givigi, S. N., “Automatic Crack Detection and Measurement Based on Image Analysis,” IEEE Trans. Instrum. Meas. 65 (3), 583–590, (2016).CrossRefGoogle Scholar
K&J Magnetics Inc. Orignial magnet calculator, Available a: https://www.kjmagnetics.com/calculator.asp, access November 10, 2016.Google Scholar
DS8911HV Servo, Available at: http://www.jramericas.com/116293/JRPS8911HV/, access November 10, 2016.Google Scholar
Forsyth, D. A. and Ponce, J., Computer Vision: A Modern Approach, 2nd Edition, Nov 5, 2011 (Prentice Hall, Upper Saddle River, NJ).Google Scholar
Brown, M. and Lowe, D. G., “Automatic panoramic image stitching using invariant features,” Int. J. Comput. Vis. 74 (1), 59–73 (2007).CrossRefGoogle Scholar
Besl, P. and McKay, N., “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Mach. Intell. 14 (2), 239–256 (1992).CrossRefGoogle Scholar
Rusu, R. B., Blodow, N. and Beetz, M., “Fast Point Feature Histograms (FPFH) for 3D registration,” Proceedings of the IEEE International Conference on Robotics and Automation, ICRA '09, (2009) pp. 3212–3217.Google Scholar
Chen, Y. and Medioni, G., “Object Modelling by Registration of Multiple Range Images,” Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3 (2015) pp. 2724–2729.Google Scholar
Holz, D., Ichim, A. E., Tombari, F., Rusu, R. B. and Behnke, S., “Registration with the point cloud library: A modular framework for aligning in 3-D,” IEEE Robot. Autom. Mag. 2 (4), 110–124 (2015).CrossRefGoogle Scholar
Yu, Y.-H., Kwok, N. M. and Ha, Q. P., “Color tracking for multiple robot control using a system-on-programmable-chip,” Autom. Constr. 20 (6), 669–676 (2011).CrossRefGoogle Scholar
Dinh, T. H., Pham, M. T., Phung, M. D., Nguyen, D. M. and Tran, Q. V., “Image Segmentation Based on Histogram of Depth and an Application in Driver Distraction Detection,” Proceedings of the 13th International Conference on Control Automation Robotics & Vision (ICARCV) (2014) pp. 969–974.Google Scholar
Otsu, N., “A threshold selection method from gray-level histograms,” IEEE Trans. Syst. Man Cybern. 9 (1), 62–66 (1979).CrossRefGoogle Scholar
Dirami, A., Hammouche, K., Diaf, M. and Siarry, P., “Fast multilevel thresholding for image segmentation through a multiphase level set method,” Signal Proces., 93 (1), 139–153 (2013).CrossRefGoogle Scholar
Yuan, X. C., Wu, L. S. and Peng, Q., “An improved Otsu method using the weighted object variance for defect detection,” Appl. Surface Sci. 349, 472–484 (2015).CrossRefGoogle Scholar
Sato, Y., Nakajima, S., Shiraga, N., Atsumi, H., Yoshida, S., Koller, T., Gerig, G. and Kikinis, R., “Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images,” Medical Image Anal. 2 (2), 143–168 (1998).CrossRefGoogle ScholarPubMed
Fujita, Y. and Hamamoto, Y., “A robust automatic crack detection method from noisy concrete surfaces,” Mach. Vis. Appl. 22 (2), 245–254 (2011).CrossRefGoogle Scholar
Zhao, Y. Q., Wang, X. H., Wang, X. F. and Shih, F. Y., “Retinal vessels segmentation based on level set and region growing,” Pattern Recognit. 47 (7), 2437–2446 (2014).CrossRefGoogle Scholar
Sezan, M. I., “A peak detection algorithm and its application to histogram-based image data reduction.” Comput. Vis. Graph. Image Process. 49 (1), 36–51 (1990).CrossRefGoogle Scholar
Yuan, B. and Liu, M., “Power histogram for circle detection on images,” Pattern Recognit. 48 (10) 3268–3280 (2015).CrossRefGoogle Scholar
Sauvola, J. and Pietikäinen, M., “Adaptive document image binarization,” Pattern Recognit. 33 (2), 225–236 (2000).CrossRefGoogle Scholar
Pham, N. H. and La, H. M., “Design and Implementation of an Autonomous Robot for Steel Bridge Inspection,” Proceedings of the 54th Annual Allerton Conference on Communication, Control, and Computing, Urbana-Champaign, Illinois, USA (Sep. 27–30, 2016) pp. 556–562.Google Scholar
Nguyen, L. V., La, H. M., Sanchez, J. and Vu, T., “A Smart Shoe for Building a Real-Time 3D Map,” Elsevier J. Autom. Constr. 71, 2–12 (Sep. 2016).CrossRefGoogle Scholar
Pham, N. H., La, H. M., Ha, Q. P., Dang, S. N., Vo, A. H. and Dinh, Q. H., “Visual and 3D mapping for steel bridge inspection using a climbing robot,” Proceedings of the 33rd International Symposium on Automation and Robotics in Construction and Mining (ISARC), Auburn, Alabama, USA (July 18–21, 2016).Google Scholar
Dinh, T. H., Ha, Q. P. and La, H. M., “Computer vision-based method for concrete crack detection,” Proceedings of the 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), Phuket, Thailand (Nov. 13–15, 2016).Google Scholar
La, H. M., Sheng, W. and Chen, J., “Cooperative and active sensing in mobile sensor networks for scalar field mapping,” IEEE Trans. Syst. Man Cybern.: Syst. 45 (1), 1–12 (Jan. 2015).CrossRefGoogle Scholar
La, H. M., Lim, R. and Sheng, W., “Multi-robot cooperative learning for predator avoidance,” IEEE Trans. Control Syst. Technol. 23 (1), 52–63 (Jan. 2015).CrossRefGoogle Scholar
La, H. M. and Sheng, W., “Multi-agent motion control in cluttered and noisy environments,” J. Commun. 8 (1), 32–46 (Jan. 2013).CrossRefGoogle Scholar
La, H. M. and Sheng, W., “Dynamic targets tracking and observing in a mobile sensor network,” Elsevier J. Robot. Autom. Syst. 60 (7), 996–1009 (Jul. 2012).CrossRefGoogle Scholar
La, H. M. and Sheng, W., “Distributed sensor fusion for scalar field mapping using mobile sensor networks,” IEEE Trans. Cybern. 43 (2), 766–778 (April, 2013).Google ScholarPubMed