Aimed at the challenges of wide-angle mobile robot visual perception for diverse field applications, we present the spherical robot visual system that uses a 360° field of view (FOV) for realizing real-time object detection. The spherical robot image acquisition system model is developed with optimal parameters, including camera spacing, camera axis angle, and the distance of the target image plane. Two 180$^{\circ}$-wide panoramic FOVs, front and rear view, are formed using four on-board cameras. The speed of the SURF algorithm is increased for feature extraction and matching. For seamless fusion of the images, an improved fade-in and fade-out algorithm is used, which not only improves the seam quality but also improves object detection performance. The speed of the dynamic image stitching is significantly enhanced by using a cache-based sequential image fusion method. On top of the acquired panoramic wide FOVs, the YOLO algorithm is used for real-time object detection. The panoramic visual system for the spherical robot is then tested in real time, which outputs panoramic views of the scene at an average frame rate of 21.69 fps and panoramic views with object detection at an average of 15.39 fps.

The rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our FdIngred328 dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.

Coverage path planning (CPP) is a subfield of path planning problems in which free areas of a given domain must be visited by a robot at least once while avoiding obstacles. In some situations, the path may be optimized for one or more criteria such as total distance traveled, number of turns, and total area covered by the robot. Accordingly, the CPP problem has been formulated as a multi-objective optimization (MOO) problem, which turns out to be a challenging discrete optimization problem, hence conventional MOO algorithms like Non-dominated Sorting Genetic Algorithm-2 (NSGA-II) do not work as it is. This study implements a modified NSGA-II to solve the MOO problem of CPP for a mobile robot. In this paper, the proposed method adopted two objective functions: (1) the total distance traveled by the robot and (2) the number of turns taken by the robot. The two objective functions are used to calculate energy consumption. The proposed method is compared to the hybrid genetic algorithm (HGA) and the traditional genetic algorithm (TGA) in a rectilinear environment containing obstacles of various complex shapes. In addition, the results of the proposed algorithm are compared to those generated by HGA, TGA, oriented rectilinear decomposition, and spatial cell diffusion and family of spanning tree coverage in existing research papers. The results of all comparisons indicate that the proposed algorithm outperformed the existing algorithms by reducing energy consumption by 5 to 60%. This paper provides the facility to operate the robot in different modes.

The integration of camera and LiDAR technologies has the potential to significantly enhance construction robots’ perception capabilities by providing complementary construction information. Structured light cameras (SLCs) are a desirable alternative as they provide comprehensive information on construction defects. However, fusing these two types of information depends largely on the sensors’ relative positions, which can only be established through extrinsic calibration. This paper introduces a novel calibration algorithm considering a customized board for SLCs and repetitive LiDARs, which are designed to facilitate the automation of construction robots. The calibration board is equipped with four symmetrically distributed hemispheres, whose centers are obtained by fitting the spheres and adoption with the geometric constraints. Subsequently, the spherical centers serve as reference features to estimate the relationship between the sensors. These distinctive features enable our proposed method to only require one calibration board pose and minimize human intervention. We conducted both simulation and real-world experiments to assess the performance of our algorithm. And the results demonstrate that our method exhibits enhanced accuracy and robustness.

Visual simultaneous localisation and mapping (vSLAM) has shown considerable promise in positioning and navigating across a variety of indoor and outdoor settings, significantly enhancing the mobility of robots employed in industrial and everyday services. Nonetheless, the prevalent reliance of vSLAM technology on the assumption of static environments has led to suboptimal performance in practical implementations, particularly in unstructured and dynamically noisy environments such as substations. Despite advancements in mitigating the influence of dynamic objects through the integration of geometric and semantic information, existing approaches have struggled to strike an equilibrium between performance and real-time responsiveness. This study introduces a lightweight, multi-modal semantic framework predicated on vSLAM, designed to enable intelligent robots to adeptly navigate the dynamic environments characteristic of substations. The framework notably enhances vSLAM performance by mitigating the impact of dynamic objects through a synergistic combination of object detection and instance segmentation techniques. Initially, an enhanced lightweight instance segmentation network is deployed to ensure both the real-time responsiveness and accuracy of the algorithm. Subsequently, the algorithm’s performance is further refined by amalgamating the outcomes of detection and segmentation processes. With a commitment to maximising performance, the framework also ensures the algorithm’s real-time capability. Assessments conducted on public datasets and through empirical experiments have demonstrated that the proposed method markedly improves both the accuracy and real-time performance of vSLAM in dynamic environments.

In this paper, a method of planning the expanded S-curve trajectory of robotic manipulators is proposed to minimize the execution time as well as to achieve the smoother trajectory generation in the deceleration stage for point-to-point motions. An asymmetric parameter is added to the piecewise sigmoid function for an improved jerk profile. This asymmetric profile is continuous and infinitely differentiable. Based on this profile, two analytical algorithms are presented. One is applied to determine the suitable time intervals of trajectory satisfying the time optimality under the kinematic constraints, and the other is to determine the asymmetric parameter generating the minimum execution time. Also, the calculation procedure for the time-scaled synchronization for all joints is given to decrease unnecessary loads onto the actuators. The velocity, acceleration, jerk and snap (the derivative of jerk) of the joints and the end-effector are equal to zero at two end points of motion. The simulation results through 3 DOF and 6 DOF robotic manipulators show that our approach reduces the jerk and snap of the deceleration stage effectively while decreasing the total execution time. Also, the analysis for a single DOF mass-spring-damper system indicates that the residual vibration could be reduced to 10% more than the benchmark techniques in case velocity, acceleration and jerk are limited to 1.24 m/s, 6 m/s2 and 80 m/s3, respectively and displacement is set to 0.8m. These results manifest that the performance of reducing residual vibrations is good and demonstrate an important characteristic of the proposed profile suitable for point-to-point motion.

In practical applications, many robots equipped with embedded devices have limited computing capabilities. These limitations often hinder the performance of existing dynamic SLAM algorithms, especially when faced with occlusions or processor constraints. Such challenges lead to subpar positioning accuracy and efficiency. This paper introduces a novel lightweight dynamic SLAM algorithm designed primarily to mitigate the interference caused by moving object occlusions. Our proposed approach combines a deep learning object detection algorithm with a Kalman filter. This combination offers prior information about dynamic objects for each SLAM algorithm frame. Leveraging geometric techniques like RANSAC and the epipolar constraint, our method filters out dynamic feature points, focuses on static feature points for pose determination, and enhances the SLAM algorithm’s robustness in dynamic environments. We conducted experimental validations on the TUM public dataset, which demonstrated that our approach elevates positioning accuracy by approximately 54% and boosts the running speed by 75.47% in dynamic scenes.

A negative pressure wall-climbing robot is a special robot for climbing vertical walls, which is widely used in construction, petrochemicals, nuclear energy, shipbuilding, and other industries. The mobility and adhesion of the wheel-track wall-climbing robot with steering-straight mode are significantly decreased on the cylindrical wall, especially during steering. The reason is that the suction chamber may separate from the wall and the required driving force for movement increases, during steering. In this paper, a negative pressure wall-climbing robot with omnidirectional movement mode is developed. By introducing a compliant adjusting suction mechanism and omni-belt wheels, an omnidirectional movement mode is formed instead of the steering-straight mode, and the performances of adhesion and mobility are improved. We establish the safety adhesion model for the robot on a cylindrical wall and obtain the safety adhesion forces. We designed and manufactured an experimental prototype based on the analysis. Experiments showed that the robot has the ability of full maneuverability in cylindrical walls.

In the application of rotorcraft atmospheric environment detection, to reflect the distribution of atmospheric pollutants more realistically and completely, the sampling points must be spread throughout the entire three-dimensional space, and the cooperation of multiple unmanned aerial vehicles (multi-UAVs) can ensure real-time performance and increase operational efficiency. In view of the problem of coordinated detection by multi-UAVs, the region division and global coverage path planning of the stereo space to be detected are studied. A whale optimization algorithm based on the simulated annealing-whale optimization algorithm (SA-WOA) is proposed, which introduces adaptive weights with the Levy flight mechanism, improves the metropolis criterion, and introduces an adaptive tempering mechanism in the SA stage. Path smoothing is subsequently performed with the help of nonuniform rational B-spline (NURBS) curves. The comparison of algorithms using the eil76 dataset shows that the path length planned by the SA-WOA algorithm in this paper is 10.15% shorter than that of the WOA algorithm, 13.25% shorter than the SA planning result, and only 0.95% difference from the optimal path length in the dataset. From the perspective of planning time, its speed is similar to WOA, with a relative speed increase of 27.15% compared to SA, proving that the algorithm proposed in this paper has good planning performance. A hardware system platform is designed and built, and environmental gas measurement experiments were conducted. The experimental results indicate that the multi-UAV collaborative environment detection task planning method proposed in this paper has certain practical value in the field of atmospheric environment detection.

The existing single-mode posture adjustment equipment for solar wing docking is only suitable for a limited number of satellite dimensions; it could not meet the diverse development trends of satellite models. The working range requirements are different when different-sized satellites dock with the solar wing, and the docking process is divided into two stages in this paper. While the DOFs required for the two stages are different, a movable heavy-load reconfigurable redundant posture adjustment platform (RrPAP) with dual motion modes is proposed in this paper. The RrPAP consists of a wheeled mobile platform and a reconfigurable parallel posture adjustment mechanism (PAM). The micro-motion PAM limb types are synthesized, and the comprehensive load-bearing index is proposed to select the mechanism types for heavy-load conditions. A decentralized four-limb six-degree-of-freedom (6-DOF) parallel micro-motion PAM is designed. In the macro-motion stage, for the PAM to still have a defined motion after being released from ground constraints, a serial coupling sub-chain is designed between adjacent limbs to restrict relative movement between them. A type synthesis method for symmetrically coupled mechanisms based on mechanism decoupling and motion distribution is proposed. Four types of symmetrically coupled mechanisms with multi-loop consisting of serial coupling sub-chains are synthesized by using this method. The feasibility of the proposed method is demonstrated through an example using the constraint synthesis method based on screw theory. This work provides a foundation for subsequent refinement and expansion of type synthesis theories and the selection of new types of mechanisms.

A decoupling method is proposed for the elastic stiffness modeling of hybrid robots based on the rigidity principle, screw theory, strain energy, and Castigliano’s second theorem. It enables the decoupling of parallel and serial modules, as well as the individual contributions of each elastic component to the mechanism’s stiffness performance. The method is implemented as follows: (1) formulate limb constraint wrenches and corresponding limb stiffness matrix based on the screw theory and strain energy, (2) formulate the overall stiffness matrix of parallel and serial modules corresponding to end of the hybrid robots based on the rigidity principle, principle of virtual work, the wrench transfer formula, and strain energy methods, and (3) obtain and decouple the overall stiffness matrix and deflection of the robot based on the Castigliano’s second theorem. Finally, A planar hybrid structure and the 4SRRR + 6R hybrid robot are used as illustrative examples to implement the proposed method. The results indicate that selectively enhancing the stiffness performance of the mechanism is the most effective approach.

Robots have the capability to perform precise and minimally invasive surgeries. For the robot-assisted teleoperated fracture reduction surgery, the operating accuracy largely depends on visual reference through fluoroscopy. The operator needs to adjust several times according to computed tomography (CT) image. During the robot reduction surgery, there are large muscle forces generated by the numerous muscles surrounding the fractured segments. However, there is no effective reduction force feedback to the master robot. In this paper, in order to improve the operating accuracy of the fracture reduction with teleoperated surgery mode, six-dimensional constraints of the master robot are studied by utilizing the virtual fixture method, which can restrict the position and orientation through the force and visual guidance. The six-dimensional force sensor is used to collect information of the reduction force. For the master robot, a motor stall control method based on the current loop is adopted to provide feedback of the reduction force, which can enhance the surgeon’s sense of operational presence. To verify the effectiveness of virtual fixture and force feedback, the fracture reduction experiments are conducted on the fractured model with simulating lager muscle force. Experimental results show that the reduction errors are within acceptable ranges: $0.03\pm 0.73\textrm{mm}$, $0.54\pm 0.43\textrm{mm}$, $0.46\pm 1.05\textrm{mm}$, $1.05\pm 1.31^{\circ}$, $1.15\pm 1.91^{\circ}$, $1.09\pm 2.61^{\circ}$. The number of fluoroscopy procedures required ranges from 1 to 2 and the average operation time is approximately 170 s. Compared to traditional methods and other teleoperation methods, the fracture reduction accuracy and surgical efficiency of method in this paper are significantly improved.

Diabetic retinopathy (DR) is a complication of diabetes that causes blindness and the early detection of diabetics using retinopathy images remains a challenging task. Hence, a novel, earlier smart prediction of diabetic retinopathy from fundus image under Innovative ResNet Optimization is introduced to effectively detect the earlier stage of DR from Fundus image. Initially, the fundus image is scaled during preprocessing and converted into a grayscale format. As the existing studies neglect some deserving unique features that are crucial for predicting the earliest signs of DR, a novel Fractional Radon Transform with Visibility Graph is introduced for extracting the novel features such as microaneurysms count, dot and blot hemorrhages count, statistical measures, and retinal layer thickness, in which a Generalized Cosine Fractional Radon Transform is used to capture the image’s fine-scale texture information thereby effectively capturing the statistical measures, while a weighted Horizontal Visibility Graph is made to examine the apparent spatial relationships between pixel pairs in the image based on the values of the pixels’ gray levels. Further, the existing works failed to identify the small fine dark areas that were ignored throughout the morphological opening process. In order to overcome this issue, a Morphological Black Hat Transform with Optimized ResNet Algorithm is implemented, where segmentation is made through Enriched Black Hat Transform-based Morphological operation to identify fine dark regions among the pixels inside the eye samples, and the classification is done by using ResNet-driven S-GOA (Socio Grasshopper Optimization Algorithm), to optimally predict the stages of DR. The result obtained showed that the proposed model outperforms existing techniques with high performance and accuracy.

The early applications of Visual Simultaneous Localization and Mapping (VSLAM) technology were primarily focused on static environments, relying on the static nature of the environment for map construction and localization. However, in practical applications, we often encounter various dynamic environments, such as city streets, where moving objects are present. These dynamic objects can make it challenging for robots to accurately understand their own position. This paper proposes a real-time localization and mapping method tailored for dynamic environments to effectively deal with the interference of moving objects in such settings. Firstly, depth images are clustered, and they are subdivided into sub-point clouds to obtain clearer local information. Secondly, when processing regular frames, we fully exploit the structural invariance of static sub-point clouds and their relative relationships. Among these, the concept of the sub-point cloud is introduced as novel idea in this paper. By utilizing the results computed based on sub-poses, we can effectively quantify the disparities between regular frames and reference frames. This enables us to accurately detect dynamic areas within the regular frames. Furthermore, by refining the dynamic areas of keyframes using historical observation data, the robustness of the system is further enhanced. We conducted comprehensive experimental evaluations on challenging dynamic sequences from the TUM dataset and compared our approach with state-of-the-art dynamic VSLAM systems. The experimental results demonstrate that our method significantly enhances the accuracy and robustness of pose estimation. Additionally, we validated the effectiveness of the system in dynamic environments through real-world scenario tests.

Hyper-redundant manipulators (HRMs) exhibit promising adaptability and superior dexterity in cavity detection tasks, owing to their snake-like segmented backbones. Due to the safety concern in contactless operating tasks, reliable motion planning in a confined environment for HRMs is very challenging. However, existing expanding-based obstacle avoidance methods are not feasible in narrow environments, as they will excessively occupy free spaces required for maneuvering. In this work, a local collision-free motion planning strategy based on dynamic safety envelope (DSE) is proposed for HRMs. First, the local motion of HRMs is analyzed in detail, and DSE is proposed for the first time to describe the boundary of the collision-free area. Then, to maximize the efficient utilization of narrow spaces, a reference trajectory for HRM is roughly planned without expanding obstacles. Further, a tip-guided trajectory tracking method based on configuration prediction is proposed by considering the discrete characteristics of rigid links to avoid obstacles. During the tracking process, DSEs are applied to evaluate collision risk and optimize the configuration. Finally, to validate the effectiveness of our proposed method, simulations are conducted, followed by experiments by using a 18-degrees of freedom mobile HRM prototype system.

Motion and constraint identification are the fundamental issue throughout the development of parallel mechanisms. Aiming at meaningful result with heuristic and visualizable process, this paper proposes a machine learning-based method for motions and constraints modeling and further develops the automatic software for mobility analysis. As a preliminary, topology of parallel mechanism is characterized by recognizable symbols and mapped to the motion of component limb through programming algorithm. A predictive model for motion and constraint with their nature meanings is constructed based on neural network. An increase in accuracy is obtained by the novel loss function, which combines the errors of network and physical equation. Based on the predictive model, an automatic framework for mobility analysis of parallel mechanisms is constructed. A software is developed with WebGL interface, providing the result of mobility analysis as well as the visualizing process particularly. Finally, five typical parallel mechanisms are taken as examples to verify the approach and its software. The method facilitates to attain motion/constraint and mobility of parallel mechanisms with both numerical and geometric features.

Collaborative robotics is a field of growing industrial interest, within which understanding the energetic behavior of manipulators is essential. In this work, we present the electro-mechanical modeling of the UR5 e-series robot through the identification of its dynamics and electrical parameters. By means of the identified robot model, it is then possible to compute and optimize the energy consumption of the robot during prescribed trajectories. The proposed model is derived from data acquired from the robot controller during bespoke experimental tests, using model identification procedures and datasheet provided by manipulator, motors, and gearbox manufacturers. The entire procedure does not require the use of any additional sensor, so it can be easily replicated with an off-the-shelf manipulator, and applied to other robots of the same family.

Path planning for the unmanned aerial vehicle (UAV) is to assist in finding the proper path, serving as a critical role in the intelligence of a UAV. In this paper, a path planning for UAV in three-dimensional environment (3D) based on enhanced gravitational search algorithm (EGSA) is put forward, taking the path length, yaw angle, pitch angle, and flight altitude as considerations of the path. Considering EGSA is easy to fall into local optimum and convergence insufficiency, two factors that are the memory of current optimal and random disturbance with chaotic levy flight are adopted during the update of particle velocity, improving the balance between exploration and exploitation for EGSA through different time-varying characteristics. With the identical cost function, EGSA is compared with seven peer algorithms, such as moth flame optimization algorithm, gravitational search algorithm, and five variants of gravitational search algorithm. The experimental results demonstrate that EGSA is superior to the seven comparison algorithms on CEC 2020 benchmark functions and the path planning method based on EGSA is more valuable than the other seven methods in diverse environments.