Imitation from Observation (IfO) prompts the robot to imitate tasks from unlabeled videos via reinforcement learning (RL). The performance of the IfO algorithm depends on its ability to extract task-relevant representations since images are informative. Existing IfO algorithms extract image representations by using a simple encoding network or pre-trained network. Due to the lack of action labels, it is challenging to design a supervised task-relevant proxy task to train the simple encoding network. Representations extracted by a pre-trained network such as Resnet are often task-irrelevant. In this article, we propose a new approach for robot IfO via multimodal observations. Different modalities describe the same information from different sides, which can be used to design an unsupervised proxy task. Our approach contains two modules: the unsupervised cross-modal representation (UCMR) module and a self-behavioral cloning (self-BC)-based RL module. The UCMR module learns to extract task-relevant representations via a multimodal unsupervised proxy task. The Self-BC for further offline policy optimization collects successful experiences during the RL training. We evaluate our approach on the real robot pouring water task, quantitative pouring task, and pouring sand task. The robot achieves state-of-the-art performance.

Expert drivers possess the ability to execute high sideslip angle maneuvers, commonly known as drifting, during racing to navigate sharp corners and execute rapid turns. However, existing model-based controllers encounter challenges in handling the highly nonlinear dynamics associated with drifting along general paths. While reinforcement learning-based methods alleviate the reliance on explicit vehicle models, training a policy directly for autonomous drifting remains difficult due to multiple objectives. In this paper, we propose a control framework for autonomous drifting in the general case, based on curriculum reinforcement learning. The framework empowers the vehicle to follow paths with varying curvature at high speeds, while executing drifting maneuvers during sharp corners. Specifically, we consider the vehicle’s dynamics to decompose the overall task and employ curriculum learning to break down the training process into three stages of increasing complexity. Additionally, to enhance the generalization ability of the learned policies, we introduce randomization into sensor observation noise, actuator action noise, and physical parameters. The proposed framework is validated using the CARLA simulator, encompassing various vehicle types and parameters. Experimental results demonstrate the effectiveness and efficiency of our framework in achieving autonomous drifting along general paths. The code is available at https://github.com/BIT-KaiYu/drifting.

In this study, a novel kinematic modeling method of parallel mechanism is proposed. It can obtain position and posture space simultaneously in a single model. Compared with the traditional method only based on inverse kinematics, the novel method can significantly improve computational performance. The original evaluation metric $\mathfrak{R}$ is proposed to evaluate the performance of the two modeling methods. Three groups of experiments with different calculation times are carried out for the classical PPU-3RUS parallel mechanism, and the new RS-3UPRU parallel mechanism after the effectiveness and wide applicability of the novel modeling method is proved. The calculation time and output rate are recorded, respectively, and then the respective $\mathfrak{R}$ values are obtained by weighting. The results show that the novel modeling method has better performance.

Robot pick-and-place for unknown objects is still a very challenging research topic. This paper proposes a multi-modal learning method for robot one-shot imitation of pick-and-place tasks. This method aims to enhance the generality of industrial robots while reducing the amount of data and training costs the one-shot imitation method relies on. The method first categorizes human demonstration videos into different tasks, and these tasks are classified into six types to symbolize as many types of pick-and-place tasks as possible. Second, the method generates multi-modal prompts and finally predicts the action of the robot and completes the symbolic pick-and-place task in industrial production. A carefully curated dataset is created to complement the method. The dataset consists of human demonstration videos and instance images focused on real-world scenes and industrial tasks, which fosters adaptable and efficient learning. Experimental results demonstrate favorable success rates and loss results both in simulation environments and real-world experiments, confirming its effectiveness and practicality.

In confined multi-obstacle environments, generating feasible paths for continuum robots is challenging due to the need to avoid obstacles while considering the kinematic limitations of the robot. This paper deals with the path-planning algorithm for continuum robots in confined multi-obstacle environments to prevent their over-deformation. By modifying the tree expansion process of the Rapidly-exploring Random Tree Star (RRT*) algorithm, a path-planning algorithm called the continuum-RRT* algorithm herein is proposed to achieve fewer iterations and faster convergence as well as generating desired paths that adhere to the kinematic limitations of the continuum robots. Then path planning and path tracking are implemented on a tendon-driven four-section continuum robot to validate the effectiveness of the path-planning algorithm. The path-planning results show that the path generated by the algorithm indeed has fewer transitions, and the path generated by the algorithm is closer to the optimal path that satisfies the kinematic limitations of the continuum robot. Furthermore, path-tracking experiments validate the successful navigation of the continuum robot along the algorithm-generated path, exhibiting an error range of 2.51%–3.91%. This attests to the effectiveness of the proposed algorithm in meeting the navigation requirements of continuum robots.

In response to the complex and challenging task of long-distance inspection of small-diameter and variable-diameter mine holes, this paper presents a design for an adaptive small-sized mine hole robot. First, focusing on the environment of small-diameter mine holes, the paper analyzes the robot’s functions and overall structural framework. A two-wheeled wall-pressing robot with good mobility, arranged in a straight line, is designed. Furthermore, an adaptive variable-diameter method is devised, which involves constructing an adaptive variable-diameter model and proposing a control method based on position and force estimators, enabling the robot to perceive external forces. Lastly, to verify the feasibility of the structural design and adaptive variable-diameter method, performance tests and analyses are conducted on the robot’s mobility and adaptive variable-diameter capabilities. Experimental results demonstrate that the robot can move within small-diameter mine holes at any inclination angle, with a maximum horizontal crawling speed of 3.96 m/min. By employing the adaptive variable-diameter method, the robot can smoothly navigate convex platform obstacles and slope obstacles in mine holes with diameters ranging from 70 mm to 100 mm, achieving the function of adaptive variable-diameter within 2 s. Thus, it can meet the requirements of moving inside mine holes under complex conditions such as steep slopes and small and variable diameters.

The two-wheeled legged robot combines the advantages of legged robot and wheeled robot and has high terrain adaptability. Spherical robots are highly resistant to interference during detection. In this paper, a new sphere-wheel-legged robot is designed by combining these three motion modes. This paper begins by introducing the mechanical design, hardware, and software. Then, kinematics and dynamics of wheel-legged motion and spherical motion are analyzed in detail. Subsequently, the controllers for wheel-legged balancing motion, wheel-legged jumping motion, and sphere rolling motion are developed, respectively. Finally, experiments are carried out for different modes. The results demonstrate that the designed robot has excellent locomotor capabilities over different terrains.

This paper presents an eight wire-driven parallel robot (WDPR-8) designed to serve as a suspension manipulator for aircraft models during wind tunnel testing. The precision of these tests is significantly influenced by the system’s stability and workspace, both of which are shaped by the geometric configuration of the structure and the tension in the wires. To acquire the efficiency principle of the suspension scheme design for the model, a kinematics model for a WDPR-8 was established. Based on the kinematics model, the stiffness of a WDPR-8 was theoretically studied, and the analytical expression of stiffness matrix of a WDPR was deduced. The stiffness matrix was composed of two terms, one of which is determined by the configuration of suspension system and the other term is determined by the wire tension. Based on the analysis result, a set of suspension scheme was discussed under the calculation of stiffness matrix and workspace analysis. In the discussion process, in addition to the stiffness-maximum calculation, another criterion as force closure is presented, which is useful for increasing the stiffness and workspace of the robot. Finally, a prototype was established according to the analysis result, and the workspace experiments are conducted. Test results indicate that the workspace meets the design requirements, validating the system suspension design method of a WDPR for aircraft model suspension in wind tunnel test considering of the systematic stiffness and workspace.

Lower limb rehabilitation robots based on linkage-based mechanisms have recently drawn significant attention in the field due to their numerous advantages. The control of previously proposed linkage-based gait rehabilitation robotic orthoses has been achieved using constant speed control without consideration for the interaction forces. However, such an approach can be harmful to people with stroke since the level of disability varies among individuals, and it may cause potential injuries when excessive force is applied by the robot. To overcome this limitation and improve the rehabilitation process, it is necessary to recognize the force exerted by the person during walking and adjust the robot’s assistive torque accordingly, to provide synchronized motion. Thus, in this work, a human-cooperative approach based on a stiffness control strategy for the six-bar linkage-based gait rehabilitation robot is presented. The proposed methodology can serve as a solid foundation for developing a human-cooperative approach for linkage-based lower limb rehabilitation robotic orthoses. The control was validated and tested with eight healthy human subjects. As a result, customized robotic assistance with this mechanism can be provided during training to meet the individual needs of stroke patients, which can lead to increased engagement and contribution, thus improving treatment outcomes.

This paper proposes an online robust self-learning terminal sliding mode control (RS-TSMC) with stability guarantee for balancing control of reaction wheel bicycle robots (RWBR) under model uncertainties and disturbances, which improves the balancing control performance of RWBR by optimising the constrained output of TSMC. The TSMC is designed for a second-order mathematical model of RWBR. Then robust adaptive dynamic programming based on an actor-critic algorithm is used to optimise the TSMC only by data sampled online. The system closed-loop stability and convergence of the neural network weights are guaranteed based on the Lyapunov analysis. The effectiveness of the proposed algorithm is demonstrated through simulations and experiments.

The tribological behavior can be informative about the incipient faults of robot manipulators. This study explores the evolution of friction characteristics from cold start to thermal equilibrium through a series of steady-state friction experiments. Based on these experimental observations, a friction-based fault diagnosis framework is proposed. The fault diagnosis process primarily involves defining the healthy state, decomposing friction curves and their features, and anomaly detection. Given the dependence of friction characteristics on different sources of faults, the parameters of steady-state experimental friction model are divided into two categories: one associated with contact interactions and the other related to non-contact regimes. Subsequently, confidence regions corresponding to distinguishable friction characteristics are independently constructed. These regions encapsulate the statistical description of the healthy state, characterized by mean values and the covariance of the friction characteristic parameter vectors during the unloaded state. In addition, we conduct experiments that consider the influence of applied loads on friction behavior. These experiments serve as a test set for comparison against nominal statistics. Leveraging the similarity between the effects of wear and load on friction, we introduce equivalent load thresholds to assess the severity of joint degradation. The results demonstrate the feasibility of employing confidence region views based on friction characteristic classification for fault detection and isolation.

The large number of patients with ankle injuries and the high incidence make ankle rehabilitation an urgent health problem. However, there is a certain degree of difference between the motion of most ankle rehabilitation robots and the actual axis of the human ankle. To achieve more precise ankle joint rehabilitation training, this paper proposes a novel 3-PUU/R parallel ankle rehabilitation mechanism that integrates with the human ankle joint axis. Moreover, it provides comprehensive ankle joint motion necessary for effective rehabilitation. The mechanism has four degrees of freedom (DOFs), enabling plantarflexion/dorsiflexion, eversion/inversion, internal rotation/external rotation, and dorsal extension of the ankle joint. First, based on the DOFs of the human ankle joint and the variation pattern of the joint axes, a 3-PUU/R parallel ankle joint rehabilitation mechanism is designed. Based on the screw theory, the inverse kinematics inverse, complete Jacobian matrix, singular characteristics, and workspace analysis of the mechanism are conducted. Subsequently, the motion performance of the mechanism is analyzed based on the motion/force transmission indices and the constraint indices. Then, the performance of the mechanism is optimized according to human physiological characteristics, with the motion/force transmission ratio and workspace range as optimization objectives. Finally, a physical prototype of the proposed robot was developed, and experimental tests were performed to evaluate the above performance of the proposed robot. This study provides a good prospect for improving the comfort and safety of ankle joint rehabilitation from the perspective of human-machine axis matching.

In recent years, unmanned aerial vehicles (UAVs) have been applied in underground mine inspection and other similar works depending on their versatility and mobility. However, accurate localization of UAVs in perceptually degraded mines is full of challenges due to the harsh light conditions and similar roadway structures. Due to the unique characteristics of the underground mines, this paper proposes a semantic knowledge database-based localization method for UAVs. By minimizing the spatial point-to-edge distance and point-to-plane distance, the relative pose constraint factor between keyframes is designed for UAV continuous pose estimation. To reduce the accumulated localization errors during the long-distance flight in a perceptual-degraded mine, a semantic knowledge database is established by segmenting the intersection point cloud from the prior map of the mine. The topological feature of the current keyframe is detected in real time during the UAV flight. The intersection position constraint factor is constructed by comparing the similarity between the topological feature of the current keyframe and the intersections in the semantic knowledge database. Combining the relative pose constraint factor of LiDAR keyframes and the intersection position constraint factor, the optimization model of the UAV pose factor graph is established to estimate UAV flight pose and eliminate the cumulative error. Two UAV localization experiments conducted on the simulated large-scale Edgar Mine and a mine-like indoor corridor indicate that the proposed UAV localization method can realize accurate localization during long-distance flight in degraded mines.

Due to the flexibility obtained through both materials and structures, soft robots have wide potential applications in complicated internal and external environments. This paper presents a new soft crawling robot with multiple locomotion patterns that integrate inchworm motion and various turning motions. First, the conceptual design of the proposed robot is presented by introducing thick-panel origami into the synthesis of a crawling robot, resulting in a Waterbomb-structure-inspired hybrid mechanism. Second, all locomotion patterns of the robot are precisely described and analyzed by screw theory in an algebraic manner, which include inchworm motion, restricted planar motion, quantitative turning motion, and marginal exploration motion. Then, the output motion parameter for each locomotion pattern is analytically modeled as a function of the robotic dimensional parameters, and the robot can thus be designed and controlled in a customized way for the expected output motion. Finally, the theoretical analysis and derivations are validated by simulation and physical prototype building, which lay the foundations for the design and manufacture of small-scale soft crawling robots with precise output motions in a complex planar environment.

In order to improve the global convergence performance of the super-twisting sliding mode control (STSMC) for the uncertain hybrid mechanism, especially in the high-speed scenario, and enhance the robustness of hybrid mechanism system to the uncertainties with a wide range of changes, an intelligent fixed-time super-twisting sliding mode control (IFTSTSMC) is proposed. Firstly, a fixed-time super-twisting sliding mode control (FTSTSMC) algorithm is designed by adding the exponential power terms with the fixed-time performance parameters in sliding variables and the transcendental function of the super-twisting algorithm in order to enhance the global convergence performance of the STSMC. Secondly, the existence condition of FTSTSMC for the uncertain hybrid mechanism is analyzed. The IFTSTSMC is designed by introducing RBF neural network to break through the limited range of uncertainties in FTSTSMC and enhance the robustness of hybrid mechanism system to the uncertainties with a wide range of changes. Then, the Lyapunov stability of the proposed method and the global fixed-time convergence of the system are proved theoretically. Finally, the effectiveness and superiority of the proposed control method are verified by the simulation and the automobile electro-coating conveying prototype experiment comparing with two classical finite-time sliding mode control methods.

Accurate dynamic model is essential for the model-based control of robotic systems. However, on the one hand, the nonlinearity of the friction is seldom treated in robot dynamics. On the other hand, few of the previous studies reasonably balance the calculation time-consuming and the quality for the excitation trajectory optimization. To address these challenges, this article gives a Lie-theory-based dynamic modeling scheme of multi-degree-of-freedom (DoF) serial robots involving nonlinear friction and excitation trajectory optimization. First, we introduce two coefficients to describe the Stribeck characteristics of Coulomb and static friction and consider the dependency of friction on load torque, so as to propose an improved Stribeck friction model. Whereafter, the improved friction model is simplified in a no-load scenario, a novel nonlinear dynamic model is linearized to capture the features of viscous friction across the entire velocity range. Additionally, a new optimization algorithm of excitation trajectories is presented considering the benefits of three different optimization criteria to design the optimal excitation trajectory. On the basis of the above, we retrieve a feasible dynamic parameter set of serial robots through the hybrid least square algorithm. Finally, our research is supported by simulation and experimental analyses of different combinations on the seven-DoF Franka Emika robot. The results show that the proposed friction has better accuracy performance, and the modified optimization algorithm can reduce the overall time required for the optimization process while maintaining the quality of the identification results.

To address the issues of low positioning accuracy and weak robustness of prior visual simultaneous localization and mapping (VSLAM) systems in dynamic environments, a semantic VSLAM (Sem-VSLAM) approach based on deep learning is proposed in this article. The proposed Sem-VSLAM algorithm adds semantic segmentation threads in parallel based on the open-source ORB-SLAM2’s visual odometry. First, while extracting the ORB features from an RGB-D image, the frame image is semantically segmented, and the segmented results are detected and repaired. Then, the feature points of dynamic objects are eliminated by using semantic information and motion consistency detection, and the poses are estimated by using the remaining feature points after the dynamic feature elimination. Finally, a 3D point cloud map is constructed by using tracking information and semantic information. The experiment uses Technical University of Munich public data to show the usefulness of the Sem-VSLAM algorithm. The experimental results show that the Sem-VSLAM algorithm can reduce the absolute trajectory error and relative attitude error of attitude estimation by about 95% compared to the ORB-SLAM2 algorithm and by about 14% compared to the VO-YOLOv5s in a highly dynamic environment and the average time consumption of tracking each frame image reaches 61 ms. It is verified that the Sem-VSLAM algorithm effectively improves the robustness and positioning accuracy in high dynamic environment and owning a satisfying real-time performance. Therefore, the Sem-VSLAM has a better mapping effect in a highly dynamic environment.

The cable-driven parallel mechanism (CDPM) is known as an interesting application in industry to pick and place objects owing to its advantages such as large workspaces. In addition to the advantages of this mechanism, there are some challenges to improving performance by considering constraints in different components, such as the behavior of cables, shape, size of the end effector and base, and model of pulleys and actuators. Moreover, the impact of online geometry reconfiguration must be analyzed. This paper demonstrates the impact of these constraints on the performance of reconfigurable CDPM. The methodology is based on the systematic review and meta-analysis guidelines to report the results. The databases used to find the papers are extracted from Scopus and Google Scholar, using related keywords. As a result, the impact of physical constraints on system performance is discussed. A total of 90 and 37 articles are selected, respectively. After removing duplicates and unrelated papers, 88 studies that met the inclusion criteria are selected for review. Even when considering the physical constraints in modeling the mechanism, simplifications in designing a model for the reconfigurable CDPM generate errors. There is a gap in designing high-performance controllers to track desired trajectories while reconfiguring the geometry, and the satisfaction of physical constraints needs to be satisfied. In conclusion, this review presents several constraints in designing a controller to track desired trajectories and improve performance in future work. This paper presents an integrated controller architecture that includes physical constraints and predictive control.