A distributed UAV (unmanned aerial vehicle) flocking control method based on vision geometry is proposed, in which only monocular RGB (red, green, blue) images are used to estimate the relative positions and velocities between drones. It does not rely on special visual markers and external infrastructure, nor does it require inter-UAV communication or prior knowledge of UAV size. This method combines the advantages of deep learning and classical geometry. It adopts a deep optical flow network to estimate dense matching points between two consecutive images, uses segmentation technology to classify these matching points into background and specific UAV, and then maps the classified matching points to Euclidean space based on the depth map information. In 3D matching points, also known as 3D feature point pairs, each of their classifications is used to estimate the rotation matrix, translation vector, velocity of the corresponding UAV, as well as the relative position between drones, based on RANSAC and least squares method. On this basis, a flocking control model is constructed. Experimental results in the Microsoft Airsim simulation environment show that in all evaluation metrics, our method achieves almost the same performance as the UAV flocking algorithm based on ground truth cluster state.

Flow fields are determined from image sequences obtained in an experiment in which benthic macrofauna, Arenicola marina, causes water flow and the images depict the distribution of a tracer that is carried with the flow. The experimental setup is such that flow is largely two-dimensional, with a localised region where the Arenicola resides, from which flow originates. Here, we propose a novel parametric framework that quantifies such flow that is dominant along the image plane. We adopt a Bayesian framework so that we can impart certain physical constraints on parameters into the estimation process via prior distribution. The primary aim is to approximate the mean of the posterior distribution to present the parameter estimate via Markov Chain Monte Carlo. We demonstrate that the results obtained from the proposed method provide more realistic flows (in terms of divergence magnitude) than those computed from classical approaches such as the multi-resolution Horn–Schunk method. This highlights the usefulness of our approach if motion is largely constrained to the image plane with localised fluid sources.

Markley variables have advantages of slow variation, easy numerical integration and high precision in describing the attitude of spinning spacecraft. Previous attitude estimation methods based on Markley variables for spinning spacecraft usually employ a sun vector from the sun sensor, a magnetic vector from the magnetometer, or the angular rate from the gyro as the measurement. This paper proposes a Markley variables-based attitude estimation method using optical flow and a star vector from a star sensor as the measurement, where optical flow provides rate information and the star vector provides direction information. This method can estimate the direction of the spin axis and spin angular rate very well by using only one star sensor. In addition, the star sensor has higher accuracy than the traditional sun sensor and magnetometer, and the star sensor can also replace the gyro in case the gyro is out of action. The impact factors of this method are also analysed, which include spin angular rate, spin axis orientation and spacecraft moment of inertia tensor error.

In this paper, a fault-tolerant velocity estimation method is proposed for quadrotors in a GPS denied environment. A novel filter is developed in light of the quadrotor model and measurements from optical flow and inertial sensors. The proposed filter is capable of detecting and isolating the optical flow sensor faults, by which the velocity estimation accuracy and stability will be improved. It is also demonstrated that the wind velocity is observable in the proposed filter. Therefore, the new filter can also be implemented in a windy environment, which is a significant improvement to the previous model-aided inertial sensor estimator. At the end, some simulations are carried out to verify the advantages of our method.

An efficient method for tracking a target using a single Pan-Tilt-Zoom (PTZ) camera is proposed. The proposed Scale-Invariant Optical Flow (SIOF) method estimates the motion of the target and rotates the camera accordingly to keep the target at the center of the image. Also, SIOF estimates the scale of the target and changes the focal length relatively to adjust the Field of View (FoV) and keep the target appear in the same size in all captured frames. SIOF is a feature-based tracking method. Feature points used are extracted and tracked using Optical Flow (OF) and Scale-Invariant Feature Transform (SIFT). They are combined in groups and used to achieve robust tracking. The feature points in these groups are used within a twist model to recover the 3D free motion of the target. The merits of this proposed method are (i) building an efficient scale-invariant tracking method that tracks the target and keep it in the FoV of the camera with the same size, and (ii) using tracking with prediction and correction to speed up the PTZ control and achieve smooth camera control. Experimental results were performed on online video streams and validated the efficiency of the proposed method SIOF, comparing with OF, SIFT, and other tracking methods. The proposed SIOF has around 36% less average tracking error and around 70% less tracking overshoot than OF.

Push recovery is one of the most challenging problems for the current humanoid robots. The importance of push recovery can be well observed in the real environment. The critical issue for a humanoid is to maintain and recover its balance against any disturbances. In this research a new stereovision approach is proposed to estimate the robot deviation angle and consequently, the movement of center of mass of the robot is calculated. Then, two novel strategies have been devised to recover the balance of the humanoid which are called “knee strategy” and “knee-hip strategy.” Also, a mathematical model validates the efficiency of the proposed strategies as demonstrated in the paper. Experiments have been conducted on a humanoid robot and demonstrate that the predicted robot deviation angle, using stereovision technique, converges to the actual deviation angle. Stable regions of proposed strategies illustrate that the humanoid can recover its stability in a robust manner. Vision-based estimation also shows a higher correlation to actual deviation angle and a lower fluctuation compared with the output of the acceleration sensor.

In this paper, a vision-based scheme for the autonomous hovering of a miniature quad-rotor is developed. Cameras are used to estimate the position and the translational velocity of the vehicle. The dynamic model of the miniature quad-rotor is developed using the Newton–Euler approach. A nonlinear controller based on a separated saturation control strategy for a miniature quad-rotor is presented. To validate the theoretical results, an embedded control system for the miniature quad-rotor has been developed. Thus, the analytic results are supported by experimental tests. Experimental results have validated the proposed control strategy.

This article describes the implementation of a simple wavelet-based optical-flow motion estimator dedicated to continuous motions such as fluid flows. The wavelet representation of the unknown velocity field is considered. This scale-space representation, associated to a simple gradient-based optimization algorithm, sets up a well-defined multiresolution framework for the optical flow estimation. Moreover, a very simple closure mechanism, approaching locally the solution by high-order polynomials is provided by truncating the wavelet basis at fine scales. Accuracy and efficiency of the proposed method is evaluated on image sequences of turbulent fluid flows.

Integrating the visual navigation mechanism of flying insects with a nonlinear Kalman filter, this paper proposes a novel navigation algorithm. New concepts of entropic map and entropy flow are presented, which can characterize topographic features and measure changes of the image respectively. Meanwhile, an auto-selecting algorithm of assessment threshold is proposed to improve computational accuracy and efficiency of global motion estimation. The simulation results suggest that the navigation algorithm can perform real-time rectification of the missile's trajectory well, and can reduce the cost of the missile's hardware.

When negotiating a narrow gap, honeybees tend to fly through the middle of the gap, balancing the distances to the boundary on either side. To investigate the basis of this “centering response,” bees were trained to fly through a tunnel on their way to a feeding site and back, while their flight trajectories were filmed from above. The wall on either side carried a visual pattern. When the patterns were stationary vertical gratings, bees tended to fly through the middle of the tunnel, i.e. along its longitudinal axis. However, when one of the gratings was in motion, bees flying in the same direction as the moving grating tended to fly closer to while bees flying in the opposite direction tended to fly closer to the stationary grating. This demonstrates, directly and unequivocally, that flying bees estimate the distances of surfaces in terms of the apparent motion of their images. A series of further experiments revealed that the distance to the gratings is gauged in terms of their apparent angular speeds, and that the visual system of the bee is capable of measuring angular speed largely independently of the spatial period, intensity profile, or contrast of the grating. Thus, the motion-sensitive mechanisms mediating range perception appear to be qualitatively different from those that mediate the well-known optomotor response in insects, or those involved in motion detection and ocular tracking in man.

Visual motion, i.e. the pattern of changes on the retinae caused by the motion of objects or the observer through the environment, contains important cues for the accurate perception of the three-dimensional layout of the visual scene. In this study, we investigate if neurons in the visual system, specifically in area MT of the macaque monkey, are able to differentiate between various velocity gradients. Our stimuli were random dot patterns designed to eliminate stimulus variables other than the orientation of a velocity gradient. We develop a stimulus space (“deformation space”) that allows us to easily parameterize our stimuli. We demonstrate that a substantial proportion of MT cells show tuned responses to our various velocity gradients, often exceeding the response evoked by an optimized flat velocity profile. This suggests that MT cells are able to represent complex aspects of the visual environment and that their properties make them well suited as building blocks for the complex receptive-field properties encountered in higher areas, such as area MST to which many cells in area MT project.

The face is a rich source of information about human behavior. Available methods for coding facial displays, however, are human-observer dependent, labor intensive, and difficult to standardize. To enable rigorous and efficient quantitative measurement of facial displays, we have developed an automated method of facial display analysis. In this report, we compare the results with this automated system with those of manual FACS (Facial Action Coding System, Ekman & Friesen, 1978a) coding. One hundred university students were videotaped while performing a series of facial displays. The image sequences were coded from videotape by certified FACS coders. Fifteen action units and action unit combinations that occurred a minimum of 25 times were selected for automated analysis. Facial features were automatically tracked in digitized image sequences using a hierarchical algorithm for estimating optical flow. The measurements were normalized for variation in position, orientation, and scale. The image sequences were randomly divided into a training set and a cross-validation set, and discriminant function analyses were conducted on the feature point measurements. In the training set, average agreement with manual FACS coding was 92% or higher for action units in the brow, eye, and mouth regions. In the cross-validation set, average agreement was 91%, 88%, and 81% for action units in the brow, eye, and mouth regions, respectively. Automated face analysis by feature point tracking demonstrated high concurrent validity with manual FACS coding.