This paper proposes a computational model for selecting a suitable interlocutor of socially interactive robots in a situation interacting with multiple persons. To support this, a hybrid approach incorporating gaze control criteria and perceptual measurements for social cues is applied to the robot. For the perception part, representative non-verbal behaviors indicating human-interaction intent are designed based on the psychological analysis of human–human interaction, and these behavioral features are quantitatively measured by core perceptual components including visual, auditory, and spatial modalities. In addition, each aspect of recognition performance is improved through temporal confidence reasoning as a post-processing step. On the other hand, two factors of the physical space and conversational intimacy are tactically applied to the model calculation as a way of strengthening social gaze control effect of the robot. Interaction experiments with performance evaluation are given to verify that the proposed model is suitable to assess intended behaviors of individuals and perform gaze behavior about multiple persons. By showing a success rate of 93.3% in human decision-making criteria, it confirms a potential to establish socially acceptable gaze control in multiple-person interaction.

Historically, the principal function of vision has been to provide the information needed to support action. Visually mediated actions rely on three systems: the gaze system responsible for locating and fixating task-relevant objects, the motor system of the limbs to carry out the task, and the visual system to supply information to the other two. All three systems are under the control of a fourth system, the schema system, which specifies the current task and plans the overall sequence of actions. These four systems have separate but interconnected cortical representations. The way these systems interact in time and space is discussed here in relation to two studies of the gaze changes and manipulations made during two ordinary food preparation tasks. The main conclusions are that complex action sequences consist of a succession of individual object-related actions, each of which typically involve a turn toward the object (if needed), followed by fixation and finally manipulation monitored by vision. Gaze often moves on to the next object just before manipulation is complete. Task-irrelevant objects are hardly ever fixated, implying that the control of fixation comes principally from top-down instructions from the schema system, not bottom-up salience. Single fixations have identifiable functions (locating, directing, guiding, and checking) related to the action to be taken. Several variants of the basic object-related action scheme are discussed, including single-action events in ball sports involving only one anticipatory gaze shift, continuous production loops in text and music reading, and storage–action alternation in copying tasks such as portrait sketching.