This chapter presents an overview of several recent developments in vision science and outlines some of their implications for the management of visual attention in graphic displays. These include ways of sending attention to the right item at the right time, techniques to improve attentional efficiency, and possibilities for offloading some of the processing typically done by attention onto nonattentional mechanisms. In addition it is argued that such techniques not only allow more effective use to be made of visual attention but also open up new possibilities for human–machine interaction.

Introduction

Graphic displays such as maps, diagram and visual interfaces have long been used to present information in a form intended to be easy to comprehend (e.g., Massironi 2002; Tufte 2001; Ware 2008). While it is clear that such a goal is important, it is not so clear that it has always been achieved. Are current displays for the most part effective – do they enable user performance to be rapid, easy and accurate? Are they optimally so? Or are better designs possible?

These concerns are discussed here in the context of how to manage visual attention in graphic displays (including visual displays). This chapter is not directly concerned with the design of displays that respond effectively to the user (e.g., Roda and Thomas 2006; Vertegaal 2003). Rather, it focuses on the complementary perspective: how to design a display so that the user responds effectively to it. Results here apply equally well to static, dynamic and interactive displays.

By transforming the Web into a massive social space, Web 2.0 has opened a vast set of opportunities for people to interact with one another using online social networking, blogs, wikis or social bookmarking. But at the same time such a phenomenon has created the conditions for a massive social interaction overload: people are being overwhelmed by solicitations and opportunities to engage in social exchange but they have few means by which to deal effectively with this new level of interaction. The objective of this chapter is to investigate the use of ICT (information and communication technologies) to support online social interactions in a more attention-effective way. This is achieved by adapting to a social context a general model (Roda and Nabeth 2008) which defines four levels of attention support: perception, deliberation, operation and metacognition. We then describe how the support of social attention has been operationalized with the implementation of the attention-aware social platform AtGentNet, and tested in the context of communities of learners and professionals. After discussing the results of the experimentation, this chapter concludes by reflecting on how this approach can be generalized to support the interaction of people in the social web in general.

Introduction: addressing the social interaction overload

The social web, an essential component of the Web 2.0 vision, which refers to the use of the Internet for facilitating online social activities (Chi 2008), has totally reinvented the Web as a massive participatory social space.

The interactive relation and equivalence between working memory and attentional processes has been demonstrated by experimental, developmental, educational and clinical studies on preschoolers, schoolchildren, adolescents, younger adults and the elderly. It is important to understand the features of working memory from the ground theory of human cognitive architecture and its derived evolutionary educational psychology, which argue that the constraints of working memory are virtually necessary for both human survival and learning. Based on our knowledge of cognitive architecture and empirical research on effective instruction design that is in accordance with the functioning of working memory and related cognitive structures, cognitive load theory has been developed during recent decades to provide a number of principles for teaching and learning in a variety of settings. Much of this work has been carried out in a digital supported environment. In this chapter, recommendations based on cognitive load perspectives are presented along with further explorations of the potential for constructing digital supporting systems and tools.

Introduction

Digital technologies bring many capabilities to the teaching and learning environment. Anyone with access to the Internet can easily and quickly locate multimedia information. Text, images, sound and video can be accessed with the movement of a mouse or at the stroke of a key. Synchronous (e.g., video teleconferencing, chat sessions) and asynchronous (via bulletin boards, emails and the like) collaboration is possible.

This chapter addresses how an attention-management system can provide personalized support for self-regulated learning and what the effects of this support are on learning. An attention-management system can provide personalized support by capturing and interpretating information from the student's environment. A framework is proposed that will interpret the information and provide dynamic scaffolding for the learner. The essential elements are diagnosing, calibrating and fading scaffolds to the context of the learner. An intervention model supports self-regulated learning processes. In two studies, we have found evidence that an attention-management system can effectively give form to dynamic scaffolding. Dynamic scaffolding has a small- to medium-sized effect on students' performance and a small effect on students' metacognitive knowledge acquisition.

Introduction

E-learning has incrementally changed education in recent decades. Many new tools and instruments have been introduced to support existing educational practices. Yet only on a small scale have we seen transformative processes in schools. The large changes which have taken place in other sectors have not yet been achieved in education. This can partially be explained by the fact that e-learning solutions are not yet flexible enough to cater for learners' individual needs and demands. We see personalization in many sectors today, but education still seems to hold on to the ‘one size fits all’ paradigm, even though we know that personalized education is more effective than standardized education (Bloom 1984).

Remembering planned activities, resuming tasks previously interrupted, recalling the names of colleagues, sustaining focused performance under the pressure of interruptions, ensuring that we don't miss important information…these are only a few examples of critical activities whose performance is guided by attentional processes. This chapter proposes that knowledge about attentional processes can help us design systems that support users in situations such as those described above. The first part of the chapter gives an overview of some of the essential theoretical findings about human attention. The second part analyses attentional breakdowns and how those theoretical findings may be applied in order to design systems that either help avoid attentional breakdowns or assist in recovering from them.

Introduction

Current information and communication technologies concentrate on providing services to users performing focused activities. However, focused activity is no longer the norm. Users are often interrupted, they switch between the contexts of different devices and tasks, maintain awareness about the activity of distant collaborators and manage very large quantities of information. All this results in high cognitive load that may hinder users' overall achievements.

In order to address interaction in a more realistic manner, we have been working on the development of systems that are capable of supporting the processes that govern human cognitive resources allocation: attentional processes.

This chapter describes how intelligent embodied agents may react according to end-users' attention states and how these agents may adapt their interventions to encourage end users to participate actively in virtual environments such as collaboration platforms or e-learning modules. Attention-related data are taken into account by adapting generically defined interventions (templates) to particular contexts through the use of scripting and markup languages. This chapter introduces the Living Actor™ technology, whose main purpose is to provide end users with high-quality adaptive embodied agents, or avatars. Living Actor™ technology receives input from software components' reasoning on users' attention states and adjusts the actions of its embodied agents. The result is the creation of embodied agents, or avatars, that are capable of natural, intuitive, autonomous and adaptive behaviours that account for variations in emotion, gesture, mood, voice, culture and personality.

Introduction

According to Gartner (April 2007), by the end of 2011, 80 per cent of active Internet users (and Fortune 500 enterprises) will have a ‘second life’, not necessarily in the virtual world, called Second Life. Users' virtual lives will be represented by embodied agents in the form of avatars, or virtual representations of the self which allow users to express themselves with a personalized identity of their own creation. In the present chapter the authors define agents as ‘soft- and/or hardware that is intended to represent a complete person, animal, or personality’ (Sengers 2004: 4).

This chapter presents a number of software applications that make use of an eye tracker. It builds on the knowledge of visual attention and its control mechanisms as presented in chapters 3 and 5. It provides a tour through the years, showing how the use of eye gaze as indicator of visual attention has developed from being an additional input modality, supporting the disambiguation of fuzzy signals, to an interaction enhancement technique that allows software systems to work proactively and retrieve information without the user giving explicit commands.

Introduction

Our environment provides far more perceptual information than can be effectively processed. Hence, the ability to focus our attention on the essential is a crucial skill in a world full of visual stimuli. What we see is determined by what we attend to: the direction of our eye gaze, the focus of our visual attention, has a close relationship with the focus of our attention.

Eye trackers, which are used to measure the point of gaze, have developed rapidly during recent years. The history of eye-tracking equipment is long. For decades, eye trackers have been used as diagnostic equipment in medical laboratories and to enable and help communication with severely disabled people (see, e.g., Majaranta and Räihä 2007). Only recently have eye trackers reached the level of development where they can be considered as input devices for commonly used computing systems.

In recent years it has been increasingly recognized that the advent of information and communication technologies has dramatically shifted the balance between the availability of information and the ability of humans to process information. During the last century information was a scarce resource. Now, human attention has become the scarce resource whereas information (of all types and qualities) abounds. The appropriate allocation of attention is a key factor determining the success of creative activities, learning, collaboration and many other human pursuits. A suitable choice of focus is essential for efficient time organization, sustained deliberation and, ultimately, goal achievement and personal satisfaction. Therefore, we must address the problem of how digital systems can be designed so that, in addition to allowing fast access to information and people, they also support human attentional processes. With the aim of responding to this need, this book proposes an interdisciplinary analysis of the issues related to the design of systems capable of supporting the limited cognitive abilities of humans by assisting the processes guiding attention allocation. Systems of this type have been referred to in the literature as Attention-Aware Systems (Roda and Thomas 2006), Attentive User Interfaces (Vertegaal 2003) or Notification User Interfaces (McCrickard, Czerwinski and Bartram 2003) and they engender many challenging questions (see, for example, Wood, Cox and Cheng 2006).

This chapter reviews the results of the Salience Project, a cross-disciplinary research project focused on understanding how humans direct attention to salient stimuli. The first objective of the project was theoretical: that is, to understand behaviourally and electrophysiologically how humans direct attention through time to semantically and emotionally salient visual stimuli. Accordingly, we describe the glance-look model of the attentional blink. Notably, this model incorporates two levels of meaning, both of which are based upon latent semantic analysis, and, in addition, it incorporates an explicit body-state subsystem in which emotional experience manifests. Our second major objective has been to apply the same glance-look model to performance analysis of human–computer interaction. Specifically, we have considered a class of system which we call stimulus-rich reactive interfaces (SRRIs). Such systems are characterized by demanding (typically) visual environments, in which multiple stimuli compete for the user's attention, and a variety of physiological measures are employed to assess the user's cognitive state. In this context, we have particularly focused on electroencephalogram (EEG) feedback of stimulus perception. Moreover, we demonstrate how the glance-look model can be used to assess the performance of a variety of such reactive computer interfaces. Thus, the chapter contributes to the study of attentional support and adaptive interfaces associated with digital environments.

Introduction

Humans are very good at prioritizing competing processing demands. In particular, perception of a salient environmental event can interrupt ongoing processing, causing attention, and accompanying processing resources, to be redirected to the new event.