The previous two chapters of the book have focused on the basic issue: perception of the environment from two different viewpoints. Specifically, Chapter 3 focused on perception viewed as power-spectrum estimation, which is applicable to applications such as cognitive radio, where spectrum sensing of the radio environment is of paramount importance. In contrast, Chapter 4 focused on Bayesian filtering for state estimation, which is applicable to another class of applications exemplified by cognitive radar, where estimating the parameters of a target in a radar environment is the issue of interest. Although those two chapters address entirely different topics, they also have a common perspective: they both pertain to the perceptor of a cognitive dynamic system, the basic function of which is to account for perception in the perception–action cycle. In this chapter, we move on to the action part of this cycle, which is naturally the responsibility of the actuator of the cognitive dynamic system.

Just as the Bayesian framework provides the general formalism for the perceptor to perceive the environment by estimating the hidden state of the environment, Bellman's dynamic programming provides the general formalism for the actuator to act in the environment through control or decision-making.

Dynamic programming is a technique that deals with situations where decisions are made in stages (i.e. different time steps), with the outcome of each decision being predictable to some extent before the next decision is made.

This quotation is taken from a point-of-view article that appeared in the Proceedings of the Institute of Electrical and Electronics Engineers (Haykin, 2006a). In retrospect, it is perhaps more appropriate to refer to Cognitive Dynamic Systems as an “integrative field” rather than a “discipline.”

By speaking of cognitive dynamic systems as an integrative field, we mean this in the sense that its study integrates many fields that are rooted in neuroscience, cognitive science, computer science, mathematics, physics, and engineering, just to name a few. Clearly, the mixture of fields adopted in the study depends on the application of interest. However, irrespective of the application, the key question is

In this book, we adopt human cognition as the frame of reference. As for applications, the book focuses on cognitive radar and cognitive radio.

In Chapter 1 of his book entitled Cortex and Mind: Unifying Cognition, Fuster (2003) introduces a new term called the cognit, which is used to characterize the cognitive structure of a cortical network; it is defined as follows:

Towards the end of Chapter 1 of the book, Fuster goes on to say:

These two quotes taken from Fuster's book provide us with a cohesive statement on the cortical functions of cognition in the human brain; both are important. In particular, the second quotation highlights the interrelationships between the five processes involved in cognition: perception, memory, attention, language, and intelligence. Just as language plays a distinctive role in the human brain, so it is in a cognitive dynamic system where language provides the means for effective and efficient communication among the different parts of the system. However, language is outside the scope of this book.

In my Point-of-View article, entitled “Cognitive dynamic systems,” Proceedings of the IEEE, November 2006, I included a footnote stating that a new book on this very topic was under preparation. At long last, here is the book that I promised then, over four years later.

Just as adaptive filtering, going back to the pioneering work done by Professor Bernard Widrow and his research associates at Stanford University, represents one of the hallmarks of the twentieth century in signal processing and control, I see cognitive dynamic systems, exemplified by cognitive radar, cognitive control, and cognitive radio and other engineering systems, as one of the hallmarks of the twenty-first century.

The key question is: How do we define cognition? In this book of mine, I look to the human brain as the framework for cognition. As such, cognition embodies four basic processes:

• perception–action cycle,

• memory,

• attention, and

• intelligence,

each of which has a specific function of its own. In identifying this list of four processes. I have left out language, the fifth distinctive characteristic of human cognition, as it is outside the scope of this book. Simply put, there is no better framework than human cognition, embodying the above four processes, for the study of cognitive dynamic systems, irrespective of application.

Interest in a new generation of engineering systems enabled with cognition, started with cognitive radio, a term that was coined by Mitola and McGuire (1999). In that article, the idea of cognitive radio was introduced within the software-defined radio (SDR) community. Subsequently, Mitola (2000) elaborated on a so-called “radio knowledge representation language” in his own doctoral dissertation. Furthermore, in a short section entitled “Research issues” at the end of his doctoral dissertation, Mitola went on to say the following:

Then, in Haykin (2005a), the first journal paper on cognitive radio, detailed expositions of signal processing, control, learning and adaptive processes, and game-theoretic ideas that lie at the heart of cognitive radio were presented for the first time. Three fundamental cognitive tasks, embodying the perception–action cycle of cognitive radio, were identified in that 2005 paper:

• radio-scene analysis of the radio environment performed in the receiver;

• transmit-power control and dynamic spectrum management, both performed in the transmitter; and

• global feedback, enabling the transmitter to act and, therefore, control data transmission across the forward wireless (data) channel in light of information about the radio environment fed back to it by the receiver.

Perception of the environment, viewed as a problem in spectrum estimation as discussed in Chapter 3, is most appropriate for applications where spectrum sensing of the environment is crucial to the application at hand; cognitive radio is one such important application. A distinctive aspect of spectrum estimation is the fact that it works directly on environmental measurements (i.e. observables). However, in many other environments encountered in the study of cognitive dynamic systems, perception of the environment boils down to state estimation, which is the focus of attention in this chapter.

We defer a formal definition of the state to Section 4.4, where the issue of state estimation is taken up. For now, it suffices to say that estimating the state of a physical environment is compounded by two practical issues:

1. (1) The state of the environment is hidden from the observer, with information about the state being available only indirectly through dependence of the observables (measurements) on the state.

2. (2) Evolution of the state across time and measurements on the environment are both corrupted by the unavoidable presence of physical uncertainties in the environment.

To tackle perception problems of the kind just described, the first step is to formulate a probabilistic model that accounts for the underlying physics of the environment. Logically, the model consists of a pair of equations:

• a system equation, which accounts for evolution of the environmental state across time, and

• a measurement equation, which describes dependence of the measurements on the state.

Introduction

Stackelberg games are increasingly important for informing real-world decision making, including a growing body of work that applies these techniques in security domains such as critical infrastructure protection (Bier, 2007; Sandler and D. G. A. M., 2003), computer networks (Alpcan and Basar, 2003; Nguyen and Basar 2009), and robot patrolling strategies (Agmon et al., 2009; Basilico, Gatti, and Amigoni, 2009; Gatti, 2008). Two software systems that use this type of game modeling are in use by the the Los Angeles International Airport (LAX) (Pita et al., 2008) and the Federal Air Marshals Service (FAMS) (Tsai et al., 2009) to assist with resource allocation decision. A key issue that has arisen in these applications is whether the models can accurately represent the uncertainty that domain experts have about the inputs used to construct the game models, including the preferences and capabilities of terrorist adversaries.

To apply game-theoretic reasoning, the first step in the analysis is to construct a precise game model. The typical approach (e.g., in the LAX and FAMS applications) is to construct a model using a combination of the available data and expert opinions. Unfortunately, the data is often limited or imprecise, especially in regard to information about the terrorist adversaries. For example, it can be difficult to predict precisely how attackers will weigh casualties, economic consequences, media exposure, and other factors when selecting targets.

Introduction

Transportation networks such as buses, trains, and airplanes carry millions of people per day and from to their destinations, making them a prime target for terrorists and extremely difficult for law enforcement agencies to protect. In 2001, the 9/11 attack on the World Trade Center in New York City with commercial airliners resulted in $27.2 billion of direct short term costs (Looney 2002) as well as a government-reported 2,974 lives lost. The 2004 Madrid commuter train bombings resulted in 191 lives lost, 1755 people wounded, and an estimated cost of 212 million Euros (Blanco et al., 2007). Finally, in the 2005 London subway and bus bombings, 52 innocent lives were lost, 700 others were injured, and the estimated economic cost was 2 billion pounds (Thornton, 2005).

In addition to preboarding security checkpoints patrols aboard the vehicles are another key defensive measure used by many organizations in these domains (Billante, 2003; Kenney, 1989). In all these networks, there are hundreds or thousands of vehicles to protect, making it difficult to create patrol schedules. Furthermore, since motivated aggressors will attempt to observe law-enforcement patterns and try to exploit the schedule, law-enforcement organizations have embraced the use of randomization in their scheduling practices. Also, in all these networks, it is not possible to simply assign law-enforcement personnel to vehicles in isolation, without considering the route the vehicle takes as well as departure and arrival times.

Introduction

Protecting the national infrastructure, such as airports, historical landmarks, or locations of political or economic importance, is a challenging task for police and security agencies around the world, one that is exacerbated by the threat of terrorism. The protection of important locations includes such tasks as monitoring all entrances or inbound roads and checking inbound traffic. However, limited resources mean that it is typically impossible to provide full security coverage at all times. Furthermore, adversaries can observe security arrangements over time and exploit any predictable patterns to their advantage. Randomizing schedules for patrolling, checking, or monitoring is thus an important tool in the police arsenal to avoid the vulnerability that comes with predictability. Even beyond protecting infrastructure, randomized patrolling is important in tasks varying from security on university campuses to normal police beats to border or maritime security (Billante 2003; Paruchuri et al. 2007; Ruan et al. 2005).

This paper focuses on a deployed software-assistant agent that can aid police or other security agencies in randomizing their security schedules. We face at least three key challenges in building such a software assistant. First, the assistant must provide quality guarantees in randomization by appropriately weighing the costs and benefits of the different options available. For example, if an attack on one part of an infrastructure would cause economic damage while an attack on another could potentially cost human lives, we must weigh the two options differently – giving higher weight (probability) to guarding the latter.

Introduction

Security, commonly defined as the ability to deal with intentional threats from other agents, is a major challenge for agents deployed in adversarial environments (Paruchuri et al., 2006). In this paper, we focus on adversarial domains in which the agents have limited information about the adversaries. Such adversarial scenarios arise in a wide variety of situations that are becoming increasingly important, such as patrol agents providing security for a group of houses or regions (Carroll et al., 2005; Paruchuri et al., 2007), UAVs monitoring a humanitarian mission (Beard and Mclain, 2003; Paruchuri et al., 2006), agents assisting in routine security checks at airports (Poole and Passantino, 2003), agents providing privacy in sensor network routing (Ozturk, Zhang, and Trappe, 2004), and agents maintaining anonymity in peer-to-peer networks (Borisov and Waddle, 2005).

This paper brings together some of our recent work on how to plan for agents acting in uncertain environments in the presence of adversaries (Paruchuri et al., 2006, 2007, 2008). This research has introduced two very different approaches to increasing security in agent systems and has lead to the ARMOR (Assistant for Randomized Monitoring over Routes) system, which has been deployed for security scheduling at the LAX airport since August 2007 (Murr, 2007; Paruchuri et al., 2008; Pita et al., 2008). Here we will present the main results and algorithms proposed in these two approaches and highlight the relationship between them.

Introduction

The history of plots and attacks on LAX suggests that it has achieved a level of importance in the minds of a diverse collection of terrorist organizations. The counterterrorism community is challenged by an intelligent and adaptive adversary who is prone to return to targets and continues to demonstrate sustained intent and an evolving attack methodology. LAX seems to present itself as an elusive trophy, as homeland security forces struggle, along with the rest of the world, to achieve and maintain a level of resistance capable of thwarting a man-enabled catastrophe.

“Over 5,000 deaths have resulted from terrorist attacks on civil aviation since 1980; about 200 deaths occurred in attacks on airports themselves, as opposed to aircraft.” Although the aviation domain has become the hardest critical infrastructure since the terror attack on September 11, 2001, it remains the most desirable for the attacker. There are essentially three fundamental strategies for attacking the aviation system – hijackings, bombings, and airport assaults. Security countermeasures designed to reduce the threat of in-flight incidents suggest attacks on airport facilities are probable.

A Brief History of Significant Events

Terrorism has long been a serious threat to the air transportation system of the United States and other nations. In 1972, three members of the Japanese Red Army initiated what is now known as the Lod Airport Massacre.