Introduction: Standard versus Nonstandard Theories of Agents and Patients

Moral situations commonly involve agents and patients. let us define the class A of moral agents as the class of all entities that can in principle qualify as sources or senders of moral action, and the class P of moral patients as the class of all entities that can in principle qualify as receivers of moral action. A particularly apt way to introduce the topic of this paper is to consider how ethical theories (macroethics) interpret the logical relation between those two classes. There can be five logical relations between A and P; see Figure 12.1.

It is possible, but utterly unrealistic, that A and P are disjoint (alternative 5). On the other hand, P can be a proper subset of A (alternative 3), or A and P can intersect each other (alternative 4). These two alternatives are only slightly more promising because they both require at least one moral agent that in principle could not qualify as a moral patient. Now this pure agent would be some sort of supernatural entity that, like Aristotle's God, affects the world but can never be affected by it. Yet being in principle “unaffectable” and irrelevant in the moral game, it is unclear what kind of role this entity would exercise with respect to the normative guidance of human actions.

Introduction

Although traditional models of decision making in ai have focused on utilitarian theories, there is considerable psychological evidence that these theories fail to capture the full spectrum of human decision making (e.g. Kahneman and Tversky 1979; Ritov and Baron 1999). Current theories of moral decision making extend beyond pure utilitarian models by relying on contextual factors that vary with culture. In particular, research on moral reasoning has uncovered a conflict between normative outcomes and intuitive judgments. This has led some researchers to propose the existence of deontological moral rules; that is, some actions are immoral regardless of consequences, which could block utilitarian motives. Consider the starvation scenario (from Ritov and Baron [1999]) that follows:

A convoy of food trucks is on its way to a refugee camp during a famine in Africa. (Airplanes cannot be used.) You find that a second camp has even more refugees. If you tell the convoy to go to the second camp instead of the first, you will save one thousand people from death, but one hundred people in the first camp will die as a result.

Would you send the convoy to the second camp?

The utilitarian decision would send the convoy to the second camp, but 63 percent of participants did not divert the truck.

Making these types of decisions automatically requires an integrated approach, including natural language understanding, qualitative reasoning, analogical reasoning, and first-principles reasoning.

The Rational Architecture

As a basis for analysis, let us use a simplistic model of the workings of an AI mind. The model simply divides the thinking of the AI into two parts:

1. A world model (WM) contains the sum of its objective knowledge about the world and can be used to predict the effects of actions, plan actions to achieve given goals, and the like.

2. A utility function (UF) that establishes a preference between world states with which to rank goals.

In practice, the workings of any computationally realistic AI faced with real-world decisions will be intertwined, heuristic, and partial, as indeed are the workings of a human mind. At present, only programs dealing with limited, structured domains such as chess playing are actually formalized to the extent of separating the WM and the UF. However, it can be shown as one of the fundamental theorems of economics that any agent whose preference structure is not equivalent to a single real-valued total function of world states can be offered a series of voluntary transactions that will make it arbitrarily worse off – even by its own reckoning! To put it another way, any agent that doesn't act as if it had a coherent UF would be an incompetent decision maker. So as we increasingly use AIs for decisions that matter, we should try to build them to match the model as closely as possible as an ideal.

The first book report i ever gave, to mrs. slatin's first grade class in Lake, Mississippi, in 1961, was on a slim volume entitled You Will Go to the Moon. I have spent the intervening years thinking about the future.

The four decades that have passed have witnessed advances in science and physical technology that would be incredible to a child of any other era. I did see my countryman Neil Armstrong step out onto the moon. The processing power of the computers that controlled the early launches can be had today in a five-dollar calculator. The genetic code has been broken and the messages are being read – and in some cases, rewritten. Jet travel, then a perquisite of the rich, is available to all.

That young boy that I was spent time on other things besides science fiction. My father was a minister, and we talked (or in many cases, I was lectured and questioned!) about good and evil, right and wrong, and what our duties were to others and to ourselves.

In the same four decades, progress in the realm of ethics has been modest. Almost all of it has been in the expansion of inclusiveness, broadening the definition of who deserves the same consideration one always gave neighbors. I experienced some of this first hand as a schoolchild in 1960s Mississippi.

Introduction

In this paper i will argue that computer systems are moral entities but not, alone, moral agents. In making this argument I will navigate through a complex set of issues much debated by scholars of artificial intelligence, cognitive science, and computer ethics. My claim is that those who argue for the moral agency (or potential moral agency) of computers are right in recognizing the moral importance of computers, but they go wrong in viewing computer systems as independent, autonomous moral agents. Computer systems have meaning and significance only in relation to human beings; they are components in socio-technical systems. What computer systems are and what they do is intertwined with the social practices and systems of meaning of human beings. Those who argue for the moral agency (or potential moral agency) of computer systems also go wrong insofar as they overemphasize the distinctiveness of computers. Computer systems are distinctive, but they are a distinctive form of technology and have a good deal in common with other types of technology.

On the other hand, those who claim that computer systems are not (and can never be) moral agents also go wrong when they claim that computer systems are outside the domain of morality. To suppose that morality applies only to the human beings who use computer systems is a mistake.

Introduction

Robots have been a part of our work environment for the past few decades, but they are no longer limited to factory automation. The additional range of activities they are being used for is growing. Robots are now automating a wide range of professional activities such as: aspects of the health-care industry, white collar office work, search and rescue operations, automated warfare, and the service industries.

A subtle but far more personal revolution has begun in home automation as robot vacuums and toys are becoming more common in homes around the world. As these machines increase in capability and ubiquity, it is inevitable that they will impact our lives ethically as well as physically and emotionally. These impacts will be both positive and negative, and in this paper I will address the moral status of robots and how that status, both real and potential, should affect the way we design and use these technologies.

Morality and Human-Robot Interactions

As robotics technology becomes more ubiquitous, the scope of human-robot interactions will grow. At the present time, these interactions are no different than the interactions one might have with any piece of technology, but as these machines become more interactive, they will become involved in situations that have a moral character that may be uncomfortably similar to the interactions we have with other sentient animals.

Introduction

Intelligent robots must be both proactive and responsive. that requirement is the main challenge facing designers and developers of robot architectures. A robot in an active environment changes that environment in order to meet its goals and it, in turn, is changed by the environment. In this chapter we propose that these concerns can best be addressed by using constraint satisfaction as the design framework. This will allow us to put a firmer technical foundation under various proposals for codes of robot ethics.

Constraint Satisfaction Problems

We will start with what we might call Good Old-Fashioned Constraint Satisfaction (GOFCS). Constraint satisfaction itself has now evolved far beyond GOFCS. However, we initially focus on GOFCS as exemplified in the constraint satisfaction problem (CSP) paradigm. The whole concept of constraint satisfaction is a powerful idea. It arose in several applied fields roughly simultaneously; several researchers, in the early 1970s, abstracted the underlying theoretical model. Simply, many significant sets of problems of interest in artificial intelligence can each be characterized as a CSP. A CSP has a set of variables; each variable has a domain of possible values, and there are various constraints on some subsets of those variables, specifying which combinations of values for the variables involved are allowed (Mackworth 1977). The constraints may be between two variables or among more than two variables. A familiar CSP example is the Sudoku puzzle.

When our mobile robots are free-ranging critters, how ought they to behave? What should their top-level instructions look like?

The best known prescription for mobile robots is the Three Laws of Robotics formulated by Isaac Asimov (1942):

1. A robot may not injure a human being, or through inaction, allow a human being to come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second law.

Let's leave aside “implementation questions” for a moment. (No problem, Asimov's robots have “positronic brains”.) These three laws are not suitable for our magnificent robots. These are laws for slaves.

We want our robots to behave more like equals, more like ethical people. (See Figure 14.1.) How do we program a robot to behave ethically? Well, what does it mean for a person to behave ethically?

People have discussed how we ought to behave for centuries. Indeed, it has been said that we really have only one question that we answer over and over: What do I do now? Given the current situation what action should I take?

Colin allen, wendell wallach, and iva smit maintain in “why Machine Ethics?” that it is time to begin adding ethical decision making to computers and robots. They point out that “[d]riverless [train] systems put machines in the position of making split-second decisions that could have life or death implications” if people are on one or more tracks that the systems could steer toward or avoid. The ethical dilemmas raised are much like the classic “trolley” cases often discussed in ethics courses. “The computer revolution is continuing to promote reliance on automation, and autonomous systems are coming whether we like it or not,” they say. Shouldn't we try to ensure that they act in an ethical fashion?

Allen et al. don't believe that “increasing reliance on autonomous systems will undermine our basic humanity” or that robots will eventually “enslave or exterminate us.” However, in order to ensure that the benefits of the new technologies outweigh the costs, “we'll need to integrate artificial moral agents into these new technologies … to uphold shared ethical standards.” It won't be easy, in their view, “but it is necessary and inevitable.”

It is not necessary, according to Allen et al., that the autonomous machines we create be moral agents in the sense that human beings are. They don't have to have free will, for instance. We only need to design them “to act as if they were moral agents … we must be confident that their behavior satisfies appropriate norms.”

Introduction

In this paper i will discuss some of the broad philosophical issues that apply to the field of machine ethics. ME is often seen primarily as a practical research area involving the modeling and implementation of artificial moral agents. However this shades into a broader, more theoretical inquiry into the nature of ethical agency and moral value as seen from an AI or information-theoretical point of view, as well as the extent to which autonomous AI agents can have moral status of different kinds. We can refer to these as practical and philosophical ME respectively.

Practical ME has various kinds of objectives. Some are technically well defined and relatively close to market, such as the development of ethically responsive robot care assistants or automated advisers for clinicians on medical ethics issues. Other practical ME aims are more long term, such as the design of a general purpose ethical reasoner/advisor – or perhaps even a “genuine” moral agent with a status equal (or as equal as possible) to human moral agents.

In our early work on attempting to develop ethics for a machine, we first established that it is possible to create a program that can compute the ethically correct action when faced with a moral dilemma using a well-known ethical theory (Anderson et al. 2006). The theory we chose, Hedonistic Act Utilitarianism, was ideally suited to the task because its founder, Jeremy Bentham (1781), described it as a theory that involves performing “moral arithmetic.” Unfortunately, few contemporary ethicists are satisfied with this teleological ethical theory that bases the rightness and wrongness of actions entirely on the likely future consequences of those actions. It does not take into account justice considerations, such as rights and what people deserve in light of their past behavior; such considerations are the focus of deontological theories like Kant's Categorical Imperative, which have been accused of ignoring consequences. The ideal ethical theory, we believe, is one that combines elements of both approaches.

The prima facie duty approach to ethical theory, advocated by W.D. Ross (1930), maintains that there isn't a single absolute duty to which we must adhere, as is the case with the two aforementioned theories, but rather a number of duties that we should try to follow (some teleological and others deontological), each of which could be overridden on occasion by one of the other duties.

We describe and justify the use of a schema for contextualized attention metadata (CAM) and a framework for capturing and exploiting such data. CAM are data about computer-related activities and the foci of attention for computer users. As such, they are a prerequisite for the personalization of both information and task environments. We outline the possibilities of utilizing CAM, with a focus on technology-enhanced learning (TEL) scenarios, presenting the MACE system for architecture education as a CAM test bed.

Introduction

Contextualized attention metadata

The contextualized attention metadata (CAM) format, defined by an XML schema, is a format for data about the foci of attention and activities of computer users. Contextualized attention metadata describe which data objects attract the users' attention, which actions users perform with these objects and what the use contexts are. As such, they are a prerequisite for generating context-specific user profiles that help to personalize and optimize task and information environments. They can be employed for annotating data objects with information about their users and usages, thereby rendering possible object classifications according to use frequency, use contexts and user groups. Moreover, they can be crucial for supporting cooperative work: they may be utilized for monitoring distributed task processing, for identifying and sharing knowledge of critical information, and for bringing together working groups (Schuff et al. 2007; Hauser et al. 2009; Adomavicius and Tuzhilin 2005, among others).

Analogous to the introduction of colour displays, 3D displays hold the potential to expand the information that can be displayed without increasing clutter. This is in addition to the application of 3D technology to showing volumetric data. Going beyond colour, separation by depth has in the past been shown to enable very fast (‘parallel’) visual search (Nakayama and Silverman 1986), something that separation by colour alone does not do. The ability to focus attention exclusively on a depth plane provides a potentially powerful (and relatively practical) extension to command-and-control displays. Just one extra depth layer can declutter the display. For this reason we have developed a ‘Dual-layer’ display with two physically separated layers. As expected, conjunction search times become parallel when information is split into two depth layers but only when the stimuli are simple and non-overlapping; complex and overlapping imagery in the rear layer still interferes with visual search in the front layer. With the Dual-layer cockpit display, it is possible to increase information content significantly without substantially affecting ease-of-search. We show experimentally that the secondary depth cues (accommodation and parallax) boost this advantage.

We expect the primary ‘declutter’ market to lie in applications that do not tolerate the overlooking of crucial information, in environments that are space limited, and in mobile displays. Note that the use of 3D to declutter fundamentally differs from the use of 3D to show volumetric spatial relationships.

Human attention is a complex phenomenon (or a set of related phenomena) that occurs at different levels of cognition (from low-level perceptual processes to higher perceptual and cognitive processes). Since the dawn of modern psychology through cognitive sciences to fields like Human–Computer Interaction (HCI), attention has been one of the most controversial research topics. Attempts to model attentional processes often show their authors' implicit construal of related cognitive phenomena and even their overall meta-theoretical stands about what cognition is. Moreover, the modelling of attention cannot be done in isolation from related cognitive phenomena like curiosity, motivation, anticipation and awareness, to mention but a few. For these reasons we believe that attention models are best presented within a complete cognitive architecture where most authors' assumptions will be made explicit.

In this chapter we first present several attempts to model attention within a complete cognitive architecture. Several known cognitive architectures (ACT-R, Fluid Concepts, LIDA, DUAL, Novamente AGI and MAMID) are reviewed from the point of view of their treatment of attentional processes. Before presenting our own take on attention modelling, we briefly present the meta-theoretical approach of interactivism as advocated by Mark Bickhard.

We then give a description of a cognitive architecture that we have been developing in the last ten years. We present some of the cognitive phenomena that we have modelled (expectations, routine behaviour, planning, curiosity and motivation) and what parts of the architecture can be seen as involved in the attentional processes.