This paper presents an overview on the role of agents in the Semantic Web, which was the topic of the AgentLink Technical Forum Group on ‘Semantic Web Agents’, aimed at fostering closer collaboration between the European communities working in these areas. The paper is structured in three main sections. In the first, we argue how agents are an essential component of the Semantic Web, then we provide a brief history, by no means comprehensive, of how the Semantic Web vision—that includes agents—has evolved over the past fifteen years. We then conclude, reporting on the topics presented and discussed during the Technical Forum.

The problem of traffic congestion incurs numerous social and economical repercussions and has thus become a central issue in every major city in the world. For this work we look at the transportation domain from a multiagent system perspective, where every driver can be seen as an autonomous decision-making agent. We explore how learning approaches can help achieve an efficient outcome, even when agents interact in a competitive environment for sharing common resources. To this end, we consider the route choice problem, where self-interested drivers need to independently learn which routes minimise their expected travel costs. Such a selfish behaviour results in the so-called user equilibrium, which is inefficient from the system’s perspective. In order to mitigate the impact of selfishness, we present Toll-based Q-learning (TQ-learning, for short). TQ-learning employs the idea of marginal-cost tolling (MCT), where each driver is charged according to the cost it imposes on others. The use of MCT leads agents to behave in a socially desirable way such that the is attainable. In contrast to previous works, however, our tolling scheme is distributed (i.e., each agent can compute its own toll), is charged a posteriori (i.e., at the end of each trip), and is fairer (i.e., agents pay exactly their marginal costs). Additionally, we provide a general formulation of the toll values for univariate, homogeneous polynomial cost functions. We present a theoretical analysis of TQ-learning, proving that it converges to a system-efficient equilibrium (i.e., an equilibrium aligned to the system optimum) in the limit. Furthermore, we perform an extensive empirical evaluation on realistic road networks to support our theoretical findings, showing that TQ-learning indeed converges to the optimum, which translates into a reduction of the congestion levels by 9.1%, on average.

This paper presents a parameterized gait generator based on linear inverted pendulum model (LIPM) theory, which allows users to generate a natural gait pattern with desired step sizes. Five types of zero moment point (ZMP) components are proposed for formulating a natural ZMP reference, where ZMP moves continuously during single support phases instead of staying at a fixed point in the sagittal and lateral plane. The corresponding center of mass (CoM) trajectories for these components are derived by LIPM theory. To generate a parameterized gait pattern with user-defined parameters, a gait planning algorithm is proposed, which determines related coefficients and boundary conditions of the CoM trajectory for each step. The proposed parameterized gait generator also provides a concept for users to generate gait patterns with self-defined ZMP references by using different components. Finally, the feasibility of the proposed method is validated by the experimental results with a teen-sized humanoid robot, David, which won first place in the sprint event at the 20th Federation of International Robot-soccer Association (FIRA) RoboWorld Cup.

Expert critiquing systems present an interesting and important complement to conventional expert systems. The basic idea is to emphasize the support for the user's own decision making rather than to let the system independently suggest a solution to a given problem. Thus it is assumed that the user initially proposes a decision or course of action. The system then reviews this suggestion relative to known circumstance, tries to evaluate the proposed solution, provides suggestions for improvements, draws attention to possible risks, indicates alternatives, and evaluates their merits, etc. A critiquing system is characterized both by its approach to problem solving and by its style of interaction with the user, as will be detailed below.

This paper explores the implications of research results in behavioural decision theory on knowledge engineering. Behavioural decision theory, with its performance (versus process) orientation, can tell us a great deal about the validity of human expert knowledge, and when it should be modelled. A brief history of behavioural decision theory is provided. Implications for knowledge elicitation and representation are discussed. An approach to knowledge engineering is proposed that takes into account these implications.

Mechanical engineering design is a broad subject area covering many topics and bas influences upon many other engineering disciplines and activities. Computer support for mechanical engineering design activity has been in draughting Systems and analysis packages, but there has been little in conceptual design assistance. This paper presents a number of areas of work in which AI techniques and developments are being used, sometimes in conjunction with traditional methods, to improve the support of design. The approaches to design and design Systems are covered, along with some techniques that are used. Specifie design Systems illustrate progress, and integration issues and simultaneous engineering Systems indicate the way research is moving. Finally, discussion of the trends and future topics indicates where and how effort may be applied in the future.

The concept of holonic systems has its roots in the desire to understand the structure of natural systems (e.g. living organisms and social organisations) and in particular their ability to behave in a stable yet flexible manner in the face of change. It is not surprising that the lessons learned from these natural systems could help with the design and control of complex man-made systems. However, a key issue is, how can one translate holonic concepts to real industrial environments? For example, one of the key holonic concepts, the holon, can be described as a self-contained autonomous and cooperative entity; when deciding how to implement holons, software agents appear to be the logical choice. In this paper, we summarise the presentations and discussions from a workshop held at the recent International Conference on Autonomous Agents that focused on this issue and brought together researchers from both the holonic systems and the multi-agents systems communities.

Automated modelling is a young research field and is attracting increasingly more attention. It is a cross-disciplinary field involving simulation, modelling, qualitative reasoning, bond graphs and systems dynamics. It is an investigation of the modelling process with the purpose of developing computer tools which will automatically follow modelling principles. In addition, these tools will take into account the details of an application and generate the most appropriate model for the application. Its objective is to develop computer modelling tools which will have perception of model correctness, completeness and appropriateness and can perform modelling automatically. One way to achieve this objective is to introduce well-defined models and automate the process of assembling submodels into models to create well-defined models. This paper reviews the motivation and background behind this new field, its theory and current state of the art, compares existing approaches and discusses the underlying issues. It is hoped that more researchers will become aware of this field and be encouraged to work in it.

The scope of the Technical Forum Group (TFG) on Agents in Bioinformatics (BIOAGENTS) was to inspire collaboration between the agent and bioinformatics communities with the aim of creating an opportunity to propose a different (agent-based) approach to the development of computational frameworks both for data analysis in bioinformatics and for system modelling in computational biology. During the day, the participants examined the future of research on agents in bioinformatics primarily through 12 invited talks selected to cover the most relevant topics. From the discussions, it became clear that there are many perspectives to the field, ranging from bio-conceptual languages for agent-based simulation, to the definition of bio-ontology-based declarative languages for use by information agents, and to the use of Grid agents, each of which requires further exploration. The interactions between participants encouraged the development of applications that describe a way of creating agent-based simulation models of biological systems, starting from an hypothesis and inferring new knowledge (or relations) by mining and analysing the huge amount of public biological data. In this report we summarize and reflect on the presentations and discussions.

Traditional approaches to plan representation have focused on the generation of a sequence of actions and orderings. Knowledge rich models, which incorporate plan rationale, provide benefits to the planning process in a number of ways. The use of rationale in planning is reviewed in terms of causality, dependencies, and decisions. Each dimension addresses practical issues in the planning process, and adds value to the resultant plan. The contribution of this paper is to explore this categorisation, and to motivate the need to explicitly record and represent rationale knowledge for situated, mixed-initiative planning systems.

Recent progress in reinforcement learning (RL) using self-play has shown remarkable performance with several board games (e.g., Chess and Go) and video games (e.g., Atari games and Dota2). It is plausible to hypothesize that RL, starting from zero knowledge, might be able to gradually approach a winning strategy after a certain amount of training. In this paper, we explore neural Monte Carlo Tree Search (neural MCTS), an RL algorithm that has been applied successfully by DeepMind to play Go and Chess at a superhuman level. We try to leverage the computational power of neural MCTS to solve a class of combinatorial optimization problems. Following the idea of Hintikka’s Game-Theoretical Semantics, we propose the Zermelo Gamification to transform specific combinatorial optimization problems into Zermelo games whose winning strategies correspond to the solutions of the original optimization problems. A specially designed neural MCTS algorithm is then introduced to train Zermelo game agents. We use a prototype problem for which the ground-truth policy is efficiently computable to demonstrate that neural MCTS is promising.

Conventional reinforcement learning focuses on problems with single objective. However, many problems have multiple objectives or criteria that may be independent, related, or contradictory. In such cases, multi-objective reinforcement learning is used to propose a compromise among the solutions to balance the objectives. TOPSIS is a multi-criteria decision method that selects the alternative with minimum distance from the positive ideal solution and the maximum distance from the negative ideal solution, so it can be used effectively in the decision-making process to select the next action. In this research a single-policy algorithm called TOPSIS Q-Learning is provided with focus on its performance in online mode. Unlike all single-policy methods, in the first version of the algorithm, there is no need for the user to specify the weights of the objectives. The user’s preferences may not be completely definite, so all weight preferences are combined together as decision criteria and a solution is generated by considering all these preferences at once and user can model the uncertainty and weight changes of objectives around their specified preferences of objectives. If the user only wants to apply the algorithm for a specific set of weights the second version of the algorithm efficiently accomplishes that.

Openness is a challenging property that may characterize multi-agent systems (MAS). It refers to their ability to deal with entities leaving and joining agent society over time. This property makes the MAS behaviour complex and difficult to study and analyze, hence the need for a representative model allowing its understanding. In this context, many models were defined in the literature and we propose to classify them into three categories: structural models, functional models and interactional models. The existing models were proposed either for representing structural openness or for modelling functional or interactional ones independently. But, none of them was oriented to represent MAS openness in a global way while considering its three aspects at once. Besides, each one was defined in order to realize a specific objective and in a particular domain of application. In this paper, we propose an evolving KAGR graph. The latter provides a common understanding of openness and unifies its structural, functional and interactional aspects in a generic way. Our model is finally tested and validated on a multi-agent rescue simulator.

Regulatory information has a direct and significant effect on the safety, economics and quality of many industries' operations and end-products. However, standards documents are often voluminous and can be complex. Users of such standards need to invest considerable time and effort to become familiar with the structure and content of the documents. Therefore, it has long been recognized that computer handling of this information is a potentially useful application area. This paper provides an overview of the work that has been done to provide computer systems that will help both users and authors of regulations. The emphasis is on using advanced symbolic computing techniques that would help in accessing, interpreting and applying the information.

This paper discusses the development of Information Presentation Systems (IPS). It identifies that these systems have had two development paths; generic and application-specific. It discusses the main pieces of early work, and identifies the significant problems and shortcomings associated with them. The development of later systems is traced through to the present day, and outstanding research issues are identified.

The techniques of qualitative reasoning are now becoming sufficiently mature to be applied to real world problems. In order to better understand which techniques are being used successfully for real world applications, and which application areas can be suitably addressed using qualitative reasoning techniques, it is helpful to have a summary of what application oriented work has been done to date. This helps to provide a picture of the application areas in which the techniques are being applied, and who is working in each application domain. In this paper, we summarize over 40 relevant projects.

An appropriately designed architecture of a neural network is essential to many realistic pattern-recognition tasks. A choice of just the right number of neurons, and their interconnections, can cut learning costs by orders of magnitude, and still warrant high classification accuracy. Surprisingly, textbooks often neglect this issue. A specialist seeking systematic information will soon realize that relevant material is scattered over diverse sources, each with a different perspective, terminology and goals. This brief survey attempts to rectify the situation by explaining the involved aspects, and by describing some of the fundamental techniques.

Potential-based reward shaping is a commonly used approach in reinforcement learning to direct exploration based on prior knowledge. Both in single and multi-agent settings this technique speeds up learning without losing any theoretical convergence guarantees. However, if speed ups through reward shaping are to be achieved in multi-agent environments, a different shaping signal should be used for each context in which agents have a different subgoal or when agents are involved in a different interaction situation.

This paper describes the use of context-aware potential functions in a multi-agent system in which the interactions between agents are sparse. This means that, unknown to the agents a priori, the interactions between the agents only occur sporadically in certain regions of the state space. During these interactions, agents need to coordinate in order to reach the global optimal solution.

We demonstrate how different reward shaping functions can be used on top of Future Coordinating Q-learning (FCQ-learning); an algorithm capable of automatically detecting when agents should take each other into consideration. Using FCQ-learning, coordination problems can even be anticipated before the actual problems occur, allowing the problems to be solved timely. We evaluate our approach on a range of gridworld problems, as well as a simulation of air traffic control.

Nakamoto proposed a new solution to transact value via the internet. And since 2009, blockchain technology has expanded and diversified. It has, however, proven to be inefficient in the way it achieves its outcomes, especially through the proof of work protocol. Other developers are promoting alternative methods but, as yet, none has superseded proof of work. The competing protocols illuminate a key feature of the blockchain community, namely, its inability to create consensus in a decentralized community. Because of this lack of consensus, the formation of standards is particularly difficult to achieve. At best standards are contested sites, and we examine three such sites where some form of agreement over standards will be essential if blockchain is to evolve successfully. These three sites are blockchain governance, smart contracts, and interoperability of blockchains. We argue that because standards’ formation is a contested and contingent process, the blockchain community will persist in creating difficulties and barriers for itself until it is able to resolve internal conflicts.

This paper deals with the implications of showing the storyteller’s perspective when telling stories in virtual environments. The paper proposes an open and reusable architecture for the construction of virtual guides who tell stories about the worlds they inhabit from their own perspective. The system consists of two main components: the structure of a knowledge base to code the narrative knowledge about the worlds and a novel hybrid algorithm for the generation of narratives showing the storyteller’s viewpoint. The complete architecture has been developed. Furthermore, we carried out a study with users in order to both validate the implemented storytelling system and test whether the users consider it significantly better or worse (in terms of several aspects related to believability) adding the virtual guide perspective (as both text and animations showing emotions) to the guide discourse. The interesting results are discussed.