Schulte-Geers and Stadje [Journal of Applied Probability, 2015, 52: 180–190] gave several closed form expressions for the exact distribution of the all-time maximal percentage in Pólya's urn model. But all these expressions corresponded to an integer parameter taking the value 1. Here, we derive much more general closed form expressions applicable for all possible values of the integer parameter. We also illustrate their computational efficiency.

This paper deals with two ideas appeared during the last developing phase in Artificial Intelligence: Reservoir Computing (RC) and Random Neural Networks. Both have been very successful in many applications. We propose a new model belonging to the first class, taking the structure of the second for its dynamics. The new model is called Echo State Queuing Network. The paper positions the model in the global Machine Learning area, and provides examples of its use and performances. We show on largely used benchmarks that it is a very accurate tool, and we illustrate how it compares with standard RC models.

There are n customers that need to be served by m parallel servers (n≥m). Customer i will only wait in queue for an exponentially distributed time with rate λi before departing the system. The service time on server j is exponentially distributed with rate μj for all customers, and upon completion of service of customer i a positive reward ri is earned. The non-preemptive problem is to choose, after each service completion, which currently in queue customer to serve next. The preemptive problem is to decide when to preempt a service, and to choose, after each service completion or preemption, which currently in queue customer to serve next. The objective of both problems is to maximize the expected total return. We give conditions under which a list policy is optimal for both problems.

This paper surveys research on mathematical models that predict the performance of digital devices that operate with intermittent energy sources. The approach taken in this work is based on the “Energy Packet Network” paradigm where the arrival of data to be processed or transmitted, and the energy to operate the system, are modeled as discrete random processes. Our assumption is that these devices will capture energy from intermittent ambient sources such as vibrations, heat or light, and capture it onto electrical energy that may be stored in batteries or capacitors. The devices consume this energy intermittently for processing and for wired or wireless transmission. Thus, both the arrival of energy to the device, and the devices workload, are modeled as random processes. Based on these assumptions, we discuss probability models based on Markov chains that can be used to predict the effective rates at which such devices operate. We also survey related work that models networks of such systems.

The knowledge gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision-making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including a propensity to take actions which are dominated with respect to both exploitation and exploration. We propose variants of KG which avoid such errors. These new policies include an index heuristic, which deploys a KG approach to develop an approximation to the Gittins index. A numerical study shows this policy to perform well over a range of MABs including those for which index policies are not optimal. While KG does not take dominated actions when bandits are Gaussian, it fails to be index consistent and appears not to enjoy a performance advantage over competitor policies when arms are correlated to compensate for its greater computational demands.

This paper discusses the stochastic monotonicity property of the conditional order statistics from independent multiple-outlier scale variables in terms of the likelihood ratio order. Let X1, …, Xn be a set of non-negative independent random variables with Xi, i=1, …, p, having common distribution function F(λ1x), and Xj, j=p+1, …, n, having common distribution function F(λ2x), where F(·) denotes the baseline distribution. Let Xi:n(p, q) be the ith smallest order statistics from this sample. Denote by $X_{i,n}^{s}(p,q)\doteq [X_{i:n}(p,q)|X_{i-1:n}(p,q)=s]$. Under the assumptions that xf′(x)/f(x) is decreasing in x∈ℛ+, λ1≤λ2 and s1≤s2, it is shown that $X_{i:n}^{s_{1}}(p+k,q-k)$ is larger than $X_{i:n}^{s_{2}}(p,q)$ according to the likelihood ratio order for any 2≤i≤n and k=1, 2, …, q. Some parametric families of distributions are also provided to illustrate the theoretical results.

This paper presents a probability model of an energy-harvesting wireless sensor node, with the objective of linking quality of sensed data to energy consumption and self-sustainability. The model departs from the common energy discretization framework used in the literature, and instead uses a diffusion process modulated by discrete packet arrival and transmission processes for the detailed representation of renewable energy supply, consumption and storage. An analytical–numerical method is developed to compute the average time until the node experiences an outage, due to lack of energy, for a given workload and ambient energy characteristics, battery capacity and initial charge. The results are illustrated with numerical examples.

The concept of Energy Packet Network (EPN) proposed by Gelenbe, is a new framework for modeling power grids that takes distributed energy generation such as renewable energy sources into consideration, and which contributes to modeling the smart grid. Based on G-network theory, this paper presents a simplified model of EPN and formulates energy-distribution as an optimization problem. We analyze it theoretically, and detail its optimal solutions. In addition to using existing optimization algorithms, a heuristic algorithm is proposed to solve for EPN optimization. The optimal solutions and efficacy of the algorithm are illustrated with numerical experiments. Further, we present an EPN with disconnections and a similar optimization problem is investigated. Optimal solutions are presented, and numerical results using the analytic optimal solutions, random solutions, a cooperative particle swarm optimizer and a heuristic algorithm illustrate the power of different approaches for solving energy-distribution problems using the EPN formalism.

G-Network queueing network models, and in particular the random neural network (RNN), are useful tools for decision making in complex systems, due to their ability to learn from measurements in real time, and in turn provide real-time decisions regarding resource and task allocation. In particular, the RNN has led to the design of the cognitive packet network (CPN) decision tool for the routing of packets in the Internet, and for task allocation in the Cloud. Thus in this paper, we present recent research on how to dynamically create the means for quality of service (QoS) to end users of the Internet and in the Cloud. The approach is based on adapting the decisions so as to benefit users as the conditions in the Internet and in Cloud servers vary due to changing traffic and workload. We present an overview of the algorithms that were designed based on the RNN, and also detail the experimental results that were obtained in three areas: (i) traffic routing for real-time applications, which have strict QoS constraints; (ii) routing approaches, which operate at the overlay level without affecting the Internet infrastructure; and (iii) the routing of tasks across servers in the Cloud through the Internet.

The paper provides an analysis of G-network with positive customers and signals when signals arriving to the system move customer to another system or destroy in it a group of customers, reducing their number to a random value that is given by a probability distribution. The signal arriving to the system, in which there are no positive customers, does not exert any influence on the queueing network and immediately disappears from it. Streams of positive customers and signals arriving to each of the network systems are independent. Customer in the transition from one system to another brings the latest some revenue, and the revenue of the first system is reduced by this amount. A method of finding the expected revenues of the systems of such a network has been proposed. The case when the revenues from transitions between network states are deterministic functions depending on its states has been considered. A description of the network is given, all possible transitions between network states, transition probabilities, and revenues from state transitions are indicated. A system of difference-differential equations for the expected revenues of network systems has been obtained. To solve it, we propose a method of successive approximations, combined with the method of series. It is proved that successive approximations converge to the stationary solution of such a system of equations, and the sequence of approximations converges to a unique solution of the system. Each approximation can be represented as a convergent power series with an infinite radius of convergence, the coefficients of which are related by recurrence relations. Therefore, it is convenient to use them for calculations on a PC. The obtained results can be applied in forecasting losses in information and telecommunication systems and networks from the penetration of computer viruses into it and conducting computer attacks.