Monte Carlo is one of the most versatile and widely used numerical methods. Its convergence rate, O(N−1/2), is independent of dimension, which shows Monte Carlo to be very robust but also slow. This article presents an introduction to Monte Carlo methods for integration problems, including convergence theory, sampling methods and variance reduction techniques. Accelerated convergence for Monte Carlo quadrature is attained using quasi-random (also called low-discrepancy) sequences, which are a deterministic alternative to random or pseudo-random sequences. The points in a quasi-random sequence are correlated to provide greater uniformity. The resulting quadrature method, called quasi-Monte Carlo, has a convergence rate of approximately O((logN)kN−1). For quasi-Monte Carlo, both theoretical error estimates and practical limitations are presented. Although the emphasis in this article is on integration, Monte Carlo simulation of rarefied gas dynamics is also discussed. In the limit of small mean free path (that is, the fluid dynamic limit), Monte Carlo loses its effectiveness because the collisional distance is much less than the fluid dynamic length scale. Computational examples are presented throughout the text to illustrate the theory. A number of open problems are described.

This is a survey of nonlinear approximation, especially that part of the subject which is important in numerical computation. Nonlinear approximation means that the approximants do not come from linear spaces but rather from nonlinear manifolds. The central question to be studied is what, if any, are the advantages of nonlinear approximation over the simpler, more established, linear methods. This question is answered by studying the rate of approximation which is the decrease in error versus the number of parameters in the approximant. The number of parameters usually correlates well with computational effort. It is shown that in many settings the rate of nonlinear approximation can be characterized by certain smoothness conditions which are significantly weaker than required in the linear theory. Emphasis in the survey will be placed on approximation by piecewise polynomials and wavelets as well as their numerical implementation. Results on highly nonlinear methods such as optimal basis selection and greedy algorithms (adaptive pursuit) are also given. Applications to image processing, statistical estimation, regularity for PDEs, and adaptive algorithms are discussed.

It used to be good enough to bound absolute of matrix eigenvalues and singular values. Not any more. Now it is fashionable to bound relative errors. We present a collection of relative perturbation results which have emerged during the past ten years.

No need to throw away all those absolute error bound, though. Deep down, the derivation of many relative bounds can be based on absolute bounds. This means that relative bounds are not always better. They may just be better sometimes – and exactly when depends on the perturbation.

In this paper we review results on asymptotic stability of stationary states of PDEs. After scaling, our normal form is ut = Pu + ε f(u, ux,…) + F(x, t), where the (vector-valued) function u(x, t) depends on the space variable x and time t. The differential operator P is linear, F(x, t) is a smooth forcing, which decays to zero for t → ∞, and εf(u, …) is a nonlinear perturbation. We will discuss conditions that ensure u → 0 for t → ∞ when |ε| is sufficiently small. If this holds, we call the problem asymptotically stable.

While there are many approaches to show asymptotic stability, we mainly concentrate on the resolvent technique. However, comparisons with the Lyapunov technique will also be given. The emphasis on the resolvent technique is motivated by the recent interest in pseudospectra.

Many different procedures have been proposed for optimization calculations when first derivatives are not available. Further, several researchers have contributed to the subject, including some who wish to prove convergence theorems, and some who wish to make any reduction in the least calculated value of the objective function. There is not even a key idea that can be used as a foundation of a review, except for the problem itself, which is the adjustment of variables so that a function becomes least, where each value of the function is returned by a subroutine for each trial vector of variables. Therefore the paper is a collection of essays on particular strategies and algorithms, in order to consider the advantages, limitations and theory of several techniques. The subjects addressed are line search methods, the restriction of vectors of variables to discrete grids, the use of geometric simplices, conjugate direction procedures, trust region algorithms that form linear or quadratic approximations to the objective function, and simulated annealing. We study the main features of the methods themselves, instead of providing a catalogue of references to published work, because an understanding of these features may be very helpful to future research.

For the approximation of functions and data, it is often appropriate to minimize a norm. Many norms have been considered, and a review is presented of methods for solving a range of problems using a wide variety of norms.