6 - Density and Representation

Summary

In this chapter we consider the question of the density of linear combinations of ridge functions in the set of continuous functions on Rⁿ, in the topology of uniform convergence on compact subsets of Rⁿ. In Section 6.1 we consider a set of fixed directions and obtain necessary and sufficient conditions on the direction set for when we have density of linear combinations of all possible ridge functions with these directions. In Section 6.2 we discuss this same question with regard to generalized ridge functions. We discuss the question of density when permitting variable directions in Section 6.3. In Section 6.4 we ask for conditions on when a specific function in C(Rn) can be approximated by a linear combination of ridge function with given directions, without the density assumption. Much of the presentation of Sections 6.1, 6.2 and 6.4 is taken from Lin and Pinkus [1993]. In Section 6.5 we study a related but different question. Here we permit all possible directions, but restrict the set of permissible functions. We look at the class of ridge functions generated by shifts of a single function. This question is relevant in ridgelet analysis and in the MLP model in neural networks.

Density with Fixed Directions

In this section we consider the question of the density of linear combinations of ridge functions with a given set of directions. Let Ω denote any subset of vectors in Rⁿ, and set

M(Ω) := span{f(a · x) : a ∊ Ω, f ∊ C(R)}.

That is, in M(Ω) we vary over all a ∊ Ω and all f ∊ C(R), and take arbitrary linear combinations of the elements of this set. This is our approximating set. The question we ask is one of density: when can elements of the above set approximate arbitrarily well any function in our class?