12 - Kernel and More Independent Component Methods

Summary

As we know, there are known knowns; there are things we know we know. We also know, there are known unknowns, that is to say, we know there are some things we do not know. But there are also unknown unknowns, the ones we do not know we do not know (Donald Rumsfeld, Department of Defense news briefing, 12 February 2002).

Introduction

The classical or pre-2000 developments in Independent Component Analysis focus on approximating the mutual information by cumulants or moments, and they pursue the relationship between independence and non-Gaussianity. The theoretical framework of these early independent component approaches is accompanied by efficient software, and the FastICA solutions, in particular, have resulted in these approaches being recognised as among the main tools for calculating independent and non-Gaussian directions. The computational ease of FastICA solutions, however, does not detract from the development of other methods that find non-Gaussian or independent components. Indeed, the search for new ways of determining independent components has remained an active area of research.

This chapter looks at a variety of approaches which address the independent component problem. It is impossible to do justice to this fast-growing body of research; I aim to give a flavour of the diversity of approaches by introducing the reader to a number of contrasting methods. The methods I describe are based on a theoretical framework, but this does not imply that heuristically based approaches are not worth considering.