6 - Data smoothing: density estimation

Summary

The astronomical context

The goal of density estimation is to estimate the unknown probability density function of a random variable from a set of observations. In more familiar language, density estimation smooths collections of individual measurements into a continuous distribution, smoothing dots on a scatterplot by a curve or surface.

The problem arises in a wide variety of astronomical investigations. Galaxy or lensing distributions can be smoothed to trace the underlying dark matter distribution. Photons in an X-ray or gamma-ray image can be smoothed to visualize the X-ray or gamma-ray sky. Light curves from episodically observed variable stars or quasars can be smoothed to understand the nature of their variability. Star streams in the Galaxy's halo can be smoothed to trace the dynamics of cannibalized dwarf galaxies. Orbital parameters of Kuiper Belt Objects can be smoothed to understand resonances with planets.

Astronomical surveys measure properties of large samples of sources in a consistent fashion, and the objects are often plotted in low-dimensional projections to study characteristics of (sub)populations. Photometric color-magnitude and color-color plots are well-known examples, but parameters may be derived from spectra (e.g. emission-line ratios to measure gas ionization, velocity dispersions to study kinematics) or images (e.g. galaxy morphology measures). In these situations, it is often desirable to estimate the density for comparison with astrophysical theory, to visualize relationships between variables, or to find outliers of interest.

Concepts of density estimation

When the parametric form of the distribution is known, either from astrophysical theory or from a heuristic choice of some simple mathematical form, then the distribution function can be estimated by fitting the model parameters.