Published online by Cambridge University Press: 23 February 2017
Let $X$ be a vector space and let
$\unicode[STIX]{x1D711}:X\rightarrow \mathbb{R}\cup \{-\infty ,+\infty \}$ be an extended real-valued function. For every function
$f:X\rightarrow \mathbb{R}\cup \{-\infty ,+\infty \}$, let us define the
$\unicode[STIX]{x1D711}$-envelope of
$f$ by
$$\begin{eqnarray}f^{\unicode[STIX]{x1D711}}(x)=\sup _{y\in X}\unicode[STIX]{x1D711}(x-y)\begin{array}{@{}c@{}}-\\ \cdot \end{array}f(y),\end{eqnarray}$$
$\begin{array}{@{}c@{}}-\\ \\ \\ \cdot \end{array}$ denotes the lower subtraction in
$\mathbb{R}\cup \{-\infty ,+\infty \}$. The main purpose of this paper is to study in great detail the properties of the important generalized conjugation map
$f\mapsto f^{\unicode[STIX]{x1D711}}$. When the function
$\unicode[STIX]{x1D711}$ is closed and convex,
$\unicode[STIX]{x1D711}$-envelopes can be expressed as Legendre–Fenchel conjugates. By particularizing with
$\unicode[STIX]{x1D711}=(1/p\unicode[STIX]{x1D706})\Vert \cdot \Vert ^{p}$, for
$\unicode[STIX]{x1D706}>0$ and
$p\geqslant 1$, this allows us to derive new expressions of the Klee envelopes with index
$\unicode[STIX]{x1D706}$ and power
$p$. Links between
$\unicode[STIX]{x1D711}$-envelopes and Legendre–Fenchel conjugates are also explored when
$-\unicode[STIX]{x1D711}$ is closed and convex. The case of Moreau envelopes is examined as a particular case. In addition to the
$\unicode[STIX]{x1D711}$-envelopes of functions, a parallel notion of envelope is introduced for subsets of
$X$. Given subsets
$\unicode[STIX]{x1D6EC}$,
$C\subset X$, we define the
$\unicode[STIX]{x1D6EC}$-envelope of
$C$ as
$C^{\unicode[STIX]{x1D6EC}}=\bigcap _{x\in C}(x+\unicode[STIX]{x1D6EC})$. Connections between the transform
$C\mapsto C^{\unicode[STIX]{x1D6EC}}$ and the aforestated
$\unicode[STIX]{x1D711}$-conjugation are investigated.