QUALITATIVE UNCERTAINTY PRINCIPLE ON CERTAIN LIE GROUPS

Abstract

In this article, we study the recent development of the qualitative uncertainty principle on certain Lie groups. In particular, we consider that if the Weyl transform on certain step-two nilpotent Lie groups is of finite rank, then the function has to be zero almost everywhere as long as the nonvanishing set for the function has finite measure. Further, we consider that if the Weyl transform of each Fourier–Wigner piece of a suitable function on the Heisenberg motion group is of finite rank, then the function has to be zero almost everywhere whenever the nonvanishing set for each Fourier–Wigner piece has finite measure.

1 Introduction

The uncertainty principle for the Fourier transform states that a function and its Fourier transform cannot be concentrated simultaneously. Let $L_f$ denote the set of Lebesgue points of $f\in L^1(\mathbb R^d).$ We call $A_f=\{x\in \mathbb R^d\cap L_f:\, f(x)\neq 0\}$ the nonvanishing set for $f.$ It is well known that if $f\in L^1(\mathbb R^d),$ then almost all points are Lebesgue points of $f.$ Therefore, without loss of generality, we write $A_f=\{x\in \mathbb R^d:\, f(x)\neq 0\}$ as the nonvanishing set for $f.$ A finer version of the uncertainty principle emphasizes that the nonvanishing sets of a nonzero function and its Fourier transform cannot be of finite measure simultaneously. In [4], Benedicks studied the ‘qualitative uncertainty principle (QUP)’ on the Euclidean space $\mathbb R^n,$ and therefore extended the classical Paley–Wiener theorem for the compactly supported functions to the class of integrable functions on $\mathbb R^n.$ More precisely, for $f\in L^1(\mathbb R^n),$ let $A=\{x\in \mathbb R^n: f(x)\neq 0\}$ and ${B=\{\xi \in \mathbb R^n : \hat f(\xi )\neq 0\}.}$ If $0<m(A)m(B)<\infty ,$ then $f=0$ almost everywhere (a.e.), where m stands for the Lebesgue measure on $\mathbb R^n.$ In [1], Amrein and Berthier proved the same result with some implications in a slightly different way by employing Hilbert space theory.

In 1997, Arnal and Ludwig [2] extended the notion of QUP to unimodular groups. Let G be a unimodular group and $\widehat G$ be the unitary dual of $G.$ Let $\nu $ and $\widehat \nu $ denote the Haar measure and Plancherel measure on G and $\widehat G,$ respectively. For $f\in L^1(G),$ if $\nu \{x\in G:f(x)\neq 0\}<\nu (G)$ and $\int _{\widehat G}\text {rank}(\pi (f))d\widehat \nu <\infty ,$ then $f=0$ a.e. Let us denote $\mathbb {R}^\ast =\mathbb{R}\smallsetminus \{0\}.$ Then for the Heisenberg group $\mathbb H^n$ (see [24]), the above conditions from [2] boil down to the fact that the set $\{\lambda \in \mathbb R^\ast :\hat f(\lambda )\neq 0\}$ has finite Plancherel measure in addition to almost all $\hat f(\lambda )$ having finite rank. In [16], the authors obtained a result on the Heisenberg group for partially compactly supported functions whose Fourier transform is of finite rank. Later, Vemuri [26] relaxed the partial compact support condition to finite Lebesgue measure of the nonvanishing set. However, the above problem is of further interest if $f\in L^1(\mathbb H^n)$ is nonvanishing on a subset of $\mathbb H^n$ having finite Haar measure while $\hat f(\lambda )$ satisfies some appropriate general conditions. We are still clueless about this question. However, instead of this, a series of analogous results related to Benedicks’ theorem has been obtained in various contexts, including the Heisenberg group as well as the Euclidean motion group, see, for example [3, 9, 17, 19, 22]. There are many formulations of the uncertainty principle, and we refer to an excellent survey article by Folland and Sitaram [7] and also the monograph by Thangavelu [25].

We consider analogous results on certain step two nilpotent Lie groups related to the Amrein–Berthier–Benedicks theorem in the sense of Narayanan and Ratnakumar [16]. That is, if the Weyl transform on step-two nilpotent Lie groups of $MW$ -type is of finite rank, then the function has to be zero almost everywhere as long as the nonvanishing set for the function has finite measure.

It is well known that $(\mathbb H^n\rtimes U(n), U(n))$ is a Gelfand pair (for instance, see [5]). So, the Fourier transform of a nonzero $U(n)$ -bi-invariant integrable function f on $\mathbb H^n\rtimes U(n)$ has rank one, irrespective of the support of $f.$ Thus, an exact analog of the Benedicks-type theorem on the Heisenberg group, as in [16], is not possible for the Heisenberg motion group. However, a close observation reveals that the Fourier–Wigner representation of the function thinly conflicts with the Peter–Weyl representation in the following sense. That is, if $g\in L^2(\mathbb C^n\rtimes U(n))$ is $U(n)$ -bi-invariant, then g need not fall into the trivial Fourier–Wigner representation in Equation (4-5) as compared with the Gelfand pair argument. This has been illustrated in a one-dimensional Heisenberg motion group $\mathbb H^1\rtimes U(1)$ by Ghosh and Srivastava [8]. Thus, if the Weyl transform of each Fourier–Wigner piece of a suitable function on the Heisenberg motion group is of finite rank, then the function has to be zero almost everywhere as long as the nonvanishing set for each Fourier–Wigner piece has finite measure. In the case of the Heisenberg motion group, Ghosh and Srivastava [8] have proved a similar result using Hilbert space theory, however, the proof in this article is different and we draw some comparisons through Remark 4.10.

We organize the paper as follows. In Section 2, we study a version of Benedicks’ theorem for the Weyl transform on the certain step two nilpotent Lie groups introduced by Moore and Wolf. In Section 3, we recall the necessary preliminaries regarding group Fourier transform and the Plancherel formula on the Heisenberg motion group. Finally, in Section 4, we explore the qualitative uncertainty principle for the Heisenberg motion group.

2 Uniqueness results on certain step two nilpotent Lie group

2.1 Preliminaries

In this section, we study an analogous result of [16, Theorem 2.2] of Benedicks’ theorem for the Weyl transform on certain step-two nilpotent Lie groups introduced by Moore and Wolf. A typical example of these groups are Métivier groups (see [14]). When Métivier groups are quotiented by the hyperplane in the center, they become Heisenberg groups. The Heisenberg-type groups introduced by Kaplan (see [12]) are examples of Métivier groups. However, there are Métivier groups which are distinct from the Heisenberg-type (or H-type) groups. For more details, see [15].

Let G be a connected, simply connected, step-two nilpotent Lie group whose Lie algebra $\mathfrak g$ has the orthogonal decomposition $\mathfrak g=\mathfrak b\oplus {\mathfrak {z}}$ with $[\mathfrak b,\mathfrak b]\subset {\mathfrak {z}}$ and ${[\mathfrak g,[\mathfrak g,\mathfrak g]]=\{0\},}$ where ${\mathfrak {z}}$ is the center of $\mathfrak g.$ Since $\mathfrak g$ is nilpotent, the exponential map $\exp :\mathfrak g\rightarrow G$ is surjective. Thus, G can be expressed using exponential coordinates.

Consider an orthonormal basis $\{V_1,\ldots , V_m, Z_1,\ldots , Z_k\}$ such that $\mathfrak b$ is spanned by $ \{V_1,\ldots , V_m\}$ over $\mathbb R$ and ${\mathfrak {z}}$ is spanned by $ \{Z_1,\ldots ,Z_k\}$ over $\mathbb R.$ For any element $g\in G,$ we can identify it with a point $V+Z\in \mathfrak b\oplus {\mathfrak {z}}$ so that $g=\exp (V+Z)$ and denote it by $(V,Z).$ Since $[\mathfrak b,\mathfrak b]\subset {\mathfrak {z}}$ and $[\mathfrak g,[\mathfrak g,\mathfrak g]]=\{0\},$ by the Baker–Campbell–Hausdorff formula, the group law on G can be expressed by

$$ \begin{align*} (V,Z)(V',Z')=(V+V',Z+Z'+({1}/{2})[V,V']) \end{align*} $$

for all $V+Z, V'+Z'\in \mathfrak b\oplus {\mathfrak {z}}.$ Let $dV$ and $dZ$ be the Lebesgue measures on $\mathfrak b$ and ${\mathfrak {z}}$ , respectively. Then the left-invariant Haar measure on G can be expressed by ${dg=dVdZ.}$

Let ${\mathfrak {z}}^\ast $ denote the dual of ${\mathfrak {z}}.$ Next, for every $\omega \in {\mathfrak {z}}^\ast ,$ consider the skew-symmetric bilinear form $B_\omega $ on $\mathfrak b$ by letting

$$ \begin{align*} B_\omega(X,Y)=\omega([X,Y]). \end{align*} $$

Let $m_\omega $ be the orthogonal complement of $s_\omega =\{X\in \mathfrak b:B_\omega (X,Y)=0~\text { for all }~ Y\in \mathfrak b\}$ in  $\mathfrak b.$ Then $B_\omega $ is called a nondegenerate bilinear form when $s_\omega $ is trivial. Let ${\Lambda =\{\omega \in {\mathfrak {z}}^\ast : \dim m_\omega ~\text {is maximum}\},}$ which is a Zariski open subset of ${\mathfrak {z}}^\ast .$ If $B_\omega $ is nondegenerate for all $\omega \in \Lambda ,$ then G is called an MW group.

Since $m_\omega $ is invariant under the skew-symmetric bilinear form $B_\omega ,$ it follows that the dimension of $m_\omega $ is even; let $\text {dim}~m_\omega =2n.$ Then there exists an orthonormal almost symplectic basis $\{X_i(\omega ),Y_j(\omega ): i=1,\ldots ,n\}$ of $\mathfrak b$ and $d_i(\omega )>0$ such that

$$ \begin{align*}\omega[X_i(\omega),Y_j(\omega)]= \begin{cases} \delta_{ij}d_i(\omega) & \mbox{when } X_i\neq Y_j;\\ 0 & \mbox{otherwise}. \end{cases} \end{align*} $$

Let $\zeta _\omega =\text {span}\{X_i(\omega ): i=1,\ldots ,n\}$ and $\eta _\omega =\text {span}\{Y_j(\omega ): j=1,\ldots ,n\}$ be two real vector spaces. Then we can write $\mathfrak b=\zeta _\omega \oplus \eta _\omega $ and each $(X, Y, Z)\in G$ can be represented by

$$ \begin{align*}(X, Y, Z)=\sum_{i=1}^nx_i(\omega)X_i(\omega)+\sum_{i=1}^ny_i(\omega)Y_i(\omega)+\sum_{i=1}^kt_i(\omega)Z_i(\omega),\end{align*} $$

where $x=(x_1,\ldots ,x_n)$ , $y=(y_1,\ldots ,y_n)$ are in $\mathbb R^n$ and $t=(t_1,\ldots ,t_k)\in \mathbb R^k.$ Hence, a typical element $(X, Y, Z)\in G$ can be identified with $(x,y,t),$ where $x,y\in \mathbb R^n$ and $t\in \mathbb R^k.$ That is, in these coordinates, G can be realized as $\mathbb R^{2n+k}.$ Moreover, the left-invariant Haar measure $dg$ on G can be expressed by $dg=dx\,dy\,dt$ , which is a Lebesgue measure on $\mathbb R^{2n+k}.$ For more details, we refer to [6, 13, 14, 20].

Next, we briefly introduce the unitary irreducible representation of the MW group $G.$ There are two types of representations of $G.$ The trivial representations on $\text {exp}~{\mathfrak {z}}$ can be parameterized by $\mathfrak b^\ast $ , which are basically scalars, and the nontrivial representations of G on $\text {exp}~\mathfrak b$ can be parameterized by $\Lambda $ , which appears in the Plancherel formula. More precisely, each $\omega \in \Lambda $ induces an irreducible unitary representation $\pi _{\omega }$ of G on $L^2(\eta _\omega )$ by letting

$$ \begin{align*} (\pi_{\omega}(x,y,t)\phi)(\xi)=e^{i\sum_{j=1}^k\omega_jt_j +i\sum_{j=1}^n d_j(\omega)(x_j\xi_j+({1}/{2})x_jy_j)}\phi(\xi+y), \end{align*} $$

where $\phi \in L^2(\eta _\omega ).$ We write $v=(x,y).$ The group Fourier transform of $f\in L^1(G)$ is defined by

$$ \begin{align*} \hat f(\omega)=\int_{{\mathfrak{z}}}\int_{\mathfrak{b}}f(v,t)\pi_{\omega}(v,t)\,dv\,dt, \end{align*} $$

where $\omega \in \Lambda .$ The Fourier inversion of f in the variable t is given by $f^\omega (v)=\int _{{\mathfrak {z}}}e^{i\sum _{j=1}^k\omega _jt_j}f(v,t)dt.$ Then for suitable functions f and g on $\mathfrak b,$ we can define the $\omega $ -twisted convolution of f and g by

$$ \begin{align*} f\ast_\omega g(v)=\int_b f(v-v')g(v')e^{({i}/{2})\omega([v,v'])}\,dv'. \end{align*} $$

It is immediate that $(f\ast g)^\omega =f^\omega \ast _\omega g^\omega .$ Let $p(\omega ):=\Pi _{i=1}^nd_i(\omega )$ be the symmetric function of degree n corresponding to $B_\omega .$ For any $f\in L^1\cap L^2(G),$ the Fourier transform $\hat f(\omega )$ is a Hilbert–Schmidt operator that satisfies

$$ \begin{align*} p(\omega)\|\hat f(\omega)\|_{\mathrm{HS}}^2=(2\pi)^{n}\int_{\mathfrak b}|f^\omega(v)|^2\,dv. \end{align*} $$

If we write $\pi _{\omega }(v)=\pi _{\omega }(v,o),$ the function $f^\omega $ can be recovered from the identity

$$ \begin{align*}f^\omega(v)=(2\pi)^{-n}p(\omega)~\text{tr}(\pi_{\omega}(v)^\ast\hat f(\omega)).\end{align*} $$

2.2 QUP for the Weyl transform

For $\omega \in \Lambda $ and $h\in L^1\cap L^2(\mathfrak b),$ the Weyl transform $W_{\omega }(h)$ is defined by

$$ \begin{align*} W_{\omega}(h)=\int_{\mathfrak b}h(v)\pi_{\omega}(v)\,dv. \end{align*} $$

It is well known that $W_{\omega }(h)$ is a Hilbert–Schmidt operator on $L^2(\eta _\omega )$ which satisfies the following Plancherel formula (see [18] for more general cases). For $h\in L^2(\mathfrak b),$ the following equality holds:

$$ \begin{align*} \|W_{\omega}(h)\|_{\mathrm{HS}}^2=c(\omega)\int_{\mathfrak b}|h(v)|^2\,dv, \end{align*} $$

where $c(\omega )=(2\pi )^{n}~p(\omega )^{-1}.$ Notice that for $h\in L^1\cap L^2(\mathfrak b),$ we have that $W_{\omega }(h^\ast )=W_{\omega }(h)^\ast $ and hence $W_{\omega }(h^\ast \ast _\omega h) =W_{\omega }(h)^\ast W_{\omega }(h),$ where $h^\ast (v)=\overline {h(v^{-1})}.$

Next, we recall that for $\phi ,\psi \in L^2(\mathbb {R}^n),$ the Fourier–Wigner transform of $\phi , \psi $ is a function on $\mathbb C^n$ and is defined by

$$ \begin{align*} T(\phi,\psi)(z)=\langle\pi(z)\phi,\psi\rangle. \end{align*} $$

It is observed in [10, 11] that Fourier–Wigner transforms of nontrivial functions will never be nonvanishing on a set of finite Lebesgue measure in $\mathbb {R}^{2n}.$ This in turn (as noted in [16]), implies that if the Weyl transform of function $F\in L^1(\mathbb C^n)$ is of rank one, then the function has to be zero almost everywhere as long as the nonvanishing set for the function has finite measure.

Theorem 2.1 [10, 11].

For $\phi ,\psi \in L^2(\mathbb {R}^n),$ write $X=T(\phi ,\psi ).$ If $\{z\in \mathbb C^n:~X(z)~\neq ~0\}$ has finite Lebesgue measure, then X is zero almost everywhere.

By abuse of notation, we use the same notation for the Fourier–Wigner transform on $G.$ For $\phi ,\psi \in L^2(\eta _\omega ),$ the Fourier–Wigner transform of $\phi $ and $\psi $ is a function on $\mathfrak b$ defined by

$$ \begin{align*}T(\phi,\psi)(v)=\langle\pi_{\omega}(v)\phi,\psi\rangle.\end{align*} $$

Then the following orthogonality relation holds (see Wolf [27]).

Lemma 2.2 [27].

Let $\phi _j,\psi _j\in L^2(\eta _\omega )$ be such that $T(\phi _j,\psi _j);~j=1,2$ are square integrable on $\mathfrak b.$ Then

$$ \begin{align*} \int_{\mathfrak b}T(\phi_1,\psi_1)(v)\overline{T(\phi_2,\psi_2)(v)}\,dv= c(\omega)\langle \phi_1,\phi_2 \rangle\overline{\langle \psi_1,\psi_2 \rangle}. \end{align*} $$

We observe that the functions $T(\phi ,\psi )$ form an orthonormal basis for $L^2(\mathfrak b).$ Let $\{\varphi _j:j\in \mathbb N\}$ be an orthonormal basis for $L^2(\eta _\omega ).$

Proposition 2.3. The set $\{T(\varphi _i,\varphi _j):i,j\in \mathbb N\}$ is an orthonormal basis for $L^2(\mathfrak b).$

Proof. It is immediate from Lemma 2.2 that $\{T(\varphi _i,\varphi _j):i,j\in \mathbb N\}$ is an orthonormal set. Now, it only remains to verify the completeness. For this, let $f\in L^2(\mathfrak b)$ be such that $\langle f,T(\varphi _i,\varphi _j)\rangle =0,$ whenever $i,j\in \mathbb N.$ A simple calculation shows that

$$ \begin{align*} \langle W_{\omega}(\bar f)\phi_i,\phi_j\rangle = \langle f,T(\phi_i,\phi_j)\rangle=0 \end{align*} $$

and hence $W_{\omega }(\bar f)=0.$ Thus, by the Plancherel formula, f is zero almost everywhere.

The following analog of Theorem 2.1 for the Fourier–Wigner transform on G holds true.

Proposition 2.4. Let $F=T(\phi ,\psi ),$ where $\phi ,\psi \in L^2(\eta _\omega ).$ If the set $\{v\in \mathfrak b: F(v)\neq 0\}$ has finite Lebesgue measure, then F is zero almost everywhere.

Proof. The proof of Proposition 2.4 is similar to the proof of Theorem 2.1 and hence we omit it here.

Let $\mathcal E$ and $\mathcal F$ be measurable subsets of $\zeta _\omega $ and $\eta _\omega $ , respectively, such that $0<m(\mathcal E)m(\mathcal F)<\infty .$ Denote $\Sigma =\mathcal E\times \mathcal F.$ The following result is crucial in proving Theorem 2.6.

Lemma 2.5. For $h_j\in L^2(\eta _\omega ),$ write $K_y(\xi )=\sum \nolimits _{j=1}^N h_j(\xi +y)\overline {h_j(\xi )},$ where $y\in \eta _\omega .$ If $K_y(\xi )=0$ for all $y\in \eta _\omega \smallsetminus \mathcal F$ and for almost all $\xi \in \eta _\omega ,$ then each $h_j$ is nonvanishing on a set of finite measure.

Proof. Since $h_j\in L^2(\eta _\omega ),$ the functions $|h_j|$ are finite almost everywhere on $\eta _\omega .$ Define a function $\chi $ on $\eta _\omega $ by $\chi =(h_1,\ldots ,h_N).$ Then, we get that

$$ \begin{align*} K_y(\xi)=\langle\chi(\xi+y),\chi(\xi)\rangle_{\mathbb C^N} \end{align*} $$

for almost every $\xi \in \eta _\omega .$ By assumption, $K_y=0$ for all $y\in \eta _\omega \smallsetminus \mathcal F.$ Thus, it follows that

$$ \begin{align*} \langle\chi(\xi+y),\chi(\xi)\rangle_{\mathbb C^N}=0 \end{align*} $$

for almost every $\xi \in \eta _\omega $ and $y\in \eta _\omega \smallsetminus \mathcal F.$ In contrast, assume that the nonvanishing set $S:=\{\zeta \in \mathbb R^n:\, \chi (\zeta )\neq 0\}$ has infinite Lebesgue measure. Since $\mathcal F$ has finite measure, there exists $v_1\in S\cap (\eta _\omega \smallsetminus \mathcal F).$ Observe that $\chi (v_1)\neq 0$ and

$$ \begin{align*} \langle\chi(\xi+v_1),\chi(\xi)\rangle_{\mathbb C^N}=0. \end{align*} $$

Next, take $v_2\in S\cap (\eta _\omega \smallsetminus (\mathcal F+v_1)),$ then $\chi (v_2)\neq 0,$ and since $v_2-v_1\notin \mathcal F,$ it is immediate that $\langle \chi (\xi +v_2-v_1),\chi (\xi )\rangle _{\mathbb C^N}=0.$ In particular, $\langle \chi (v_2),\chi (v_1)\rangle _{\mathbb C^N}=0.$ In this way, after m steps, we get that $\{v_j:~1\leq j\leq m\}$ such that $\chi (v_j)\neq 0$ and

(2-1) $$ \begin{align} \langle\chi(\xi+v_j-v_{j'}),\chi(\xi)\rangle_{\mathbb C^N}=0\quad\text{for}~1\leq j\neq j'\leq m. \end{align} $$

If we consider $v_{m+1}\in S\cap (\eta _\omega \smallsetminus \bigcup \limits _{j=1}^{m}(\mathcal F+v_j)),$ then $\chi (v_{m+1})\neq 0$ and for $j\leq m,$

(2-2) $$ \begin{align} \langle\chi(\xi+v_{m+1}-v_j),\chi(\xi)\rangle_{\mathbb C^N}=0. \end{align} $$

In particular, taking $\xi =v_{j'}$ in Equation (2-1) and $\xi =v_{j}$ in Equation (2-2),

$$ \begin{align*}\langle\chi(v_j),\chi(v_{j'})\rangle_{\mathbb C^N}=0\quad\text{for}~1\leq j\neq j'\leq m+1.\end{align*} $$

For $m=N,$ we obtain $N+1$ nonzero mutually orthogonal vectors in $\mathbb C^N,$ which is a contradiction. It follows that S must have finite measure and hence all $h_j$ terms are nonvanishing on $S.$

Next, we prove our main result of this section which is motivated by [16, Theorem 2.2].

Theorem 2.6. Let $h\in L^1\cap L^2(\mathfrak b)$ be nonvanishing on $\Sigma $ in $\mathfrak b.$ If $~W_{\omega }(h)$ is a finite rank operator, then $h=0$ a.e.

Proof. Let $\bar \tau =h^\ast \ast _\omega h,$ where $h^\ast (v)=\overline {h(v^{-1})}.$ Then $W_\omega (\bar \tau )=W_\omega (h)^\ast W_\omega (h)$ is a positive and finite rank operator on $L^2(\eta _\omega ).$ By the spectral theorem, there exists an orthonormal set $\{\phi _j\in L^2(\eta _\omega ): j=1,\ldots ,N\}$ and scalars $a_j\geq 0$ such that

$$ \begin{align*}W_{\omega}(\bar\tau)\phi=\sum_{j=1}^Na_j\langle \phi,\phi_j\rangle \phi_j,\end{align*} $$

whenever $\phi \in L^2(\eta _\omega ).$ Now, for $\psi \in L^2(\eta _\omega ),$ the orthogonality relation gives

(2-3) $$ \begin{align} \langle W_{\omega}(\bar\tau)\phi,\psi\rangle &=\sum_{j=1}^Na_j\langle\phi,\phi_j\rangle\langle\phi_j,\psi\rangle\nonumber\\ &= c(\omega)^{-1}\sum_{j=1}^Na_j\int_{\mathfrak b}T(\phi,\psi)(v)\overline{T(\phi_j,\phi_j)(v)}\,dv. \end{align} $$

Further, by definition of $W_{\omega }(\bar \tau ),$

(2-4) $$ \begin{align} \langle W_{\omega}(\bar\tau)\phi,\psi\rangle=\int_{\mathfrak b}\bar\tau(v)T(\phi,\psi)(v)\,dv. \end{align} $$

Hence, by comparing Equation (2-3) with Equation (2-4) in view of Proposition 2.3, it follows that

(2-5) $$ \begin{align} \tau=\sum\limits_{j=1}^N T(h_j,h_j), \end{align} $$

where $h_j=c(\omega )^{-1/2}\sqrt {a_j}~\phi _j\in L^2(\eta _\omega ).$ Now, for $v=(x,y),$ write $\tau _y(x)=\tau (x,y).$ Then Equation (2-5) becomes

$$ \begin{align*} \tau_y(x)=\int_{\eta_\omega}e^{i\sum\nolimits_{j=1}^nd_j(\omega) (x_j\xi_j+({1}/{2})x_jy_j)}K_y(\xi)\,d\xi. \end{align*} $$

Since $\bar \tau $ is nonvanishing on $\mathcal E\times \mathcal F$ , it follows that $K_y(\xi )=0$ for almost every $\xi $ and for all $y\in \eta _\omega \smallsetminus \mathcal F.$ Then in view of Lemma 2.5, it follows that each $h_j$ is nonvanishing on a set of finite measure and hence each $K_y$ is nonvanishing on a set of finite measure. Since $\tau _y$ is nonvanishing on $\mathcal E,$ whenever $y\in \eta _\omega ,$ we infer that $\tau _y$ is zero for all $y\in \eta _\omega .$ Now, by the Plancherel formula, we conclude that $h=0.$ This completes the proof.

Remark 2.7

1. (i) For the rank-one case (that is, $N=1$ ), instead of taking $\Sigma =\mathcal E\times \mathcal F$ with $0<m(\mathcal E)m(\mathcal F)<\infty ,$ if we assume that $\Sigma $ has finite Lebesgue measure in $\mathfrak b=\zeta _\omega \oplus \eta _\omega ,$ in view of Equation (2-5), it is immediate from Proposition 2.4 that $\tau =0.$ Hence, $h=0.$

2. (ii) Instead of taking the set $\mathcal E\times \mathcal F$ in $\mathfrak b,$ if we consider a finite Lebesgue measure set $\Sigma $ in $\mathfrak b,$ then the projection of $\Sigma $ on $\zeta _\omega $ or $\eta _\omega $ need not be a set of finite Lebesgue measure. Thus, the idea of the proof of Theorem 2.6 does not work in this case.

3 Preliminaries on the Heisenberg motion group

The Heisenberg group $\mathbb H^n=\mathbb C^n\times \mathbb R$ is a step two nilpotent Lie group having center $\mathbb R$ equipped with the group law

$$ \begin{align*}(z,t)\cdot(w,s)=(z+w,t+s+\tfrac{1}{2}\text{Im}(z\cdot\bar w)).\end{align*} $$

By the Stone–von Neumann theorem, the infinite-dimensional irreducible unitary representations of $\mathbb H^n$ can be parameterized by $\mathbb R^\ast .$ That is, each $\lambda \in \mathbb R^\ast $ defines a Schrödinger representation $\pi _\lambda $ of $\mathbb H^n$ via

$$ \begin{align*}\pi_\lambda(z,t)\varphi(\xi)=e^{i\lambda t}e^{i\lambda(x\cdot\xi+({1}/{2})x\cdot y)}\varphi(\xi+y),\end{align*} $$

where $z=x+iy$ and $\varphi \in L^2(\mathbb {R}^n).$

The Heisenberg motion group G consists of isometries of $\mathbb H^n$ that commute with the sub-Laplacian $\mathcal L$ on $\mathbb H^n.$ Since the unitary group $K=U(n)$ acts on $\mathbb H^n$ by the automorphism $k\cdot (z,t)=(kz,t),$ where $k\in K,$ the group G can be expressed as the semidirect product of $\mathbb H^n$ and $K,$ that is, $\mathbb H^n\rtimes K.$ Hence, the group law on G can be understood by

$$ \begin{align*}(z, t, k)\cdot(w, s, h)=(z+kw, t+s+\tfrac{1}{2}\text{Im}(z\cdot\overline{kw}), kh).\end{align*} $$

Since a right K-invariant function on G can be thought of as a function on $\mathbb H^n$ and the Lebesgue measure $dz\,dt$ is the Haar measure on $\mathbb H^n,$ we infer that the Haar measure on G is $dg=dk\,dz\,dt,$ where $dk$ stands for the normalized Haar measure on  $K.$

For $k\in K,$ define a new set of irreducible representations of the Heisenberg group $\mathbb H^n$ through $\pi _{\lambda ,k}(z,t)=\pi _\lambda (kz,t).$ Since $\pi _{\lambda ,k}$ agrees with $\pi _\lambda $ on the center of $\mathbb H^n,$ it follows by the Stone–Von Neumann theorem for the Schrödinger representation that $\pi _{\lambda ,k}$ is equivalent to $\pi _\lambda .$ Hence, there exists an intertwining operator $\mu _\lambda (k)$ satisfying

$$ \begin{align*} \pi_\lambda(kz,t)=\mu_\lambda(k)\pi_\lambda(z,t)\mu_\lambda(k)^\ast. \end{align*} $$

Also, $\mu _\lambda (k)$ is unitary as well and could be chosen to represent K in $L^2(\mathbb R^n),$ known as a metaplectic representation. Let $\phi _\alpha ^\lambda (x)=|\lambda |^{\frac {n}{4}}\phi _\alpha (\!\sqrt {|\lambda |}x);~\alpha \in \mathbb Z_+^n,$ where the $\phi _\alpha $ terms are the Hermite functions on $\mathbb R^n.$ Then for each $\lambda \in \mathbb R^\ast ,$ the set $\{\phi _\alpha ^\lambda : \alpha \in \mathbb Z_+^n \}$ forms an orthonormal basis for $L^2(\mathbb R^n).$ By letting $P_m^\lambda =\text {span}\{\phi _\alpha ^\lambda :~ |\alpha |=m\}$ , $\mu _\lambda $ becomes an irreducible unitary representation of K on $P_m^\lambda .$ Hence, the action of $\mu _\lambda $ can be realized on $P_m^\lambda $ by

$$ \begin{align*} \mu_\lambda(k)\phi_\alpha^\lambda=\sum_{|\gamma|=|\alpha|}\eta_{\gamma\alpha}^\lambda(k)\phi_\gamma^\lambda, \end{align*} $$

where the $\eta _{\gamma \alpha }^\lambda $ terms are the matrix coefficients of $\mu _\lambda (k).$ For details on the metaplectic representation and spherical function, see [5]. Let $(\sigma ,\mathcal H_\sigma )$ be an irreducible unitary representation of $K,$ and $\{e_j^\sigma :1\leq j\leq d_\sigma \}$ be an orthonormal basis for $\mathcal H_\sigma .$ Denote the matrix coefficients of $\sigma \in \widehat K$ by

$$ \begin{align*}\varphi_{ij}^\sigma(k)=\langle\sigma(k)e_j^\sigma, e_i^\sigma\rangle.\end{align*} $$

Define a set of bilinear forms $\phi _{\alpha }^\lambda \otimes e_i^\sigma $ on $L^2(\mathbb R^n)\times \mathcal H_\sigma $ by $\phi _{\alpha }^\lambda \otimes e_i^\sigma =\phi _{\alpha }^\lambda ~e_i^\sigma .$ Then ${\mathcal B_\sigma =\{\phi _{\alpha }^\lambda \otimes e_i^\sigma :1\leq i\leq d_\sigma , \alpha \in \mathbb N^n\}}$ forms an orthonormal basis for $L^2(\mathbb R^n)\otimes \mathcal {H}_\sigma .$ Denote $\mathcal H_\sigma ^2:=L^2(\mathbb R^n)\otimes \mathcal {H}_\sigma .$ Define a representation $\rho _\sigma ^\lambda $ of G on $\mathcal H_\sigma ^2$ by

(3-1) $$ \begin{align} \rho_\sigma^\lambda(z,t,k)=\pi_\lambda(z,t)\mu_\lambda(k)\otimes\sigma(k). \end{align} $$

Then $\rho _\sigma ^\lambda $ are all possible irreducible unitary representations of G that appear in the Plancherel formula [21]. Thus, in view of the above discussion, we denote the partial dual of the group G by $ G'\cong \mathbb R^\ast \times \widehat K.$ For $(\lambda ,\sigma )\in G',$ the Fourier transform of $f\in L^1(G),$ defined by

(3-2) $$ \begin{align} \hat f(\lambda,\sigma) =\int_K\int_{\mathbb{R}}\int_{\mathbb{C}^n} f(z,t,k)\rho_\sigma^\lambda(z,t,k)\,dz\,dt\,dk, \end{align} $$

is a bounded linear operator on $\mathcal H_\sigma ^2.$ Let $f^\lambda $ be the inverse Fourier transform of the function f in the variable t which is given by $f^\lambda (z,k)=\int _{\mathbb {R}}f(z,t,k)e^{i\lambda t}\,dt.$ Then Equation (3-2) reduces to

$$ \begin{align*} \hat f(\lambda,\sigma)=\int_K\int_{\mathbb{C}^n} f^\lambda(z,k)\rho_\sigma^\lambda(z,k)\,dz\,dk, \end{align*} $$

where $\rho _\sigma ^\lambda (z,k)=\rho _\sigma ^\lambda (z,0,k).$ Since $\mathcal B_\sigma $ is an orthonormal basis for $\mathcal {H}_\sigma ^2,$ the action of $\hat f(\lambda ,\sigma )$ is given by

$$ \begin{align*}\hat f(\lambda,\sigma)(\phi_{\gamma}^\lambda\otimes e_i^\sigma)=\sum_{|\alpha|=|\gamma|}\int_K\eta_{\gamma\alpha}^\lambda(k) \int_{\mathbb{C}^n} f^\lambda(z,k)(\pi_\lambda(z)\phi_{\alpha}^\lambda\otimes\sigma(k)e_i^\sigma)\,dz\,dk.\end{align*} $$

Moreover, if $f\in L^1\cap L^2(G),$ then $\hat f(\lambda ,\sigma )$ becomes a Hilbert–Schmidt operator satisfying the Plancherel formula [21]:

$$ \begin{align*} \int_K\int_{\mathbb{H}^n} |f(z,t,k)|^2 \,dz\,dt\,dk=(2\pi)^{-n}\sum_{\sigma\in\widehat K}d_\sigma \int_{\mathbb{R}\setminus\{0\}}\|\hat f(\lambda,\sigma)\|^2_{\mathrm{HS}}|\lambda|^nd\lambda. \end{align*} $$

4 Uniqueness results on the Heisenberg motion group

As in the case of a Heisenberg group, in a natural way, one can define the Weyl transform on $G^\times :=\mathbb C^n\rtimes K.$ For $(\lambda , \sigma )\in { G'},$ define the Weyl transform $W_\sigma ^\lambda $ on $L^1(G^\times )$ by letting

$$ \begin{align*} W_\sigma^\lambda(F)=\int_{K}\int_{\mathbb{C}^n}F(z,k)\rho_\sigma^\lambda(z,k)\,dz\,dk. \end{align*} $$

Let $f, g$ be two functions on the Heisenberg motion group $G.$ Then the convolution of f and g is defined by

$$ \begin{align*}(f\ast g)(z,t,k)=\int_{K}\int_{\mathbb{H}^n}f((z,t,k)(-w,-s, h^{-1}))g(w,s,h)\,dw\,ds\,dh.\end{align*} $$

By the definition of the Fourier transform on $G,$ it is easy to see that $\widehat {(f\ast g)}(\lambda ,\sigma )=\hat f(\lambda ,\sigma )\hat g(\lambda ,\sigma ).$ Recall that $f^\lambda (z,k)$ is the inverse Fourier transform of f in the variable t. A simple computation shows that

(4-1) $$ \begin{align} (f\ast g)^\lambda(z, k)=\int_{K}\int_{\mathbb{C}^n}f^\lambda(z-kw, kh^{-1})g^\lambda(w, h)e^{({i\lambda}/{2})\text{Im} (z\cdot\overline{kw})}\,dw\,dh. \end{align} $$

The right-hand expression in Equation (4-1) is called the $\lambda $ -twisted convolution of the functions $f^\lambda , g^\lambda ,$ and it is denoted by $f^\lambda \times _\lambda g^\lambda .$ Since $\widehat f(\lambda , \sigma )=W_\sigma ^\lambda (f^\lambda ),$ it is immediate that $W_\sigma ^\lambda (f^\lambda \times _\lambda g^\lambda )=W_\sigma ^\lambda (f^\lambda )W_\sigma ^\lambda (g^\lambda ).$ In a more general way, the $\lambda $ -twisted convolutions of $F,H\in L^1\cap L^2(G^\times )$ can be defined by letting

$$ \begin{align*}F\times_\lambda H(z, k)=\int_{K}\int_{\mathbb{C}^n}F(z-kw, kh^{-1})H(w, h)e^{({i\lambda}/{2})\text{Im} (z\cdot\overline{kw})}\,dw\,dh.\end{align*} $$

For $\lambda =1,$ we use the notation $F\times H$ instead of $F\times _1 H$ and simply call it the twisted convolution of F and $ H.$ A simple observation shows that $W_\sigma ^\lambda (F^\ast )=W_\sigma ^\lambda (F)^\ast ,$ where $F^\ast (z, k)=\overline {F((z, k)^{-1})},$ and $W_\sigma ^\lambda (F\times _\lambda H)=W_\sigma ^\lambda (F)W_\sigma ^\lambda (H).$ We identify $W_\sigma $ with the Weyl transform on $L^1(G^\times )$ whenever $\lambda =1.$ Next, we derive the Plancherel formula for $W_\sigma $ and the general case follows similarly.

Proposition 4.1. If $F\in L^2(G^\times ),$ then the following holds:

$$ \begin{align*}\sum_{\sigma\in\widehat K}d_\sigma||W_\sigma(F)||_{\mathrm{HS}}^2=(2\pi)^n\int_{K}\int_{\mathbb{C}^n}|F(z,k)|^2\,dz\,dk.\end{align*} $$

Proof. Since $L^1\cap L^2(G^\times )$ is dense in $L^2(G^\times ),$ it is enough to prove the result for $L^1\cap L^2(G^\times ).$ For convenience, let $\phi _{\alpha , i}^{\sigma }=\phi _{\alpha }^\lambda \otimes e_i^\sigma $ and $\phi _{\alpha \beta }=(2\pi )^{{n}/{2}}\phi _{\alpha \beta }^\lambda $ whenever $\lambda =1.$ By the Parseval identity,

$$ \begin{align*} \|W_\sigma(F)\phi_{\gamma,i}^\sigma\|_{\mathcal{H}_\sigma^2}^2 &= \sum_{\beta\in\mathbb N^n}\sum_{j=1}^{d_\sigma}|\langle W_\sigma(F)\phi_{\gamma,i}^\sigma,\phi_{\beta,j}^\sigma\rangle|^2\\ &=(2\pi)^n\sum_{\beta\in\mathbb N^n}\sum_{j=1}^{d_\sigma}\bigg|\sum_{|\alpha|=|\gamma|}\int_{K} \eta_{\gamma\alpha}(k)\int_{\mathbb{C}^n}F(z,k)\phi_{\alpha\beta}(z)\varphi_{ji}^\sigma(k)\,dz\,dk\bigg|^2. \end{align*} $$

Recall that the matrix coefficient functions $\phi _{ij}^\nu $ of a $d_\nu $ -dimensional unitary irreducible representation $(\nu ,\mathcal H_\nu )$ of K satisfy the identity

(4-2) $$ \begin{align} \sum_{q=1}^{d_\nu}\bigg(\sum_{j=1}^{d_\nu}c_j\phi_{qj}^\nu(k)\sum_{n=1}^{d_\nu}\overline{a_n} \overline{\phi_{qn}^\nu(k)}\bigg) =\sum_{n=1}^{d_\nu}c_n\overline{a_n}, \end{align} $$

where $a_j, c_j\in \mathbb C; 1\leq j\leq d_\nu $ and $k\in K.$ Also, the matrix coefficients of $(\nu ,\mathcal H_\nu )$ satisfy the orthogonality condition

(4-3) $$ \begin{align} d_\nu\langle\phi_{qj}^\nu,\phi_{ln}^\nu\rangle=\delta_{ql}\delta_{jn}. \end{align} $$

Let $\eta _{\gamma \alpha }$ represent the ${(\alpha ,\gamma )}$ th matrix coefficient of $\mu _\lambda \vert _{P_m^\lambda }.$ In view of Equation (4-2), it follows that

(4-4) $$ \begin{align} \sum_{|\gamma|=m}\bigg|\sum_{|\alpha|=m}c_\alpha\eta_{\gamma\alpha}(k)\bigg|^2 =\sum_{|\alpha|=m}|c_\alpha|^2, \end{align} $$

where $k\in K$ and $c_\alpha \in \mathbb C.$ Now, by Plancherel theorem for $L^2(K)$ and the identity in Equation (4-4), we infer that

$$ \begin{align*} \sum_{\sigma\in\widehat K}d_\sigma||W_\sigma(F)||_{\mathrm{HS}}^2 &= (2\pi)^n\sum_{\beta, \gamma\in\mathbb N^n}\int_{K}\bigg|\sum_{|\alpha|=|\gamma|}\eta_{\gamma\alpha}(k)\int_{\mathbb{C}^n}F(z,k)\phi_{\alpha\beta}(z)\,dz\bigg|^2\,dk\\ &= (2\pi)^n\sum_{\alpha,\beta\in\mathbb N^n}\int_{K}\bigg|\int_{\mathbb{C}^n}F(z,k)\phi_{\alpha\beta}(z)\,dz\bigg|^2\,dk\\ &= (2\pi)^n\int_{K}\int_{\mathbb{C}^n}|F(z,k)|^2\,dz\,dk.\\[-42pt] \end{align*} $$

4.1 Fourier–Wigner transform

For $\sigma \in \widehat K,$ define the Fourier–Wigner transform of $f,g\in \mathcal {H}_\sigma ^2$ on $G^\times $ by letting

$$ \begin{align*}V_f^g(z,k)=\langle\rho_\sigma(z,k)f,g\rangle.\end{align*} $$

Then $V_f^g$ satisfies the following orthogonality relation.

Lemma 4.2. For $f_l, g_l\in \mathcal {H}_\sigma ^2,~l=1,2,$ the following identity holds:

$$ \begin{align*} \int_{K}\int_{\mathbb{C}^n}V_{f_1}^{g_1}(z,k)\overline{V_{f_2}^{g_2}(z,k)}\,dz\,dk= (2\pi)^nd_\sigma^{-1}\langle f_1,f_2 \rangle\overline{\langle g_1,g_2 \rangle}. \end{align*} $$

Proof. Since $f_l, g_l\in \mathcal {H}_\sigma ^2,$ we can express the functions $f_l, g_l$ as

$$ \begin{align*} f_l=\sum _{\gamma \in \mathbb {N}^n}\sum _{1\leq i\leq d_\sigma } f_{\gamma ,i}^l\phi _\gamma \otimes e_i^\sigma \end{align*} $$

and

$$ \begin{align*} g_l=\sum _{\beta \in \mathbb {N}^n}\sum _{1\leq j\leq d_\sigma }g_{\beta ,j}^l\phi _\beta \otimes e_j^\sigma ,\quad ~l=1,2, \end{align*} $$

where $f_{\gamma ,i}^l$ and $g_{\beta ,j}^l$ are constants. Thus, $V_{f_l}^{g_l}$ takes the form

$$ \begin{align*} V_{f_l}^{g_l}(z,k)=(2\pi)^{{n}/{2}}\sum_{\alpha,\beta\in\mathbb N^n}\sum_{1\leq i,j\leq d_\sigma}\sum_{|\gamma|=|\alpha|} f_{\gamma,i}^l\overline{g_{\beta,j}^l}\eta_{\gamma\alpha}(k)\phi_{\alpha\beta}(z)\varphi_{ji}^\sigma(k). \end{align*} $$

By orthogonality of the functions $\phi _{\alpha \beta }$ and Equation (4-2), it follows that

$$ \begin{align*}\int_{\mathbb{C}^n}V_{f_1}^{g_1}(z,k)\overline{V_{f_2}^{g_2}(z,k)}\,dz= (2\pi)^n\sum_{\gamma,\beta\in\mathbb N^n}\bigg[\sum_{i,j=1}^{d_\sigma}(f_{\gamma,i}^1\overline{g_{\beta,j}^1})\phi_{ji}^\sigma(k) \sum_{i,j=1}^{d_\sigma}(\overline{f_{\gamma,i}^2}g_{\beta,j}^2)\overline{\phi_{ji}^\sigma(k)}\bigg].\end{align*} $$

On integrating the above equation with respect to k and using Equation (4-3),

$$ \begin{align*} d_\sigma\int_{K}\int_{\mathbb{C}^n}V_{f_1}^{g_1}(z,k)\overline{V_{f_2}^{g_2}(z,k)}\,dz\,dk &= (2\pi)^n\bigg(\sum_{\gamma\in\mathbb N^n}\sum_{1\leq i\leq d_\sigma}f_{\gamma,i}^1\overline{f_{\gamma,i}^2}\bigg)\bigg(\sum_{\beta\in\mathbb N^n} \sum_{1\leq j\leq d_\sigma}g_{\beta,j}^2\overline{g_{\beta,j}^1}\bigg)\\ &= (2\pi)^n\langle f_1,f_2 \rangle\overline{\langle g_1,g_2 \rangle}. \\[-32pt]\end{align*} $$

Note that for $f,g\in \mathcal {H}_\sigma ^2,$ Lemma 4.2 implies that $V_f^g\in L^2(G^\times ).$ Consider the set $V_\sigma =\overline {\text {span}}\{V_f^g: f,g\in \mathcal {H}_\sigma ^2\}.$ Since $\mathcal B_\sigma $ forms an orthonormal basis for $\mathcal {H}_\sigma ^2,$ it follows from Lemma 4.2 that

$$ \begin{align*}V_{B_\sigma}=\{V_{\psi_{\alpha,i}^\sigma}^{\psi_{\beta,j}^\sigma}:~ \psi_{\alpha,i}^\sigma,\psi_{\beta,j}^\sigma\in {\mathcal B}_\sigma\} \end{align*} $$

forms an orthonormal basis for $V_\sigma .$ We need to recall the following Peter–Weyl theorem, see [23].

Theorem 4.3. Let $\widehat K$ be the unitary dual of a compact Lie group $K.$ Then the set $\{\!\sqrt {d_\sigma }\phi _{ij}^\sigma :1\leq i,j\leq d_\sigma , \sigma \in \widehat K\}$ is an orthonormal basis for $L^2(K).$

Proposition 4.4. The set $V_{\mathcal B}=\{V_{\mathcal B_\sigma }: \sigma \in \widehat K\}$ is a complete orthogonal set for $L^2(G^\times ).$ Moreover, $L^2(G^\times )=\bigoplus \limits _{\sigma \in \widehat K}V_{\sigma }.$

Proof. By Lemma 4.2 and Theorem 4.3, it follows that $V_{\mathcal B}$ is an orthogonal set. For completeness, suppose $F\in V_{\mathcal B}^\perp .$ Then

$$ \begin{align*} \langle W_\sigma(\overline{F})\psi_{\alpha,i}^\sigma,\psi_{\beta,j}^\sigma\rangle &= \int_{K}\int_{\mathbb{C}^n}\overline{F}(z,k)V_{\psi_{\alpha,i}^\sigma}^{{\psi_{\beta,j}^\sigma}}(z,k)\,dz\,dk\\ &= \langle F,V_{\psi_{\alpha,i}^\sigma}^{\psi_{\beta,j}^\sigma} \rangle=0, \end{align*} $$

whenever $\psi _{\alpha ,i}^\sigma ,\psi _{\beta ,j}^\sigma \in \mathcal B_\sigma .$ Hence, $W_\sigma (\overline {F})=0$ for arbitrary $\sigma \in \widehat K.$ Thus, by Proposition 4.1, we conclude that $F=0.$ Since $V_{\mathcal B_\sigma }$ is a complete orthogonal set for $V_{\sigma }$ , $L^2(G^\times )$ is the direct sum of $V_{\sigma }$ terms.

4.2 QUP for the Heisenberg motion group

Let $\text {A}$ and $\text {B}$ be Lebesgue measurable subsets of $\mathbb {R}^n$ with $0<m(\text {A})m(\text {B})$ $<\infty .$ Write $\overline {\tau _\lambda }={f^\lambda }^\ast \times _\lambda f^\lambda ,$ where $g^\ast (v)=\overline {g(v^{-1})}.$ Then $W_\sigma ^\lambda (\overline {\tau _\lambda })={\hat f(\lambda ,\sigma )}^\ast \hat f(\lambda ,\sigma ).$ From Proposition 4.4, we can express

(4-5) $$ \begin{align} \tau_\lambda=\bigoplus\limits_{\sigma\in\widehat K}\tau_{\lambda,\sigma}, \end{align} $$

which we call the Fourier–Wigner decomposition, and $\tau _{\lambda ,\sigma }$ the Fourier–Wigner representation.

Those functions in $L^1(G)$ that are K-bi-invariant form a commutative convolution algebra. Therefore, the Fourier transform of a K-bi-invariant integrable function has rank one. However, these functions differ from the Fourier–Wigner transform in terms of the Fourier–Wigner representations.

Though the decomposition in Equation (4-5) follows from the Peter–Weyl theorem, it is finer than the usual Peter–Weyl decomposition of functions on $K,$ which might be due to the presence of the metaplectic representation. As an effect, even if $f\in L^2(G^\times )$ is K-bi-invariant on $G^\times ,$ it need not fall into the trivial Fourier–Wigner representation. Thus, the question of uncertainty arises in the sense that if the Weyl transform of each Fourier–Wigner piece of $f\in L^2(G^\times )$ is of finite rank, then f is zero almost everywhere as long as the nonvanishing set for each Fourier–Wigner piece of f has finite measure.

The following QUP holds for the Heisenberg motion group.

Theorem 4.5. Let $f\in L^1\cap L^2(G)$ be such that each $\tau _{\lambda ,\sigma }$ is nonvanishing on the set $\Sigma _\sigma \times K.$

1. (i) If $~\Sigma _\sigma $ has finite measure, and each $W_\sigma ^\lambda (\overline {\tau _\lambda })(\cdot )=a_0\langle \cdot , \phi \otimes \psi \rangle \phi \otimes \psi $ for some $\phi \otimes \psi \in \mathcal {H}_\sigma ^2$ and $a_0\geq 0,$ then $f=0$ a.e.

2. (ii) If $\Sigma _\sigma =\text {A}\times \text {B}\subset \mathbb R^n\times \mathbb R^n$ has finite measure, and each $W_\sigma ^\lambda (\overline {\tau _\lambda })(\cdot )=\sum _{j=1}^Na_j\langle \cdot , f_j\rangle f_j,$ where $a_j\geq 0$ and $\{f_j=\phi _j\otimes \psi _j\in \mathcal {H}_\sigma ^2: 1\leq j\leq N\}$ is an orthonormal set, then $f = 0$ a.e.

As a corollary to Theorem 2.1, the following analog holds for the Fourier–Wigner transform on $G^\times .$

Proposition 4.6. Let $f_j=\phi _j\otimes h_j\in {\mathcal H}_\sigma ^2;~j=1,2,$ and $F=V_{f_1}^{f_2}.$ If for each $k\in K$ the set $\{z\in \mathbb C^n : F(z,k)~\neq ~0\}$ has finite Lebesgue measure, then $F=0.$

Proof. It follows from Equation (3-1) that

$$ \begin{align*} F(z,k) &= \langle\pi(z)\mu(k)\otimes\sigma(k)(\phi_1\otimes h_1),\phi_2\otimes h_2\rangle \\ &= \langle\pi(z)\mu(k)\phi_1,\phi_2\rangle\langle\sigma(k)h_1,h_2\rangle\\ &= T(\mu(k)\phi_1,\phi_2)(z)\langle\sigma(k)h_1,h_2\rangle,\\ &= X(z,k)\langle\sigma(k)h_1,h_2\rangle, \end{align*} $$

where $X(z,k)=T(\mu (k)\phi _1,\phi _2)(z).$ If $\langle \sigma (k)h_1,h_2\rangle =0,$ then $F(.,k)=0.$ However, if $\langle \sigma (k)h_1,h_2\rangle \neq 0$ for some $k\in K,$ then $T(\mu (k)\phi _1,\phi _2)$ is nonvanishing on a set of finite Lebesgue measure. Hence, Theorem 2.1 yields that $F=0.$

For proving our main results of Theorems 4.5 and 4.9, we need the following crucial result.

Proposition 4.7. Let $\phi _1,\ldots ,\phi _N\in L^2(\mathbb {R}^n)$ and $k\in U(n).$ Define $b_j(k)=\langle \sigma (k)\psi _j,\psi _j\rangle ,$ where $\psi _j\in \mathcal H_\sigma .$ For $z\in \mathbb C^n,$ let

$$ \begin{align*}\psi(z,k)=\sum\limits_{j=1}^Nb_j(k)\langle\pi(z)\mu(k)\phi_j,\phi_j\rangle.\end{align*} $$

Let $\mathcal E,\mathcal F$ be two subsets of $\mathbb R^n$ of finite Lebesgue measure. If $\psi $ is nonvanishing on $\mathcal E\times \mathcal F\times U(n),$ then $\psi \equiv 0.$

Proof. We prove the proposition in the following two steps.

Step I. In this step, we show that all $\phi _j;~1\leq j\leq N$ are nonvanishing on a set of finite Lebesgue measure. Let $k=e\in U(n)$ be the identity matrix. Then $\mu (e)=I,$ the identity operator on $L^2(\mathbb {R}^n).$ For $z=x+iy\in \mathbb C^n,$ we introduce the function ${\psi _y(x)=\psi (z,e).}$ Since $\phi _j\in L^2(\mathbb {R}^n),$ except on a set of measure zero, $|\phi _j|$ is finite on $\mathbb {R}^n.$ Let us introduce a function $\chi $ on $\mathbb R^n$ by $\chi (\xi )=(\|\psi _1\|\phi _1(\xi ),\ldots , \|\psi _N\|\phi _N(\xi )),$ and hence

$$ \begin{align*}K_y(\xi):=\sum\limits_{j=1}^N\|\psi_j\|^2\phi_j(\xi+y) \overline{\phi_j(\xi)}=\langle\chi(\xi+y),\chi(\xi)\rangle_{\mathbb C^N}\end{align*} $$

for almost every $\xi \in \mathbb R^n,$ so that

$$ \begin{align*}\psi_y(x)=\int_{\mathbb{R}^n}e^{i(x\cdot\xi +({1}/{2})x\cdot y)}K_y(\xi)\,d\xi\end{align*} $$

is the Fourier transform of $K_y$ (up to the factor $e^{ix\cdot y/2}$ ). By assumption, it follows that $\psi _y=0$ for all $y\in \mathbb R^n\smallsetminus \mathcal F.$ Hence, we infer that $K_y=0.$ Thus, we have shown that

$$ \begin{align*}\langle\chi(\xi+y),\chi(\xi)\rangle_{\mathbb C^N}=0\end{align*} $$

for almost every $\xi \in \mathbb R^n$ and $y\in \mathbb R^n\smallsetminus \mathcal F.$

Assume toward a contradiction that $S:=\{\zeta \in \mathbb R^n:\chi (\zeta )\neq 0\}$ has infinite Lebesgue measure. Since $\mathcal F$ has finite measure, without loss of generality, there exists $s_1\in S\cap (\mathbb R^n\smallsetminus \mathcal F).$ Note that $\chi (s_1)\neq 0$ and

$$ \begin{align*}\langle\chi(\xi+s_1),\chi(\xi)\rangle_{\mathbb C^N}=0.\end{align*} $$

Next, take $s_2\in S\cap (\mathbb R^n\smallsetminus (\mathcal F+s_1)),$ then $\chi (s_2)\neq 0,$ and since $s_2-s_1\notin \mathcal F,$ we have that $\langle \chi (\xi +s_2-s_1),\chi (\xi )\rangle _{\mathbb C^N}=0.$ In particular, $\langle \chi (s_2),\chi (s_1)\rangle _{\mathbb C^N}=0.$ In this way, after m steps, we get that $\{s_j:~1\leq j\leq m\}$ such that $\chi (s_j)\neq 0$ and

(4-6) $$ \begin{align} \langle\chi(\xi+s_j-s_{j'}),\chi(\xi)\rangle_{\mathbb C^N}=0\quad\text{for}~1\leq j\neq j'\leq m. \end{align} $$

If we consider $s_{m+1}\in S\cap (\mathbb R^n\smallsetminus \bigcup \nolimits _{j=1}^{m}(\mathcal F+s_j)),$ then $\chi (s_{m+1})\neq 0$ and for $j\leq m,$

(4-7) $$ \begin{align} \langle\chi(\xi+s_{m+1}-s_j),\chi(\xi)\rangle_{\mathbb C^N}=0. \end{align} $$

In particular, taking $\xi =s_{j'}$ in Equation (4-6) and $\xi =s_{j}$ in Equation (4-7),

$$ \begin{align*}\langle\chi(s_j),\chi(s_{j'})\rangle_{\mathbb C^N}=0\quad\text{for}~1\leq j\neq j'\leq m+1.\end{align*} $$

For $m=N$ , we obtain $N+1$ nonzero mutually orthogonal vectors in $\mathbb C^N,$ which is a contradiction. It follows that S must have finite measure and hence all $\phi _j$ terms are nonvanishing on $S.$

Step II. From Step I, it follows that $K_y$ is nonvanishing on a set of finite Lebesgue measure for all $y\in \mathbb R^n.$ Since $\psi _y$ is nonvanishing on $\mathcal E,$ by Benedicks’ theorem on $\mathbb R^n,$ we get that $K_y=0$ for all $y\in \mathbb R^n.$ Hence, $\psi (x+iy,e)=0$ for all $x,y\in \mathbb R^n.$

Let $k\in U(n)$ and for $z=x+iy\in \mathbb C^n,$ consider the function $\psi _{y,k}(x)=\psi (z,k).$ If we write

$$ \begin{align*}H_{y, k}(\xi):=\sum\limits_{j=1}^N b_j(k)(\mu(k)\phi_j)(\xi+y)\overline{\phi_j(\xi)}~\text{for~almost~every}~\xi\in\mathbb R^n,\end{align*} $$

then $\psi _{y,k}(x)=\int _{\mathbb {R}^n}e^{i(x\cdot \xi +1/2x\cdot y)}H_{y,k}(\xi )\,d\xi $ is the Fourier transform of $H_{y,k}$ up to the factor $e^{ix\cdot (y/2)}.$ Recall that each $\phi _j$ is nonvanishing on a set of finite Lebesgue measure on $\mathbb R^n,$ and $H_{y,k}$ is nonvanishing on a set of finite measure for all $y\in \mathbb R^n.$ Since $\psi _{y,k}$ is nonvanishing on $\mathcal E,$ by Benedicks’ theorem, we get that $H_{y,k}=0$ for all $y\in \mathbb R^n.$ Hence, $\psi (x+iy,k)=0$ for all $x,y\in \mathbb R^n$ and $k\in U(n).$

Remark 4.8. Instead of the rectangle $\mathcal E\times \mathcal F$ in $\mathbb R^{2n},$ if we consider a set E of finite Lebesgue measure in $\mathbb R^{2n},$ then the projection of E on $\mathbb R^n$ need not be a set of finite measure. Hence, the above proof of Proposition 4.7 does not work.

4.3 QUP for the Weyl transform

Let $\mathcal E$ and $\mathcal F$ be Lebesgue measurable sets in $\mathbb R^n$ satisfying the condition $0<m(\mathcal E)m(\mathcal F)$ $<\infty .$ We write $\Sigma '=\mathcal E\times \mathcal F$ and $F^\ast (v)=\overline {F(v^{-1})}.$ Recall from Proposition 4.4 that every $\tau \in L^2(G^\times )$ can be expressed as $\tau =\bigoplus \nolimits _{\sigma \in \widehat K}\tau _\sigma .$ Then the following QUP holds for the Weyl transform on the Heisenberg motion group.

Theorem 4.9. Let $f\in L^1\cap L^2(G^\times )$ and let $\bar \tau =f^\ast \times f$ be such that each $\tau _\sigma $ is nonvanishing on the set of the type $\Sigma '\times K.$ If for each $\sigma \in \hat K,$ the Weyl transform of $\bar \tau $ has the form

$$ \begin{align*} W_\sigma(\overline\tau)(\cdot)=\sum_{j=1}^Na_j\langle\cdot, f_j\rangle f_j, \end{align*} $$

where $a_j\geq 0$ and $\mathcal B_\sigma ^N=\{f_j=\phi _j\otimes \psi _j\in \mathcal {H}_\sigma ^2: 1\leq j\leq N\}$ is an orthonormal set, then $f=0$ a.e.

Proof. By hypothesis, $W_\sigma (\bar \tau )=W_\sigma (f)^\ast W_\sigma (f)$ is a positive operator which satisfies $W_\sigma (\bar \tau )f_j=a_jf_j$ with $a_j\geq 0$ and $f_j=\phi _j\otimes \psi _j.$ Now, for $f,g\in \mathcal {H}_\sigma ^2,$ it is immediate that

(4-8) $$ \begin{align} \langle W_\sigma(\bar\tau)f,g\rangle &= \sum_{j=1}^Na_j\langle f,f_j\rangle \langle f_j,g\rangle\nonumber\\&= (2\pi)^{-n}\sum_{j=1}^Na_j \int_{K} \int_{\mathbb{C}^n}V_{f}^{g}(z,k) \overline{V_{f_j}^{f_j}(z,k)}\,dz\,dk. \end{align} $$

In view of the decomposition $\tau =\bigoplus \nolimits _{\sigma \in \widehat K}\tau _\sigma $ and by the definition of $W_\sigma (\bar \tau ),$ we can write

(4-9) $$ \begin{align} \langle W_\sigma(\bar\tau)f,g\rangle &= \int_{K}\int_{\mathbb{C}^n}\bar\tau(z,k)\langle\rho_\sigma^1(z,k)f,g\rangle \,dz\,dk\nonumber\\&= \int_{K}\int_{\mathbb{C}^n}\overline{{\tau_\sigma}}(z,k)V_{f}^{g}(z,k)\,dz\,dk. \end{align} $$

Hence, by comparing Equation (4-8) with Equation (4-9), it follows that

(4-10) $$ \begin{align} {\tau_\sigma}=\sum_{j=1}^N V_{h_j}^{h_j}, \end{align} $$

where $h_j=(2\pi )^{-{n}/{2}}\sqrt {a_j}f_j\in \mathcal {H}_\sigma ^2.$ Now, let $h_j=\phi _j\otimes \psi _j$ for some $\phi _j\in L^2(\mathbb {R}^n)$ and $\psi _j~\in ~{\mathcal H}_\sigma .$ Then from Equation (4-10), we have that

$$ \begin{align*} {\tau_\sigma}(z,k) = \sum_{j=1}^N\langle\rho_\sigma(z,k)h_j,h_j\rangle = \sum_{j=1}^N b_j(k)\langle\pi(z)\mu(k)\phi_j,\phi_j\rangle, \end{align*} $$

where $b_j(k)=\langle \sigma (k)\psi _j,\psi _j\rangle .$ Since $\tau _\sigma $ is nonvanishing on a set of finite Lebesgue measure in the $\mathbb C^n$ -variable, by Proposition 4.7, it follows that $\tau _\sigma =0,$ whenever $\sigma \in \widehat K.$ Thus, Proposition 4.1 yields that $f=0$ a.e.

Remark 4.10. (i) Observe that in Theorem 4.9, if we assume that f is nonvanishing on the set $\mathcal E\times \mathcal F\times K$ so that $0<m(\mathcal E)m(\mathcal F)$ $<\infty ,$ then $\bar \tau =f^\ast \times f$ is nonvanishing on the set $\mathcal E\times \mathcal F$ of finite measure, but from the decomposition $\tau =\bigoplus \nolimits _{\sigma \in \widehat K}\tau _\sigma ,$ each Fourier–Wigner piece $\tau _\sigma $ need not be nonvanishing on a set of finite measure. In the case above, the proof of Theorem 4.9 does not work. Moreover, if each $\tau _\sigma $ is nonvanishing on a set of finite measure, then $\tau $ may or may not be nonvanishing on a set of finite measure. Thus, the hypothesis of Theorem 4.9 is different as compared with [16, Theorem 2.2], but still we have that $f=0$ a.e.

(ii) If $\tau \in L^2(G^\times )$ is K-bi-invariant and nonvanishing on a set of finite measure, then $\tau $ need not be identical with $\tau _{\sigma }$ for any $\sigma \in \widehat K.$ This fact is verified by considering the example of the one-dimensional Heisenberg motion group $\mathbb H^1\rtimes U(1)$ in the article by Ghosh and Srivastava [8].

4.4 Proof of Theorem 4.5

Proof. (i) For $f\in L^1\cap L^2(G),$ since $\overline {\tau _\lambda }={f^\lambda }^\ast \times f^\lambda $ and it is given that $W_\sigma ^\lambda (\overline {\tau _\lambda })$ has rank one, it is enough to show that $f^\lambda =0.$ Consider the case when $\lambda =1,$ and for simplicity, we use the notation $\tau $ and $\tau _\sigma $ instead of $\tau _1$ and $\tau _{1,\sigma }.$ By hypothesis, we have that $W_\sigma (\bar \tau )h=\langle h,f_1\rangle f_2$ for all $h\in \mathcal {H}_\sigma ^2,$ where $f_1=\phi \otimes \psi $ and $f_2=a_0f_1.$ Hence, for $g,h\in \mathcal {H}_\sigma ^2,$ Lemma 4.2 yields

(4-11) $$ \begin{align} \langle W_\sigma(\bar\tau)h,g\rangle &= \langle h,f_1\rangle \langle f_2,g\rangle\nonumber\\&= (2\pi)^{-n}\int_{K}\int_{\mathbb{C}^n}V_{h}^{g}(z,k)\overline{V_{f_1}^{f_2}(z,k)}\,dz\,dk. \end{align} $$

Let $\tau =\bigoplus \limits _{\sigma \in \widehat K}\tau _\sigma ,$ where $\tau _\sigma \in V_{B_\sigma }.$ Then by definition of $W_\sigma (\bar \tau ),$ it follows that

(4-12) $$ \begin{align} \langle W_\sigma(\bar\tau)h,g\rangle = \int_{K}\int_{\mathbb{C}^n}\overline{\tau_\sigma}(z,k)V_{h}^{g}(z,k)\,dz\,dk. \end{align} $$

Now, by comparing Equation (4-11) with Equation (4-12) in view of Proposition 4.4, we infer that $\tau _\sigma =(2\pi )^{-n}V_{f_1}^{f_2}.$ Since each $\tau _\sigma $ is nonvanishing on $\Sigma _\sigma \times K,$ it follows from Proposition 4.6 that $\tau _\sigma =0$ for all $\sigma \in \widehat K.$ That is, $\tau =0$ and hence $f^1=0.$ Similarly, we can show that $f^\lambda =0$ for all $\lambda \in \mathbb R^\ast .$ Thus, we conclude that $f=0$ a.e.

(ii) For $\lambda =1,$ it follows from Theorem 4.9 that $f^1=0.$ Similarly, it can be shown that $f^\lambda =0$ for all $\lambda \in \mathbb R^\ast .$ Thus, we conclude that $f=0$ a.e.