Equivariant Brill–Noether theory for elliptic operators and superrigidity of J-holomorphic maps

Abstract

The space of Fredholm operators of fixed index is stratified by submanifolds according to the dimension of the kernel. Geometric considerations often lead to questions about the intersections of concrete families of elliptic operators with these submanifolds: Are the intersections nonempty? Are they smooth? What are their codimensions? The purpose of this article is to develop tools to address these questions in equivariant situations. An important motivation for this work are transversality questions for multiple covers of J-holomorphic maps. As an application, we use our framework to give a concise exposition of Wendl’s proof of the superrigidity conjecture.

Introduction

Let X and Y be two finite-dimensional vector spaces. The space $\operatorname {\mathrm {Hom}}(X,Y)$ is stratified by the submanifolds

of codimension

$$ \begin{align*} \operatorname{\mathrm{codim}} \mathscr{H}_r = (\dim X-r)(\dim Y-r). \end{align*} $$

This generalizes to infinite dimensions as follows. Let X and Y be two Banach spaces. The space of Fredholm operators from X to Y, denoted by $\mathscr {F}(X,Y)$ , is stratified by the submanifolds

of codimension

$$ \begin{align*} \operatorname{\mathrm{codim}} \mathscr{F}_{d,e} = de. \end{align*} $$

In many geometric problems, especially in the study of moduli spaces in algebraic geometry, gauge theory and symplectic topology, one is led to consider families of Fredholm operators $D\mskip 0.5mu\colon \thinspace \mathscr {P} \to \mathscr {F}(X,Y)$ parametrized by a Banach manifold $\mathscr {P}$ , and to analyze the subsets $D^{-1}(\mathscr {F}_{d,e})$ .

The archetypal example is Brill–Noether theory in algebraic geometry. Let $\Sigma $ be a closed, connected Riemann surface of genus g. Denote by $\operatorname {\mathrm {Pic}}(\Sigma )$ the Picard group of isomorphism classes of holomorphic line bundles $\mathscr {L} \to \Sigma $ . Brill–Noether theory is concerned with the study of the subsets $G_d^r \subset \operatorname {\mathrm {Pic}}(\Sigma )$ , called the Brill–Noether loci, defined by

The fundamental results of this theory deal with the questions of whether $G_d^r$ is nonempty, smooth and of the expected complex codimension.

This connects to the previous discussion as follows. Let L be a Hermitian line bundle of degree d over $\Sigma $ . Denote by $\mathscr {A}(L)$ the space of unitary connections on L. The complex gauge group $\mathscr {G}^{\mathbf {C}}(L)$ acts on $\mathscr {A}(L)$ and the quotient $\mathscr {A}(L)/\mathscr {G}^{\mathbf {C}}(L)$ is biholomorphic to $\operatorname {\mathrm {Pic}}^d(\Sigma )$ , the component of $\operatorname {\mathrm {Pic}}(\Sigma )$ parametrizing holomorphic line bundles of degree d. Define the family of Fredholm operators

$$ \begin{align*} \bar{\partial}\mskip0.5mu\colon\thinspace \mathscr{A}(L) \to \mathscr{F}(\Gamma(L),\Omega^{0,1}(\Sigma,L)) \end{align*} $$

by assigning to every connection A the Dolbeault operator $\bar {\partial }_A = \nabla _A^{0,1}$ . Set

It follows from the Riemann–Roch theorem and Hodge theory that the Brill–Noether loci can be described as the quotients

$$ \begin{align*} G_d^r = \tilde G_d^r/\mathscr{G}^{\mathbf{C}}(L). \end{align*} $$

If $G_d^r$ is nonempty, then

$$ \begin{align*} \operatorname{\mathrm{codim}}_{\mathbf{C}} G_d^r = \operatorname{\mathrm{codim}}_{\mathbf{C}} \tilde G_d^r \leqslant (r+1)(g-d+r). \end{align*} $$

This is an immediate consequence of the definition of $\tilde G_d^r$ and $\operatorname {\mathrm {codim}}_{\mathbf {C}} \mathscr {F}_{d,e} = de$ . Ideally, every $G_d^r$ is smooth of complex codimension $(r+1)(g-d+r)$ . This is not always true, but Gieseker [Gie82] proved that it holds for generic $\Sigma $ ; see also [EH83; Laz86]. Furthermore, Kempf [Kem71] and Kleiman and Laksov [KL72; KL74] proved that if $(r+1)(g-d+r) \leqslant g$ , then $G_d^r$ is nonempty. For an extensive discussion of Brill–Noether theory in algebraic geometry we refer the reader to [ACGH85].

By analogy, for a family of Fredholm operators $D\mskip 0.5mu\colon \thinspace \mathscr {P} \to \mathscr {F}(X,Y)$ one might ask:

1. (1) When are the subsets $D^{-1}(\mathscr {F}_{d,e})$ nonempty?

2. (2) When are they smooth submanifolds of $\mathscr {P}$ ?

3. (3) What are their codimensions?

Index theory and the theory of spectral flow sometimes give partial results regarding (1). A simple answer to (2) and (3) is that $D^{-1}(\mathscr {F}_{d,e})$ is smooth and of codimension $de$ if the map D is transverse to $\mathscr {F}_{d,e}$ . However, for many naturally occurring families of elliptic operators this condition does not hold. For example, if D is a family of elliptic operators over a manifold M and ${\underline V}$ is a local system, then the family $D^{\underline V}$ of the elliptic operators D twisted by ${\underline V}$ often is not transverse to $\mathscr {F}_{d,e}$ even if D is. Related issues arise for families of elliptic operators pulled back by a covering map $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ . The purpose of this article is to give useful tools for answering (2) and (3) which apply to these equivariant situations. This theory is developed in Part 2.

The issues discussed above are well known to arise from multiple covers in the theory of J-holomorphic maps in symplectic topology. In fact, our motivation for writing this article came from trying to understand Wendl’s proof of Bryan and Pandharipande’s superrigidity conjecture for J-holomorphic maps [Wen19b]. The theory developed in Part 2 is essentially an abstraction of Wendl’s ideas, some of which can themselves be traced back to Taubes [Tau96] and Eftekhary [Eft16]. In Part 2, we use this theory to give a concise exposition of the proof of the superrigidity conjecture. The main results of Part 2 are contained in [Wen19b], and most of the proofs closely follow Wendl’s approach. There are, however, two key differences:

1. (1) Our discussion consistently uses the language of local systems. This appears to us to be more natural for the problem at hand. It also avoids the use of representation theory and covering theory. In particular, there is no need to take special care of nonnormal covering maps.

2. (2) Our approach to dealing with branched covering maps is geometric: Branched covering maps between Riemann surfaces are reinterpreted as unbranched covering maps between orbifold Riemann surfaces. This is to be compared with Wendl’s analytic approach which uses suitable weighted Sobolev spaces on punctured Riemann surfaces. One feature of our approach is that it leads to a simple proof of the crucial index theorem; cf. Section 2.B and [Wen19b, Theorem 4.1].

We expect the theory developed in Part 1 to have many applications outside of the theory of J-holomorphic maps. In future work, we plan to study transversality for multiple covers of calibrated submanifolds in manifolds with special holonomy, such as associative submanifolds in $G_2$ -manifolds.

1 Equivariant Brill–Noether theory

Throughout this part, let $(M,g)$ be a closed, connected, oriented Riemannian orbifold of dimension $\dim M = n$ , and let E and F be Euclidean vector bundles of rank $\operatorname {\mathrm {rk}} E = \operatorname {\mathrm {rk}} F = r$ over M equipped with orthogonal covariant derivatives. ¹ Here and throughout this article, $\dim $ and $\operatorname {\mathrm {rk}}$ denote the dimension and rank over the real numbers. If dimension and rank are to be taken over a different field, then this is indicated by a subscript. $\Gamma (E)$ denotes the space of smooth sections of E. For an open subset $U \subset M$ , $\Gamma (U,E)$ and $\Gamma _c(U,E) \subset \Gamma (E)$ denote the spaces of smooth sections of E defined over U and with support in U respectively. For $k \in {\mathbf {N}}_0 = {\lbrace }{0,1,2,\ldots }{\rbrace }$ , denote by $W^{k,2}\Gamma (E)$ the Sobolev completion of $\Gamma (E)$ with respect to the $W^{k,2}$ norm induced by the Euclidean metric and covariant derivative. For $k \in {\mathbf {N}}$ set . Set . Denote by $L^1\Gamma (E)$ the completion of $\Gamma (E)$ with respect to the $L^1$ norm. (Analogous notation is used, instead of E, for F, etc.)

1.1 Brill–Noether loci

Let us begin by discussing the nonequivariant theory.

Definition 1.1.1. Let $k \in {\mathbf {N}}_0$ . A family of linear elliptic differential operators of order k consists of a Banach manifold $\mathscr {P}$ and a smooth map

$$ \begin{align*} D \mskip0.5mu\colon\thinspace \mathscr{P} \to \mathscr{F}(W^{k,2}\Gamma(E),L^2\Gamma(F)) \end{align*} $$

such that for every $p \in \mathscr {P}$ the operator is a linear elliptic differential operator. ² , ³

Definition 1.1.2. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $d,e \in {\mathbf {N}}_0$ define the Brill–Noether locus $\mathscr {P}_{d,e}$ by

Remark 1.1.3. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic operators of index $i \in \mathbf {Z}$ . If $\mathscr {P}_{d,e} \neq \varnothing $ , then $d-e = i$ ; in particular: $d \geqslant i$ and $e \geqslant -i$ .

The following elementary fact from the theory of Fredholm operators reduces the discussion to the finite-dimensional case. As in the introduction, if X and Y are Banach spaces, then $\mathscr {L}(X,Y)$ denotes the Banach space of bounded linear maps from X to Y equipped with the operator norm, and $\mathscr {F}(X,Y) \subset \mathscr {L}(X,Y)$ denotes the open subset of Fredholm operators from X to Y.

Lemma 1.1.4 (cf. Koschorke [Kos68, Chapter I §1.b]).

Let X and Y be Banach spaces. For every $L \in \mathscr {F}(X,Y)$ , there is an open neighborhood $\mathscr {U} \subset \mathscr {F}(X,Y)$ and a smooth map $\mathscr {S} \mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}(\ker L,\operatorname {\mathrm {coker}} L)$ such that for every $T \in \mathscr {U}$ there are isomorphisms

$$ \begin{align*} \ker T \cong \ker \mathscr{S}(T) \quad\text{and}\quad \operatorname{\mathrm{coker}} T \cong \operatorname{\mathrm{coker}} \mathscr{S}(T); \end{align*} $$

furthermore, the derivative of S at L,

$$ \begin{align*} \mathrm{d}_L\mathscr{S}\mskip0.5mu\colon\thinspace T_L\mathscr{F}(X,Y) \to \operatorname{\mathrm{Hom}}(\ker L,\operatorname{\mathrm{coker}} L) \end{align*} $$

satisfies

$$ \begin{align*} \mathrm{d}_L\mathscr{S}(\hat L)s = \hat L s \quad\mod \operatorname{\mathrm{im}} L \end{align*} $$

for every $\hat L \in T_L\mathscr {F}(X,Y) = \mathscr {L}(X,Y)$ .

Proof. Pick a complement $\operatorname {\mathrm {coim}} L$ of $\ker L$ in X, and a lift of $\operatorname {\mathrm {coker}} L$ to Y. With respect to the splittings $X = \operatorname {\mathrm {coim}} L \oplus \ker L$ and $Y = \operatorname {\mathrm {im}} L \oplus \operatorname {\mathrm {coker}} L$ every $T \in \mathscr {F}(X,Y)$ can be written as

$$ \begin{align*} T = \begin{pmatrix} T_{11} & T_{12} \\ T_{21} & T_{22} \end{pmatrix}. \end{align*} $$

By construction, $L_{11}$ is invertible, and the remaining components of L vanish.

Choose an open neighborhood $\mathscr {U}$ of L in $\mathscr {F}(X,Y)$ such that for every $T \in \mathscr {U}$ the operator $T_{11}$ is invertible. Define $\mathscr {S}\mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}(\ker L,\operatorname {\mathrm {coker}} L)$ by

A brief computation shows that for every $T \in \mathscr {U}$

hence, $\ker T \cong \ker \mathscr {S}(T)$ and $\operatorname {\mathrm {coker}} T \cong \operatorname {\mathrm {coker}} \mathscr {S}(T)$ .

The formula for $\mathrm {d}_L S$ is evident from the fact that $L_{21}$ and $L_{12}$ vanish.

Lemma 1.1.4 together with the regular value theorem immediately imply the following.

Theorem 1.1.5. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $p \in \mathscr {P}$ define $\Lambda _p \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \operatorname {\mathrm {Hom}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$ by

Let $d,e \in {\mathbf {N}}_0$ and $p \in \mathscr {P}_{d,e}$ . If $\Lambda _p$ is surjective, then following hold:

1. (1) There is an open neighborhood $\mathscr {U}$ of $p \in \mathscr {P}$ such that $\mathscr {P}_{d,e} \cap \mathscr {U}$ is a submanifold of codimension

   $$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e} \cap \mathscr{U}) = de. \end{align*} $$

2. (2) $\mathscr {P}_{\tilde d,\tilde e} \neq \varnothing $ for every $\tilde d,\tilde e \in {\mathbf {N}}_0$ with $\tilde d \leqslant d, \tilde e \leqslant e$ and $\tilde d-\tilde e = d-e$ .

Remark 1.1.6. If E and F are Hermitian vector bundles and $(D_p)_{p \in \mathscr {P}}$ is a family of complex linear elliptic differential operators, then the map $\Lambda _p$ factors through $\operatorname {\mathrm {Hom}}_{\mathbf {C}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$ . Therefore, the hypothesis of Theorem 1.1.5 cannot be satisfied (unless it holds trivially). Of course, this issue is rectified by replacing $\mathbf {R}$ with ${\mathbf {C}}$ throughout the above discussion.

Example 1.1.7 (Brill–Noether theory for holomorphic line bundles over a Riemann surface).

Let $\Sigma $ be a closed, connected Riemann surface of genus g. Let L be a Hermitian line bundle of degree d over $\Sigma $ . Denote by $\mathscr {A}(L)$ the space of unitary connections on L. ⁴ Define the family of complex linear elliptic differential operators

$$ \begin{align*} \bar{\partial}\mskip0.5mu\colon\thinspace \mathscr{A}(L) \to \mathscr{F}(W^{1,2}\Gamma(L),L^2\Omega^{0,1}(\Sigma,L)) \end{align*} $$

by assigning to every connection A the Dolbeault operator .

Let $A \in \mathscr {A}(L)$ . The map $\Lambda _A\mskip 0.5mu\colon \thinspace T_A\mathscr {A}(L) \to \operatorname {\mathrm {Hom}}_{\mathbf {C}}(\ker \bar {\partial }_A, \operatorname {\mathrm {coker}} \bar {\partial }_A)$ can be described concretely as follows. Since the derivative of the map $\bar {\partial }$ is $\mathrm {d}_A\bar {\partial }(a) = a^{0,1}$ , the map $\Lambda _A$ factors through the isomorphism $T_A\mathscr {A}(L) = \Omega ^1(\Sigma ,i\mathbf {R}) \cong \Omega ^{0,1}(\Sigma )$ defined by $a \mapsto a^{0,1}$ . Denote by $\mathscr {L}$ the holomorphic line bundle associated with $\bar {\partial }_A$ . By Serre duality,

$$ \begin{align*} \operatorname{\mathrm{coker}} \bar{\partial}_A = H^1(\Sigma,\mathscr{L}) \cong H^0(\Sigma,K_{\Sigma}\otimes_{\mathbf{C}} \mathscr{L}^*)^*; \end{align*} $$

hence,

$$ \begin{align*} \operatorname{\mathrm{Hom}}_{\mathbf{C}}(\ker \bar{\partial}_A, \operatorname{\mathrm{coker}} \bar{\partial}_A) \cong H^0(\Sigma,\mathscr{L})^* \otimes_{\mathbf{C}} H^0(\Sigma,K_{\Sigma}\otimes_{\mathbf{C}} \mathscr{L}^*)^*. \end{align*} $$

The Petri map

(1.1.8) $$ \begin{align} \varpi_{\mathscr{L}} \mskip0.5mu\colon\thinspace H^0(\Sigma,\mathscr{L}) \otimes_{\mathbf{C}} H^0(\Sigma,K_{\Sigma}\otimes \mathscr{L}^*) \to H^0(\Sigma,K_{\Sigma}) \end{align} $$

is induced by the isomorphism $\mathscr {L} \otimes _{\mathbf {C}} \mathscr {L}^* \cong \mathscr {O}_{\Sigma }$ . The adjoint of $\Lambda _A$ is the composition of the Petri map $\varpi _{\mathscr {L}}$ with the inclusion $H^0(\Sigma ,K_{\Sigma }) \hookrightarrow \Omega ^{1,0}(\Sigma )$ . Here, the duality between $\Omega ^{1,0}(\Sigma )$ and $\Omega ^{0,1}(\Sigma )$ is given by integration.

As a consequence, $\Lambda _A$ is surjective if and only if $\varpi _{\mathscr {L}}$ is injective. If $\varpi _{\mathscr {L}}$ is injective for every $[\mathscr {L}] \in \operatorname {\mathrm {Pic}}^d(\Sigma )$ , then

is a complex submanifold of codimension $(r+1)(g-d+r)$ ; therefore, so is the Brill–Noether locus

cf. [ACGH85, Lemma 1.6, Chapter IV].

This example motivates the following definitions, which are particularly appropriate for first order operators appearing in geometric applications.

Definition 1.1.9. Let $U \subset M$ be an open subset. A family of linear elliptic differential operators $(D_p)_{p \in \mathscr {P}}$ is flexible in U at $p_{\star } \in \mathscr {P}$ if for every $A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$ there is a $\hat p \in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}} D(\hat p)s = As \quad\mod \operatorname{\mathrm{im}} D_{p_{\star}} \end{align*} $$

for every $s \in \ker D_{p_{\star }}$ .

Definition 1.1.10. Let $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ be a linear differential operator. Set

The formal adjoint of D is the linear differential operator $D^{\dagger } \mskip 0.5mu\colon \thinspace L^2\Gamma (F^{\dagger }) \to W^{-k,2}\Gamma (E^{\dagger })$ characterized by

$$ \begin{align*} \int_M {\langle}{s,D^{\dagger} t}{\rangle} = \int_M {\langle}{Ds,t}{\rangle} \end{align*} $$

for $s \in \Gamma (E)$ and $t \in \Gamma (F^{\dagger })$ . Here, ${\langle }{\cdot ,\cdot }{\rangle }$ denotes the canonical pairings $E\otimes E^{\dagger } \to \Lambda ^nT^* M$ and $F\otimes F^{\dagger } \to \Lambda ^nT^* M$ .

Definition 1.1.11. The Petri map $\varpi \mskip 0.5mu\colon \thinspace \Gamma (E)\otimes \Gamma (F^{\dagger }) \to \Gamma (E\otimes F^{\dagger })$ is defined by

Let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ satisfies Petri’s condition in U if the map

$$ \begin{align*} \varpi_{D,U} \mskip0.5mu\colon\thinspace \ker D \otimes \ker D^{\dagger} \to L^1\Gamma(U,E\otimes F^{\dagger}) \end{align*} $$

induced by the Petri map is injective.

Proposition 1.1.12. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. Let $U \subset M$ be an open subset. If $(D_p)_{p \in \mathscr {P}}$ is flexible in U at $p_{\star } \in \mathscr {P}$ and $D_{p_{\star }}$ satisfies Petri’s condition in U, then the map $\Lambda _{p_{\star }}$ defined in Theorem 1.1.5 is surjective.

Proof. Define the map $\mathrm {ev}_p \mskip 0.5mu\colon \thinspace \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F)) \to \operatorname {\mathrm {Hom}}(\ker D_p,\operatorname {\mathrm {coker}} D_p)$ by

$(D_p)_{p \in \mathscr {P}}$ is flexible in U at $p_{\star } \in \mathscr {P}$ if and only $\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }} \subset \operatorname {\mathrm {im}} \Lambda _{p_{\star }}$ . $D_{p_{\star }}$ satisfies Petri’s condition in U if and only if $\mathrm {ev}_{p_{\star }}$ is surjective. To see this, observe the following. Since $\ker D_{p_{\star }}^{\dagger } \cong (\operatorname {\mathrm {coker}} D_{p_{\star }})^*$ , the pairing $\operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }}) \otimes (\ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }) \to \mathbf {R}$ induced by

is perfect, that is, it induces an isomorphism $\operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }})^* \cong \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$ . There is a canonical perfect pairing ${\langle }{\cdot ,\cdot }{\rangle }\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}(E,F) \otimes (E\otimes F^{\dagger }) \to \Lambda ^n T^*M$ . Evidently,

$$ \begin{align*} t(\mathrm{ev}_{p_{\star}}(A)s) = {\langle}{A,\varpi(s\otimes t)}{\rangle}. \end{align*} $$

Therefore, an element $B \in \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$ annihilates $\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }}$ if and only if

$$ \begin{align*} {\langle\!\langle}{\mathrm{ev}_{p_{\star}}(A),B}{\rangle\!\rangle} = \int_M {\langle}{A,\varpi(B)} {\rangle} = 0 \end{align*} $$

for every $A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$ ; that is, $\varpi (B) = 0$ in U. Therefore, $(\operatorname {\mathrm {im}} \mathrm {ev}_{p_{\star }})^{\perp } = \ker \varpi _{D_{p_{\star }},U}$ .

Remark 1.1.13. In Example 1.1.7, $\operatorname {\mathrm {im}} \Lambda _p = \operatorname {\mathrm {im}} \mathrm {ev}_p$ (with $U=\Sigma $ , and $\mathbf {R}$ replaced with ${\mathbf {C}}$ ). Therefore, $\Lambda _p$ being surjective is equivalent to Petri’s condition. Furthermore, tracing through the isomorphisms identifies the restriction of Petri map $\varpi $ to $\ker \bar {\partial }_A\otimes _{\mathbf {C}} \ker \bar {\partial }_A^*$ with the Petri map $\varpi _{\mathscr {L}}$ .

Flexibility is not a rare condition. Petri’s condition appears to be more subtle. (By the uniqueness theorem for ordinary differential equations, it holds for first-order linear elliptic differential operators on $1$ -manifolds. This is somewhat useful; see, for example, [Eft19].) The upcoming Remark 1.1.14(3) hints at the connection between Petri’s condition and the unique continuation property. In Section 1.6, we revisit Petri’s condition and discuss an algebraic criterion due to Wendl for Petri’s condition to hold away from a subset of infinite codimension.

Remark 1.1.14. Assume the situation of Theorem 1.1.5. The following observations are useful in situations where the primary objective is to estimate the codimension of $\mathscr {P}_{d,e}$ .

1. (1) Every $p \in \mathscr {P}_{d,e}$ has an open neighborhood $\mathscr {U}$ in $\mathscr {P}$ such that $\mathscr {P}_{d,e}\cap \mathscr {U}$ is contained in a submanifold of codimension $\operatorname {\mathrm {rk}} \Lambda _p$ .

2. (2) Let $\rho \in {\mathbf {N}}$ , and let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies Petri’s condition up to rank $\rho $ in U if for every nonzero $B \in \ker D\otimes \ker D^{\dagger }$ of rank at most $\rho $ the section $\varpi (B)$ does not vanish on U. (A simple tensor is nonzero tensor of the form $v\otimes w$ . Every tensor B is a sum of simple tensors. The rank of B is the minimal number of simple tensors that sum to B.) If $D_{p_{\star }}$ satisfies this condition and $(D_p)_{p\in \mathscr {P}}$ is flexible in U at $p_{\star } \in \mathscr {P}$ , then

   $$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \min{\lbrace}{\rho,d,e}{\rbrace}\max{\lbrace}{d,e}{\rbrace}.\\[-30pt] \end{align*} $$

   Proof. Set . If $d \leqslant e$ , then choose an injection $\mathbf {R}^{\sigma } \hookrightarrow \ker D_{p_{\star }}$ and set ; otherwise, choose a surjection $\operatorname {\mathrm {coker}} D_{p_{\star }} \twoheadrightarrow \mathbf {R}^{\sigma }$ and set . In either case, composition defines a surjection

   $$ \begin{align*} \pi_{p_{\star}}\mskip0.5mu\colon\thinspace \operatorname{\mathrm{Hom}}(\ker D_{p_{\star}},\operatorname{\mathrm{coker}} D_{p_{\star}}) \to H. \end{align*} $$

   The subspace $\operatorname {\mathrm {im}} \pi _{p_{\star }}^* \subset \operatorname {\mathrm {Hom}}(\ker D_{p_{\star }},\operatorname {\mathrm {coker}} D_{p_{\star }})^* \cong \ker D_{p_{\star }} \otimes \ker D_{p_{\star }}^{\dagger }$ consists of elements of rank at most $\sigma \leqslant \rho $ . The argument of the proof of Proposition 1.1.12 thus shows that $\pi _{p_{\star }} \circ \Lambda _{p_{\star }}$ is surjective. Therefore, $\operatorname {\mathrm {rk}} \Lambda _{p_{\star }} \geqslant \dim H = \min {\lbrace }{\rho ,d,e}{\rbrace }\max {\lbrace }{d,e}{\rbrace }$ .

3. (3) (This is due to Eftekhary [Eft16, Proof of Lemma 4.4].) Let $U \subset M$ be a nonempty open subset. Let $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ be a first linear elliptic differential operator of first order. If $\ker D$ and $\ker D^{\dagger }$ consists of continuous sections, and D and $D^{\dagger }$ have the weak unique continuation property, then D satisfies Petri’s condition up to rank three in U. ⁵ , ⁶

   Proof. Every $B \in \ker D\otimes \ker D^{\dagger }$ can be written as $B = s_1\otimes t_1 + \cdots + s_{\rho }\otimes t_{\rho }$ with , and $s_1,\ldots ,s_{\rho } \in \ker D$ and $t_1,\ldots ,t_{\rho } \in \ker D^{\dagger }$ linearly independent. If $\rho = 1$ and $\varpi (B) = 0$ , then $s_1$ or $t_1$ vanishes on an open subset; hence, by unique continuation, $s_1 = 0$ or $t_1 = 0$ : a contradiction.

   Henceforth, assume that $\rho \geqslant 2$ . Define $\delta ,\varepsilon \mskip 0.5mu\colon \thinspace U \to {\mathbf {N}}_0$ by and . By unique continuation, $\delta $ and $\varepsilon $ are positive on a dense open subset. In fact, $\delta ,\varepsilon \geqslant 2$ on a dense open subset. To see this, observe that if $\delta = 1$ on a nonempty open subset, then there is a nonempty open subset $V \subset U$ and a function $f \in C^{\infty }(V)$ such that $s_1(x) = f(x)s_2(x)$ for every $x \in V$ . Therefore, $\sigma (\mathrm {d} f)s_2 = 0$ with $\sigma $ denoting the symbol of D. Since D is elliptic, f must be constant: a contradiction to $s_1$ and $s_2$ being linearly independent. The same argument applies to $\varepsilon $ .

   If $\rho = 2$ , then there exists an $x \in U$ such that $\delta (x) = \varepsilon (x) = 2$ ; therefore: $\varpi (B)$ does not vanish at x. If $\rho = 3$ , then there is an $x \in U$ such that $\min {\lbrace }{\delta (x),\varepsilon (x)}{\rbrace } \geqslant 2$ . If $\delta (x) = \varepsilon (x) = 3$ , then $\varpi (B)$ evidently does not vanishing at x; otherwise, without loss of generality, $s_1(x)$ and $s_2(x)$ are linearly independent, and $s_3(x) = \lambda _1s_1(x) + \lambda _2s_2(x)$ . In the latter case,

   $$ \begin{align*} \varpi(B)(x) = s_1(x) \otimes ({t_1(x)+\lambda_1t_3(x)}) + s_2(x)\otimes ({t_2(x) + \lambda_2t_3(x)}) \end{align*} $$

   which cannot vanish because $\varepsilon (x) \geqslant 2$ .

   There are examples of first-order linear elliptic operators which fail to satisfy Petri’s condition up to rank four; see [Wen19b, Example 5.5] or Proposition 2.5.4. Finally, a brief warning: The preceding observation is false when $\mathbf {R}$ is replaced with ${\mathbf {C}}$ or $\mathbf {H}$ . The analogue of Petri’s condition only holds up to rank one in this case. (The issue is that $\sigma ({\mathrm {d} f})s_2 =0$ does not imply $\mathrm {d} f = 0$ if f takes values in ${\mathbf {C}}$ or $\mathbf {H}$ .)

1.2 Pulling back and twisting

This section introduces two constructions which produce new linear elliptic operators from old ones: pulling back by a covering map and twisting by a Euclidean local system.

Definition 1.2.1. Let $k \in {\mathbf {N}}_0$ . Let $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ be a covering map with $\tilde M$ connected. ⁷ Let $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ be a linear differential operator of order k. The pullback of D by $\pi $ is the linear differential operator of order k

$$ \begin{align*} \pi^*D \mskip0.5mu\colon\thinspace W^{k,2}\Gamma(\pi^*E) \to L^2\Gamma(\pi^*F) \end{align*} $$

characterized by

$$ \begin{align*} (\pi^*D)(\pi^*s) = \pi^*(Ds).\\[-35pt] \end{align*} $$

Remark 1.2.2. If $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ is a branched covering map of manifolds whose ramification locus is a closed submanifold of codimension two, then $\tilde M$ and M can be equipped with orbifold structures and $\pi $ can be lifted to an unbranched covering map of orbifolds. Section 2.7 discusses this construction in the case of Riemann surfaces; the higher-dimensional case follows immediately from the two-dimensional case and the above local model.

Definition 1.2.3. A Euclidean local system on M is a Euclidean vector bundle ${\underline V}$ over M together with a flat orthogonal connection.

Remark 1.2.4. Let $x_0 \in M$ . Denote by $\pi _1(M,x_0)$ the fundamental group with base-point $x_0$ . If $*$ denotes the usual concatenation of paths, then the multiplication in $\pi _1(M,x_0)$ is defined by . ⁸ Parallel transport induces a monodromy representation

$$ \begin{align*} \mu\mskip0.5mu\colon\thinspace \pi_1(M,x_0) \to \mathrm{O}(V) \end{align*} $$

with V denoting the fiber of ${\underline V}$ over $x_0$ . ${\underline V}$ can be recovered from $\mu $ as follows. Denote by $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ the universal covering map and by $\operatorname {\mathrm {Aut}}(\pi )$ the group of deck transformations. A choice of $\tilde x_0 \in \pi ^{-1}(x_0)$ induces an anti-isomorphism from $\operatorname {\mathrm {Aut}}(\pi )$ to $\pi _1(X,x_0)$ . Therefore, $\tilde M$ is a principal $\pi _1(M,x_0)$ -bundle, and ${\underline V}$ is the associated bundle:

$$ \begin{align*} {\underline V} \cong \tilde M \times_{\mu} V. \end{align*} $$

This sets up a bijection between gauge equivalence classes $[{\underline V}]$ of Euclidean local systems of rank r and equivalence classes $[\mu ]$ of representations $\pi _1(M,x_0) \to \mathrm {O}(r)$ up to conjugation by $\mathrm {O}(r)$ . For a more detailed discussion—in particular, of how the to interpret the above in the category of orbifolds—we refer the reader to [SY19, Sections 2.4 and 2.5].

Definition 1.2.5. Let $k \in {\mathbf {N}}_0$ Let $D \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ be a linear differential operator of order k. Let ${\underline V}$ be a Euclidean local system on M. The twist of D by ${\underline V}$ is the linear differential operator of order k

$$ \begin{align*} D^{\underline V} \mskip0.5mu\colon\thinspace W^{k,2}\Gamma(E\otimes{\underline V}) \to L^2\Gamma(F\otimes{\underline V}) \end{align*} $$

characterized as follows: If U is a open subset M, $s \in \Gamma (U,E)$ , and $f \in \Gamma (U,{\underline V})$ is covariantly constant with respect to the flat connection on ${\underline V}$ , then

$$ \begin{align*} D^{\underline V} (s\otimes f) = (Ds)\otimes f.\\[-33pt] \end{align*} $$

Proposition 1.2.9 shows that the pullback $\pi ^*D$ is equivalent to the twist $D^{\underline V}$ for a suitable choice of ${\underline V}$ . Its statement requires the following preparation.

Definition 1.2.6. Let $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ be a finite covering map. Let E be a Euclidean vector bundle over $\tilde M$ . The pushforward of E by $\pi $ is the unique Euclidean vector bundle $\pi _*E$ over X such that for every $x \in X$

$$ \begin{align*} (\pi_*E)_x = \bigoplus_{\tilde x \in \pi^{-1}(x)} E_{\tilde x} \end{align*} $$

as Euclidean vector spaces and such that a section s of $\pi _*E$ is smooth if and only if the induced section $\tilde s$ of E is smooth.

Remark 1.2.7. The following facts about the construction from Definition 1.2.6 are important.

1. (1) If E is a Euclidean vector bundle over $\tilde M$ , then the sheaf $\Gamma (\cdot ,\pi _*E)$ is the sheaf-theoretic pushforward of the sheaf $\Gamma (\cdot ,E)$ ; that is: There are canonical isomorphisms

   $$ \begin{align*} \Gamma(U,\pi_*E) \cong \Gamma(\pi^{-1}(U),E) \end{align*} $$
   for every open set $U \subset M$ and these are compatible with the restriction maps.

2. (2) If E is a Euclidean local system on $\tilde M$ , then $\pi _*E$ is a Euclidean local system on M: $s \in \Gamma (U,\pi _*E)$ is covariantly constant if and only if $\tilde s \in \Gamma (\pi ^{-1}(U),E)$ is.

3. (3) Let E and F be a Euclidean vector bundles over M and $\tilde M$ respectively. For every $x \in M$ , there is a conical isomorphism

   $$ \begin{align*} \pi_*(\pi^* E\otimes F)_x \cong \bigoplus_{\tilde x \in \pi^{-1}(x)} E_x \otimes F_{\tilde x} \cong (E\otimes \pi_*F)_x. \end{align*} $$

   These assemble into the push-pull formula

   $$ \begin{align*} \pi_*(\pi^* E\otimes F) \cong E \otimes \pi_*F. \end{align*} $$

   In particular,

   $$ \begin{align*} \pi_*(\pi^*E) \cong E \otimes \pi_*{\underline{\mathbf{R}}}. \end{align*} $$

   Here, ${\underline {\mathbf {R}}}$ denotes the trivial rank one Euclidean local system on $\tilde M$ .

Definition 1.2.8. Let G be a group, and let $H < G$ be a subgroup. The normal core of H is the normal subgroup

Proposition 1.2.9. Let $k \in {\mathbf {N}}_0$ . Let $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ be a finite covering map with $\tilde M$ connected. Let $x_0 \in M$ and $\tilde x_0 \in \pi ^{-1}(x_0)$ . Denote by

the characteristic subgroup of the covering map and by N the normal core of C. Set

Denote by ${\underline {\mathbf {R}}}$ the trivial rank one Euclidean local system on $\tilde M$ . Set

Let $D \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ be a linear differential operator of order k. The following hold:

1. (1) The monodromy representation of ${\underline V}$ factors through ; indeed, it is induced by the representation of G on $\operatorname {\mathrm {Map}}(S,\mathbf {R})$ defined by

   

2. (2) The push-pull formula induces isometries

   $$ \begin{align*} \pi_*\mskip0.5mu\colon\thinspace W^{k,2}\Gamma(\pi^*E) \cong W^{k,2}\Gamma(E\otimes{\underline V}) \quad\text{and}\quad \pi_*\mskip0.5mu\colon\thinspace L^2\Gamma(\pi^*F) \cong L^2\Gamma(F\otimes{\underline V}) \end{align*} $$
   such that
   $$ \begin{align*} D^{{\underline V}} = \pi_* \circ \pi^*D \circ \pi_*^{-1}. \end{align*} $$

Remark 1.2.10. If $\pi $ is a normal covering, then $C=N$ and $G = \pi _1(M,x_0)/N$ is isomorphic to its deck transformation group. If $\pi $ has k sheets, then C has index k. Its normal core has index at most $k!$ by an elementary result known as Poincaré’s theorem. This theorem follows from the observation that the kernel of the canonical homomorphism $\pi _1(M,x_0) \to \operatorname {\mathrm {Bij}}(G/C)$ is precisely N and $\operatorname {\mathrm {Bij}}(G/C) \cong S_k$ . Here, $\operatorname {\mathrm {Bij}}(G/C)$ denotes the set of bijections of $G/C$ ,

Proof of Proposition 1.2.9.

The monodromy representation $\mu \mskip 0.5mu\colon \thinspace \pi _1(M,x_0) \to \mathrm {O}(V)$ of ${\underline V}$ is trivial on C; hence, it factors through G. Denote by $\rho \mskip 0.5mu\colon \thinspace (\hat M,\hat x_0) \to (M,x_0)$ the pointed covering map with characteristic subgroup N. $\hat M$ is a principal G-bundle and $\tilde M \cong \hat M \times _G S$ . This implies the assertion about the monodromy representation.

By Remark 1.2.7(3),

$$ \begin{align*} \pi_*\pi^* E \cong E \otimes\pi_*{\underline{\mathbf{R}}} = E\otimes{\underline V}. \end{align*} $$

Denote the resulting isomorphism $\Gamma (\pi ^*E) \cong \Gamma (\pi _*\pi ^*E) \cong \Gamma (E\otimes {\underline V})$ by $\pi _*$ . For $s \in \Gamma (E)$ and $f \in C^{\infty }(\tilde M)$

$$ \begin{align*} \pi_*((\pi^*s)f) = s \otimes \pi_*f. \end{align*} $$

Let U be an open subset of M, $s \in \Gamma (U,E)$ , and $f \in \Gamma (U,{\underline V})$ . Suppose that f is covariantly constant. This is equivalent to the corresponding function on being locally constant. By the characterizing properties of $D^{\underline V}$ and $\pi ^*D$ and since $\pi ^*D$ is a differential operator,

$$ \begin{align*} D^{\underline V} (s\otimes f) = (Ds)\otimes f \end{align*} $$

and

$$ \begin{align*} (\pi^*D)(\pi_*)^{-1} (s\otimes f) = (\pi^*D)(\pi^*s \cdot \tilde f) = \pi^*(Ds)\cdot \tilde f = (\pi_*)^{-1}({Ds\otimes f}). \end{align*} $$

This proves that $D^{{\underline V}} = \pi _* \circ \pi ^*D \circ \pi _*^{-1}$ .

1.3 Equivariant Brill–Noether loci, I: twists

Pulling back and twisting lead to families of linear elliptic differential operators which fail to satisfy the hypotheses of Theorem 1.1.5 (except for a few corner cases). In this section, we formulate a variant of this result which applies to families of twisted linear elliptic differential operators. Throughout this section, assume the following.

Situation 1.3.1. Let $x_0 \in M$ . Let ${\mathfrak V} = ({\underline V}_{\alpha })_{\alpha =1}^m$ be a finite collection of irreducible Euclidean local systems which are pairwise nonisomorphic. (A Euclidean local system is irreducible if it is not a direct sum of two nonzero Euclidean local systems.) For every $\alpha =1,\ldots ,m$ , denote by ${\mathbf {K}}_{\alpha }$ the algebra of parallel endomorphisms of ${\underline V}_{\alpha }$ and set .

Remark 1.3.2. Since ${\underline V}_{\alpha }$ is irreducible, ${\mathbf {K}}_{\alpha }$ is a division algebra; hence, by Frobenius’ Theorem it is isomorphic to either $\mathbf {R}$ , ${\mathbf {C}}$ , or $\mathbf {H}$ and $k_{\alpha } \in {\lbrace }{1,2,4}{\rbrace }$ .

If D is a linear elliptic differential operator, then the twists $D^{{\underline V}_{\alpha }}$ commute with the action of ${\mathbf {K}}_{\alpha }$ . Therefore, $\ker D^{{\underline V}_{\alpha }}$ and $\operatorname {\mathrm {coker}} D^{{\underline V}_{\alpha }}$ are left ${\mathbf {K}}_{\alpha }$ -modules and, hence, right ${\mathbf {K}}_{\alpha }^{\mathrm {op}}$ -modules. Here, ${\mathbf {K}}_{\alpha }^{\mathrm {op}}$ denotes the opposite algebra of ${\mathbf {K}}_{\alpha }$ .

Definition 1.3.3. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $d,e \in {\mathbf {N}}_0^m$ , define the ${\mathfrak V}$ -equivariant Brill–Noether locus $\mathscr {P}_{d,e}^{\mathfrak V}$ by

Remark 1.3.4. Let $i \in \mathbf {Z}^m$ . Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic operators such that $\operatorname {\mathrm {index}}_{{\mathbf {K}}_{\alpha }} D_p^{{\underline V}_{\alpha }} = i_{\alpha }$ for every $p \in \mathscr {P}$ and $\alpha = 1,\ldots ,m$ . If $\mathscr {P}_{d,e}^{\mathfrak V} \neq \varnothing $ , then $d-e = i$ ; in particular: $d_{\alpha } \geqslant i_{\alpha }$ and $e_{\alpha } \geqslant -i_{\alpha }$ .

If M is a manifold, then

$$ \begin{align*} \operatorname{\mathrm{index}}_{{\mathbf{K}}_{\alpha}} D_p^{{\underline V}_{\alpha}} = \operatorname{\mathrm{rk}}_{{\mathbf{K}}_{\alpha}} {\underline V}_{\alpha} \cdot \operatorname{\mathrm{index}} D_p \end{align*} $$

by the Atiyah–Singer index theorem; therefore, the $i_{\alpha }$ all have the same sign. If M is an orbifold, there are corrections terms in the index formula which spoil this relation between the indices; see, for example, Proposition 2.8.6.

Lemma 1.1.4 immediately implies the following.

Theorem 1.3.5. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $p \in \mathscr {P}$ define $\Lambda _p^{\mathfrak V} \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \bigoplus _{\alpha =1}^m \operatorname {\mathrm {Hom}}_{{\mathbf {K}}_{\alpha }}(\ker D_p^{{\underline V}_{\alpha }},\operatorname {\mathrm {coker}} D_p^{{\underline V}_{\alpha }})$ by

Let $d,e \in {\mathbf {N}}_0^m$ and $p \in \mathscr {P}_{d,e}^{\mathfrak V}$ . If $\Lambda _p^{\mathfrak V}$ is is surjective, then the following hold:

1. (1) There is an open neighborhood $\mathscr {U}$ of $p \in \mathscr {P}$ such that $\mathscr {P}_{d,e}^{\mathfrak V} \cap \mathscr {U}$ is a submanifold of codimension

   $$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e}^{\mathfrak V} \cap \mathscr{U}) = \sum_{\alpha=1}^m k_{\alpha} d_{\alpha} e_{\alpha}. \end{align*} $$

2. (2) $\mathscr {P}_{\tilde d,\tilde e}^{\mathfrak V} \neq \varnothing $ for every $\tilde d,\tilde e \in {\mathbf {N}}_0^m$ with $\tilde d \leqslant d, \tilde e \leqslant e$ , and $\tilde d-\tilde e = d-e$ .

Remark 1.3.6. If E and F are Hermitian vector bundles and $(D_p)_{p \in \mathscr {P}}$ is a family of complex linear elliptic differential operators, then Theorem 1.3.5 does not apply; cf. Remark 1.1.6. Again, this issue is rectified by replacing $\mathbf {R}$ with ${\mathbf {C}}$ throughout. In fact, this somewhat simplifies the discussion since ${\mathbf {C}}$ is the unique complex division algebra; hence, there is no need to introduce ${\mathbf {K}}_{\alpha }$ .

Remark 1.3.7 (Twists by arbitrary Euclidean local systems).

Every Euclidean local system ${\underline V}$ decomposes into irreducible local systems

$$ \begin{align*} {\underline V} \cong \bigoplus_{\alpha=1}^m {\underline V}_{\alpha}^{\oplus \ell_{\alpha}} \end{align*} $$

with $\ell _1,\ldots ,\ell _m \in {\mathbf {N}}_0$ for a suitable choice of ${\mathfrak V}$ . For every $\bar d,\bar e \in {\mathbf {N}}_0$ the Brill–Noether locus

is the finite disjoint union of the subsets $\mathscr {P}_{d,e}^{\mathfrak V}$ with $(d,e) \in {\mathbf {N}}_0^m\times {\mathbf {N}}_0^m$ satisfying

$$ \begin{align*} \sum_{\alpha=1}^m \ell_{\alpha} k_{\alpha} d_{\alpha} = \bar d \quad\text{and}\quad \sum_{\alpha=1}^m \ell_{\alpha} k_{\alpha} e_{\alpha} = \bar e. \end{align*} $$

Through this observation Theorem 1.3.5 can be brought to bear on families of linear elliptic differential operators twisted by ${\underline V}$ .

Definition 1.1.9, Definition 1.1.11, and Proposition 1.1.12 have the following analogues in the present situation.

Definition 1.3.8. A family of linear elliptic differential operators $(D_p)_{p \in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ if for every $A \in \Gamma _c(U,\operatorname {\mathrm {Hom}}(E,F))$ there is a $\hat p \in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}} D^{{\underline V}_{\alpha}}(\hat p)s = (A\otimes \mathrm{id}_{{\underline V}_{\alpha}})s \quad\mod \operatorname{\mathrm{im}} D_{p_{\star}}^{{\underline V}_{\alpha}} \end{align*} $$

for every $\alpha = 1,\ldots ,m$ and $s \in \ker D_{p_{\star }}^{{\underline V}_{\alpha }}$ .

Definition 1.3.9. The ${\mathfrak V}$ -equivariant Petri map

$$ \begin{align*} \varpi^{\mathfrak V} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m \Gamma(E\otimes {\underline V}_{\alpha})\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \Gamma(F^{\dagger} \otimes {\underline V}_{\alpha}^*) \to \Gamma(E\otimes F^{\dagger}) \end{align*} $$

is defined by with $\varpi _{\alpha }$ denoting the composition of the Petri map

$$ \begin{align*} \varpi_{\alpha}\mskip0.5mu\colon\thinspace \Gamma(E\otimes {\underline V}_{\alpha})\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \Gamma(F^{\dagger} \otimes {\underline V}_{\alpha}^*) \to \Gamma(E\otimes F^{\dagger} \otimes {\underline V}_{\alpha} \otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} {\underline V}_{\alpha}^*) \end{align*} $$

and the map induced by

$$ \begin{align*} \operatorname{\mathrm{tr}}\mskip0.5mu\colon\thinspace {\underline V}_{\alpha} \otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} {\underline V}_{\alpha}^* \to {\underline{\mathbf{R}}}. \end{align*} $$

Here, is the dual of ${\underline V}_{\alpha }$ . (Its algebra of parallel endomorphisms is ${\mathbf {K}}_{\alpha }^{\mathrm {op}}$ .)

Let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies the ${\mathfrak V}$ -equivariant Petri condition in U if the map

$$ \begin{align*} \varpi_{D,U}^{\mathfrak V} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m \ker D^{{\underline V}_{\alpha}} \otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \ker D^{{\underline V}_{\alpha},\dagger} \to L^1\Gamma(U,E\otimes F^{\dagger}) \end{align*} $$

induced by the ${\mathfrak V}$ -equivariant Petri map is injective. Here, .

Remark 1.3.10. The ${\mathfrak V}$ -equivariant Petri condition appears even more difficult to verify than the Petri condition. It turns out, however, that there is a general method for verifying both of these conditions simultaneously. This will be discussed in Section 1.6.

Proposition 1.3.11. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. Let $U \subset M$ be an open subset. If $(D_p)_{p \in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ and $D_{p_{\star }}$ satisfies the ${\mathfrak V}$ -equivariant Petri condition in U, then the map $\Lambda _{p_{\star }}^{\mathfrak V}$ defined in Theorem 1.3.5 is surjective.

Remark 1.3.12. There are analogues of the observations from Remark 1.1.14 in the equivariant setting.

1. (1) Every $p \in \mathscr {P}_{d,e}^{\mathfrak V}$ has an open neighborhood $\mathscr {U}$ in $\mathscr {P}$ such that $\mathscr {P}_{d,e}^{\mathfrak V}\cap \mathscr {U}$ is contained in a submanifold of codimension $\operatorname {\mathrm {rk}} \Lambda _p^{\mathfrak V}$ .

2. (2) Let $\rho \in {\mathbf {N}}_0^m$ , and let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ satisfies the ${\mathfrak V}$ -equivariant Petri condition up to rank $\rho $ in U if for every nonzero $B = (B_1,\ldots ,B_m) \in \bigoplus _{\alpha =1}^m \ker D_p^{{\underline V}_{\alpha }} \otimes _{\,{\mathbf {K}}_{\alpha }^{\mathrm {op}}} \ker D_p^{\dagger ,{\underline V}_{\alpha }^*}$ with $\operatorname {\mathrm {rk}} B_{\alpha } \leqslant \rho _{\alpha }$ for $\alpha = 1,\ldots ,m$ the section $\varpi ^{\mathfrak V}(B)$ does not vanish on U. If $D_p$ satisfies this condition and $(D_p)_{p\in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ , then

   $$ \begin{align*} \operatorname{\mathrm{rk}} \Lambda_{p_{\star}} \geqslant \sum_{\alpha=1}^m \min{\lbrace}{\rho_{\alpha},d_{\alpha},e_{\alpha}}{\rbrace}\max{\lbrace}{d_{\alpha},e_{\alpha}}{\rbrace}. \end{align*} $$

3. (3) Let $\rho \in {\mathbf {N}}_0^m$ , and let $U \subset M$ be an open subset. Every first-order linear elliptic differential operator $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ satisfies the ${\mathfrak V}$ -equivariant Petri condition up to rank $\rho $ on U provided

   (1.3.13) $$ \begin{align} \sum_{\alpha=1}^m \operatorname{\mathrm{rk}}_{\mathbf{R}} V_{\alpha} \cdot \rho_{\alpha} \leqslant 3. \end{align} $$

   Proof. Set . Denote by $\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$ the universal covering map. Every $s \in \ker D^{{\underline V}_{\alpha }}$ can be regarded as an element $\tilde s \in \Gamma (\pi ^*E\otimes V_{\alpha })^G$ in the space of G-invariant sections, with $G \to \mathrm {O}(V_{\alpha })$ denoting the monodromy representation of ${\underline V}_{\alpha }$ . This section can be regarded as sections $s_1,\ldots ,s_{r_{\alpha }}$ of $\pi ^*E$ . For every $\alpha =1,\ldots ,m$ let $s_1^{\alpha },\ldots ,s_{q_{\alpha }}^{\alpha } \in \ker D^{{\underline V}_{\alpha }}$ be linearly independent over ${\mathbf {K}}_{\alpha }$ . The resulting collection of sections $s_{j,k}^{\alpha } \in \ker \pi ^*D$ ( $\alpha = 1,\ldots ,m, j=1,\ldots ,q_{\alpha }, k=1,\ldots ,r_{\alpha }$ ) are linearly independent. The latter is a consequence of Proposition 1.3.14 applied to $W = \ker \pi ^*D$ . (Analogous statements hold for $D^{\dagger }$ instead of D.) At this point, one can apply the argument in Remark 1.1.14(3).

   Unfortunately, this is not as useful as Remark 1.1.14(3) because equation (1.3.13) is very restrictive; however, it is what lies at the heart of Eftekhary’s proof of the $4$ -rigidity conjecture [Eft16].

We end this section with a useful fact of representation theory, which is essentially a consequence of Schur’s lemma. This fact is needed to justify Remark 1.3.12(3) above. More importantly, it plays a crucial role in verifying the ${\mathfrak V}$ -equivariant Petri condition in Section 1.6, specifically in Proposition 1.6.6.

Let G be a group. For a vector space V with an action of G, denote by $V^G$ the subspace of G-invariant vectors, and set and with W a further vector space with an action of G. We do not assume here that G is finite or that V or W are finite dimensional. For example, as in Remark 1.3.12(3), G could be the monodromy representation of a local system and V and W kernels of linear elliptic operators twisted by that local system.

Proposition 1.3.14. Let $({V_{\alpha }})_{\alpha =1}^m$ be a finite collection of irreducible finite-dimensional orthogonal representations which are pairwise nonisomorphic. Set . For every representation W of G, the map

$$ \begin{align*} \operatorname{\mathrm{tr}} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} (V_{\alpha}\otimes W)^G \to W \end{align*} $$

induced by the trace maps $V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \to \mathbf {R}$ is injective.

Proof of Proposition 1.3.14.

If $\operatorname {\mathrm {tr}}$ is not injective, then there are finite-dimensional ${\mathbf {K}}_{\alpha }$ -linear subspaces $X_{\alpha } \subset (V_{\alpha }\otimes W)^G$ such that

$$ \begin{align*} \operatorname{\mathrm{tr}} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} X_{\alpha} \to W \end{align*} $$

is not injective. (W need not be finite dimensional.)

G acts on $V_{\alpha }^*$ via the contragredient representation and trivially on $X_{\alpha }$ . Choose a ${\mathbf {K}}_{\alpha }$ -sesquilinear inner product on $X_{\alpha }$ (e.g., by choosing a basis $X_{\alpha } \cong {\mathbf {K}}_{\alpha }^{d_{\alpha }}$ ). This exhibits $V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} X_{\alpha }$ as an orthogonal representation. Since $\operatorname {\mathrm {tr}}$ is G-equivariant,

$$ \begin{align*} \ker \operatorname{\mathrm{tr}} \subset \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} X_{\alpha} \end{align*} $$

is an orthogonal subrepresentation.

Since $\ker \operatorname {\mathrm {tr}}$ is an orthogonal representation, it decomposes into irreducible orthogonal representations

$$ \begin{align*} \ker \operatorname{\mathrm{tr}} \cong \bigoplus_{\beta = 1}^n V_{\beta}^* \otimes_{{\mathbf{K}}_{\beta}} {\mathbf{K}}_{\beta}^{d_{\beta}}. \end{align*} $$

A priori, $(V_{\beta })_{\beta =1}^n$ might not be a subset of $(V_{\alpha })_{\alpha =1}^m$ . However, for every copy of $V_{\beta }^*$ appearing in the above decomposition, the induced map

$$ \begin{align*} V_{\beta}^* \to \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} X_{\alpha} \end{align*} $$

is injective. Therefore, by Schur’s lemma, $\beta $ is among the $\alpha $ . Moreover, the image of the of the above map is $V_{\beta }^* \otimes _{{\mathbf {K}}_{\beta }} L$ for some $L \subset X_{\beta }$ with $\dim _{{\mathbf {K}}_{\beta }} L = 1$ . Denote by $S_{\beta } \subset X_{\beta }$ the ${\mathbf {K}}_{\beta }$ -linear subspace spanned by the corresponding lines L. The upshot of this discussion is that

$$ \begin{align*} \ker \operatorname{\mathrm{tr}} = \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} S_{\alpha}. \end{align*} $$

For every nonzero $T \in S_{\alpha }$ , there is a $v^* \in V_{\alpha }^*$ with $\operatorname {\mathrm {tr}}(v^* \otimes T) \neq 0$ . Therefore, $S_{\alpha } = 0$ ; hence, $\ker \operatorname {\mathrm {tr}} = 0$ .

Proposition 1.3.15. In the situation of Proposition 1.3.14, suppose that we have two representations V and W of G. In that case, the trace map

$$ \begin{align*} \operatorname{\mathrm{tr}} \colon \bigoplus_{\alpha=1}^m (V_{\alpha}^*\otimes V)^G \otimes_{\, {\mathbf{K}}_{\alpha}} (V_{\alpha} \otimes W)^G \to (V\otimes W)^G \end{align*} $$

is injective.

Proof. By Proposition 1.3.14, the map

$$ \begin{align*} \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} (V_{\alpha}\otimes W)^G \to W \end{align*} $$

is injective. Tensoring with the identity map $V \to V$ , we get that

$$ \begin{align*} \bigoplus_{\alpha=1}^m (V_{\alpha}^*\otimes V) \otimes_{\,{\mathbf{K}}_{\alpha}} (V_{\alpha}\otimes W)^G \to V\otimes W \end{align*} $$

is injective. This map is G-equivariant, where the action on $(V_{\alpha }\otimes W)^G$ is trivial. Thus, it induces an injective map on the G-invariant parts.

1.4 Equivariant Brill–Noether loci, II: pullbacks

In this section, we formulate a variant of Theorem 1.1.5 which applies to families of linear elliptic differential operators pulled back by a finite normal covering map. While this is not needed in Part 2, this approach to equivariant Brill–Noether theory for elliptic operators is similar in spirit to the framework used by Wendl in his proof of the superrigidity conjecture. In Section 1.5, we relate this approach to the one from Section 1.3.

Throughout this section, assume the following.

Situation 1.4.1. Let $x_0 \in M$ . Let G be the quotient of $\pi _1(M,x_0)$ by a finite index normal subgroup N. Denote by $\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$ a pointed covering map with characteristic subgroup N. Let

$$ \begin{align*} \mu_{\alpha}\mskip0.5mu\colon\thinspace G \to \mathrm{O}(V_{\alpha}) \quad (\alpha = 1,\ldots,m = m(G)) \end{align*} $$

be the irreducible representations of G (up to isomorphism). Set

If $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ is a linear elliptic differential operator of order k, then $\ker \pi ^*D$ and $\operatorname {\mathrm {coker}} \pi ^*D$ are representations of G. Every representation V of G can be decomposed into irreducible representations. The evaluation map defines a G-equivariant isomorphism

(1.4.2) $$ \begin{align} \mathrm{ev}\mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_G(V_{\alpha},V) \otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha} \cong V. \end{align} $$

Hence,

In particular, $d = (d_1,\ldots ,d_m) \in {\mathbf {N}}^m$ determines V up to isomorphism.

Definition 1.4.3. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $d,e \in {\mathbf {N}}_0^m$ define the G -equivariant Brill–Noether locus $\mathscr {P}_{d,e}^G$ by

Remark 1.4.4. Let $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ is a linear elliptic differential operator. The G-equivariant index of $\pi ^*D$ is . Here, $R(G)$ denotes the representation ring of G; its elements are formal differences of isomorphism classes of representations of G. It is a consequence of the above discussion that $R(G) \cong \mathbf {Z}^m$ as abelian groups.

For families of linear elliptic operators with G-equivariant index corresponding to $i \in \mathbf {Z}^m$ , what was said in Remark 1.3.4 applies in the present situation as well.

Lemma 1.1.4 has the following refinement for G-equivariant Fredholm operators.

Lemma 1.4.5. Let X and Y be two Banach spaces equipped with G-actions. Denote by $\mathscr {F}_G(X,Y)$ the space of G-equivariant Fredholm operators. For every $L \in \mathscr {F}_G(X,Y)$ , there is an open neighborhood $\mathscr {U} \subset \mathscr {F}_G(X,Y)$ and a smooth map $\mathscr {S} \mskip 0.5mu\colon \thinspace \mathscr {U} \to \operatorname {\mathrm {Hom}}_G(\ker L,\operatorname {\mathrm {coker}} L)$ such that for every $T \in \mathscr {U}$ there are G-equivariant isomorphisms

$$ \begin{align*} \ker T \cong \ker \mathscr{S}(T) \quad\text{and}\quad \operatorname{\mathrm{coker}} T \cong \operatorname{\mathrm{coker}} \mathscr{S}(T); \end{align*} $$

furthermore, $\mathrm {d}_L\mathscr {S} \mskip 0.5mu\colon \thinspace T_L\mathscr {F}_G(X,Y) \to \operatorname {\mathrm {Hom}}_G(\ker L,\operatorname {\mathrm {coker}} L)$ satisfies

$$ \begin{align*} \mathrm{d}_L\mathscr{S}(\hat L)s = \hat L s \quad\mod \operatorname{\mathrm{im}} L. \end{align*} $$

Proof. The proof of Lemma 1.1.4 carries over provided $\operatorname {\mathrm {coim}} L$ and the lift of $\operatorname {\mathrm {coker}} L$ are chosen G-invariant.

Lemma 1.4.5 immediately implies the following.

Theorem 1.4.6. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. For $p \in \mathscr {P}$ define $\Lambda _p^G \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \operatorname {\mathrm {Hom}}_G(\ker \pi ^*D_p,\operatorname {\mathrm {coker}} \pi ^*D_p)$ by

Let $d,e \in {\mathbf {N}}_0^m$ and $p \in \mathscr {P}_{d,e}^G$ . If $\Lambda _p^G$ is surjective, then the following hold:

1. (1) There is an open neighborhood $\mathscr {U}$ of $p \in \mathscr {P}$ such that $\mathscr {P}_{d,e}^G \cap \mathscr {U}$ is a submanifold of codimension

   $$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_{d,e}^G \cap \mathscr{U}) = \sum_{\alpha=1}^m k_{\alpha} d_{\alpha} e_{\alpha}. \end{align*} $$

2. (2) $\mathscr {P}_{\tilde d,\tilde e}^G \neq \varnothing $ for every $\tilde d,\tilde e \in {\mathbf {N}}_0^m$ with $\tilde d \leqslant d, \tilde e \leqslant e$ and $\tilde d-\tilde e = d-e$ .

Remark 1.4.7 (Pullbacks by arbitrary covering maps).

Suppose that $\pi \mskip 0.5mu\colon \thinspace \tilde M \to M$ is a finite covering map with characteristic subgroup $C < \pi _1(M,x_0)$ . Denote by N the normal core of C, denote by $\rho \mskip 0.5mu\colon \thinspace (\hat M,\hat x_0) \to (M,x_0)$ the pointed covering map with characteristic subgroup N and set . For the decomposition (1.4.2) of $\operatorname {\mathrm {Map}}(S,\mathbf {R})$ is

$$ \begin{align*} \operatorname{\mathrm{Map}}(S,\mathbf{R}) \cong \bigoplus_{\alpha=1}^m (V_{\alpha}^*)^C\otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha}; \end{align*} $$

indeed: The map $\mathrm {ev}_{[\mathbf {1}]}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },\operatorname {\mathrm {Map}}(S,\mathbf {R})) \to V_{\alpha }^*$ defined by is injective and its image is $(V_{\alpha }^*)^C$ . Therefore, by Proposition 1.2.9(2)

$$ \begin{align*} \ker \pi^*D &\cong \bigoplus_{\alpha=1}^m V_{\alpha}^C\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\ker \rho^*D) \quad\text{and} \\ \operatorname{\mathrm{coker}} \pi^*D &\cong \bigoplus_{\alpha=1}^m V_{\alpha}^C\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\operatorname{\mathrm{coker}} \rho^*D). \end{align*} $$

With the above in mind, Theorem 1.4.6 can be brought to bear on nonnormal covering maps; cf. Remark 1.3.7.

Definition 1.1.9, Definition 1.1.11, and Proposition 1.1.12 have the following analogues in the present situation.

Definition 1.4.8. A family of linear elliptic differential operators $(D_p)_{p \in \mathscr {P}}$ is G -equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ if for every $A \in \Gamma (\operatorname {\mathrm {Hom}}(E,F))$ supported in U there is a $\hat p \in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}} (\pi^*D)(\hat p)s = (\pi^*A)s \quad\mod \operatorname{\mathrm{im}} \pi^*D_{p_{\star}} \end{align*} $$

for every $s \in \ker \pi ^*D_{p_{\star }}$ .

Definition 1.4.9. Let $U \subset M$ be an open subset. A linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ satisfies the G -equivariant Petri condition in U if the map

$$ \begin{align*} \varpi_{D,U}^G\mskip0.5mu\colon\thinspace ({\ker \pi^*D \otimes \ker \pi^*D^{\dagger}})^G \to \Gamma(\pi^{-1}(U),\pi^*E\otimes \pi^*F^{\dagger})^G \end{align*} $$

induced by the Petri map is injective.

Remark 1.4.10. Remark 1.3.10 applies to the the G-equivariant Petri condition as well.

Proposition 1.4.11. Let $(D_p)_{p \in \mathscr {P}}$ be a family of linear elliptic differential operators. Let $U \subset M$ be an open subset. If $(D_p)_{p \in \mathscr {P}}$ is G-equivariantly flexible in U at $p_{\star } \in \mathscr {P}$ and $D_{p_{\star }}$ satisfies the G-equivariant Petri condition in U, then the map $\Lambda _{p_{\star }}^G$ defined in Theorem 1.4.6 is surjective.

1.5 Equivariant Brill–Noether loci, III: comparison

This section discusses the relation between the two approaches to Brill–Noether theory of equivariant elliptic operators: using local systems, discussed in Section 1.3 and using group actions, discussed in Section 1.4. Results from these sections are not used in Part 2. Throughout this section, assume Situation 1.4.1. This yields an instance of Situation 1.3.1 by setting

Denote by $\sigma \in S_m$ the permutation such that $V_{\alpha }^* \cong V_{\sigma (\alpha )}$ . The following summarizes the what lies at the heart of the relation.

Proposition 1.5.1. Let $D\mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (E) \to L^2\Gamma (F)$ be a linear differential operator of order k. The following hold:

1. (1) The action of G by deck transformations of $\pi $ induces a G-action on the local system

   

   The isomorphisms $\pi _* \mskip 0.5mu\colon \thinspace W^{k,2}\Gamma (\pi ^*E) \cong W^{k,2}\Gamma (E\otimes {\underline V})$ and $\pi _* \mskip 0.5mu\colon \thinspace L^2\Gamma (\pi ^*F) \cong L^2\Gamma (F\otimes {\underline V})$ from Proposition 1.2.9(2) are G-equivariant.

2. (2) There is a G-equivariant isomorphism

   $$ \begin{align*} \phi \mskip0.5mu\colon\thinspace {\underline V} \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} {\underline V}_{\alpha}. \end{align*} $$

   Here, G acts on $V_{\alpha }^*$ .

3. (3) Denote by

   $$ \begin{align*} \psi_E \mskip0.5mu\colon\thinspace W^{k,2}\Gamma(\pi^*E) \cong W^{k,2}\Gamma(E \otimes {\underline V}) \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} W^{k,2}\Gamma(E \otimes {\underline V}_{\alpha}) \end{align*} $$
   the G-equivariant isomorphisms induced by $\pi _*$ and $\phi $ (and analogously for F and $F^{\dagger }$ ). The composition
   $$ \begin{align*} \psi_F \circ \pi^*D \circ \psi_E^{-1} \end{align*} $$
   agrees with
   $$ \begin{align*} \bigoplus_{\alpha=1}^m \mathrm{id}_{V_{\alpha}^*} \otimes_{\,{\mathbf{K}}_{\alpha}} D^{{\underline V}_{\alpha}} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m V_{\alpha}^*\otimes_{\,{\mathbf{K}}_{\alpha}} W^{k,2}\Gamma(E\otimes {\underline V}_{\alpha}) \to \bigoplus_{\alpha=1}^m V_{\alpha}^*\otimes_{\,{\mathbf{K}}_{\alpha}} L^2\Gamma(F\otimes {\underline V}_{\alpha}). \end{align*} $$

4. (4) Define $\gamma $ to be the composition of the isomorphisms

   

   Here, $(\star )$ is induced by the identification

   $$ \begin{align*} ({V_{\alpha}^* \otimes V_{\sigma(\beta)}})^G = \begin{cases} {\mathbf{K}}_{\alpha}^{\mathrm{op}} & \text{if } \alpha = \sigma(\beta) \\ 0 & \text{otherwise}. \end{cases} \end{align*} $$

   The following diagram commutes:

   (1.5.2)

   

Proof. Let $g \in G$ . Denote by $\delta _g$ the corresponding deck transformation: . There is a canonical isomorphism ${\underline {\mathbf {R}}} \cong ({\delta _g})_*{\underline {\mathbf {R}}}$ identifying ${\underline {\mathbf {R}}}_{\tilde x} = \mathbf {R} = ({(\delta _g)_*{\underline {\mathbf {R}}}})_{\tilde x} = {\underline {\mathbf {R}}}_{\tilde xg}$ . This defines an isomorphism

(1.5.3) $$ \begin{align} {\underline V} = \pi_*{\underline{\mathbf{R}}} \cong \pi_*(\delta_g)_*{\underline{\mathbf{R}}} = \pi_*{\underline{\mathbf{R}}} \cong {\underline V}. \end{align} $$

A moment’s thought shows that this isomorphism maps $v \in {\underline V}_x = C^{\infty }(\pi ^{-1}(x),\mathbf {R})$ to $gv \in {\underline V}_x$ defined by . These isomorphisms (1.5.3) assemble into a G-action on ${\underline V}$ . This description makes (1) evident.

The left and right regular representations of G on

are defined by

respectively. By Proposition 1.2.9(1), the monodromy representation of ${\underline V}$ is $\lambda $ ; that is, ${\underline V} \cong \tilde M \times _{\lambda } \mathbf {R}[G]$ . Since $\lambda $ and $\rho $ commute, $\rho $ defines an action of G on ${\underline V}$ . This is precisely the action described above.

Since $\lambda _g$ and $\rho _h$ commute, $(g,h) \mapsto \lambda _g\circ \rho _h$ defines a representation of $G\times G$ on $\mathbf {R}[G]$ . $G\times G$ also acts on $V_{\alpha }^*\otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha }$ via $(g,h) \mapsto \mu _{\alpha }(h^{-1})^* \otimes \mu _{\alpha }(g)$ . The isomorphism (1.4.2) corresponding to $\lambda $ reads

$$ \begin{align*} \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_G(V_{\alpha},\mathbf{R}[G]) \otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha} \cong \mathbf{R}[G]. \end{align*} $$

$\operatorname {\mathrm {Hom}}_G(V_{\alpha },\mathbf {R}[G])$ inherits a G-action from $\rho $ . The map $\mathrm {ev}_{\mathbf {1}}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },\mathbf {R}[G]) \to V_{\alpha }^*$ defined by is a G-equivariant isomorphism. This yields the $G\times G$ -equivariant Peter–Weyl isomorphism

(1.5.4) $$ \begin{align} \psi\mskip0.5mu\colon\thinspace \mathbf{R}[G] \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha}. \end{align} $$

It induces a G-equivariant isomorphism

(1.5.5) $$ \begin{align} {\underline V} \cong \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} {\underline V}_{\alpha}. \end{align} $$

This proves (2).

(2) and Proposition 1.2.9(2) imply (3).

It suffices to prove (4) for $M = {\lbrace }{\mathbf {1}}{\rbrace }$ and $\tilde M = G$ . In this case, E and $F^{\dagger }$ are vector spaces, $\Gamma (\pi ^*E) = \operatorname {\mathrm {Map}}(G,E) = \mathbf {R}[G]\otimes E$ with the G-action induced by $\rho $ , $\Gamma (E \otimes {\underline V}_{\alpha }) = E \otimes V_{\alpha }$ (and analogously for $F^{\dagger }$ and $E\otimes F^{\dagger }$ ). The diagram (1.5.2) becomes

(1.5.6)

Since every map in this diagram has a factor $\mathrm {id}_{E\otimes F^{\dagger }}$ , it suffices to prove that it commutes for $E = F^{\dagger } = \mathbf {R}$ .

The map $\varpi \mskip 0.5mu\colon \thinspace (\mathbf {R}[G]\otimes \mathbf {R}[G])^G \to (\mathbf {R}[G])^G$ is the pointwise multiplication and the map $(\pi ^*)^{-1}\mskip 0.5mu\colon \thinspace \mathbf {R}[G]^G \to \mathbf {R}$ is evaluation at $\mathbf {1}$ . Therefore,

$$ \begin{align*} (\pi^*)^{-1} \circ \varpi \, \bigg({\sum\nolimits_{g,h \in G} a_{g,h} \cdot g \otimes h}\bigg) = a_{\mathbf{1},\mathbf{1}}. \end{align*} $$

The computation of the composition $\varpi ^{\mathfrak V} \circ \gamma \circ (\psi \otimes \psi )$ relies on the following.

Proposition 1.5.7. After identifying $V_{\alpha }^* \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } = \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha })$ , the Peter–Weyl isomorphism (1.5.4) is given by

$$ \begin{align*} \psi(g) = \bigoplus_{\alpha=1}^m \frac{\dim_{{\mathbf{K}}_{\alpha}} V_{\alpha}}{{\lvert}{G}{\rvert}} \cdot \mu_{\alpha}(g). \end{align*} $$

Proof. The inverse of the evaluation map $\mathrm {ev}\mskip 0.5mu\colon \thinspace \bigoplus _{\alpha =1}^m \operatorname {\mathrm {Hom}}_G(V_{\alpha },V) \otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \cong V$ is the map $\Pi = (\Pi _1,\ldots ,\Pi _m)$ with

Here,

, $e_{\alpha ,i}$ ( $i=1,\ldots ,r_{\alpha }$ ) is a basis of $V_{\alpha }$ and $e_{\alpha ,i}^*$ ( $i=1,\ldots ,r_{\alpha }$ ) is the dual basis of $V_{\alpha }^*$ . Indeed,

$$ \begin{align*} \mathrm{ev} \circ \Pi(v) &= \sum_{\alpha=1}^m \frac{\dim_{{\mathbf{K}}_{\alpha}} V_{\alpha}}{{\lvert}{G}{\rvert}} \sum_{g \in G} \operatorname{\mathrm{tr}}(\mu_{\alpha}(g^{-1})) \cdot \mu(g) v \\ &= \frac{1}{{\lvert}{G}{\rvert}} \sum_{\alpha=1}^m \sum_{g \in G} \operatorname{\mathrm{tr}}(\lambda(g^{-1})) \cdot \mu(g) v \\ &= v. \end{align*} $$

Here, the the first identity follows by direct inspection, the second uses the existence of the Peter–Weyl isomorphism (1.5.4), and the last identity follows by direct computation of ${\operatorname {\mathrm {tr}}} \circ \lambda $ . (The composition of $\Pi _{\alpha }$ with $\mathrm {ev}_{\alpha } \mskip 0.5mu\colon \thinspace \operatorname {\mathrm {Hom}}_G(V_{\alpha },V)\otimes _{\,{\mathbf {K}}_{\alpha }} V_{\alpha } \to V$ is the projection to the $V_{\alpha }$ -isotypic component.)

The Peter–Weyl isomorphism (1.5.4) is the composition

$$ \begin{align*} \psi \mskip0.5mu\colon\thinspace \mathbf{R}[G] \xrightarrow{\Pi} \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_G(V_{\alpha},\mathbf{R}[G]) \otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha} \xrightarrow{\bigoplus_{\alpha=1}^m {\mathrm{ev}_{\mathbf{1}}}\otimes \mathrm{id}_{V_{\alpha}}} \bigoplus_{\alpha=1}^m V_{\alpha}^* \otimes_{\,{\mathbf{K}}_{\alpha}} V_{\alpha}. \end{align*} $$

By direct computation

$$ \begin{align*} \psi(g) = \bigg({ \frac{\dim_{{\mathbf{K}}_{\alpha}} V_{\alpha}}{{\lvert}{G}{\rvert}} \sum_{i=1}^{r_{\alpha}} \mu_{\alpha}^*(g^{-1}) e_{\alpha,i}^* \otimes e_{\alpha,i} : \alpha=1,\ldots,m }\bigg). \end{align*} $$

This yields the asserted expression for $\psi $ .

In the definition of $\gamma $ , the map $(V_{\alpha }^* \otimes V_{\alpha })^G \to {\mathbf {K}}_{\alpha }$ in $(\star )$ is induced by the composition

$$ \begin{align*} V_{\alpha}^* \otimes V_{\alpha} \to V_{\alpha}^* \otimes_{{\mathbf{K}}_{\alpha}} V_{\alpha} \xrightarrow{\frac{\operatorname{\mathrm{tr}}_{{\mathbf{K}}_{\alpha}}}{\dim_{{\mathbf{K}}_{\alpha}} V_{\alpha}}} {\mathbf{K}}_{\alpha}. \end{align*} $$

Therefore, $\gamma $ is induced by $1/\dim _{{\mathbf {K}}_{\alpha }} V_{\alpha }$ times the map $\operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha }) \otimes \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }^{\mathrm {op}}}(V_{\alpha }^*) \to \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha })$ , $A\otimes B \mapsto A \circ B^*$ . The Petri map $\varpi ^{\mathfrak V}$ is the sum of the traces $\operatorname {\mathrm {tr}}\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {End}}_{{\mathbf {K}}_{\alpha }}(V_{\alpha }) \to \mathbf {R}$ . Therefore,

$$ \begin{align*} \varpi^{\mathfrak V} \circ \gamma \circ (\psi\otimes \psi) \bigg({\sum\nolimits_{g,h \in G} a_{g,h} \cdot g \otimes h}\bigg) &= \sum_{g,h \in G} a_{g,h} \sum_{\alpha=1}^m \frac{\dim_{{\mathbf{K}}_{\alpha}} V_{\alpha}}{{\lvert}{G}{\rvert}^2} \cdot \operatorname{\mathrm{tr}}(\mu_{\alpha}(gh^{-1})) \\ &= \frac{1}{{\lvert}{G}{\rvert}} \sum_{g \in G} a_{g,g} \\ &= a_{\mathbf{1},\mathbf{1}}. \end{align*} $$

Here, the second identity follows as in the proof of Proposition 1.5.7 above, and the third identity uses the G-invariance: $a_{g,g} = a_{\mathbf {1},\mathbf {1}}$ .

With Proposition 1.5.1 in hand, the discussions in Section 1.3 and Section 1.4 can be related as follows:

1. (1) By Proposition 1.5.1(3), For every $\alpha =1,\ldots ,m$ the isomorphisms $\psi _E$ and $\psi _F$ induce isomorphisms

   $$ \begin{align*} \operatorname{\mathrm{Hom}}_G(V_{\alpha},\ker \pi^*D) \cong \ker D^{{\underline V}_{\alpha}^*} \quad\text{and}\quad \operatorname{\mathrm{Hom}}_G(V_{\alpha},\operatorname{\mathrm{coker}} \pi^*D) \cong \operatorname{\mathrm{coker}} D^{{\underline V}_{\alpha}^*} \end{align*} $$

   (and analogously for $\pi ^* D^{\dagger }$ and $D^{{\underline V}_{\alpha },\dagger }$ ). If V and W are representations of G, then equation (1.4.2) induces isomorphisms

   $$ \begin{align*} \operatorname{\mathrm{Hom}}_G(V,W) &\cong \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,V),\operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,W)) \quad\text{and} \\ (V\otimes W)^G &\cong \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_G(V_{\alpha}^*,V) \otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \operatorname{\mathrm{Hom}}_G(V_{\alpha},W). \end{align*} $$

   Hence, there are isomorphisms

   $$ \begin{align*} \eta\mskip0.5mu\colon\thinspace \operatorname{\mathrm{Hom}}_G(\ker \pi^*D,\operatorname{\mathrm{coker}} \pi^*D) &\to \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\ker D^{{\underline V}_{\alpha}},\operatorname{\mathrm{coker}} D^{{\underline V}_{\alpha}}) \quad\text{and} \\ \tau\mskip0.5mu\colon\thinspace ({\ker \pi^*D \otimes \ker \pi^*D^{\dagger}})^G &\to \bigoplus_{\alpha=1}^m \ker D^{{\underline V}_{\alpha}}\otimes_{\,{\mathbf{K}}_{\alpha}^{\mathrm{op}}} \ker D^{{\underline V}_{\alpha},\dagger}. \end{align*} $$

2. (2) In the situation of Definition 1.3.3 and Definition 1.4.3,

   $$ \begin{align*} \mathscr{P}_{d,e}^G = \mathscr{P}_{\sigma^*d,\sigma^*e}^{\mathfrak V} \end{align*} $$
   with $(\sigma ^*d)_{\alpha } = d_{\sigma (\alpha )}$ and $(\sigma ^*e)_{\alpha } = e_{\sigma (\alpha )}$ .

3. (3) In the situation of Theorem 1.3.5 and Theorem 1.4.6,

   $$ \begin{align*} \Lambda_p^{\mathfrak V} = \eta \circ \Lambda_p^G. \end{align*} $$

4. (4) In the situation of Definition 1.4.8, the maps

   $$ \begin{align*} \mathrm{ev}_p^{\mathfrak V}&\mskip0.5mu\colon\thinspace \Gamma_c(U,\operatorname{\mathrm{Hom}}(E,F)) \to \bigoplus_{\alpha=1}^m \operatorname{\mathrm{Hom}}_{{\mathbf{K}}_{\alpha}}(\ker D_p^{{\underline V}_{\alpha}},\operatorname{\mathrm{coker}} D_p^{{\underline V}_{\alpha}}) \quad\text{and} \\ \mathrm{ev}_p^G &\mskip0.5mu\colon\thinspace \Gamma_c(U,\operatorname{\mathrm{Hom}}(E,F)) \to \operatorname{\mathrm{Hom}}_G(\ker \pi^*D_p,\operatorname{\mathrm{coker}} \pi^*D_p) \end{align*} $$
   defined by
   
   satisfy
   $$ \begin{align*} \mathrm{ev}_p^{\mathfrak V} = \eta \circ \mathrm{ev}_p^G. \end{align*} $$

   Therefore, $(D_p)_{p \in \mathscr {P}}$ is G-equivariantly flexible in U at $p \in \mathscr {P}$ if and only if it is ${\mathfrak V}$ -equivariantly flexible in U at p.

5. (5) By Proposition 1.5.1(4), in the situation of Definition 1.4.9, the map $\varpi _{D,U}^G$ satisfies

   (1.5.8) $$ \begin{align} \varpi_{D,U}^G = \pi^* \circ \varpi_{D,U}^{\mathfrak V} \circ \tau. \end{align} $$

   Therefore, D satisfies the G-equivariant Petri condition in U if and only if it satisfies the ${\mathfrak V}$ -equivariant Petri condition in U.

1.6 Petri’s condition revisited

While Petri’s condition typically is hard to verify for any particular elliptic operator, one can sometimes prove that it is satisfied for a generic element of a family of operators. Theorem 1.6.17 provides a useful tool for proving such statements. This result has been developed by Wendl [Wen19b, Section 5.2] and was the essential innovation which allowed Wendl to prove the superrigidity conjecture.

Throughout this section, let $x \in M$ and, furthermore, amend Definition 1.1.1 as follows.

Definition 1.6.1. Let $k \in {\mathbf {N}}_0$ . A family of linear elliptic differential operators of order k with smooth coefficients is a family of linear elliptic differential operators $(D_p)_{p \in \mathscr {P}}$ of the form

$$ \begin{align*} D(p) = \sum_{\ell = 0}^k a_{\ell}(p,\cdot) \nabla^{\ell} \end{align*} $$

with $a_{\ell }$ a smooth section of $\mathrm {pr}_2^*\operatorname {\mathrm {Hom}}(T^*M^{\otimes \ell } \otimes E,F)$ over $\mathscr {P} \times M$ ( $\ell = 0,\ldots ,k$ ).

Let us begin by introducing the following algebraic variant of Petri’s condition.

Definition 1.6.2. Denote by $\mathscr {E}$ the sheaf of sections of E and by $\mathscr {E}_x$ its stalk at x; that is,

If $s \in \mathscr {E}_x$ vanishes at x, then its derivative at x does not depend on the choice of a local trivialization and defines an element $\mathrm { d}_xs \in \operatorname {\mathrm {Hom}}(T_xM,E_x)$ . If $\mathrm {d}_xs = 0$ , then s has a second derivative $\mathrm {d}_x^2s \in \operatorname {\mathrm {Hom}}(S^2T_xM,E_x)$ at x. Here, $S^j T_x M$ is the j-th symmetric tensor power. In general, if $s(x), \mathrm {d}_xs, \ldots , \mathrm {d}_x^{j-1}s$ vanish, then s is said to vanish to $(j-1)^{\text {st}}$ order and its $j^{\text {th}}$ derivative

$$ \begin{align*} \mathrm{d}_x^j s \in \operatorname{\mathrm{Hom}}(S^j T_xM,E_x) \end{align*} $$

is defined. The vanishing order filtration $\mathscr {V}_{\bullet }\mathscr {E}_x$ on $\mathscr {E}_x$ is defined by

for $j \in {\mathbf {N}}_0$ and

for $j \in {\mathbf {N}}$ . For $\ell \in {\mathbf {N}}_0$ , the $\ell $ -jet space of E at x is

The $\infty $ -jet space of E at x is

For $\ell \in {\mathbf {N}}_0\cup {\lbrace }{\infty }{\rbrace }$ , the $\ell $ -jet of a linear differential operator $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ of order k with smooth coefficients is the linear map

$$ \begin{align*} J_x^{\ell} D \mskip0.5mu\colon\thinspace J_x^{k+\ell}E \to J_x^{\ell} F. \end{align*} $$

induced by D.

Definition 1.6.3. The $\infty $ -jet of a linear elliptic differential operator $J_x^{\infty } D\mskip 0.5mu\colon \thinspace J_x^{\infty } E \to J_x^{\infty } F$ satisfies the $\infty $ -jet Petri condition if the map

$$ \begin{align*} \varpi_{J_x^{\infty} D}\mskip0.5mu\colon\thinspace \ker J_x^{\infty} D \otimes \ker J_x^{\infty} D^{\dagger} \to J_x^{\infty}(E \otimes F^{\dagger}) \end{align*} $$

induced by the Petri map is injective.

The $\infty $ -jet Petri condition and the equivariant Petri conditions are related by the following proposition.

Definition 1.6.4. Let $x \in M$ . Let U be an open neighborhood of $x \in M$ . A differential operator $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ has the strong unique continuation property at x in U if the map

$$ \begin{align*} \ker({D \mskip0.5mu\colon\thinspace \Gamma(U,E) \to \Gamma(U,F)}) \to \ker J_x^{\infty} D \end{align*} $$

is injective.

Remark 1.6.5. If D has smooth coefficients, satisfies $D^*D = \nabla ^*\nabla + \text {lower order terms}$ and U is connected, then it has the strong unique continuation property at $x \in U$ [Cor56; Aro57]; see also [GL87; Kaz88] for streamlined proofs using Almgren’s frequency function.

Proposition 1.6.6. Assume Situation 1.3.1 (or Situation 1.4.1). Let $x \in M$ . Let $U \subset M$ be an open neighborhood of x. Let $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ be a linear elliptic differential operator with smooth coefficients. Suppose that D and $D^{\dagger }$ possess the strong unique continuation property at x in U. If $J_x^{\infty } D$ satisfies the $\infty $ -jet Petri condition, then D satisfies the ${\mathfrak V}$ -equivariant (or G-equivariant) Petri condition in U.

Proof. By Section 1.5, it suffices to consider Situation 1.3.1. Set

. Denote by $\pi \mskip 0.5mu\colon \thinspace (\tilde M,\tilde x_0) \to (M,x_0)$ the universal covering map. Set

. The upcoming arguments prove that

$$ \begin{align*} \varpi_{\pi^*D,\tilde U} \mskip0.5mu\colon\thinspace \ker \pi^*D \otimes \ker (\pi^*D)^{\dagger} \to \Gamma(\tilde U,\pi^*E\otimes \pi^*F^{\dagger}) \end{align*} $$

is injective. Let $\tilde x \in \pi ^{-1}(x)$ . Since $\pi $ is a covering map,

$$ \begin{align*} J_x^{\infty} D = J_{\tilde x}^{\infty} \pi^*D \quad\text{and}\quad J_x^{\infty} D^{\dagger} = J_{\tilde x}^{\infty} \pi^*D^{\dagger}. \end{align*} $$

Therefore, there is a commutative diagram

Since D and $D^{\dagger }$ have the strong unique continuation property at $x \in U$ , $\pi ^*D$ and $\pi ^*D^{\dagger }$ have the strong unique continuation property at $\tilde x \in \tilde U$ . Consequently, the left vertical map is injective. Therefore, since $\varpi _{J_x^{\infty } D}$ is injective, so is $\varpi _{\pi ^*D,\tilde U}$ .

Every $s \in \Gamma (E \otimes {\underline V}_{\alpha })$ can be regarded as an element $\tilde s \in ({\Gamma (\pi ^*E) \otimes V_{\alpha }})^G$ . This establishes an isomorphism $\ker D_p^{{\underline V}_{\alpha }} \cong (\ker \pi ^*D_p \otimes V_{\alpha })^G$ (and similarly for $D^{\dagger }$ ). Consider the commutative diagram

Here, $\operatorname {\mathrm {tr}}$ is the sum of the maps induced by the trace maps $V_{\alpha } \otimes V_{\alpha }^* \to \mathbf {R}$ . It is a consequence of Proposition 1.3.14 that the map $\operatorname {\mathrm {tr}}$ is injective. Therefore, since $\varpi _{\pi ^*D,\tilde U}$ is injective, so is $\varpi _{D,U}^{\mathfrak V}$ .

The failure of the $\infty $ -jet Petri condition manifests itself at the level of symbols as follows.

Definition 1.6.7. Let $k \in {\mathbf {N}}_0$ . A symbol of order k is an element $\sigma \in S^kT_xM\otimes \operatorname {\mathrm {Hom}}(E_x,F_x)$ . Since every $v \in T_xM$ defines a derivation $\partial _v$ on the polynomial algebra $S^{\bullet } T_x^* M$ , every symbol $\sigma $ defines a formal differential operator

$$ \begin{align*} \hat\sigma\mskip0.5mu\colon\thinspace S^{\bullet} T_x^* M\otimes E_x \to S^{\bullet} T_x^* M\otimes F_x. \end{align*} $$

The adjoint symbol $\sigma ^{\dagger } \in S^kT_xM\otimes \operatorname {\mathrm {Hom}}(F_x^{\dagger },E_x^{\dagger })$ is $(-1)^k$ -times the image of $\sigma $ under the the map induced by the canonical isomorphism $\operatorname {\mathrm {Hom}}(E_x,F_x) \cong \operatorname {\mathrm {Hom}}(F_x^{\dagger },E_x^{\dagger })$ .

Remark 1.6.8. Here is an explicit description of the above provided a basis $(\partial _1,\ldots ,\partial _n)$ of $T_xM$ has been chosen. Denote the dual basis of $T_x^*M$ by $(x_1,\ldots ,x_n)$ and set (= $\delta _{ij}$ ). This exhibits $S^{\bullet } T_x^*M$ as the polynomial ring $\mathbf {R}[x_1,\ldots ,x_n]$ . Evidently, $\partial _i$ acts on $\mathbf {R}[x_1,\ldots ,x_n]$ by differentiation. If the symbol $\sigma $ is

$$ \begin{align*} \sigma = \sum_{{\lvert}{\alpha}{\rvert} = k} \partial^{\alpha} \otimes a_{\alpha} \end{align*} $$

with the sum taken over all multi-indices $\alpha \in {\mathbf {N}}_0^n$ of length ${\lvert }{\alpha }{\rvert } = k$ , then

$$ \begin{align*} \hat\sigma = \sum_{{\lvert}{\alpha}{\rvert} = k} a_{\alpha} \cdot \partial_1^{\alpha_1} \cdots \partial_n^{\alpha_n}.\\[-40pt] \end{align*} $$

The symbol of a linear differential operator $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ of order k is a section $\sigma (D) \in \Gamma (S^k TM \otimes \operatorname {\mathrm {Hom}}(E,F))$ . Its value $\sigma _x(D)$ at $x \in M$ is a symbol in the above sense and depends only on $J_x^0 D$ . Furthermore, $\sigma _x(D^{\dagger }) = \sigma _x(D)^{\dagger }$ .

Definition 1.6.9. The polynomial Petri map $\hat \varpi \mskip 0.5mu\colon \thinspace (S^{\bullet } T_x^* M\otimes E_x) \otimes (S^{\bullet } T_x^* M\otimes F_x^{\dagger }) \to S^{\bullet } T_x^* M\otimes E_x\otimes F_x^{\dagger }$ is defined by

A symbol $\sigma \in S^kT_xM\otimes \operatorname {\mathrm {Hom}}(E_x,F_x)$ satisfies the polynomial Petri condition if the map

$$ \begin{align*} \hat\varpi_{\sigma}\mskip0.5mu\colon\thinspace \ker \hat\sigma \otimes \ker\hat\sigma^{\dagger} \to S^{\bullet} T_x^* M\otimes E_x\otimes F_x^{\dagger} \end{align*} $$

induced by the polynomial Petri map is injective.

Proposition 1.6.10. If $J_x^{\infty } D$ fails to satisfy the $\infty $ -jet Petri condition, then $\sigma _x(D)$ fails to satisfy the polynomial Petri condition.

The proof of this result and the upcoming discussion require the following algebraic definitions, constructions, and facts:

1. (1) Let V be a vector space equipped with a filtration $\mathscr {F}_{\bullet } V$ . The order of $\mathscr {F}_{\bullet } V$ is the map $\operatorname {\mathrm {ord}}\mskip 0.5mu\colon \thinspace V \to \mathbf {Z} \cup {\lbrace }{\infty ,-\infty }{\rbrace }$ defined by

   

   $\mathscr {F}_{\bullet } V$ is called exhaustive if $\operatorname {\mathrm {ord}}^{-1}(-\infty ) = \varnothing $ or, equivalently, $\bigcup _{j \in \mathbf {Z}} \mathscr {F}_j V = V$ . $\mathscr {F}_{\bullet } V$ is separated if $\operatorname {\mathrm {ord}}^{-1}(\infty ) = 0$ or, equivalently, $\bigcap _{j \in \mathbf {Z}} \mathscr {F}_j V = 0$ .

2. (2) The associated graded vector space of $\mathscr {F}_{\bullet } V$ is

   

   Define $[\cdot ]\mskip 0.5mu\colon \thinspace \operatorname {\mathrm {ord}}^{-1}(\mathbf {Z}) \to \operatorname {\mathrm {gr}} V$ by

   

   This is map is not linear and not even continuous (except for a few corner cases). It is appropriate to regard $[v]$ as the leading order term of v.

3. (3) Let W be a further vector space equipped with a filtration $\mathscr {F}_{\bullet } W$ . A linear map $f\mskip 0.5mu\colon \thinspace V \to W$ is of order $k \in \mathbf {Z}$ if $f(\mathscr {F}_jV) \subset \mathscr {F}_{j+k}W$ for every $j \in \mathbf {Z}$ , but the same does not hold for $k+1$ instead of k. If this is the case, then f induces a linear map

   $$ \begin{align*} \operatorname{\mathrm{gr}} f\mskip0.5mu\colon\thinspace \operatorname{\mathrm{gr}} V \to \operatorname{\mathrm{gr}} W \end{align*} $$

   of degree k. There is a canonical inclusion $\operatorname {\mathrm {gr}} \ker f \hookrightarrow \ker \operatorname {\mathrm {gr}} f$ and and a canonical projection $\operatorname {\mathrm {coker}} \operatorname {\mathrm {gr}} f \twoheadrightarrow \operatorname {\mathrm {gr}} \operatorname {\mathrm {coker}} f$ . These maps are typically not isomorphisms.

4. (4) The tensor product $V\otimes W$ inherits the tensor product filtration defined by

   

   There is a canonical graded isomorphism

   (1.6.11) $$ \begin{align} \operatorname{\mathrm{gr}} (V\otimes W) \cong \operatorname{\mathrm{gr}} V\otimes \operatorname{\mathrm{gr}} W. \end{align} $$

   If $\mathscr {F}_{\bullet } V$ and $\mathscr {F}_{\bullet } W$ are both separated (exhaustive), then so is $\mathscr {F}_{\bullet }(V\otimes W)$ .

Let $\ell \in {\mathbf {N}}_0\cup {\lbrace }{\infty }{\rbrace }$ . The vanishing order filtration on $\mathscr {E}_x$ descends to a filtration on $J_x^{\ell } E$ . Taylor expansion defines an isomorphism

(1.6.12) $$ \begin{align} T_x^{\ell} \mskip0.5mu\colon\thinspace \operatorname{\mathrm{gr}} J_x^{\ell} E \to \bigoplus_{j=0}^{\ell} S^j T_x^* M\otimes E_x; \end{align} $$

in particular:

(1.6.13) $$ \begin{align} \dim J_x^{\ell} E = r\cdot\binom{n+\ell}{n} \end{align} $$

(and similarly for $E^{\dagger }$ , F and $F^{\dagger }$ instead of E). If $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (F)$ is a linear differential operator and $\sigma _x(D)$ denotes its symbol at x, then

$$ \begin{align*} T_x^{\infty} \circ \operatorname{\mathrm{gr}} J_x^{\infty} D = \hat \sigma_x(D) \circ T_x^{\infty}. \end{align*} $$

Furthermore,

$$ \begin{align*} T_x^{\infty} \circ \operatorname{\mathrm{gr}} J_x^{\infty}\varpi = \hat\varpi \circ (T_x^{\infty} \otimes T_x^{\infty}). \end{align*} $$

This implies corresponding identities for $\ell \in {\mathbf {N}}_0$ instead of $\infty $ provided $\hat \sigma _x(D)$ and $\hat \varpi $ are appropriately truncated.

Proof of Proposition 1.6.10.

The vanishing order filtration on $J_x^{\infty } E$ and $J_x^{\infty } F$ is exhaustive and separated. Therefore, if $B \in \ker \varpi _{J_x^{\infty } D}$ is nonzero, then

$$ \begin{align*} [B] \in (\ker \operatorname{\mathrm{gr}} J_x^{\infty} D \otimes \ker \operatorname{\mathrm{gr}} J_x^{\infty} D^{\dagger}) \cap \ker \operatorname{\mathrm{gr}} J_x^{\infty} \varpi \end{align*} $$

is defined and nonzero. By the preceding discussion, $T_x^{\infty }$ induces an isomorphism

$$ \begin{align*} (\ker\operatorname{\mathrm{gr}} J_x^{\infty} D \otimes \ker\operatorname{\mathrm{gr}} J_x^{\infty} D^{\dagger}) \cap \ker \operatorname{\mathrm{gr}} J_x^{\infty} \varpi \cong \ker \hat\varpi_{\sigma} \quad\text{with}\quad \sigma = \sigma_x(D).\\[-38pt]\end{align*} $$

Proposition 1.6.10 is probably not terribly useful for establishing the $\infty $ -jet Petri condition. The polynomial Petri condition fails for real Cauchy–Riemann operators (see [Wen19b, Example 5.5] and Proposition 2.5.4), and we suspect that it typically fails. However, this is no reason to despair. It can be shown that every $\hat B \in \ker \hat \sigma _x(D)\otimes \ker \hat \sigma _x(D^{\dagger })$ admits some (but not a unique) lift to an element $B \in \ker J_x^{\infty } D\otimes \ker J_x^{\infty } D^{\dagger }$ . However, if $\hat B \in \ker \hat \varpi $ , then this does not imply that $B \in \ker J_x^{\infty }\varpi $ . In fact, it is reasonable to expect that typically the higher-order terms will prevent the vanishing of $J_x^{\infty }\varpi (B)$ . The upcoming theorem shows that this heuristic is valid assuming an algebraic hypothesis on symbol level.

Definition 1.6.14. Let $k,\ell \in {\mathbf {N}}_0$ . A family of linear elliptic differential operators $(D_p)_{p\in \mathscr {P}}$ of order k with smooth coefficients is $\ell $ -jet flexible at x and $p_{\star } \in \mathscr {P}$ if for every $A \in J_x^{\ell }\operatorname {\mathrm {Hom}}(E,F)$ there is a $\hat p\in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}}J_x^{\ell} D(\hat p) s = As \end{align*} $$

for every $s \in J_x^{k+\ell } E$ .

Definition 1.6.15. Let $k \in {\mathbf {N}}_0$ . A symbol $\sigma \in S^kT_xM\otimes \operatorname {\mathrm {Hom}}(E_x,F_x)$ satisfies Wendl’s condition if there are $c_0\mskip 0.5mu\colon \thinspace {\mathbf {N}}_0\times {\mathbf {N}} \to (0,\infty )$ and $\ell _0\mskip 0.5mu\colon \thinspace {\mathbf {N}}_0\times {\mathbf {N}} \to {\mathbf {N}}_0$ such that for every every homogeneous $B \in \ker \hat \varpi _{\sigma }$ the following hold: there are right-inverses $\hat R$ and $\hat R^{\dagger }$ of $\hat \sigma $ and $\hat \sigma ^{\dagger }$ such that the linear map

defined by

satisfies

for every $\ell \geqslant \ell _0(d,\rho )$ with

and

. Here,

denotes the truncation of

.

Remark 1.6.16. The reader is by no means expected to understand the significance of Wendl’s condition at this point. The following remarks might help clarify the definition:

1. (1) Proposition 1.6.20 proves that $\hat \sigma $ and $\hat \sigma ^{\dagger }$ have right-inverses provided $\sigma $ is elliptic.

2. (2) The maps play a crucial role in the proof of Theorem 1.6.17. Their ranks provide lower bounds for the ranks of certain map between jet spaces tied to the failure of the $\infty $ -jet Petri condition.

3. (3) The dimension of the codomain of grows like $\ell ^n$ ; therefore, is assumed to have maximal growth rate.

4. (4) Unfortunately, it appears not to be easy to verify whether a given symbol $\sigma $ satisfies Wendl’s condition or not. In fact, even determining $\ker \hat \varpi _{\sigma }$ is a nontrivial task. Theorem 2.5.3 proves that the symbol $\sigma $ of a real Cauchy–Riemann operator satisfies Wendl’s condition. As far as we know, it is possible that every elliptic symbol satisfies Wendl’s condition.

Theorem 1.6.17 (Wendl [Wen19b, Section 5.2]).

Let $(D_p)_{p\in \mathscr {P}}$ be a family of linear elliptic differential operators with smooth coefficients of order k. Set

Let $p_{\star } \in \mathscr {P}$ . If

1. (1) $(D_p)_{p\in \mathscr {P}}$ is $\ell $ -jet flexible at x and $p_{\star } \in \mathscr {P}$ for every $\ell \in {\mathbf {N}}_0$ , and

2. (2) the symbol $\sigma _x(D_{p_{\star }})$ satisfies Wendl’s condition,

then for every $c \in {\mathbf {N}}_0$ there is an open neighborhood $\mathscr {U}$ of $p_{\star } \in \mathscr {P}$ such that $\mathscr {R} \cap \mathscr {U}$ has codimension at least c. ⁹

The remainder of this section is devoted to the proof of Theorem 1.6.17. The following observation decomposes $\mathscr {R}$ into pieces whose codimensions can be estimated using the hypotheses of the theorem.

Proposition 1.6.18. For $d \in {\mathbf {N}}_0$ and $\rho \in {\mathbf {N}}$ , set

The set $\mathscr {R}$ satisfies

The proof relies on the following fact.

Proposition 1.6.19. Let V be a vector space and equipped with a filtration $\mathscr {F}_{\bullet } V$ . Set

If $R \subset Q$ is a finite-dimensional subspace, then there is an $\ell _0 \in {\mathbf {N}}_0$ such that for every $\ell \geqslant \ell _0$ the restriction of the composition $R \to Q \to Q_{\ell }$ is injective.

Proof. is a decreasing sequence of finite-dimensional vector spaces with $\varprojlim K_{\ell } = 0$ . Therefore, $K_{\ell } = 0$ for $\ell \gg 1$ .

Proof of Proposition 1.6.18.

If $p \in \mathscr {R}$ , then there exists a nonzero $B \in \ker \varpi _{J_x^{\infty } D_p}$ . Set

,

and write B as

$$ \begin{align*} B = \sum_{i=1}^{\rho} s_i\otimes t_i \end{align*} $$

with $s_1,\ldots ,s_{\rho }$ and $t_1,\ldots ,t_{\rho }$ linearly independent. Since

$$ \begin{align*} J_x^{\infty} E = \varprojlim J_x^{\ell} E \end{align*} $$

and by Proposition 1.6.19, there is an $\ell _0 \in {\mathbf {N}}_0$ such that for every $\ell \geqslant \ell _0$ the $(k+\ell )$ -jets $\tilde s_1,\ldots ,\tilde s_{\rho } \in J_x^{k+\ell }E$ and $\tilde t_1,\ldots ,\tilde t_{\rho } \in J_x^{k+\ell }F^{\dagger }$ are linearly independent. By construction,

satisfies

$$ \begin{align*} \operatorname{\mathrm{ord}}(\tilde B) = d \quad\text{and}\quad \operatorname{\mathrm{rk}} \tilde B = \rho. \end{align*} $$

Therefore, $p \in \mathscr {R}_{d,\rho }^{\ell }$ for every $\ell \geqslant \ell _0$ .

To estimate the codimension of $\mathscr {R}_{d,\rho }^{\ell }$ , we require the following. Recall that $\dim M = n$ and $\operatorname {\mathrm {rk}} E = \operatorname {\mathrm {rk}} F = r$ .

Proposition 1.6.20. Let $J_x^{\infty } D$ be the $\infty $ -jet of an elliptic differential operator D of order k. The following hold:

1. (1) The formal differential operator $\hat \sigma _x(D)$ is surjective.

2. (2) For every $\ell \in {\mathbf {N}}_0\cup {\lbrace }{\infty }{\rbrace }$ , the $\ell $ -jet $J_x^{\ell } D$ is surjective.

3. (3) For every $\ell \in {\mathbf {N}}_0$ ,

   $$ \begin{align*} \dim \ker J_x^{\ell} D = r\cdot \left[{\binom{n+k+\ell}{n}-\binom{n+\ell}{n}}\right]. \end{align*} $$

Proof. Since D is elliptic, the restriction $\hat \sigma _x^k(D)\mskip 0.5mu\colon \thinspace S^k T_x^*M \otimes E_x \to F_x$ is surjective. Choose a basis $(\xi _1,\ldots ,\xi _n)$ of $T_x^*M$ . For a multi-index $\alpha \in {\mathbf {N}}_0^n$ set . A moment’s thought shows that

$$ \begin{align*} \hat\sigma_x(D)(\xi_1^k\xi^{\alpha}\otimes e) = \binom{k+\alpha_1}{\alpha_1} \xi^{\alpha} \otimes \hat\sigma_x(D)(\xi_1^k\otimes e) + R \end{align*} $$

with R denoting a sum of tensors of the form $\xi ^{\beta } \otimes w$ with $\beta _1> \alpha _1$ . Therefore, the image of $\hat \sigma _x(D)$ contains every tensor product of the form $\xi _1^m\otimes f$ . Descending induction on $\alpha _1$ starting at m proves that the image of $\hat \sigma _x(D)$ contains every tensor product of the form $\xi ^{\alpha } \otimes w$ with ${\lvert }{\alpha }{\rvert } = m$ . This proves (1).

Since $\operatorname {\mathrm {coker}} \operatorname {\mathrm {gr}} J_x^{\ell } D \twoheadrightarrow \operatorname {\mathrm {gr}} \operatorname {\mathrm {coker}} J_x^{\ell } D$ , (1) implies (2).

Finally, (2) implies $\dim \ker J_x^{\ell } D = \dim J_x^{k+\ell } E - \dim J_x^{\ell } F$ . Therefore, (3) follows from equation (1.6.13).

Proof of Theorem 1.6.17.

Let $d \in {\mathbf {N}}_0$ , $\rho \in {\mathbf {N}}$ , and $\ell \geqslant \ell _0(d,\rho )$ . By Proposition 1.6.20,

are vector bundles over $\mathscr {P}$ of rank

$$ \begin{align*} \operatorname{\mathrm{rk}} \mathscr{K}^{\ell} = \operatorname{\mathrm{rk}} \mathscr{C}^{\ell} = r\cdot \left[{\binom{n+k+\ell}{n}-\binom{n+\ell}{n}}\right]. \end{align*} $$

Therefore,

is a fiber bundle over $\mathscr {P}$ with fibers of dimension

$$ \begin{align*} 2\rho r \cdot \left[{ \binom{n+k+\ell}{n}-\binom{n+\ell}{n} }\right] - \rho^2 \leqslant c(n,k) \rho \ell^{n-1}. \end{align*} $$

Denote by $\pi \mskip 0.5mu\colon \thinspace \mathscr {T}_{d,\rho }^{\ell } \to \mathscr {P}$ the projection map. By construction,

$$ \begin{align*} \mathscr{R}_{d,\rho}^{\ell} = \pi({(J_x^{k+\ell}\varpi\circ\mathrm{pr}_2)^{-1}(0)}). \end{align*} $$

The upcoming discussion proves that for every $(p_{\star },B) \in (J_x^{k+\ell }\varpi \circ \mathrm {pr}_2)^{-1}(0)$

$$ \begin{align*} \operatorname{\mathrm{rk}} \mathrm{d}_{(p_{\star},B)}(J_x^{k+\ell}\varpi\circ\mathrm{pr}_2) \geqslant c_0(d,\rho)\ell^n. \end{align*} $$

Therefore, there is an open neighborhood $\mathscr {U}$ of $p_{\star } \in \mathscr {P}$ such that for every $(p,B) \in (J_x^{k+\ell }\varpi \circ \mathrm {pr}_2)^{-1}(0)$ with $p \in \mathscr {U}$ the analogous condition hold. Consequently, by Proposition 1.B.2, $\mathscr {R}_{d,\rho }^{\ell } \cap \mathscr {U}$ has codimension at least

$$ \begin{align*} (c_0(d,\rho)\ell - c(n,k) \rho) \ell^n. \end{align*} $$

This immediately implies the theorem.

Let $(p_{\star },B) \in (J_x^{k+\ell }\varpi \circ \mathrm {pr}_2)^{-1}(0)$ . Set

. Denote by $\hat R$ and $\hat R^{\dagger }$ the right-inverses of $\hat \sigma $ and $\hat \sigma ^{\dagger }$ from Definition 1.6.15. Denote by R and $R^{\dagger }$ right-inverses of $J_x^{k+\ell } D_{p_{\star }}$ and $J_x^{k+\ell } D_{p_{\star }}^{\dagger }$ such that $\operatorname {\mathrm {gr}} R$ and $\operatorname {\mathrm {gr}} R^{\dagger }$ correspond to the truncations of $\hat R$ and $\hat R^{\dagger }$ with respect to equation (1.6.12). Define ${\mathbf {L}}_{p_{\star },B}\mskip 0.5mu\colon \thinspace J_x^{\ell }\operatorname {\mathrm {Hom}}(E_x,F_x) \to J_x^{k+\ell } (E_x\otimes F_x^{\dagger })$ by

Since $(D_p)_{p\in \mathscr {P}}$ is $\ell $ -jet flexible at x and $p_{\star } \in \mathscr {P}$ , for every $A \in J_x^{\ell }\operatorname {\mathrm {Hom}}(E_x,F_x)$ there is a $\hat p\in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}}J_x^{\ell} D(\hat p) s = As \end{align*} $$

for every $s \in J_x^{k+\ell } E$ . If this identity holds, then

$$ \begin{align*} (\hat p,(RA\otimes \mathbf{1} + \mathbf{1}\otimes R^{\dagger} A^{\dagger}) B) \in T_{(p_{\star},B)}\mathscr{T}_{d,\rho}^{\ell}. \end{align*} $$

Therefore,

$$ \begin{align*} \operatorname{\mathrm{rk}} \mathrm{d}_{(p,B)}(J_x^{k+\ell}\varpi\circ\mathrm{pr}_2) \geqslant \operatorname{\mathrm{rk}} {\mathbf{L}}_{p_{\star},B}. \end{align*} $$

Since $\operatorname {\mathrm {gr}} \ker {\mathbf {L}}_{p_{\star },B} \hookrightarrow \ker \operatorname {\mathrm {gr}} {\mathbf {L}}_{p_{\star },B}$ ,

$$ \begin{align*} \operatorname{\mathrm{rk}} {\mathbf{L}}_{p_{\star},B} = r\cdot\binom{n+\ell}{n} - \dim \operatorname{\mathrm{gr}} \ker {\mathbf{L}}_{p_{\star},B} \geqslant r\cdot\binom{n+\ell}{n} - \dim \ker \operatorname{\mathrm{gr}} {\mathbf{L}}_{p_{\star},B} = \operatorname{\mathrm{rk}} \operatorname{\mathrm{gr}} {\mathbf{L}}_{p_{\star},B}. \end{align*} $$

The isomorphism (1.6.12) identifies $[B]$ with $\hat B \in \ker \hat \varpi _{\sigma }$ , which is homogeneous of degree d. Furthermore, it identifies $\operatorname {\mathrm {gr}} {\mathbf {L}}_{p_{\star },B}$ with

. Therefore,

This finishes the proof.

1.A Self-adjoint operators

The purpose of this section is to summarize how the results developed so far need to be modified to become useful for families of formally self-adjoint linear elliptic differential operators. These are relevant for many geometric applications (although not for Part 2.) A particularly interesting application would be to understand the generic multiple cover phenomena for associative submanifolds in $\mathrm {G}_2$ -manifolds: The deformation theory of the latter are controlled by twisted Dirac operators.

Definition 1.A.1. Let $k \in {\mathbf {N}}_0$ . A family of formally self-adjoint linear elliptic differential operators of order k consists of a Banach manifold $\mathscr {P}$ and a smooth map

$$ \begin{align*} D \mskip0.5mu\colon\thinspace \mathscr{P} \to \mathscr{F}(W^{k,2}\Gamma(E),L^2\Gamma(E)) \end{align*} $$

such that for every $p \in \mathscr {P}$ the operator is a formally self-adjoint linear elliptic differential operator of order k.

Throughout this section, assume Situation 1.3.1 and keep the following in mind:

1. (1) The algebras ${\mathbf {K}}_{\alpha }$ carry an anti-involution $\lambda \mapsto \lambda ^*$ and an inner product . (These correspond to the standard conjugation and inner products on $\mathbf {R}$ ${\mathbf {C}}$ , and $\mathbf {H}$ .) The Euclidean metric on ${\underline V}_{\alpha }$ is ${\mathbf {K}}_{\alpha }$ -sesquilinear.

2. (2) Let W be a left ${\mathbf {K}}_{\alpha }$ -module equipped with a ${\mathbf {K}}_{\alpha }$ -sesquilinear inner product. Denote by $\operatorname {\mathrm {Sym}}_{{\mathbf {K}}_{\alpha }}(W)$ the space of self-adjoint ${\mathbf {K}}_{\alpha }$ -linear maps. W is a right ${\mathbf {K}}_{\alpha }$ -module with . Therefore, one can form the tensor product $W\otimes _{\,{\mathbf {K}}_{\alpha }} W$ and the symmetric tensor power $S_{{\mathbf {K}}_{\alpha }}^2W$ .

3. (3) Let D be a formally self-adjoint linear elliptic differential operator. The ${\mathbf {K}}_{\alpha }$ -sesquilinear inner product on ${\underline V}_{\alpha }$ induces a canonical isomorphism

   $$ \begin{align*} \ker D_p^{{\underline V}_{\alpha}} \cong \operatorname{\mathrm{coker}} D_p^{{\underline V}_{\alpha}}. \end{align*} $$

   Moreover, the map $\Lambda _p^{\alpha }$ defined in Theorem 1.3.5 takes values in

   $$ \begin{align*} \operatorname{\mathrm{Sym}}_{{\mathbf{K}}_{\alpha}}(\ker D_p^{{\underline V}_{\alpha}}). \end{align*} $$

Here is the analogue of the theory developed in Section 1.3. (It is left as an exercise for the reader to work out the analogues of Section 1.4 and Section 1.5.)

Definition 1.A.2. Let $(D_p)_{p \in \mathscr {P}}$ be a family of formally self-adjoint linear elliptic differential operators. For $d \in {\mathbf {N}}_0^m$ define the ${\mathfrak V}$ -equivariant self-adjoint Brill–Noether locus $\mathscr {P}_d^{\mathfrak V}$ by

Theorem 1.A.3. Let $(D_p)_{p \in \mathscr {P}}$ be a family of formally self-adjoint linear elliptic differential operators. For $p \in \mathscr {P}$ define $\Lambda _p^{\mathfrak V} \mskip 0.5mu\colon \thinspace T_p\mathscr {P} \to \bigoplus _{\alpha =1}^m \operatorname {\mathrm {Sym}}_{{\mathbf {K}}_{\alpha }}(\ker D_p^{{\underline V}_{\alpha }})$ by

Let $d\in {\mathbf {N}}_0^m$ and $p \in \mathscr {P}_d^{\mathfrak V}$ . If $\Lambda _p^{\mathfrak V}$ is is surjective, then the following hold:

1. (1) There is an open neighborhood $\mathscr {U}$ of $p \in \mathscr {P}$ such that $\mathscr {P}_d^{\mathfrak V} \cap \mathscr {U}$ is a submanifold of codimension

   $$ \begin{align*} \operatorname{\mathrm{codim}} (\mathscr{P}_d^{\mathfrak V} \cap \mathscr{U}) = \sum_{\alpha=1}^m d_{\alpha} + k_{\alpha}\binom{d_{\alpha}}{2}. \end{align*} $$

2. (2) $\mathscr {P}_{\tilde d}^{\mathfrak V} \neq \varnothing $ for every $\tilde d \in {\mathbf {N}}_0^m$ with $\tilde d \leqslant d$ .

Proof. There is a straightforward variation of Lemma 1.1.4 to self-adjoint Fredholm operators. This reduces the proof to the finite-dimensional situation. The latter is straightforward. The codimension formula follows from

$$ \begin{align*} \dim \operatorname{\mathrm{Sym}}_{\mathbf{K}}({\mathbf{K}}^d) = d + k \binom{d}{2} \end{align*} $$

with .

Definition 1.A.4. A family of linear elliptic differential operators $(D_p)_{p \in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly symmetrically flexible in U at $p_{\star } \in \mathscr {P}$ if for $A \in \Gamma _c(U,\operatorname {\mathrm {Sym}}(E))$ there is a $\hat p \in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}} D^{{\underline V}_{\alpha}}(\hat p)s = (A\otimes \mathrm{id}_{{\underline V}_{\alpha}})s \quad\mod \operatorname{\mathrm{im}} D_{p_{\star}}^{{\underline V}_{\alpha}} \end{align*} $$

for every $\alpha = 1,\ldots ,m$ and $s \in \ker D_p^{{\underline V}_{\alpha }}$ .

Definition 1.A.5. The ${\mathfrak V}$ -equivariant symmetric Petri map

$$ \begin{align*} \varsigma^{\mathfrak V} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m S_{{\mathbf{K}}_{\alpha}}^2 \Gamma(E\otimes {\underline V}_{\alpha}) \to \Gamma(S^2E) \end{align*} $$

is defined by with $\varsigma _{\alpha }$ denoting the composition of the Petri map

$$ \begin{align*} \varsigma_{\alpha}\mskip0.5mu\colon\thinspace S_{{\mathbf{K}}_{\alpha}} ^2\Gamma(E\otimes {\underline V}_{\alpha}) \to \Gamma(S^2 E \otimes S_{{\mathbf{K}}_{\alpha}}^2 {\underline V}_{\alpha}) \end{align*} $$

and the map induced by the inner product ${\langle }{\cdot , \cdot }{\rangle }\mskip 0.5mu\colon \thinspace S_{{\mathbf {K}}_{\alpha }}^2 {\underline V}_{\alpha } \to {\underline {\mathbf {R}}}$ . Let $U \subset M$ be an open subset. A self-adjoint linear elliptic differential operator $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (E)$ satisfies the ${\mathfrak V}$ -equivariant symmetric Petri condition in U if the map

$$ \begin{align*} \varsigma_{D,U}^{\mathfrak V} \mskip0.5mu\colon\thinspace \bigoplus_{\alpha=1}^m S_{{\mathbf{K}}_{\alpha}}^2 \ker D_p^{{\underline V}_{\alpha}} \to L^1\Gamma(U,S^2E) \end{align*} $$

induced by the ${\mathfrak V}$ -equivariant symmetric Petri map is injective.

Proposition 1.A.6. Let $(D_p)_{p \in \mathscr {P}}$ be a family of self-adjoint linear elliptic differential operators. Let $U \subset M$ be an open subset. If $(D_p)_{p \in \mathscr {P}}$ is ${\mathfrak V}$ -equivariantly symmetrically flexible in U at $p_{\star } \in \mathscr {P}$ and $D_{p_{\star }}$ satisfies the ${\mathfrak V}$ -equivariant symmetric Petri condition in U, then the map $\Lambda _{p_{\star }}^{\mathfrak V}$ defined in Theorem 1.A.3 is surjective.

Remark 1.3.12 carries over mutatis mutandis; in particular, equation (1.3.13) it is still sharp for self-adjoint operators. Finally, these are the analogues of the results from Section 1.6.

Definition 1.A.7. The $\infty $ -jet of a formally self-adjoint linear elliptic differential operator $J_x^{\infty } D\mskip 0.5mu\colon \thinspace J_x^{\infty } E \to J_x^{\infty } E$ satisfies the $\infty $ -jet symmetric Petri condition if the map

$$ \begin{align*} \varpi_{J_x^{\infty} D}\mskip0.5mu\colon\thinspace \ker S^2 J_x^{\infty} D \to J_x^{\infty} S^2E \end{align*} $$

induced by the symmetric Petri map is injective.

Proposition 1.A.8. Assume Situation 1.3.1. Let $x \in M$ . Let $U \subset M$ be an open neighborhood of x. Let $D\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (E)$ be a formally self-adjoint linear elliptic differential operator with smooth coefficients. Suppose that D possess the strong unique continuation property at x in U. If $J_x^{\infty } D$ satisfies the $\infty $ -jet symmetric Petri condition, then D satisfies the ${\mathfrak V}$ -equivariant symmetric Petri condition in U.

Definition 1.A.9. The polynomial symmetric Petri map $\hat \varsigma \mskip 0.5mu\colon \thinspace S^2(S^{\bullet } T_x^* M\otimes E_x)\to S^{\bullet } T_x^* M\otimes S^2E_x$ is defined as the restriction of the polynomial Petri map. A symmetric symbol $\sigma \in S^kT_xM\otimes \operatorname {\mathrm {Sym}}(E_x)$ satisfies the polynomial symmetric Petri condition if the map

$$ \begin{align*} \hat\varsigma_{\sigma}\mskip0.5mu\colon\thinspace S^2\ker \hat\sigma \to S^{\bullet} T_x^* M\otimes S^2E_x \end{align*} $$

induced by the polynomial symmetric Petri map is injective.

Proposition 1.A.10. If $J_x^{\infty } D$ fails to satisfy the $\infty $ -jet symmetric Petri condition, then $\sigma _x(D)$ fails to satisfy the polynomial symmetric Petri condition.

Definition 1.A.11. Let $k,\ell \in {\mathbf {N}}_0$ . A family of self-adjoint linear elliptic differential operators $(D_p)_{p\in \mathscr {P}}$ of order k is $\ell $ -jet symmetrically flexible at x and $p_{\star } \in \mathscr {P}$ if for every $A \in J_x^{\ell }\operatorname {\mathrm {Sym}}(E_x,F_x)$ there is a $\hat p\in T_{p_{\star }}\mathscr {P}$ such that

$$ \begin{align*} \mathrm{d}_{p_{\star}}J_x^{\ell} D(\hat p) s = As \end{align*} $$

for every $s \in J_x^{k+\ell } E$ .

Definition 1.A.12. Let $k \in {\mathbf {N}}_0$ . A symbol $\sigma \in S^kT_xM\otimes \operatorname {\mathrm {Sym}}(E_x)$ satisfies the symmetric Wendl condition if there are $c_0\mskip 0.5mu\colon \thinspace {\mathbf {N}}_0\times {\mathbf {N}} \to (0,\infty )$ and $\ell _0\mskip 0.5mu\colon \thinspace {\mathbf {N}}_0\times {\mathbf {N}} \to {\mathbf {N}}_0$ such that for every every homogeneous $B \in \ker \hat \varsigma _{\sigma }$ the following hold: There is a right-inverse $\hat R$ of $\hat \sigma $ such that the linear map

defined by

satisfies

for every $\ell \geqslant \ell _0(d,\rho )$ with

and

. Here,

denotes the truncation of

.

Remark 1.A.13. Verifying the symmetric Wendl condition appears to be the crucial issue in the geometric applications alluded to at the beginning of this section. In light of Theorem 2.5.3, it is tempting to conjecture that a typical twisted Dirac operator on a $3$ -manifold does satisfy this condition.

Theorem 1.A.14 (Wendl [Wen19b, Section 5.2]).

Let $(D_p)_{p\in \mathscr {P}}$ be a family of formally self-adjoint linear elliptic differential operators with smooth coefficients of order k. Set

Let $p_{\star } \in \mathscr {P}$ . If

1. (1) $(D_p)_{p\in \mathscr {P}}$ is $\ell $ -jet symmetrically flexible at x and $p_{\star } \in \mathscr {P}$ for every $\ell \in {\mathbf {N}}_0$ , and

2. (2) the symbol $\sigma _x(D_{p_{\star }})$ satisfies the symmetric Wendl condition,

then for every $c \in {\mathbf {N}}_0$ there is an open neighborhood $\mathscr {U}$ of $p_{\star } \in \mathscr {P}$ such that $\mathscr {R} \cap \mathscr {U}$ has codimension at least c.

Proof. The proof of Theorem 1.6.17 carries over with minor changes. The salient point is that

is a fiber bundle over $\mathscr {P}$ with fibers of dimension at most $c(n,k) \rho \ell ^{n-1}$ .

1.B Codimension in Banach manifolds

There are numerous possible definitions of the concept of codimension of a subset of a Banach manifold. The following is a minor variation of the definition from [BM15, Section 2.3] and particularly well-suited for the purposes of this article.

Definition 1.B.1. Let X be a Banach manifold and $c \in {\mathbf {N}}_0$ . A subset $S \subset X$ has codimension at least c if there is a $C^1$ Banach manifold Z and a $C^1$ Fredholm map $\zeta \mskip 0.5mu\colon \thinspace Z \to X$ such that

$$ \begin{align*} \sup_{z \in Z} \operatorname{\mathrm{index}} \mathrm{d}_z \zeta \leqslant -c \quad\text{and}\quad S \subset \operatorname{\mathrm{im}}\zeta. \end{align*} $$

The codimension of S is defined by

The additivity of Fredholm indices implies the following.

Proposition 1.B.2. Let $X,Y$ be Banach manifolds. If $S \subset X$ and $f\mskip 0.5mu\colon \thinspace X\to Y$ is a Fredholm map, then

$$ \begin{align*} \operatorname{\mathrm{codim}} f(S) \geqslant \operatorname{\mathrm{codim}} S - \inf_{x\in X}\operatorname{\mathrm{index}} \mathrm{d}_xf.\\[-40pt] \end{align*} $$

The codimension of a subset can be regarded as a measure of the nongenericity of its elements. In topology, one considers the following concepts.

Definition 1.B.3. Let X be a topological space and $S \subset X$ . S is meager if it is contained in a countable union of closed subsets with empty interior. S is comeager if $X{\backslash } S$ is meager.

Recall from Footnote 2 that Banach manifolds are assumed to be Hausdorff, paracompact and separable. The Baire category theorem asserts that a meager subset of a completely metrizable space (e.g., a Banach manifold) has empty interior or, equivalently, that every comeager subset of such a space is dense. In light of this, one often regards a meager subset as consisting of nongeneric points and a comeager subset as consisting of generic points.

Proposition 1.B.4. Let X be a Banach manifold and $S \subset X$ . If $\operatorname {\mathrm {codim}} S> 0$ , then S is meager. ¹⁰

Proof. Let Z and $\zeta $ be as in Definition 1.B.1. Since $\operatorname {\mathrm {index}} \mathrm {d}_z\zeta < 0$ , by the Sard–Smale theorem [Sma65, Theorem 1.3] $\operatorname {\mathrm {im}} \zeta $ is meager; hence, so is S.

In practice, one often proves that a subset is meager by proving that it has positive codimension. The latter, however, yields more precise information.

Proposition 1.B.5. Let M be a finite-dimensional manifold, and let X be a Banach manifold. For every $S \subset X$ and $k \in {\mathbf {N}}$ , the following hold:

1. (1) The subset $\widetilde S \subset C^k(M,X)$ consisting of those f such that $f^{-1}(S) \neq \varnothing $ satisfies

   $$ \begin{align*} \operatorname{\mathrm{codim}} \widetilde S \geqslant \operatorname{\mathrm{codim}} S - \dim M.\\[-15pt] \end{align*} $$

2. (2) The subset consisting of those $f \in C^k(M,X)$ for which $ \operatorname {\mathrm {codim}} f^{-1}(S) \geqslant \operatorname {\mathrm {codim}} S$ is comeager.

Proof. Suppose that S has codimension at least c, and let Z and $\zeta $ be as in Definition 1.B.1. Set

. The evaluation map $\mathrm {ev}\mskip 0.5mu\colon \thinspace F \to X$ is a $C^k$ submersion. Therefore,

is a $C^k$ Banach manifold and the map $\mathrm {pr}_1 \mskip 0.5mu\colon \thinspace \mathrm {ev}^*Z \to F$ is a Fredholm map of index at most $-c$ . The projection map $\mathrm {pr}_2\mskip 0.5mu\colon \thinspace F \to C^k(M,X)$ is a Fredholm map of index $\dim M$ .

To prove (1), observe that $\mathrm {ev}^{-1}(S) \subset \operatorname {\mathrm {im}} \mathrm {pr}_1$ . Therefore, $\operatorname {\mathrm {codim}} \mathrm {ev}^{-1}(S) \geqslant c$ ; hence, by Proposition 1.B.2, $\widetilde S = \mathrm {pr}_2(\mathrm {ev}^{-1}(S))$ has codimension at least $c-\dim M$ .

If $f \in C^k(M,X)$ is a regular value of $\mathrm {pr}_2 \circ \mathrm {pr}_1 \mskip 0.5mu\colon \thinspace \mathrm {ev}^*Z \to C^k(M,X)$ , then $(\mathrm {pr}_2 \circ \mathrm {pr}_1)^{-1}(f)$ is a $C^k$ submanifold of $\mathrm {ev}^*Z$ of dimension at most $\dim M - c$ . Therefore, its projection to M has codimension at least c. A moment’s thought shows that this projection is $f^{-1}(\operatorname {\mathrm {im}}\zeta )$ ; hence, it contains $f^{-1}(S)$ . Therefore, $\operatorname {\mathrm {codim}} f^{-1}(S) \geqslant \operatorname {\mathrm {codim}} S$ . By the Sard–Smale theorem, the set of regular values of $\mathrm {pr}_2\circ \mathrm {pr}_1$ is comeager. This implies (2).

2 Application to superrigidity

2.1 Bryan and Pandharipande’s superrigidity conjecture

The notion of superrigidity for holomorphic maps was first introduced in algebraic geometry by Bryan and Pandharipande [BP01, Section 1.2]. The purpose of this section is to recall the corresponding notion in symplectic geometry as defined by Eftekhary [Eft16, Section 1] and Wendl [Wen19b, Section 2.1].

Definition 2.1.1. Let $(M,J)$ be an almost complex manifold. A J -holomorphic map $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ is a pair consisting of a closed, connected Riemann surface $(\Sigma ,j)$ and a smooth map $u\mskip 0.5mu\colon \thinspace \Sigma \to M$ satisfying the nonlinear Cauchy–Riemann equation

(2.1.2)

Let $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a J-holomorphic map. Let $\phi \in \operatorname {\mathrm {Diff}}(\Sigma )$ be a diffeomorphism. The reparametrization of u by $\phi $ is the J-holomorphic map $u\circ \phi ^{-1}\mskip 0.5mu\colon \thinspace (\Sigma ,\phi _*j) \to (M,J)$ .

If $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (\Sigma ,j)$ is a holomorphic map of degree $\deg (\pi ) \geqslant 2$ , then the composition $u\circ \pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (M,J)$ is said to be a multiple cover of u. A J-holomorphic map is simple if it is not constant and not a multiple cover.

Superrigidity is a condition on the infinitesimal deformation theory of the images of J-holomorphic maps (up to reparametrization). To give the precise definition, let us recall the salient parts of this theory. This material is standard and details can be found, for example, in [MS12, Chapter 3] and [Wen19a].

Let $(M,J)$ be an almost complex manifold, and let $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. Set

Let $\mathscr {J}(\Sigma )$ be the space of almost complex structures on $\Sigma $ , and let $\mathscr {T}(\Sigma ) = \mathscr {J}(\Sigma ) / \operatorname {\mathrm {Diff}}_0(\Sigma )$ be the Teichmüller space. Let $\mathscr {S}$ be a slice of the $\operatorname {\mathrm {Diff}}_0(\Sigma )$ -action through j; that is, $\mathscr {S}$ is a finite-dimensional $\operatorname {\mathrm {Aut}}(\Sigma ,j)$ -invariant submanifold of $\mathscr {J}(\Sigma )$ containing j and such that the map

$$ \begin{align*} \operatorname{\mathrm{Diff}}_0(\Sigma) \times_{\operatorname{\mathrm{Aut}}(\Sigma,j)} \mathscr{S} &\to \mathscr{T}(\Sigma), \\ [\phi,k] &\mapsto [\phi_*k] \end{align*} $$

is a homeomorphism. Denote by

$$ \begin{align*} \mathrm{d}_{u,j}\bar{\partial}_J \mskip0.5mu\colon\thinspace \Gamma(u^*TM)\oplus T_j\mathscr{S} \to \Omega^{0,1}(u^*TM) \end{align*} $$

the linearization of the map $(u,j) \mapsto \bar {\partial }_J(u,j)$ restricted to $C^{\infty }(\Sigma ,M)\times \mathscr {S}$ at at $(u,j)$ . The action of $\operatorname {\mathrm {Aut}}(\Sigma ,j)$ on $C^{\infty }(\Sigma , M)\times \mathscr {S}$ preserves $\bar {\partial }_J^{-1}(0)$ . Therefore, there is an inclusion $\operatorname {\mathrm {\mathfrak {aut}}}(\Sigma ,j) \hookrightarrow \ker \mathrm {d}_{u,j}\bar {\partial }_J$ . The moduli space of J-holomorphic maps up to reparametrization containing $[u,j]$ has virtual dimension

$$ \begin{align*} \operatorname{\mathrm{index}} \mathrm{d}_{u,j}\bar{\partial}_J - \dim\operatorname{\mathrm{\mathfrak{aut}}}(\Sigma,j) = (n-3)\chi(\Sigma) + 2{\langle}{[\Sigma], u^*c_1(M,J)}{\rangle}. \end{align*} $$

Definition 2.1.3. Let $(M,J)$ be an almost complex manifold of dimension $2n$ . The index of a J-holomorphic map $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ is

(2.1.4)

Infinitesimal deformations of j do not affect $\operatorname {\mathrm {im}} u$ . Therefore, we restrict our attention to ${\mathfrak d}_{u,J}\mskip 0.5mu\colon \thinspace \Gamma (u^*TM) \to \Omega ^{0,1}(\Sigma ,u^*TM)$ , the restriction of $\mathrm {d}_{u,j}\bar {\partial }_J$ to $\Gamma (u^*TM)$ . A brief computation shows that

(2.1.5) $$ \begin{align} {\mathfrak d}_{u,J}\xi = \frac12({ \nabla \xi + J\circ (\nabla \xi) \circ j + (\nabla_{\xi} J)\circ \mathrm{d} u\circ j }). \end{align} $$

Here, $\nabla $ denotes any torsion-free connection on $TM$ and also the induced connection on $u^*TM$ . If $(u,j)$ is a J-holomorphic map, then the right-hand side of equation (2.1.5) does not depend on the choice of $\nabla $ ; see [MS12, Proposition 3.1.1]. The operator ${\mathfrak d}_{u,J}$ has the property that if $\xi \in \Gamma (T\Sigma )$ , then ${\mathfrak d}_{u,J}(\mathrm {d} u(\xi ))$ is a $(0,1)$ -form taking values in $\mathrm {d} u(T\Sigma ) \subset u^*TM$ . If u is nonconstant, then there is a unique complex subbundle

$$ \begin{align*} Tu \subset u^*TM \end{align*} $$

of rank one containing $\mathrm {d} u(T\Sigma )$ [IS99, Section 1.3]; see also [Wen10, Section 3.3] and Section 2.A for a detailed discussion. Since $Tu$ agrees with $\mathrm {d} u(T\Sigma )$ outside finitely many points, ${\mathfrak d}_{u,J}$ maps $\Gamma (Tu)$ to $\Omega ^{0,1}(\Sigma ,Tu)$ . Infinitesimal deformations along $\Gamma (Tu)$ also do not affect $\operatorname {\mathrm {im}} u$ . This leads us to the following.

Definition 2.1.6. Let $(M,J)$ be an almost complex manifold. Let $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. Set

The normal Cauchy–Riemann operator associated with u is the linear map

$$ \begin{align*} {\mathfrak d}_{u,J}^N \mskip0.5mu\colon\thinspace \Gamma(Nu) \to \Omega^{0,1}(\Sigma, Nu) \end{align*} $$

induced by ${\mathfrak d}_{u,J}$ .

The following illuminates the role of the normal Cauchy–Riemann operator in the infinitesimal deformation theory of J-holomorphic maps.

Proposition 2.1.7 ([IS99, Lemma 1.5.1; Wen10, Theorem 3]; see also Section 2.A).

Let $(M,J)$ be an almost complex manifold. Let $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. Denote by $Z(\mathrm {d} u)$ the number of critical points of u counted with multiplicity. The following hold:

1. (1) There is a surjection

   $$ \begin{align*} \ker \mathrm{d}_{u,j}\bar{\partial}_J \twoheadrightarrow \ker {\mathfrak d}_{u,J}^N \end{align*} $$
   whose kernel contains $\operatorname {\mathrm {\mathfrak {aut}}}(\Sigma ,j)$ and has dimension $\dim \operatorname {\mathrm {\mathfrak {aut}}}(\Sigma ,j) + 2Z(\mathrm {d} u)$ . ¹¹

2. (2) There is an isomorphism

   $$ \begin{align*} \operatorname{\mathrm{coker}} \mathrm{d}_{u,j}\bar{\partial}_J \cong \operatorname{\mathrm{coker}} {\mathfrak d}_{u,J}^N. \end{align*} $$

3. (3) The index of ${\mathfrak d}_{u,J}^N$ satisfies

   $$ \begin{align*} \operatorname{\mathrm{index}} {\mathfrak d}_{u,J}^N = \operatorname{\mathrm{index}}(u) - 2Z(\mathrm{d} u) \leqslant \operatorname{\mathrm{index}}(u). \end{align*} $$

Finally, everything is in place to define superrigidity.

Definition 2.1.8. Let $(M,J)$ be an almost complex manifold. A nonconstant J-holomorphic map u is rigid if $\ker {\mathfrak d}_{u,J}^N = 0$ .

A multiple cover $\tilde u$ of u may fail to be rigid, even if u itself is rigid.

Definition 2.1.9. Let $(M,J)$ be an almost complex manifold. A simple J-holomorphic map $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ is called superrigid if it is rigid and all of its multiple covers are rigid.

Remark 2.1.10. A generic simple pseudo-holomorphic map u satisfies $\operatorname {\mathrm {index}}(u) \geqslant 0$ . If u is also a rigid immersion, then Proposition 2.1.7 implies that $\operatorname {\mathrm {index}}(u)=0$ . Conversely, for a generic J every simple J-holomorphic map of index zero is a rigid immersion. In light of this, henceforth, we focus on index zero map.

Definition 2.1.11. Let M be a manifold of dimension at least six. An almost complex structure J on M is called superrigid if the following hold:

1. (1) Every simple J-holomorphic map has nonnegative index.

2. (2) Every simple J-holomorphic map of index zero is an embedding, and every two simple J-holomorphic maps of index zero either have disjoint images or are related by a reparametrization.

3. (3) Every simple J-holomorphic map of index zero is superrigid.

Remark 2.1.12. If $\dim M = 4$ , one should weaken condition (2) and require only that every simple J-holomorphic map of index zero is an immersion with transverse self-intersections and that two such maps are either transverse to one another or are related by a reparametrization. However, we will only be concerned with dimension at least six.

Let $(M,\omega )$ be a symplectic manifold. Bryan and Pandharipande [BP01, Section 1.2] conjectured that a generic almost complex structure compatible with $\omega $ is superrigid. This conjecture has recently been proved by Wendl [Wen19a]. This part of the present article is an exposition of Wendl’s proof using the theory developed in Part 1.

Theorem 2.1.13 (Wendl [Wen19b]).

Let $(M,\omega )$ be a closed, connected symplectic manifold with $\dim M \geqslant 6$ . Denote by $\mathscr {J}(M,\omega )$ the Fréchet manifold of smooth almost complex structures compatible with $\omega $ . The subspace $\mathscr {J}_{\blacklozenge }(M,\omega ) \subset \mathscr {J}(M,\omega )$ of superrigid almost complex structures is comeager.

Remark 2.1.14. By the Baire category theorem, $\mathscr {J}_{\blacklozenge }(M,\omega )$ is dense in $\mathscr {J}(M,\omega )$ . Therefore, every $J \in \mathscr {J}(M,\omega )$ can be arbitrarily slightly perturbed into a superrigid almost complex structure. The analogue is for paths $(J_t)_{t\in [0,1]}$ fails. This is discussed in detail in Section 2.10.

The proof of Theorem 2.1.13 occupies the bulk of the remainder of Part 2. Throughout the remainder of this part, let $(M,\omega )$ be a closed, connected symplectic manifold with $\dim M \geqslant 6$ .

2.2 Floer’s $C^{\varepsilon }$ spaces and Taubes’ trick

A technical but important issue in the proof of Theorem 2.1.13 is that $\mathscr {J}(M,\omega )$ is not a Banach manifold; hence, the theory developed in Part 1 cannot directly be brought to bear on it. The initial impulse might be to work with $C^k$ (instead of smooth) almost complex structures. However, Section 1.6 requires the linear elliptic differential operators under consideration to have smooth coefficients. The solution of this conundrum is to work with the $C^{\varepsilon }$ spaces introduced by Floer [Flo88, Section 5] and employ Taubes’ trick, which allows one to pass from the $C^{\varepsilon }$ topology to the $C^{\infty }$ topology; see [Tau96, Section 5] and [Wen19a, Appendix B] for applications to superrigidity. Wendl’s blog post [Wen21] clarifies how to properly use the $C^{\varepsilon }$ spaces. The purpose of this section is to marshal the salient facts required for the proof of Theorem 2.1.13.

Definition 2.2.1. Denote by

the set of sequences in $(0,\infty )$ . Define the preorder $\preccurlyeq $ on ${\mathfrak s}$ by

$$ \begin{align*} (\varepsilon_k) \preccurlyeq (\delta_k) \quad\text{if and only if}\quad \limsup_{k \to \infty} \frac{\varepsilon_k}{\delta_k} < \infty. \end{align*} $$

The following observation is nearly trivial but crucial.

Proposition 2.2.2. Every countable subset ${\mathfrak s}_0 \subset {\mathfrak s}$ has a lower bound; that is, there is a $\delta \in {\mathfrak s}$ with $\delta \preccurlyeq \varepsilon $ for every $\varepsilon \in {\mathfrak s}_0$ .

Proof. Enumerate ${\mathfrak s}_0 = {\lbrace }{ \varepsilon ^{\ell } : \ell \in {\mathbf {N}} }{\rbrace }$ and set .

Definition 2.2.3. Suppose that a Riemannian metric on M has been chosen. Let E be a Euclidean vector bundle over M equipped with an orthogonal connection. For $\varepsilon = (\varepsilon _k) \in {\mathfrak s}$ and $s \in \Gamma (E)$ set

The vector space

equipped with the norm ${\|}{\cdot }{\|}_{C^{\varepsilon }}$ is a separable Banach space [Wen19a, Theorems B.2, B.5].

Proposition 2.2.4. For every $\varepsilon \in {\mathfrak s}$ the inclusion $C^{\varepsilon }\Gamma (E) \to \Gamma (E)$ is continuous. Moreover,

$$ \begin{align*} \Gamma(E) = \bigcup_{\varepsilon \in {\mathfrak s}} C^{\varepsilon}\Gamma(E). \end{align*} $$

Proof. It is obvious that the inclusion $C^{\varepsilon }\Gamma (E) \to \Gamma (E)$ is continuous. If $s \in \Gamma (E)$ , then $s \in C^{\varepsilon }\Gamma (E)$ with . (Indeed, in light of Proposition 2.2.2, every countable subset of $\Gamma (E)$ is contained in $C^{\varepsilon }\Gamma (E)$ for some $\varepsilon \in {\mathfrak s}$ .)

The above spaces are used as follows. The tangent space to $\mathscr {J}(M,\omega )$ at $J_0$ is given by

$$ \begin{align*} T_{J_0}\mathscr{J}(M,\omega)= {\lbrace}{ \hat J \in \Gamma(\operatorname{\mathrm{End}}(TM)) : \hat JJ_0 + J_0\hat J = 0 \text{ and } \omega(\hat J\cdot,\cdot) + \omega(\cdot,\hat J\cdot) = 0 }{\rbrace}. \end{align*} $$

This means that $T_{J_0}\mathscr {J}$ consists of antilinear endomorphisms which are skew-adjoint with respect to $\omega $ . There is a $\delta> 0$ (independent of $J_0)$ such that the map

$$ \begin{align*} \exp_{J_0} \mskip0.5mu\colon\thinspace \big{\lbrace}{ \hat J \in T_{J_0}\mathscr{J}(M,\omega) : {\|}{\hat J}{\|}_{C^0} < \delta}\big{\rbrace} \to \mathscr{J}(M,\omega) \end{align*} $$

defined by

is a diffeomorphism.

Definition 2.2.5. For $\varepsilon \in {\mathfrak s}$ , define the Banach manifold $\mathscr {U}(M,\omega ;J_0,\varepsilon )$ by

covered by a single chart $\exp _{J_0}(\hat J) \mapsto \hat J$ .

For every $J_0 \in \mathscr {J}$ and $\varepsilon \in {\mathfrak s}$ , the inclusion $\mathscr {U}(M,\omega ;J_0,\varepsilon ) \subset \mathscr {J}(M,\omega )$ is continuous (but by no means open).

An almost complex structure $J \in \mathscr {J}(M,\omega )$ fails to be superrigid if there is a simple J-holomorphic map violating one of the conditions in Definition 2.1.21. The equivalence classes of theses offending pseudo-holomorphic maps form a topological space $\mathscr {M}^{\times }$ with a canonical projection map $\Pi \mskip 0.5mu\colon \thinspace \mathscr {M}^{\times } \to \mathscr {J}(M,\omega )$ . Therefore, proving Theorem 2.1.23 amounts to establishing that $\operatorname {\mathrm {im}} \Pi $ is meager. $\mathscr {M}^{\times }$ itself stands little chance to be Banach manifold, but its restriction to $\mathscr {U}(M,\omega ;J_0,\varepsilon )$ does. The upcoming definition and proposition extend Definition 1.B.1 and Proposition 1.B.4; the latter is an abstract version of Taubes’ trick mentioned earlier.

Definition 2.2.6. Let X be a topological space. Let $c \in {\mathbf {N}}_0$ . A subset $S \subset X$ has codimension at least c if there are:

1. (1) a preordered set $(A,\preccurlyeq )$ such that every countable subset $B \subset A$ has a lower bound; that is, there is an $\alpha \in A$ with $\alpha \preccurlyeq \beta $ for every $\beta \in B$ ;

2. (2) for every $x_0 \in X$ and $\alpha \in A$ a subset $U_{\alpha }(x_0) \subset X$ with $x_0 \in U_{\alpha }(x_0)$ and the structure of a $C^1$ Banach manifold on the set $U_{\alpha }(x_0)$ such that the inclusion $U_{\alpha }(x_0) \subset X$ is continuous;

3. (3) a metrizable topological space Z and a continuous map $\zeta \mskip 0.5mu\colon \thinspace Z \to X$ such that

   $$ \begin{align*} S \subset \operatorname{\mathrm{im}} \zeta \end{align*} $$
   and the following conditions hold:

   • (a) The map $\zeta $ is $\sigma $ -proper; that is, there is a countable cover $Z = \bigcup _{k \in {\mathbf {N}}} Z_k$ such that for every $k \in {\mathbf {N}}$ the restriction $\zeta |_{Z_k}$ is proper.

   • (b) For every $x_0 \in X$ the fiber $\zeta ^{-1}(x_0) \subset Z$ is separable.

   • (c) For every $z_0 \in \zeta ^{-1}(x_0)$ there is a $\beta = \beta (z_0) \in A$ and for every $\alpha \in A$ with $\alpha \preccurlyeq \beta $ , there is an open subset

     $$ \begin{align*} V_{\alpha}(z_0) \subset \zeta^{-1}(U_{\alpha}(x_0)) \end{align*} $$
     which contains $z_0$ and has the structure of a Banach orbifold such that the map $\zeta |_{V_{\alpha }(z_0)} \mskip 0.5mu\colon \thinspace V_{\alpha }(z_0) \to U_{\alpha }(x_0)$ is $C^1$ and $\operatorname {\mathrm {index}} \mathrm {d}_z \zeta \leqslant -c$ for every $z \in V_{\alpha }(z_0)$ .

Proposition 2.2.7. Let X be a completely metrizable topological space. If $S \subset X$ has codimension at least one, then S is meager.

Proof. It suffices to prove that $\zeta (Z_k)$ is closed and has empty interior. To prove that $\zeta (Z_k)$ is closed, observe that a proper map between metrizable topological spaces is closed. (Indeed, it suffices that the codomain is metrizable [Pal70].)

To prove that $\zeta (Z_k)$ has empty interior, let $x_0 \in \zeta (Z_k)$ . The task at hand is to exhibit a sequence $(x_n)$ that avoids $\zeta (Z_k)$ but converges to $x_0$ . Choose a countable dense subset ${\lbrace }{ z_m : m \in {\mathbf {N}} }{\rbrace } \subset \zeta ^{-1}(x_0)$ . Let $\alpha $ be a lower bound of ${\lbrace }{ \beta (\zeta _m) : m \in {\mathbf {N}} }{\rbrace }$ . The subset

is open and contains $\zeta _{\alpha }^{-1}(x_0)$ . By Proposition 1.B.4, the subset $\zeta (W_{\alpha }) \subset U_{\alpha }(x_0)$ is meager. By the Baire category theorem, $\zeta (W_{\alpha })$ is nowhere-dense in $U_{\alpha }(x_0)$ and there is a sequence $(x_n)$ that avoids $\zeta (W_{\alpha })$ but converges to $x_0$ in $U_{\alpha }(x_0)$ . Since the inclusion $U_{\alpha }(x_0) \subset X$ is continuous, $(x_n)$ converges to $x_0$ in X.

It remains to prove that $x_n \notin \zeta (Z_k)$ provided $n \gg 1$ . If not, then after passing to a subsequence there is a sequence $(z_n)$ in $Z_k$ with $\zeta (z_n) = x_n$ . Since $\zeta |_{Z_k}$ is proper, after passing to a further subsequence $(z_n)$ converges to a limit $z_0 \in Z_k$ with $\zeta (z_0) = x_0$ . Since $W_{\alpha }$ is open in $\zeta ^{-1}(U_{\alpha }(x_0))$ , for $n \gg 1$ , $z_n \in W_{\alpha }$ ; hence, $x_n \in \zeta (W_{\alpha })$ —a contradiction.

2.3 Flexibility

The following observation together with Proposition 2.2.2 implies that the various notions of flexibility introduced in Part 2 are satisfied. The reader might find it helpful at this point to review Definition 1.1.9 and to keep in mind that the normal Cauchy–Riemann operator of a J-holomorphic map u is an operator $\Gamma (E) \to \Gamma (F)$ , where $E = Nu$ and $F=\overline {\mathrm {Hom}}_{\mathbf {C}}(T\Sigma ,Nu)$ is the bundle of complex antilinear maps from $T\Sigma $ to $Nu$ .

Lemma 2.3.1. Let $J_0 \in \mathscr {J}(M,\omega )$ . Let $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a simple $J_0$ -holomorphic map. Consider the set of injective points

For every

$$ \begin{align*} A \in \Gamma(\operatorname{\mathrm{Hom}}(Nu,\overline{\operatorname{\mathrm{Hom}}}_{\mathbf{C}}(T\Sigma,Nu)) \end{align*} $$

with support in U, there are $\varepsilon \in {\mathfrak s}$ , $T> 0$ and a path of compatible almost complex structures $(J_t)_{t \in (-T,T)}$ through $J_0$ in $\mathscr {U}(M,\omega ;J_0,\varepsilon )$ such that:

1. (1) u is $J_t$ -holomorphic for every $t \in (-T,T)$ , and

2. (2) $\left .\frac {\mathrm {d}}{\mathrm {d} t}\right |_{t=0} {\mathfrak d}_{u,J_t}^N \xi = A \xi $ for every $\xi \in \Gamma (Nu)$ .

Proof. As discussed in Section 2.2,

$$ \begin{align*} T_{J_0}\mathscr{J}(M,\omega)= {\lbrace}{ \hat J \in \Gamma(\operatorname{\mathrm{End}}(TM)) : \hat JJ_0 + J_0\hat J = 0 \text{ and } \omega(\hat J\cdot,\cdot) + \omega(\cdot,\hat J\cdot) = 0 }{\rbrace}. \end{align*} $$

For $x \in U$ , $T_xM$ decomposes as $T_xM = T_x\Sigma \oplus N_x\Sigma $ . Given $a \in \Gamma (\overline {\mathrm {Hom}}_{\mathbf {C}}(T\Sigma ,Nu))$ , denote by $a^{\dagger }$ its adjoint with respect to $\omega $ and set

By construction, $\hat JJ + J\hat J = 0$ and $\omega (\hat J\cdot ,\cdot ) + \omega (\cdot ,\hat J\cdot ) = 0$ ; that is, $\hat J \in T_J\mathscr {J}(M,\omega )$ .

Given $A \in \Gamma (\operatorname {\mathrm {Hom}}(Nu,\overline {\operatorname {\mathrm {Hom}}}_{\mathbf {C}}(T\Sigma ,Nu))$ with support in U, pick $\hat J \in T_{J_0}\mathscr {J}(M,\omega )$ with $\hat J|_{u(\Sigma )} = 0$ and such that for every $\xi \in \Gamma (Nu)$

$$ \begin{align*} \frac12 \nabla_{\xi} \hat J = \begin{pmatrix} 0 & \left(A(\xi)j\right)^{\dagger} \\ -A(\xi)j & 0 \end{pmatrix}. \end{align*} $$

Let $T \ll 1$ , and define $(J_t)_{t \in (-T,T)}$ by

By construction, u is $J_t$ -holomorphic for every $t \in (-T,T)$ . It follows from equation (2.1.5) that

$$ \begin{align*} \left.\frac{\mathrm{d}}{\mathrm{d} t}\right|_{t=0} {\mathfrak d}_{u,J_t}^N \xi = A \xi. \end{align*} $$

for every $\xi \in \Gamma (Nu)$ . Evidently, $(J_t)_{t \in (-T,T)}$ is a path in $\mathscr {U}(M,\omega ;J_0,\varepsilon )$ provided $\varepsilon \in {\mathfrak s}$ decays sufficiently fast.

2.4 Unobstructedness and embeddedness

The purpose of this section is to take care of (1) and (2) in Definition 2.1.21.

Definition 2.4.1. Let $k\in \mathbf {Z}$ . The universal moduli space of simple J -holomorphic maps of index k over $\mathscr {J}(M,\omega )$ is the space $\mathscr {M}_k(M,\omega )$ of pairs $(J;[u,j])$ consisting of an almost complex structure $J \in \mathscr {J}(M,\omega )$ , and an equivalence class of simple J-holomorphic maps $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ of index k up to reparametrization by $\operatorname {\mathrm {Diff}}(\Sigma )$ .

Theorem 2.4.2. The subset

has codimension at least two (in the sense of Definition 2.2.6).

At its core, this is a standard transversality result for simple J-holomorphic maps; cf. [MS12, Theorem 3.1.5 (ii)]. Let us spell out its proof nevertheless because it illuminates Definition 2.2.6 and serves as a model for proofs of other parts of the superrigidity theorem.

Proof of Theorem 2.4.2.

The task at hand is to verify the conditions in Definition 2.2.6 for

with

denoting the projection map. Indeed, $\mathscr {W}_{\geqslant 0}(M,\omega ) = \operatorname {\mathrm {im}} \Pi $ .

Proposition 2.2.2 implies (1). (2) holds by construction. The fact that $\Pi $ is $\sigma $ -proper (3.a) is a consequence of the fact that quantitative bounds on the underlying Riemann surface $(\Sigma ,j)$ and the map u guarantee compactness; see [MS12, Proof of Theorem 3.1.5 (ii)] for details. The fact that the fibres $\Pi ^{-1}(J_0)$ are separable is standard.

Here is the crucial point: establishing (3.c). A neighborhood of $(J_0,[u_0,j_0]) \in \Pi ^{-1}({\mathscr {U}(M,\omega ;J_0,\varepsilon )})$ is given by

$$ \begin{align*} \mathscr{F}^{-1}(0)/\operatorname{\mathrm{Aut}}(\Sigma_0,j_0). \end{align*} $$

Here, $\mathscr {F}$ denotes the restriction of

$$ \begin{align*} W^{1,p}\Gamma(u_0^*TM)\oplus T_{j_0}\mathscr{S} \times T_{J_0}\mathscr{U}(M,\omega;J_0,\varepsilon) &\to L^p\Omega^{0,1}(u_0^*TM), \\ (\xi,j;J) &\mapsto \Phi(\xi)^{-1}\bar{\partial}_J(\exp_{u_0}(\xi),j) \end{align*} $$

to a sufficiently small $\operatorname {\mathrm {Aut}}(\Sigma _0,j_0)$ -invariant neigborhood of $0$ . Here, $p> 2$ , and

$$ \begin{align*} \Phi(\xi) \mskip0.5mu\colon\thinspace L^p(u_0^*TM) \to L^p\Gamma(\exp_{u_0}(\xi)^*TM) \end{align*} $$

is the complex bundle isomorphism induced by parallel transport. Since

$$ \begin{align*} \mathrm{d}_{u_0,j_0}\bar{\partial}_{J_0} \mskip0.5mu\colon\thinspace W^{1,p}\Gamma(u^*TM)\oplus T_{j_0}\mathscr{S} \to L^p\Omega^{0,1}(u^*TM) \end{align*} $$

is Fredholm, it has finite-dimensional cokernel. As a consequence of Proposition 2.2.2 and Lemma 2.3.1, there is a $\delta \in {\mathfrak s}$ such that every $\varepsilon \preccurlyeq \delta $ the linearization of $\mathscr {F}$ at $(0,j_0;J_0)$ is surjective. Hence, by the regular value theorem, there is an open neighborhood $\mathscr {V}([u_0,j_0];J_0,\varepsilon )$ of $([u_0,j_0];J_0) \in \Pi ^{-1}({\mathscr {U}(M,\omega ;J_0,\varepsilon )}$ which carries the structure of a Banach orbifold. For every, $([u,j];J) \in \mathscr {V}([u_0,j_0];J_0,\varepsilon )$ , we have

$$ \begin{align*} \operatorname{\mathrm{index}} \mathrm{d}_{[u,j];J} \Pi = \operatorname{\mathrm{index}} (u) \leqslant -2, \end{align*} $$

since, by the index formula (2.1.4), $\mathscr {M}_{-1}(M,\omega ) = \varnothing $ . Therefore, (3.c) holds.

Theorem 2.4.3 ([OZ09, Theorem 1.1; IP18, Proposition A.4]).

The subset

has codimension at least $2(n-2)$ .

2.5 Petri’s condition

The objective of the next five sections is to prove that

has codimension at least one in the sense of Definition 2.2.6. This will be achieved using the theory developed in Part 1 applied to certain families of elliptic operators which will be introduced in Section 2.6 and Section 2.9. The result of this section, proved by Wendl, ensures that real Cauchy–Riemann operators satisfy the algebraic condition introduced in Definition 1.6.15 and required in Theorem 1.6.17. This will guarantee that in the application the ${\mathfrak V}$ -equivariant Petri condition holds away from a subset of infinite codimension.

Definition 2.5.1. Let $(\Sigma ,j)$ be a Riemann surface, and let E be a complex vector bundle over $\Sigma $ . A real Cauchy–Riemann operator on E is a real linear first order elliptic differential operator $D \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma , E)$ satisfying

(2.5.2) $$ \begin{align} {\mathfrak d}(fs) = \bar{\partial} f \otimes_{\mathbf{C}} s + f {\mathfrak d} s \end{align} $$

for every $f \in C^{\infty }(\Sigma ,\mathbf {R})$ and $s \in \Gamma (E)$ . In the above formula, $\bar {\partial } f \in \Omega ^{0,1}(\Sigma )$ is defined by considering f as a ${\mathbf {C}}$ -valued function.

By equation (2.1.5), the normal Cauchy–Riemann operator associated with a J-holomorphic map is a real Cauchy–Riemann operator.

Theorem 2.5.3 (Wendl [Wen19b, Section 5.3]).

Every symbol of a real Cauchy–Riemann operator satisfies Wendl’s condition.

Before embarking on the proof of this result, let us remind the reader of the following fact. Let V and W be complex vector spaces. Denote by $\overline W$ the complex vector space W with scalar multiplication $(\lambda ,w)\mapsto \bar \lambda w$ . The tensor product $V\otimes W$ admits two commuting complex structures: and . $V\otimes W$ decomposes into the subspace on which $I_1=I_2$ and the subspace on which $I_1=-I_2$ . These can be identified with $V\otimes _{\mathbf {C}} W$ and $V\otimes _{\mathbf {C}} \overline W$ ; hence,

$$ \begin{align*} V\otimes W = ({V\otimes_{\mathbf{C}} W}) \oplus ({V\otimes_{\mathbf{C}} \overline W}). \end{align*} $$

The space of real linear maps $\operatorname {\mathrm {Hom}}(V,W)$ admits two commuting complex structures given by pre- and post-composition with i. This decomposes $\operatorname {\mathrm {Hom}}(V,W)$ into the space of complex linear map $\operatorname {\mathrm {Hom}}_{\mathbf {C}}(V,W)$ and the space of complex antilinear maps $\overline {\mathrm {Hom}}_{\mathbf {C}}(V,W)$ ; that is,

$$ \begin{align*} \operatorname{\mathrm{Hom}}(V,W) = \operatorname{\mathrm{Hom}}_{\mathbf{C}}(V,W) \oplus \overline{\operatorname{\mathrm{Hom}}}_{\mathbf{C}}(V, W).\\[-30pt] \end{align*} $$

Proof of Theorem 2.5.3.

The symbol

at $x \in \Sigma $ of a real Cauchy–Riemann operator ${\mathfrak d}\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E) = \Gamma (F)$ depends only on $E_x$ . Denote by $z = s+it$ a local holomorphic coordinate around x and identify $E_x = {\mathbf {C}}^r$ with

. Identifying

$$ \begin{align*} \mathbf{R}[s,t]\otimes E_x = \mathbf{R}[s,t]\otimes F_x = \mathbf{R}[s,t]\otimes E_x^{\dagger} = \mathbf{R}[s,t]\otimes F_x^{\dagger} = {\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} {\mathbf{C}}^r \end{align*} $$

the formal differential operators $\hat \sigma $ and $-\hat \sigma ^{\dagger }$ both become

$$ \begin{align*} \bar{\partial}\otimes_{\mathbf{C}} \mathrm{id}_{{\mathbf{C}}^r} \mskip0.5mu\colon\thinspace {\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} {\mathbf{C}}^r \to {\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} {\mathbf{C}}^r. \end{align*} $$

Furthermore, identifying

$$ \begin{align*} {\mathbf{C}}^r \otimes {\mathbf{C}}^r = ({{\mathbf{C}}^r \otimes_{\mathbf{C}} {\mathbf{C}}^r}) \oplus ({{\mathbf{C}}^r \otimes_{\mathbf{C}} \bar {\mathbf{C}}^r}) \end{align*} $$

via $v\otimes w \mapsto (v\otimes _{\mathbf {C}} w, v\otimes _{\mathbf {C}} \bar w)$ the polynomial Petri map $\hat \varpi $ becomes

$$ \begin{align*} (\hat\varpi_1,\hat\varpi_2) \mskip0.5mu\colon\thinspace ({{\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} {\mathbf{C}}^r})^{\otimes 2} \to {\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} ({{\mathbf{C}}^r\otimes_{\mathbf{C}} {\mathbf{C}}^r}) \oplus {\mathbf{C}}[z,\bar z]\otimes_{\mathbf{C}} ({{\mathbf{C}}^r\otimes_{\mathbf{C}} \bar{\mathbf{C}}^r}) \end{align*} $$

defined by

From this, it is evident that it suffices to consider the case $r = 1$ to prove Theorem 2.5.3.

Proposition 2.5.4. If $B \in \ker \hat \varpi _{\sigma }$ is homogeneous of degree d, then it is of the form

(2.5.5) $$ \begin{align} B = \sum_{j=0}^d b_j \big({z^j\, \otimes z^{d-j} - iz^j \otimes iz^{d-j}}\big) + b_j' \big({iz^j \otimes z^{d-j} + z^j \otimes iz^{d-j}}\big). \end{align} $$

with $b,b' \in \mathbf {R}^d$ satisfying

(2.5.6) $$ \begin{align} \sum_{j=0}^d b_j = 0 \quad\text{and}\quad \sum_{j=0}^d b_j' = 0. \end{align} $$

Proof. Every homogeneous $B \in \ker \bar {\partial } \otimes \ker \bar {\partial }$ of degree d is of the form

$$ \begin{align*} B = \sum_{j=0}^d b_j z^j\, \otimes z^{d-j} + b_j' iz^j \otimes z^{d-j} + b_j" z^j \otimes iz^{d-j} + b_j"' iz^j \otimes iz^{d-j} \end{align*} $$

with $b,b',b",b"' \in \mathbf {R}^d$ . B satisfies $\hat \varpi _2(B) = 0$ if and only if for every $j=0,\ldots ,d$

$$ \begin{align*} b_j + b_j"' = 0 \quad\text{and}\quad b_j' - b_j" = 0; \end{align*} $$

that is, B is of the form (2.5.5). If B is of this form, then $\hat \varpi _1(B) = 0$ is equivalent to equation (2.5.6).

Henceforth, let $B \in \ker \varpi _{\sigma }$ be homogeneous of degree d. The right-inverse of $\hat \sigma (=-\hat \sigma ^{\dagger })$ can be chosen as

(2.5.7) $$ \begin{align} \hat R (z^{\alpha} \bar z^{\beta}) = \frac{1}{\beta+1}z^{\alpha}\bar z^{\beta+1}. \end{align} $$

Define the map $\hat {\mathbf {L}}_{\sigma ,B} \mskip 0.5mu\colon \thinspace {\mathbf {C}}[z,\bar z] \otimes _{\mathbf {C}} \operatorname {\mathrm {Hom}}({\mathbf {C}},{\mathbf {C}}) \to {\mathbf {C}}[z,\bar z]\otimes _{\mathbf {C}} ({\mathbf {C}} \otimes {\mathbf {C}})$ by

as in Definition 1.6.15. $\operatorname {\mathrm {Hom}}({\mathbf {C}},{\mathbf {C}})$ and ${\mathbf {C}}\otimes {\mathbf {C}}$ decompose as

$$ \begin{align*} {\mathrm{Hom}}({\mathbf{C}},{\mathbf{C}}) = {\mathrm{Hom}}_{\mathbf{C}}({\mathbf{C}},{\mathbf{C}}) \oplus \overline{{\mathrm{Hom}}}_{\mathbf{C}}({\mathbf{C}},{\mathbf{C}}) \quad\text{and}\quad {\mathbf{C}}\otimes{\mathbf{C}} = ({\mathbf{C}}\otimes_{\mathbf{C}} {\mathbf{C}}) \oplus ({\mathbf{C}} \otimes_{\mathbf{C}} \overline{\mathbf{C}}). \end{align*} $$

This induces decompositions of the domain and codomain of $\hat {\mathbf {L}}_{\sigma ,B}$ . Each of the summands is isomorphic to ${\mathbf {C}}[z,\bar z]$ . With respect to these decompositions, $\hat {\mathbf {L}}_{\sigma ,B}$ is a matrix of four operators ${\mathbf {C}}[z,\bar z] \to {\mathbf {C}}[z, \bar z]$ . Denote by

$$ \begin{align*} {\mathbf{Q}}_B\mskip0.5mu\colon\thinspace {\mathbf{C}}[z,\bar z] \to {\mathbf{C}}[z,\bar z] \end{align*} $$

the bottom-right component of $\hat {\mathbf {L}}_{\sigma ,B}$ , that is, the restriction of $\hat {\mathbf {L}}_{\sigma ,B}$ to ${\mathbf {C}}[z,\bar z]\otimes _{\mathbf {C}} \overline {\mathrm {Hom}}_{\mathbf {C}}({\mathbf {C}},{\mathbf {C}})$ composed with the projection to ${\mathbf {C}}[z,\bar z]\otimes _{\mathbf {C}} ({{\mathbf {C}}\otimes _{\mathbf {C}} \overline {\mathbf {C}}})$ . ¹² (The other components of $\hat {\mathbf {L}}_{\sigma ,B}$ can be seen to vanish.) For $\ell \in {\mathbf {N}}_0$ denote by ${\mathbf {C}}[z,\bar z]^{\leqslant \ell }$ the ring of polynomials in z and $\bar z$ of degree at most $\ell $ and denote by ${\mathbf {Q}}^{\leqslant \ell } \mskip 0.5mu\colon \thinspace {\mathbf {C}}[z,\bar z]^{\leqslant \ell } \to {\mathbf {C}}[z,\bar z]^{\leqslant \ell +1}$ the truncation of ${\mathbf {Q}}_B$ . The map ${\mathbf {Q}}_B$ is complex linear. Since

$$ \begin{align*} \operatorname{\mathrm{rk}} \hat{\mathbf{L}}_{\sigma,B}^{\leqslant\ell} \geqslant \operatorname{\mathrm{rk}}_{\mathbf{C}} {\mathbf{Q}}_B^{\leqslant \ell}, \end{align*} $$

it suffices to estimate the latter.

Proposition 2.5.8. The map ${\mathbf {Q}}_B$ satisfies

$$ \begin{align*} \operatorname{\mathrm{rk}}_{\mathbf{C}} {\mathbf{Q}}_B^{\leqslant \ell} \geqslant \frac{1}{16}\ell^2 \end{align*} $$

for $\ell \geqslant 8d$ .

Proof. The map ${\mathbf {Q}}_B$ can be computed explicitly. To do so, observe that $\overline {\mathrm {Hom}}_{\mathbf {C}}({\mathbf {C}},{\mathbf {C}}) = {\mathbf {C}}$ acts on ${\mathbf {C}}$ via

and its adjoint is

; furthermore, recall that $\hat R^{\dagger } = - \hat R$ and that $\hat R$ is given by equation (2.5.7). With this in mind, it is easy to verify that for

$$ \begin{align*} A = z^{\alpha} \bar z^{\beta} \quad\text{and}\quad B = \sum_{j=0}^d b_j \big({z^j\, \otimes z^{d-j} - iz^j \otimes iz^{d-j}}\big) + b_j' \big({iz^j \otimes z^{d-j} + z^j \otimes iz^{d-j}}\big) \end{align*} $$

the map ${\mathbf {Q}}_B$ satisfies

$$ \begin{align*} {\mathbf{Q}}_B(A) &= \hat\varpi_2 \big({ (\hat R A\otimes \mathbf{1} + \mathbf{1}\otimes \hat R^{\dagger} A^{\dagger}) B }\big) \\ &= \sum_{j=0}^d 2(b_j-ib_j') \hat R(z^{\alpha}\bar z^{\beta+j}) \bar z^{d-j} - 2(b_j+ib_j') z^j \overline{\hat R(z^{\beta} \bar z^{\alpha+d-j})} \\ &= \sum_{j=0}^d \frac{2(b_j-ib_j')}{\beta+j+1} z^{\alpha}\bar z^{\beta+d+1} - \frac{2(b_j+ib_j')}{\alpha+d-j+1} z^{\alpha+d+1}\bar z^{\beta} \\ &= \bigg({p_{\beta}^B \bar z^{d+1} + q_{\alpha}^B z^{d+1}}\bigg) z^{\alpha}\bar z^{\beta} \end{align*} $$

with

The same formula holds for ${\mathbf {Q}}_B^{\leqslant \ell }$ provided $\alpha +\beta +d \leqslant \ell $ .

Set

Choose a subset $S^{\star } \subset S$ such that $\#S^{\star } \geqslant \frac 12\#S$ and such that if $(\alpha ,\beta ) \in S$ , then $(\alpha -d-1,\beta +d+1) \notin S$ . The restriction of ${\mathbf {Q}}_B^{\leqslant \ell }$ to ${\mathbf {C}}{\langle }{z^{\alpha } \bar z^{\beta } : (\alpha ,\beta ) \in S^{\star } }{\rangle }$ is injective. The latter is evident from the construction of $S^{\star }$ and

$$ \begin{align*} {\mathbf{Q}}_B\bigg({\sum_{\alpha,\beta \in S^{\star}} \lambda_{\alpha,\beta} z^{\alpha}\bar z^{\beta}}\bigg) = \sum_{\alpha,\beta \in S^{\star}} \lambda_{\alpha,\beta} \bigg({ p_{\beta}^B z^{\alpha}\bar z^{\beta+d+1} + q_{\alpha}^B z^{\alpha+d+1}\bar z^{\beta} }\bigg). \end{align*} $$

Therefore,

$$ \begin{align*} \operatorname{\mathrm{rk}}_{\mathbf{C}} {\mathbf{Q}}_B \geqslant \frac12 \#S. \end{align*} $$

It remains to find a lower bound on $\#S$ . At most d of the numbers $p_0^B,p_1^B,\ldots $ are nonzero. This is a consequence of the following. If $\beta _0,\ldots ,\beta _{d+1}$ are $d+1$ distinct positive numbers, then the matrix

$$ \begin{align*} \begin{pmatrix} \frac{1}{\beta_0+1} & \frac{1}{\beta_0+2} & \ldots & \frac{1}{\beta_0+d+1} \\ \frac{1}{\beta_0+1} & \frac{1}{\beta_1+2} & \ldots & \frac{1}{\beta_1+d+1} \\ \ldots & \ldots & \ldots & \ldots \\ \frac{1}{\beta_d+1} & \frac{1}{\beta_d+2} & \ldots & \frac{1}{\beta_d+d+1} \end{pmatrix} \end{align*} $$

is a Cauchy matrix and, therefore, invertible. If $p_{\beta _0}^B = \cdots = p_{\beta _d}^B = 0$ , then the product of the above matrix with $(b_0-ib_0,\ldots ,b_d-ib_d)$ would vanish—a contradiction. A variation of this argument shows that at most d of the numbers $q_0^B,q_1^B,\ldots $ are nonzero. Therefore,

$$ \begin{align*} \#S \geqslant \frac{(\ell - d)^2}{4} - (\ell-d)d. \end{align*} $$

This implies the assertion.

This finishes the proof of Theorem 2.5.3 with $c_0(\rho ,d) = \frac {1}{16}$ and $\ell _0(\rho ,d) = 8d$ .

2.6 Rigidity of unbranched covers

The purpose of this section is to prove the following.

Proposition 2.6.1 (cf. [GW17, Theorem 1.3]).

Denote by $\mathscr {W}_{\lozenge }(M,\omega )$ the subset of those $J \in \mathscr {J}(M,\omega )$ for which there is a simple J-holomorphic map u of index zero such that an unbranched cover of u fails to be rigid. $\mathscr {W}_{\lozenge }(M,\omega )$ has codimension at least one (in the sense of Definition 2.2.6).

This is a only warm-up because it does not account for branched covers. However, it is instructive to see the proof in the special case as it will be a model for the general case. The following discussion puts us in a position to prove Proposition 2.6.1 using Theorem 1.3.5.

Definition 2.6.2. Let $(\Sigma ,j)$ and $(\tilde \Sigma ,\tilde j)$ be Riemann surfaces, let E be a complex vector bundle over $\Sigma $ and let ${\mathfrak d}\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E)$ be a real Cauchy–Riemann operator. Let $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,j) \to (\Sigma ,j)$ be a nonconstant holomorphic map. The pullback of ${\mathfrak d}$ by $\pi $ is the real Cauchy–Riemann operator

$$ \begin{align*} \pi^{\star}{\mathfrak d} \mskip0.5mu\colon\thinspace \Gamma(\pi^*E) \to \Omega^{0,1}(\tilde \Sigma,\pi^*E) \end{align*} $$

characterized by

$$ \begin{align*} (\pi^{\star} {\mathfrak d})(\pi^*s) = \pi^*({\mathfrak d} s). \end{align*} $$

The following is proved as Proposition 2.A.3 in Section 2.A.

Proposition 2.6.3. Let $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. If $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (\Sigma ,j)$ is a nonconstant holomorphic map and , then there is an isomorphism $N(u\circ \pi ) \cong \pi ^*Nu$ with respect to which ${\mathfrak d}_{u\circ \pi ,J}^N = \pi ^{\star }{\mathfrak d}_{u,J}^N$ .

In the situation of Definition 2.6.2, if $\pi $ is covering map, then $\pi ^{\star }{\mathfrak d}$ and $\pi ^*{\mathfrak d}$ defined in Definition 1.2.1 are related by the commutative diagram

with $\pi ^*$ being an isomorphism. (This explains the intentionally confusing choice of notation.) Therefore and by Proposition 1.2.9, if $J \in \mathscr {W}_{\lozenge }$ , then there exists a simple J-holomorphic map $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ of index zero and an irreducible Euclidean local system ${\underline V}$ whose monodromy representation factors through a finite quotient of $\pi _1(\Sigma ,x_0)$ such that

$$ \begin{align*} \ker {\mathfrak d}_{u,J}^{{\underline V}} \neq 0. \end{align*} $$

Proof of Proposition 2.6.1.

As in the proof of Theorem 2.4.2, the only nontrivial part is to verify condition (3.c) from Definition 2.2.6. To that end, let $(J_0, [u_0,j_0]) \in \mathscr {M}_0(M,\omega )$ , and let $\varepsilon \in {\mathfrak s}$ be a sequence converging to zero. By Theorem 2.4.3 it suffices to consider the case when $u_0 \colon \Sigma \to M$ is an embedding. Let $\mathscr {V}$ be an open neighborhood of $(J_0, [u_0,j_0])$ in the universal moduli space of simple maps over $\mathscr {U} = \mathscr {U}(M,\omega ;J_0,\varepsilon ) \subset \mathscr {J}(M,\omega )$ with the following two properties. First, for every $(J,[u,j]) \in \mathscr {V}$ the map u is an embedding. Second, $\mathscr {V}$ is liftable in the sense that there is an $\operatorname {\mathrm {Aut}}(\Sigma ,j_0)$ -invariant slice $\mathscr {S}$ of Teichmüller space through $j_0$ such that for every $(J,[u,j]) \in \mathscr {V}$ there is a unique lift $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ with $j \in \mathscr {S}$ .

Consider the family of normal Cauchy–Riemann operators ${\mathfrak d}_{u,J}^N\mskip 0.5mu\colon \thinspace \Gamma (Nu) \to \Omega ^{0,1}(Nu)$ parametrized by $\mathscr {V}$ . Strictly speaking, this is not a family of linear elliptic differential operators as in Definition 1.1.1. Indeed, $Nu$ and $\overline {\mathrm {Hom}}_{\mathbf {C}}(T\Sigma ,Nu)$ depend on u and j and thus define vector bundles ${\mathbf {E}}$ and ${\mathbf {F}}$ over $\mathscr {V}\times \Sigma $ . However, after shrinking $\mathscr {V}$ , one can construct isomorphisms ${\mathbf {E}} \cong \mathrm {pr}_{\Sigma }^* E$ and ${\mathbf {F}} \cong \mathrm {pr}_{\Sigma }^* F$ with and for $(J_0,[u_0,j_0]) \in \mathscr {V}$ . Employing these isomorphisms, the normal Cauchy–Riemann operators form a family of linear elliptic differential operators

$$ \begin{align*} {\mathfrak d} \mskip0.5mu\colon\thinspace \mathscr{V} \to \mathscr{F}(W^{1,2}\Gamma(E),L^2\Gamma(F)) \end{align*} $$

as in Definition 1.1.1. A moment’s thought shows that the map $\Lambda _p^{\mathfrak V}$ defined in Theorem 1.3.5, ${\mathfrak V}$ -equivariant flexiblity defined in Definition 1.3.8, and the ${\mathfrak V}$ -equivariant Petri condition defined in Definition 1.3.9 are independent of the choice of isomorphisms ${\mathbf {E}} \cong \mathrm {pr}_{\Sigma }^* E$ and ${\mathbf {F}} \cong \mathrm {pr}_{\Sigma }^* F$ . Therefore, the results from Section 1.3 apply without reservation.

By Lemma 2.3.1, there is $\delta \in {\mathfrak s}$ such that ${\mathfrak d}$ is ${\mathfrak V}$ -equivariantly flexible in U provided $\varepsilon \preccurlyeq \delta $ . Furthermore, by Theorem 1.6.17 and Theorem 2.5.3, the subset of $(J,[u,j]) \in \mathscr {V}$ for which ${\mathfrak d}_{u,J}^N$ fails to satisfy the ${\mathfrak V}$ -equivariant Petri condtion in U has infinite codimension.

Let ${\underline V}$ be an irreducible Euclidean local system on $\Sigma $ whose monodromy representation factors through a finite quotient of $\pi _1(\Sigma ,x_0)$ and consider ${\mathfrak V}$ as in Situation 1.3.1 consisting only of ${\underline V}$ . Denote by

$$ \begin{align*} \mathscr{W}_{\Lambda;{\underline V}} \end{align*} $$

the subset of those

for which the map $\Lambda _p^{\mathfrak V}$ defined in Theorem 1.3.5 fails to be surjective. Evidently, $\mathscr {W}_{\Lambda ;{\underline V}}$ is closed; in particular, $\mathscr {V}{\backslash } \mathscr {W}_{\Lambda ;{\underline V}}$ is a Banach manifold. By the preceding paragraph, $\mathscr {W}_{\Lambda ;{\underline V}}$ has infinite codimension. Set

Since

$$ \begin{align*} \operatorname{\mathrm{index}} {\mathfrak d}_{u,J}^{N,{\underline V}} = \operatorname{\mathrm{rk}}{\underline V} \cdot \operatorname{\mathrm{index}} {\mathfrak d}_{u,J}^N \leqslant \operatorname{\mathrm{rk}}{\underline V} \cdot \operatorname{\mathrm{index}}(u) = 0, \end{align*} $$

by Theorem 1.3.5, $\mathscr {W}_{\lozenge ;{\underline V}}$ has codimension at least one. Since $(J_0,[u_0,j_0])$ was arbitrary and there are countably many isomorphism classes of irreducible Euclidean local systems, condition (3.c) from Definition 2.2.6 is satisfied.

The next two sections develop tools with which the above argument can be carried over to branched covering maps.

2.7 Branched covering maps as orbifold covering maps

An orbifold Riemann surface can be constructed from a smooth Riemann surface and a collection of points equipped with multiplicities. ¹³ The starting point of this construction is the following observation. Denote by ${\mathbf {D}}$ the unit disk in ${\mathbf {C}}$ . For $k \in {\mathbf {N}}$ denote by

the group of $k^{\text {th}}$ roots of unity. The map $\pi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to {\mathbf {D}}$ defined by

induces a homeomorphism ${\mathbf {D}}/\mu _k \cong {\mathbf {D}}$ . Denote by $[{\mathbf {D}}/\mu _k]$ the orbifold with ${\mathbf {D}}$ as the underlying topological space and $\pi $ as chart. The map $\pi $ also induces an orbifold map $\beta \mskip 0.5mu\colon \thinspace [{\mathbf {D}}/\mu _k] \to {\mathbf {D}}$ which induces the identity map on the underlying topological spaces. The identity map ${\mathbf {D}} \to {\mathbf {D}}$ defines an orbifold map $\hat \pi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to [{\mathbf {D}}/\mu _k]$ . This map is a covering map because ${\mathbf {D}} \cong [({\mathbf {D}}\times \mu _k)/\mu _k]$ ; cf. Footnote 7 on page 8. By construction, $\pi = \beta \circ \hat \pi $ . This can be globalized as follows.

Definition 2.7.1. Let $(\Sigma ,j)$ be a Riemann surface. A multiplicity function is a function $\nu \mskip 0.5mu\colon \thinspace \Sigma \to {\mathbf {N}}$ such that the set

is discrete. Given a multiplicity function $\nu $ , denote by $(\Sigma _{\nu },j_{\nu })$ the orbifold Riemann surface whose underlying topological space is $\Sigma $ and such that for every $x \in \Sigma $ and every holomorphic chart $\phi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to \Sigma $ with $\phi (0) = x$ the map $\phi _{\nu (x)} \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to \Sigma $ defined by

is a holomorphic orbifold chart. Denote by $\beta _{\nu }\mskip 0.5mu\colon \thinspace (\Sigma _{\nu },j_{\nu }) \to (\Sigma ,j)$ the holomorphic orbifold map given by $z \mapsto z^{\nu (x)}$ with respect to these charts. The underlying continuous map of topological spaces is the identity map $\Sigma \to \Sigma $ .

Remark 2.7.2. An orbifold is effective if the local stabilizer group of every point acts effectively. Every effective orbifold Riemann surface is isomorphic to one constructed as in Definition 2.7.1.

This construction allows us to canonically associate an orbifold cover with every branched cover of Riemann surfaces.

Proposition 2.7.3. Let $(\Sigma ,j)$ and $(\tilde \Sigma ,\tilde j)$ be Riemann surfaces, and let $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (\Sigma ,j)$ be a nonconstant holomorphic map. For every $\tilde x \in \tilde \Sigma $ denote by $r(\tilde x) \in {\mathbf {N}}$ the ramification index of $\pi $ at $\tilde x$ . Define $\nu \mskip 0.5mu\colon \thinspace \Sigma \to {\mathbf {N}}$ and $\tilde \nu \mskip 0.5mu\colon \thinspace \tilde \Sigma \to {\mathbf {N}}$ by

Let $(\tilde \Sigma _{\tilde \nu },\tilde j)$ and $(\Sigma _{\nu },j_{\nu })$ be the corresponding orbifold Riemann surfaces constructed in Definition 2.7.1. There is a unique holomorphic covering map $\hat \pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma _{\tilde \nu },\tilde j_{\tilde \nu }) \to (\Sigma _{\nu },j_{\nu })$ such that the diagram

(2.7.4)

commutes.

Proof. For every $x \in \Sigma $ , there is a holomorphic chart $\phi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to \Sigma $ with $\phi (0) = x$ and for every $\tilde x \in \pi ^{-1}(x)$ there is a holomorphic chart $\tilde \phi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to \tilde \Sigma $ such that $\tilde \phi (0) = \tilde x$ and $\pi \circ \phi (z) = \phi (z^{r(\tilde x)})$ . There is a unique orbifold map $\hat \pi \mskip 0.5mu\colon \thinspace \tilde \Sigma _{\tilde \nu } \to \Sigma _{\nu }$ which is given by the identity map with respect to the charts $\phi _{\tilde \nu (\tilde x)}$ and $\phi _{\nu (x)}$ . Evidently, this map is holomorphic. It is a covering map because

$$ \begin{align*} [{\mathbf{D}}/\mu_{\tilde\nu(\tilde x)}] \cong [{ ({ {\mathbf{D}} \times_{\mu_{\tilde \nu(\tilde x)}} \mu_{\nu(x)} })] /\mu_{\nu(x)} } \end{align*} $$

and the canonical map

$$ \begin{align*} {\mathbf{D}} \times_{\mu_{\tilde \nu(\tilde x)}} \mu_{\nu(x)} \to {\mathbf{D}} \end{align*} $$

is a $\mu _{\nu (x)}$ -equivariant covering map.

Remark 2.7.5. Every covering map of effective orbifold Riemann surfaces arises from a branched cover of the underlying smooth Riemann surfaces by the above construction.

2.8 A criterion for the failure of superrigidity

Proposition 2.8.3 below shows that the orbifoldization process from Definition 2.7.1 does not affect the kernel and cokernel of real Cauchy–Riemann operators.

Remark 2.8.1. The notion of a real Cauchy–Riemann operators from Definition 2.5.1 can be easily adapted to the orbifold setting. A complex bundle E over an orbifold Riemann surface $\Sigma $ is given, in a local orbifold chart ${\mathbf {D}} / \Gamma $ , where $\Gamma $ is the local stabilizer group, by a $\Gamma $ -equivariant bundle over the disc ${\mathbf {D}}$ . In this chart, sections of E correspond to $\Gamma $ -equivariant sections over ${\mathbf {D}}$ . A real Cauchy–Riemann operator is a linear map ${\mathfrak d} \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E)$ which for sections supported in a local orbifold chart agrees with a $\Gamma $ -equivariant real Cauchy–Riemann operator on ${\mathbf {D}}$ . The pullback construction from Definition 2.6.2 generalizes in an obvious way, by pulling back $\Gamma $ -equivariant operators in in every orbifold chart. For more details on vector bundles and elliptic operators over orbifolds, see, for example, [Kaw79; Kaw81], [SY19, Sections 2-3]; see also [Moe02, Section 5] for a groupoid perspective. ¹⁴

Definition 2.8.2. Let $(\Sigma ,j)$ be a Riemann surface with a multiplicity function $\nu $ . Given a complex vector bundle E over $\Sigma $ and a real Cauchy–Riemann operator ${\mathfrak d} \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E)$ , set

with $\beta _{\nu }^{\star }{\mathfrak d} \mskip 0.5mu\colon \thinspace \Gamma (E_{\nu }) \to \Omega ^{0,1}(\Sigma _{\nu },E_{\nu })$ as in Definition 2.6.2.

Proposition 2.8.3. If $(\Sigma ,j)$ is a closed Riemann surface with a multiplicity function $\nu $ , E is a complex vector bundle over $\Sigma $ and ${\mathfrak d} \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E)$ is a real Cauchy–Riemann operator, then

$$ \begin{align*} \ker {\mathfrak d}_{\nu} \cong \ker {\mathfrak d} \quad\text{and}\quad \operatorname{\mathrm{coker}} {\mathfrak d}_{\nu} \cong \operatorname{\mathrm{coker}} {\mathfrak d}. \end{align*} $$

Proof. The pullback map $\beta _{\nu }^* \mskip 0.5mu\colon \thinspace \Gamma (E) \to \Gamma (E_{\nu })$ induces an injection $\ker {\mathfrak d} \hookrightarrow \ker {\mathfrak d}_{\nu }$ . In fact, this map is an isomorphism. To see this, the following local consideration suffices. Let $x \in \Sigma $ , and set . Define $\beta \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to {\mathbf {D}}$ by . Choose a holomorphic chart $\phi \mskip 0.5mu\colon \thinspace {\mathbf {D}} \to \Sigma $ with $\phi (0) = x$ and a trivialization of E over $\phi ({\mathbf {D}})$ . With respect to these ${\mathfrak d}$ and ${\mathfrak d}_{\nu }$ can be written as

$$ \begin{align*} {\mathfrak d} = \bar{\partial} + {\mathfrak n} \quad\text{and}\quad {\mathfrak d}_{\nu} = \bar{\partial} + \beta^*{\mathfrak n} \end{align*} $$

for some ${\mathfrak n} \in \Omega ^{0,1}({\mathbf {D}},\operatorname {\mathrm {End}}_{\mathbf {R}}({\mathbf {C}}^r))$ . If $\tilde s \in C^{\infty }({\mathbf {D}},{\mathbf {C}}^r)$ is $\mu _k$ -invariant, then there is a bounded map $s \in C^{\infty }({\mathbf {D}}{\backslash }{\lbrace }{0}{\rbrace },{\mathbf {C}}^r)$ such that $\tilde s = s\circ \beta $ . If $(\bar {\partial } + \beta ^*{\mathfrak n})\tilde s = 0$ , then $(\bar {\partial } + {\mathfrak n})s = 0$ ; hence, s extends to ${\mathbf {D}}$ by elliptic regularity.

Since $\operatorname {\mathrm {coker}} {\mathfrak d} \cong (\ker {\mathfrak d}^{\dagger })^*$ and similarly for ${\mathfrak d}_{\nu }$ , it suffices to produce an isomorphism $\ker {\mathfrak d}^{\dagger } \cong \ker {\mathfrak d}_{\nu }^{\dagger }$ . The formal adjoints ${\mathfrak d}^{\dagger } \mskip 0.5mu\colon \thinspace \Omega ^{1,0}(\Sigma ,E^*) \to \Omega ^{1,1}(\Sigma ,E^*)$ and ${\mathfrak d}_{\nu }^{\dagger } \mskip 0.5mu\colon \thinspace \Omega ^{1,0}(\Sigma _{\nu },E_{\nu }^*) \to \Omega ^{1,1}(\Sigma _{\nu },E_{\nu }^*)$ are real Cauchy–Riemann operators acting on $(1,0)$ -forms and locally of the form ${\mathfrak d}^{\dagger } = \bar {\partial } + {\mathfrak n}$ and ${\mathfrak d}_{\nu }^{\dagger } = \bar {\partial } + \beta ^*{\mathfrak n}$ . The pullback map $\beta _{\nu }^* \mskip 0.5mu\colon \thinspace \Omega ^{1,0}(\Sigma ,E) \to \Omega ^{1,0}(\Sigma _{\nu },E_{\nu })$ induces an injection $\ker {\mathfrak d}^{\dagger } \hookrightarrow \ker {\mathfrak d}_{\nu }^{\dagger }$ . This map is an isomorphism by the following local consideration. If $\tilde s \in C^{\infty }({\mathbf {D}},{\mathbf {C}}^r)$ is such that $\tilde s \,\mathrm {d} z$ is $\mu _k$ -invariant, then there is a map $s \in C^{\infty }({\mathbf {D}}{\backslash }{\lbrace }{0}{\rbrace },{\mathbf {C}}^r)$ such that $\tilde s = kz^{k-1}s\circ \beta $ . If $(\bar {\partial } + \beta ^*{\mathfrak n})\tilde s = 0$ , then $(\bar {\partial } + {\mathfrak n})s = 0$ and a consideration of the Taylor expansion of $\tilde s$ shows that s is bounded. Therefore, s extends to ${\mathbf {D}}$ and $\tilde s \ \mathrm {d} z = \beta ^*(s \,\mathrm {d} z)$ .

This together with the discussion in Section 2.7 leads to the following criterion for the failure of superrigidity.

Definition 2.8.4. Let $(\Sigma ,j)$ be a Riemann surface with a multiplicity function $\nu $ , and let $x_0 \in \Sigma {\backslash } Z_{\nu }$ . For every $x \in Z_{\nu }$ , there is a conjugacy class of a subgroup $\mu _{\nu (x)} \leqslant \pi _1(\Sigma _{\nu },x_0)$ , generated by the homotopy class of a loop in $\Sigma {\backslash } Z_{\nu }$ based at $x_0$ which is contractible in $(\Sigma {\backslash } Z_{\nu }) \cup {\lbrace }{x}{\rbrace }$ and goes around x once. If ${\underline V}$ is a Euclidean local system on $\Sigma _{\nu }$ , then its monodromy around x is the representation $\mu _{\nu (x)} \to \mathrm {O}(V)$ induced by the monodromy representation.

Proposition 2.8.5. Let $u \mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a simple J-holomorphic map. If u is not superrigid, then there are a multiplicity function $\nu \mskip 0.5mu\colon \thinspace \Sigma \to {\mathbf {N}}$ and an irreducible Euclidean local system ${\underline V}$ on $\Sigma _{\nu }$ such that:

1. (1) the monodromy representation of ${\underline V}$ factors through a finite quotient of $\pi _1(\Sigma _{\nu },x_0)$ ,

2. (2) ${\underline V}$ has nontrivial monodromy around every point of $Z_{\nu }$ , and

3. (3) the twist

   
   has nontrivial kernel.

Proof. Let $(\tilde \Sigma ,\tilde j)$ be a closed Riemann surface and $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (\Sigma ,j)$ a nonconstant holomorphic map such that is not rigid, that is, $\ker {\mathfrak d}_{\tilde u,J}^N$ is nontrivial.

Let $\hat \pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma _{\tilde \nu },\tilde j_{\tilde \nu }) \to (\Sigma _{\nu },j_{\nu })$ be the corresponding holomorphic covering map between orbifold Riemann surfaces constructed in Proposition 2.7.3. Set and . By Proposition 2.8.3, Proposition 2.6.3 and Proposition 1.2.9,

$$ \begin{align*} \ker \tilde{\mathfrak d} \cong \ker \tilde{\mathfrak d}_{\tilde \nu} \cong \ker \hat\pi^*{\mathfrak d}_{\nu} \cong \ker {\mathfrak d}_{\nu}^{\hat\pi_*{\underline{\mathbf{R}}}}. \end{align*} $$

Therefore, $\ker {\mathfrak d}_{\nu }^{\hat \pi _*{\underline {\mathbf {R}}}}$ is nontrivial.

Since $\hat \pi _*{\underline {\mathbf {R}}}$ decomposes into irreducible local systems, there is an irreducible local system ${\underline V}$ such that $\ker {\mathfrak d}_{\nu }^{\underline V}$ is nontrivial. Define the multiplicity function $\nu '\mskip 0.5mu\colon \thinspace \Sigma \to {\mathbf {N}}$ by

${\underline V}$ descends to an irreducible local system ${\underline V}'$ on $\Sigma _{\nu '}$ with nontrivial monodromy around every $x \in Z_{\nu '}$ . By Proposition 2.8.3, $\ker {\mathfrak d}_{\nu '}^{{\underline V}'} \cong \ker {\mathfrak d}_{\nu }^{\underline V}$ .

The following index formula is the final preparation required for the proof of Theorem 2.1.23. Its proof is presented in Section 2.B.

Proposition 2.8.6. Let $(\Sigma ,j)$ be a closed Riemann surface with a multiplicity function $\nu $ , let E be a complex vector bundle over $\Sigma $ and let ${\mathfrak d}\mskip 0.5mu\colon \thinspace \Gamma (E) \to \Omega ^{0,1}(\Sigma ,E)$ a real Cauchy–Riemann operator on E. If ${\underline V}$ is a Euclidean local system on $\Sigma _{\nu }$ , then

$$ \begin{align*} \operatorname{\mathrm{index}} {\mathfrak d}_{\nu}^{\underline V} = \dim V \operatorname{\mathrm{index}} {\mathfrak d} - \operatorname{\mathrm{rk}}_{\mathbf{C}} E \sum_{x \in Z_{\nu}}\dim(V / V^{\rho_x}). \end{align*} $$

Here, $\rho _x$ denotes the monodromy of ${\underline V}$ around x, and $V^{\rho _x} \subset V$ is the subspace of $\rho _x$ -invariant vectors.

2.9 The loci of failure of superrigidity

Denote by $\mathscr {W}_{\blacklozenge }(M,\omega )$ the set of those $J \in \mathscr {J}(M,\omega )$ for which there exists an index zero J-holomorphic embedding $u \colon (\Sigma ,j) \to (M,J)$ which is not superrigid. To prove Theorem 2.1.23, it remains to prove that $\mathscr {W}_{\blacklozenge }(M,\omega )$ has codimension at least one in the sense of Definition 2.2.6. The proof makes use of the moduli spaces introduced in the following two definitions.

Definition 2.9.1. Let $k \in \mathbf {Z}$ and $s \in {\mathbf {N}}_0$ . Denote by $\mathscr {M}\mathscr {O}_{k,s}(M,\omega )$ the space of pairs $(J,[u,j;\nu ])$ consisting of an almost complex structure $J \in \mathscr {J}(M,\omega )$ , and an equivalence class $[u,j;\nu ]$ of

1. (1) a simple J-holomorphic map $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ of index k, and

2. (2) a multiplicity function $\nu \mskip 0.5mu\colon \thinspace \Sigma \to {\mathbf {N}}$ with $\#Z_{\nu } = s$

up to reparametrization by $\operatorname {\mathrm {Diff}}(\Sigma )$ . The set of multiplicity functions $\nu $ is an infinite cover of the regular subset set of the symmetric product $\operatorname {\mathrm {Sym}}^s \Sigma $ . $\mathscr {M}\mathscr {O}_{k,s}(M,\omega )$ is equipped with the topology induced by $\operatorname {\mathrm {Sym}}^s\Sigma $ and $\mathscr {M}(M,\omega )$ .

By Proposition 2.8.5, the failure of superrigidity is detected by the following subsets.

Definition 2.9.2. Let $s \in {\mathbf {N}}_0$ . Denote by $\mathscr {W}_{\blacklozenge ,s}(M,\omega )$ the subset of those $(J,[u,j;\nu ]) \in \mathscr {M}\mathscr {O}_{0,s}(M,\omega )$ for which there exists an irreducible Euclidean local system ${\underline V}$ on $\Sigma _{\nu }$ such that:

1. (1) the monodromy representation of ${\underline V}$ factors through a finite quotient of $\pi _1(\Sigma ,x_0)$ ,

2. (2) ${\underline V}$ has nontrivial monodromy around every $x \in Z_{\nu }$ , and

3. (3) $\ker {\mathfrak d}_{u,J;\nu }^{N,{\underline V}} \neq 0$ .

Denote by $\mathscr {W}_{\blacklozenge ,s}^{\text {top}}(M,\omega )$ the subset of those $(J,[u,j;\nu ])\in \mathscr {M}\mathscr {O}_{0,s}(M,\omega )$ for which there exists ${\underline V}$ with all of the above properties and satisfying additionally:

1. (4) $\dim \ker {\mathfrak d}_{u,J;\nu }^{N,{\underline V}} = 1$ ,

2. (5) if ${\underline V}'$ is any irreducible Euclidean local system on $\Sigma _{\nu }$ not isomorphic to ${\underline V}$ and whose monodromy representation factors through a finite quotient of $\pi _1(\Sigma ,x_0)$ , then

   $$ \begin{align*} \dim \ker {\mathfrak d}_{u,J;\nu}^{N,{\underline V}'} = 0. \end{align*} $$

3. (6) if $n = 3$ , then $\dim (V/V^{\rho _x}) = 1$ for every $x\in Z_{\nu }$ ; otherwise, $s = 0$ .

To prove that $\mathscr {W}_{\blacklozenge }(M,\omega )$ has codimension at least one, we will verify that condition (3.c) from Definition 2.2.6 holds; the other conditions are verified in the same way as in the proof of Theorem 2.4.2. To that end, consider, as in the proof of Proposition 2.6.1, a point $(J_0, [u_0,j_0]) \in \mathscr {M}_0(M,\omega )$ with $u_0$ being an embedding and a sequence $\varepsilon \in {\mathfrak s}$ converging to zero. Let $\mathscr {V}$ be an open neighborhood of $(J_0, [u_0,j_0])$ in the universal moduli space of simple maps over $\mathscr {U} = \mathscr {U}(M,\omega ;J_0,\varepsilon ) \subset \mathscr {J}(M,\omega )$ .

Notation 2.9.3. Given a subset $\mathscr {V} \subset \mathscr {M}_k(M,\omega )$ , denote by $\mathscr {M}\mathscr {O}_{k,s}(\mathscr {V})$ , $\mathscr {W}_{\blacklozenge ,s}(\mathscr {V})$ and $\mathscr {W}_{\blacklozenge ,s}^{\text {top}}(\mathscr {V})$ the subsets of the corresponding spaces consisting of $(J,[u,j;\nu ])$ such that $(J,[u,j]) \in \mathscr {V}$ .

In the situation at hand, we equip these spaces with the topology induced from the Banach space topology on $\mathscr {U}$ . We can choose $\varepsilon $ , $\mathscr {U}$ and $\mathscr {V}$ so that the following hold:

• • $\mathscr {V}$ maps onto $\mathscr {U}$ under the projection $\mathscr {M}_0(M,\omega ) \to \mathscr {J}(M,\omega )$ .

• • For every $(J,[u,j]) \in \mathscr {V}$ , the map u is an embedding.

• • $\mathscr {V}$ is liftable in the sense that there is an $\operatorname {\mathrm {Aut}}(\Sigma ,j_0)$ -invariant slice $\mathscr {S}$ of Teichmüller space through $j_0$ such that for every $(J,[u,j]) \in \mathscr {V}$ there is a unique lift $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ with $j \in \mathscr {S}$ .

• • $\mathscr {M}\mathscr {O}_{0,s}(\mathscr {V})$ is a Banach manifold and the map $\Pi _{0,s}\mskip 0.5mu\colon \thinspace \mathscr {M}\mathscr {O}_{0,s}(\mathscr {V}) \to \mathscr {U}$ is a Fredholm map of index $2s$ . (The existence of $\varepsilon $ and $\mathscr {V}$ satisfying this condition is proved in the same way as Theorem 2.4.2.)

Proposition 2.9.4. For every $s \in {\mathbf {N}}_0$ , there is a closed subset $\mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V}) \subset \mathscr {M}\mathscr {O}_{0,s}(\mathscr {V})$ of infinite codimension such that the following hold:

1. (1) $\mathscr {W}_{\blacklozenge ,s}^{\text {top}}(\mathscr {V}){\backslash } \mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V})$ is contained in a submanifold of codimension $2s + 1$ .

2. (2) $\mathscr {W}_{\blacklozenge ,s}(\mathscr {V}){\backslash }(\mathscr {W}_{\blacklozenge ,s}^{\text {top}}(\mathscr {V})\cup \mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V}))$ has codimension at least $2s + 2$ .

Proof. As in the proof of Proposition 2.6.1, the operators ${\mathfrak d}_{u,J;\nu }^N\mskip 0.5mu\colon \thinspace \Gamma ((Nu)_{\nu }) \to \Omega ^{0,1}(\Sigma _{\nu },(Nu)_{\nu })$ for $(J,[u,j,\nu ]) \in \mathscr {V}$ can be regarded as a family of linear elliptic operators

$$ \begin{align*} {\mathfrak d} \mskip0.5mu\colon\thinspace \mathscr{V} \to \mathscr{F}(W^{1,2}\Gamma(E),L^2\Gamma(F)) \end{align*} $$

as in Definition 1.1.1.

Let ${\underline V}_1,{\underline V}_2$ be a pair of nonisomorphic irreducible Euclidean local system whose monodromy representation factors through a finite quotient of $\pi _1(\Sigma ,x_0)$ , and consider ${\mathfrak V}$ as in Situation 1.3.1 consisting of ${\underline V}_1$ and ${\underline V}_2$ . Denote by $\mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^{\Lambda }(\mathscr {V})$ the subset of those

for which the map $\Lambda _p^{\mathfrak V}$ defined in Theorem 1.3.5 fails to be surjective. The argument from the proof of Proposition 2.6.1 shows that $\mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^{\Lambda }(\mathscr {V})$ is a closed subset of infinite codimension. The union of these subsets is $\mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V})$ . For $d \in {\mathbf {N}}_0^2$ set

By Theorem 1.3.5, $\mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^d(\mathscr {V})$ is a submanifold of codimension

By Proposition 2.8.6,

If ${\underline V}_{\alpha }$ has nontrivial monodromy around every $x \in Z_{\nu }$ , then $i_{\alpha } \leqslant -(n-1)r$ . Therefore, if $d_{\alpha } \geqslant 1$ , then

$$ \begin{align*} \operatorname{\mathrm{codim}} \mathscr{W}_{\blacklozenge,s;{\mathfrak V}}^d(\mathscr{V}) \geqslant (n-1)s+1 \geqslant 2s+1. \end{align*} $$

$\mathscr {W}_{\blacklozenge ,s}(\mathscr {V}){\backslash }\mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V})$ is the union of countably many subsets of the form $\mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^d(\mathscr {V})$ with at least one $\alpha = 1,2$ as above. Therefore, it has codimension at least $2s+1$ .

Analysing the chain of inequalities shows that $\operatorname {\mathrm {codim}} \mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^d (\mathscr {V})= 2s+1$ if and only if there is an $\alpha = 1,2$ such that:

1. (1) $d_{\alpha } = 1$ , ${\mathbf {K}}_{\alpha } = \mathbf {R}$ and $d_{\beta } = 0$ for $\beta \neq \alpha $ , and

2. (2) if $n=3$ , then $\dim (V_{\alpha }/V_{\alpha }^{\rho _x}) = 1$ for every $x \in Z_{\nu }$ ; otherwise, $Z_{\nu } = \varnothing $ .

The union of these subsets is $\mathscr {W}_{\blacklozenge ,s}^{\text {top}}(\mathscr {V}){\backslash }\mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V})$ ; hence, (1) holds. Furthermore, only $\mathscr {W}_{\blacklozenge ,s;{\mathfrak V}}^d(\mathscr {V})$ of codimension at least $2s+2$ are required to cover $\mathscr {W}_{\blacklozenge ,s}(\mathscr {V}) {\backslash } (\mathscr {W}_{\blacklozenge ,s}^{\text {top}} (\mathscr {V})\cup \mathscr {W}_{\blacklozenge ,s}^{\Lambda }(\mathscr {V}))$ . This implies (2).

Proof of Theorem 2.1.23.

The subset of $J \in \mathscr {J}(M,\omega )$ which fail to be superrigid is

$$ \begin{align*} \mathscr{W}_{\geqslant 0}(M,\omega) \cup \mathscr{W}_{\hookrightarrow}(M,\omega) \cup \mathscr{W}_{\blacklozenge}(M,\omega). \end{align*} $$

The first two subsets have already been shown to have codimension at least two. To show that $ \mathscr {W}_{\blacklozenge }(M,\omega )$ has codimension at least one, observe that, the intersection of $\mathscr {W}_{\blacklozenge }(M,\omega )$ with $\mathscr {U} = \mathscr {U}(M,\omega ;J_0,\varepsilon )$ is contained in

$$ \begin{align*} \bigcup_{s \in {\mathbf{N}}_0} \Pi_{0,s}(\mathscr{W}_{\blacklozenge,s}(\mathscr{V})). \end{align*} $$

This verifies condition (3.c) from Definition 2.2.6 since $\Pi _{s,0}$ has index $2s$ and, by Proposition 2.9.4, $\mathscr {W}_{\blacklozenge ,s}(\mathscr {V})$ has codimension at least $2s+1$ in $\mathscr {U}$ .

2.10 Superrigidity along paths of almost complex structures

The following describes in detail how superrigidity may fail along a generic path of almost complex structures. In what follows, it is convenient to use the fibered product notation. Given continuous maps of topological spaces $f \mskip 0.5mu\colon \thinspace X \to Z$ and $g \mskip 0.5mu\colon \thinspace Y \to Z$ , the fibered product of X and Y along Z is

$$ \begin{align*} X \times_Z Y = {\lbrace}{ (x,y) \in X\times Y : f(x) = g(y)}{\rbrace}. \end{align*} $$

Definition 2.10.1. Let $\boldsymbol {\mathscr {J}}(M,\omega )$ be the space of paths ${\mathbf {J}} \mskip 0.5mu\colon \thinspace [0,1]\to \mathscr {J}(M,\omega )$ which are smooth as a section over $[0,1]\times M$ . Equip $\boldsymbol {\mathscr {J}}(M,\omega )$ with the $C^{\infty }$ topology.

For ${\mathbf {J}} \in \boldsymbol {\mathscr {J}}(M,\omega )$ set . Denote by $\boldsymbol {\mathscr {J}}\!_{\blacklozenge }(M,\omega )$ the subset of all ${\mathbf {J}} \in \boldsymbol {\mathscr {J}}(M,\omega )$ for which the following conditions hold:

1. (1) The $1$ -parameter moduli space of pseudo-holomorphic maps

   
   is a $1$ -dimensional manifold with boundary. Here, the fibered product is taken with respect to the path ${\mathbf {J}} \mskip 0.5mu\colon \thinspace [0,1] \to \mathscr {J}(M,\omega )$ and the projection $\mathscr {M}_0(M,\omega ) \to \mathscr {J}(M,\omega )$ .

2. (2) For every $t \in [0,1]$ , the following hold:

   1. (a) Every simple $J_t$ -holomorphic map has nonnegative index.

   2. (b) Every simple $J_t$ -holomorphic map of index zero is an embedding, and every two simple $J_t$ -holomorphic maps of index zero either have disjoint images or are related by a reparametrization.

3. (3) The set $I_{\blacklozenge }$ of those $t \in [0,1]$ for which $J_t$ fails to be superrigid is countable; moreover, $t \in I_{\blacklozenge }$ if and only if

   $$ \begin{align*} J_t \in \bigcup_{s \in {\mathbf{N}}_0} \Pi_{0,s}(\mathscr{W}_{\blacklozenge,s}^{\mathrm{top}}(M,\omega)).\\[-47pt] \end{align*} $$

Theorem 2.10.2 (cf. [Wen19b, Section 2.4]).

$\boldsymbol {\mathscr {J}}\!_{\blacklozenge }(M,\omega )$ is a comeager subset of $\boldsymbol {\mathscr {J}}(M,\omega )$ .

Proof of Theorem 2.10.2.

The set of ${\mathbf {J}}$ satisfying conditions (1) and (2) from Definition 2.10.1 is comeager by the same arguments as in the proofs of Theorem 2.4.2 and Theorem 2.4.3; this is standard.

By Proposition 2.2.7, to prove that condition (3) from Definition 2.10.1 is satisfied for ${\mathbf {J}}$ from a comeager set it suffices to consider the following local situation. Fix ${\mathbf {J}}_0$ , $\varepsilon \in {\mathfrak s}$ , and $t_0 \in [0,1]$ , and let $\mathscr {U}(M,\omega ;{\mathbf {J}}_0;\varepsilon )$ be the $C^{\varepsilon }$ neighborhood of ${\mathbf {J}}_0$ in $\boldsymbol {\mathscr {J}}(M,\omega )$ ; here, we think of elements of $\boldsymbol {\mathscr {J}}(M,\omega )$ as sections over $[0,1]\times M$ and define the $C^{\varepsilon }$ topology in the same way as in Definition 2.2.3 and Definition 2.2.5, with M replaced by $[0,1]\times M$ . Let $\mathscr {V}$ be an open subset of the universal moduli space of simple index zero maps over $\mathscr {U}(M,\omega ,{\mathbf {J}}_0(t_0),\varepsilon )$ , with the properties listed in Section 2.9. For a sufficiently small open neighborhood $I \subset [0,1]$ of $t_0$ , there is a well-defined evaluation map

which is a submersion of Banach manifolds. By Proposition 2.9.4, the preimage of

$$ \begin{align*} \bigcup_{s \in {\mathbf{N}}_0} \Pi_{0,s}(\mathscr{W}_{\blacklozenge,s}(\mathscr{V}){\backslash}(\mathscr{W}_{\blacklozenge,s}^{\text{top}}(\mathscr{V})\cup\mathscr{W}_{\blacklozenge,s}^{\Lambda}(\mathscr{V}))) \end{align*} $$

under the evaluation map has codimension at least two in $I \times \mathscr {U}(M,\omega ;{\mathbf {J}}_0;\varepsilon )$ . Therefore, its image in $\mathscr {U}(M,\omega ;{\mathbf {J}}_0;\varepsilon )$ has codimension at least one; that is, a generic path in $\mathscr {U}(M,\omega ;{\mathbf {J}}_0;\varepsilon )$ either avoids the subsets $ \Pi _{0,s}(\mathscr {W}_{\blacklozenge ,s}(\mathscr {V}))$ or intersects them at one of the subsets $ \Pi _{0,s}(\mathscr {W}_{\blacklozenge ,s}^{\mathrm {top}}(\mathscr {V}))$ . The set of points $t \in I$ at which the latter happens is codimension one in I, and therefore countable.

2.A The normal Cauchy–Riemann operator

The normal Cauchy–Riemann operator for embedded J-holomorphic maps can be traced back to the work of Gromov [Gro85, 2.1.B]. It was observed by Ivashkovich and Shevchishin [IS99, Section 1.3] that the normal Cauchy–Riemann operator can be defined even for nonembedded J-holomorphic maps, and that it plays an important role in understanding the deformation theory of J-holomorphic curves; see also [Wen10, Section 3]. In this section, we will briefly explain the construction of $Tu$ and $Nu$ , and discuss the proof of Proposition 2.1.17.

Definition 2.A.1. Let $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. Denote by ${\mathfrak d}_{u,J}$ the linearization of the J-holomorphic map equation introduced in equation (2.1.5). Denote by $\bar {\partial }_{u,J}$ the complex linear part of ${\mathfrak d}_{u,J}$ . This is a complex Cauchy–Riemann operator and gives $u^*TM$ the structure of a holomorphic vector bundle

Denote by $\mathscr {T}\Sigma $ the tangent bundle of $\Sigma $ equipped with its natural holomorphic structure. The derivative of u induces a holomorphic map $\mathrm {d} u \mskip 0.5mu\colon \thinspace \mathscr {T}\Sigma \to \mathscr {E}$ . The quotient of this map, thought of as a morphism of sheaves,

is a coherent sheaf on $\Sigma $ . Denote by $\operatorname {\mathrm {Tor}}(\mathscr {Q})$ the torsion subsheaf of $\mathscr {Q}$ . The quotient

is torsion-free, hence, locally free. The corresponding holomorphic vector bundle $(Nu,\bar {\partial }_{Nu})$ is called the generalized normal bundle of u. The kernel

also is locally free. The corresponding holomorphic vector bundle $(Tu,\bar {\partial }_{Tu})$ is called the generalized tangent bundle of u.

Proposition 2.A.2. Denote by D the divisor of critical points of $\mathrm {d} u$ counted with multiplicity. There is a short exact sequence

$$ \begin{align*} 0 \to \mathscr{T}\Sigma \to \mathscr{T}\!u \to \mathscr{O}_D \to 0; \end{align*} $$

in particular:

$$ \begin{align*} \mathscr{T}\!u \cong \mathscr{T}\Sigma(D). \end{align*} $$

Proof. The following commutative diagram summarizes the construction of $\mathscr {T}\!u$ and $\mathscr {N}\!u$ :

Since the columns and rows are exact sequences, it follows from the snake lemma that

$$ \begin{align*} \operatorname{\mathrm{Tor}}(\mathscr{Q}) \cong \mathscr{T}\!u/\mathscr{T}\Sigma. \end{align*} $$

Thus, it remains to prove that $\operatorname {\mathrm {Tor}}(\mathscr {Q}) \cong \mathscr {O}_D$ . This is a consequence of the fact that near a critical point $z_0$ of order k we can write $\mathrm {d} u$ as $(z-z_0)^kf(z)$ with $f(z_0) \neq 0$ .

Proposition 2.A.3. Let $u\mskip 0.5mu\colon \thinspace (\Sigma ,j) \to (M,J)$ be a nonconstant J-holomorphic map. If $\pi \mskip 0.5mu\colon \thinspace (\tilde \Sigma ,\tilde j) \to (\Sigma ,j)$ is a nonconstant holomorphic map and

, then

$$ \begin{align*} \mathscr{T}\!\tilde u \cong \pi^*\mathscr{T}\!u \quad\text{and}\quad \mathscr{N}\!\tilde u \cong \pi^*\mathscr{N}\!u. \end{align*} $$

The corresponding isomorphism of vector bundles $N\tilde u \cong \pi ^*Nu$ induces a commutative diagram

Proof. $\mathscr {T}\!u \subset \mathscr {E}$ is the minimal locally free subsheaf which contains the image of $\mathscr {T}\Sigma \hookrightarrow \mathscr {E}$ . Set . There is a canonical isomorphism $\tilde {\mathscr {E}} \cong \pi ^*\mathscr {E}$ . Through this identification, $\pi ^*\mathscr {T}\!u$ can be regarded as a subsheaf of $\tilde {\mathscr {E}}$ . It is locally free and contains the image of $\mathscr {T}\tilde \Sigma \hookrightarrow \tilde {\mathscr {E}}$ . Therefore, $\mathscr {T}\!\tilde u \cong \pi ^*\mathscr {T}\!u$ . This also implies that $\mathscr {N}\!\tilde u \cong \tilde {\mathscr {E}}/\mathscr {T}\!\tilde u \cong \pi ^*(\mathscr {E}/\mathscr {T}\!u) \cong \pi ^*\mathscr {N}\!u$ .

That the isomorphism $N\tilde u \cong \pi ^*Nu$ identifies $\pi ^*{\mathfrak d}_{u,J}^N$ and ${\mathfrak d}_{\tilde u, J}^N$ is evident away from the set of critical points of $\pi $ . Since the latter is nowhere dense, the operators are identified everywhere.

Proof of Proposition 2.1.17.

Let $\mathscr {S}$ be an $\operatorname {\mathrm {Aut}}(\Sigma ,j)$ -invariant local slice of the Teichmüller space $\mathscr {T}(\Sigma )$ through j. Recall that $\mathrm {d}_{u,j}\bar {\partial }_J \mskip 0.5mu\colon \thinspace \Gamma (u^*TM)\oplus T_j\mathscr {S} \to \Omega ^{0,1}(\Sigma ,u^*TM)$ is the linearization of $\bar {\partial }_J$ , defined in equation (2.1.2), restricted to $C^{\infty }(\Sigma ,M)\times \mathscr {S}$ . Denote by $Tu$ the complex vector bundle underlying $\mathscr {T}\!u$ and by $Nu$ the complex vector bundle underlying $\mathscr {N}\!u$ . As was mentioned before Definition 2.1.16, $Tu \subset u^*TM$ is the unique complex subbundle of rank one containing $\mathrm {d} u(T\Sigma )$ . Using a Hermitian metric on $u^*TM$ , we obtain an isomorphism

$$ \begin{align*} u^*TM \cong Tu \oplus Nu. \end{align*} $$

With respect to this splitting ${\mathfrak d}_{J,u}$ , the restriction of $\mathrm {d}_{u,j}\bar {\partial }_J$ to $\Gamma (u^*TM)$ , can be written as

$$ \begin{align*} {\mathfrak d}_{J,u} = \begin{pmatrix} {\mathfrak d}_{u,J}^T & * \\ \dagger & {\mathfrak d}_{u,J}^N \end{pmatrix} \end{align*} $$

with ${\mathfrak d}_{u,J}^N$ denoting the normal Cauchy–Riemann operator introduced in Definition 2.1.16. Since

$$ \begin{align*} \bar{\partial}_{u,J}\circ \mathrm{d} u = \mathrm{d} u \circ \bar{\partial}_{T\Sigma} \quad\text{and}\quad \mathscr{T}\!u \cong \mathscr{T}\Sigma(D), \end{align*} $$

it follows that

$$ \begin{align*} {\mathfrak d}_{u,J}^T = \bar{\partial}_{Tu} \quad\text{and}\quad \dagger = 0. \end{align*} $$

Denote by $\iota \mskip 0.5mu\colon \thinspace T_j\mathscr {S} \to \Omega ^{0,1}(\Sigma ,u^*TM)$ the restriction of $\mathrm {d}_{u,j}\bar {\partial }_J$ to $T_j\mathscr {S}$ . The tangent space to the Teichmüller space $\mathscr {T}(\Sigma )$ at $[j]$ can be identified with $\operatorname {\mathrm {coker}} \bar {\partial }_{T\Sigma } \cong \ker \bar {\partial }_{T\Sigma }^*$ . With respect to this identification, $\iota $ is the restriction of $\mathrm {d} u\mskip 0.5mu\colon \thinspace T\Sigma \to u^*TM$ to $\ker \bar {\partial }_{T\Sigma }^*$ . Consequently, we can write $\mathrm {d}_{u,j}\bar {\partial }_J \mskip 0.5mu\colon \thinspace \Gamma (Tu)\oplus T_j\mathscr {S} \oplus \Gamma (Nu) \to \Gamma (Tu)\oplus \Gamma (Nu)$ as

$$ \begin{align*} \mathrm{d}_{u,j}\bar{\partial}_J = \begin{pmatrix} \bar{\partial}_{Tu} & \iota & *\\ 0 & 0 & {\mathfrak d}_{u,J}^N \end{pmatrix}. \end{align*} $$

The short exact sequence

$$ \begin{align*} 0 \to \mathscr{T}\Sigma \to \mathscr{T}\!u \to \mathscr{O}_D \to 0 \end{align*} $$

induces the following long exact sequence in cohomology

$$ \begin{align*} 0 \to H^0(\mathscr{T}\Sigma) \to H^0(\mathscr{T}\!u) \to H^0(\mathscr{O}_D) \to H^1(\mathscr{T}\Sigma) \to H^1(\mathscr{T}\!u) \to 0. \end{align*} $$

It follows that

$$ \begin{align*} \operatorname{\mathrm{index}} \bar{\partial}_{Tu} = 2\chi(\mathscr{T}\!u) = 2\chi(\mathscr{T}\Sigma) + 2h^0(\mathscr{O}_D) = \operatorname{\mathrm{index}} \bar{\partial}_{T\Sigma} + 2Z(\mathrm{d} u), \end{align*} $$

and, moreover, that $\ker \bar {\partial }_{T\Sigma } \to \ker \bar {\partial }_{Tu}$ is injective, and $\operatorname {\mathrm {coker}} \bar {\partial }_{T\Sigma } \to \operatorname {\mathrm {coker}} \bar {\partial }_{Tu}$ is surjective. The latter implies that $\bar {\partial }_{Tu} \oplus \iota $ is surjective. Therefore, there is an exact sequence

$$ \begin{align*} 0 \to \ker \bar{\partial}_{Tu} \oplus \iota \to \ker \mathrm{d}_{u,j}\bar{\partial}_J \to \ker {\mathfrak d}_{u,J}^N \to 0, \end{align*} $$

and an isomorphism

$$ \begin{align*} \operatorname{\mathrm{coker}} \mathrm{d}_{u,j}\bar{\partial}_J \cong \operatorname{\mathrm{coker}} {\mathfrak d}_{u,J}^N. \end{align*} $$

The kernel of $\bar {\partial }_{Tu} \oplus \iota $ contains $\operatorname {\mathrm {\mathfrak {aut}}}(\Sigma ,j) = \ker \bar {\partial }_{T\Sigma }$ and

$$ \begin{align*} \dim\ker \bar{\partial}_{Tu}\oplus\iota &= \operatorname{\mathrm{index}} \bar{\partial}_{Tu}\oplus\iota \\ &= \operatorname{\mathrm{index}} \bar{\partial}_{Tu} + \dim T_j\mathscr{S} \\ &= \operatorname{\mathrm{index}} \bar{\partial}_{T\Sigma} + \dim T_j\mathscr{S} + 2Z(\mathrm{d} u) \\ &= \dim \operatorname{\mathrm{\mathfrak{aut}}}(\Sigma,j) + 2Z(\mathrm{d} u). \end{align*} $$

This completes the proof of Proposition 2.1.17.

2.B Orbifold Riemann–Roch formula

The purpose of this section is to prove Proposition 2.8.6. The proof relies on Kawasaki’s orbifold Riemann–Roch theorem [Kaw79] and a result due to Ohtsuki [Oht82]. The Riemann–Roch theorem for complex orbifolds is not easy to digest; however, for orbifold Riemann surfaces it simplifies significantly and can be proved by an elementary argument based on the discussion in [FS92, Section 1; NS95, Section 1B; KM95, Sections 8(ii)–(iii)].

This argument relies on the following local considerations. Let $\rho \mskip 0.5mu\colon \thinspace \mu _k \to \operatorname {\mathrm {GL}}(V)$ be a representation, and let $\mu _k$ act on ${\mathbf {D}}\times V$ via

.

is a vector bundle over $[{\mathbf {D}}/\mu _k]$ . In fact, up to isomorphism, every vector bundle over $[{\mathbf {D}}/\mu _k]$ is of this form. $V_{\rho }$ and $V_{\sigma }$ are isomorphic if and only if the representations $\rho $ and $\sigma $ are. However, if $\rho $ is a complex representation, then the restriction $V_{\rho }$ to

is trivial. This is a consequence of the fact that $\operatorname {\mathrm {GL}}_r({\mathbf {C}})$ is connected; more concretely, it can be seen as follows. Choose an isomorphism $V\cong {\mathbf {C}}^r$ with respect to which $\rho $ is diagonal; that is,

(2.B.1) $$ \begin{align} \rho(\zeta) = \begin{pmatrix} \zeta^{w_1} & \\ & \ddots & \\ & & \zeta^{w_r} \end{pmatrix} \end{align} $$

for $w \in (\mathbf {Z}/k\mathbf {Z})^r$ . A choice of lift of w to $\tilde w \in \mathbf {Z}^r$ extends $\rho $ to a representation $\hat \rho \mskip 0.5mu\colon \thinspace {\mathbf {C}}^* \to \operatorname {\mathrm {GL}}(V)$ . The (inverse of the) map $\eta \mskip 0.5mu\colon \thinspace V_{\rho }|_{\dot {\mathbf {D}}} \to \dot {\mathbf {D}}\times V$ defined by

(2.B.2)

trivializes $V_{\rho }$ over $\dot {\mathbf {D}}$ . The trivial bundle over $[{\mathbf {D}}/\mu _k]$ with fiber V and the bundle $V_{\rho } \to [{\mathbf {D}} / \mu _k]$ have canonical holomorphic structures. Denote by $\mathscr {V}$ and $\mathscr {V}_{\rho }$ the corresponding sheaves of holomorphic orbifold sections. The map $\eta $ is holomorphic with respect to the canonical holomorphic structures. If $\tilde w$ is chosen in $(-k,0]^r$ , then $\eta $ induces a sheaf morphism $\eta \mskip 0.5mu\colon \thinspace \mathscr {V}_{\rho } \to \mathscr {V}$ . To see this, observe that if s is a germ of a section of $\mathscr {V}_{\rho }$ at $[0]$ , then $\eta (s)$ is bounded and thus defines a germ of a section of $\mathscr {V}$ at $[0]$ . Evidently, $\eta $ is injective. Furthermore, it fits into the exact sequence

(2.B.3) $$ \begin{align} \mathscr{V}_{\rho} \hookrightarrow \mathscr{V} \twoheadrightarrow V/V^{\rho}\otimes \mathscr{O}_0. \end{align} $$

Here, $\mathscr {O}_0$ denotes the structure sheaf of the point $[0]$ . To see this, it suffices to consider the case $r = 1$ . A germ of a section of $\mathscr {V}^{\rho }$ at $[0]$ is nothing but a germ of a holomorphic map $s\mskip 0.5mu\colon \thinspace {\mathbf {D}} \to {\mathbf {C}}$ which is $\mu _k$ -equivariant; that is, $s(\zeta z) = \zeta ^w s(z)$ for every $\zeta \in \mu _k$ . The map $\eta $ is given by

. If $w = 0$ , then $\eta $ is the identity and the final map in equation (2.B.3) is trivial. If $w \neq 0$ , then the final map in equation (2.B.3) is the evaluation map at $0$ . The Taylor expansion of a germ of a $\mu _k$ -invariant holomorphic map $t\mskip 0.5mu\colon \thinspace {\mathbf {D}} \to {\mathbf {C}}$ involves powers of $z^k$ . Therefore, if t vanishes at $0$ , then

is a germ of a $\mu _k$ -equivariant holomorphic map such that $\eta s = t$ . (Here it is crucial that $\tilde w \geqslant -k$ .)

Definition 2.B.4. Let $(\Sigma ,j)$ be a Riemann surface with a multiplicity function $\nu $ and let $\mathscr {E} = (E,\bar {\partial })$ be a holomorphic vector bundle over $\Sigma $ . Define $(\Sigma _{\nu },j_{\nu })$ and $\beta _{\nu } \mskip 0.5mu\colon \thinspace (\Sigma _{\nu },j_{\nu }) \to (\Sigma ,j)$ as in Definition 2.7.1, and set . Let $\rho = ({\rho _x \mskip 0.5mu\colon \thinspace \mu _{\nu (x)} \to \operatorname {\mathrm {GL}}(E_x)}_{x \in Z_{\nu }}$ be a collection of representations. A Hecke modification of $\mathscr {E}_{\nu }$ of type $\rho $ consists of a holomorphic vector bundle $\mathscr {E}_{\nu ,\rho }$ over $\Sigma _{\nu }$ together with a holomorphic map

$$ \begin{align*} \eta\mskip0.5mu\colon\thinspace \mathscr{E}_{\nu,\rho}|_{\Sigma_{\nu}{\backslash} Z_{\nu}} \to \mathscr{E}_{\nu}|_{\Sigma_{\nu}{\backslash} Z_{\nu}} \end{align*} $$

such that for every $x \in Z_{\nu }$ with respect to suitable holomorphic trivializations of $\mathscr {E}_{\nu ,\rho }$ and $\mathscr {E}_{\nu }$ around x the map $\eta $ is of the form (2.B.2) with $\rho = \rho _x$ and $\tilde w \in (-k,0]^r$ .

Remark 2.B.5. It is evident from the preceding discussion that every holomorphic vector bundle on $(\Sigma _{\nu },j_{\nu })$ can be obtained by a Hecke modification.

Theorem 2.B.6 (Orbifold Riemann–Roch formula [Kaw79]).

In the situation of Definition 2.B.4,

$$ \begin{align*} \chi(\mathscr{E}_{\nu,\rho}) = \chi(\mathscr{E}_{\nu}) - \sum_{x\in Z_{\nu}} \dim_{\mathbf{C}} (E_x/E_x^{\rho_x}). \end{align*} $$

Proof. The exact sequence (2.B.3) induces the exact sequence

$$ \begin{align*} \mathscr{E}_{\nu,\rho} \hookrightarrow \mathscr{E}_{\nu} \twoheadrightarrow \bigoplus_{x \in Z_{\nu}} (E_x/E_x^{\rho_x}) \otimes \mathscr{O}_x. \end{align*} $$

This immediately implies the assertion.

The proof of Proposition 2.8.6 requires one more piece of preparation. In the situation of Definition 2.B.4, if $\mathscr {E}_{\nu ,\rho }$ carries a holomorphic flat connection $\nabla _{\nu ,\rho }$ , then it induces a meromorphic flat connection $\nabla _{\nu }$ on $\mathscr {E}$ with simple poles. With respect to suitable local holomorphic coordinates and trivializations around x

Here, $\tilde w_i(x)$ are as in the discussion preceding Definition 2.B.4. By (a very special case of) [Oht82, Theorem 3], the degree of E and the residues $\operatorname {\mathrm {Res}}_x(\nabla _{\nu })$ are related by

(2.B.7) $$ \begin{align} \deg E = -\sum_{x\in Z_{\nu}} \operatorname{\mathrm{tr}} \mathrm{Res}_x(\nabla) = -\sum_{x\in Z_{\nu}} \sum_{i=1}^r \frac{\tilde w_i(x)}{\nu(x)}. \end{align} $$

Proof of Proposition 2.8.6.

Set and . For every $x \in Z_{\nu }$ denote by $\rho _x^{\mathbf {C}} \mskip 0.5mu\colon \thinspace \mu _{\nu (x)} \to \operatorname {\mathrm {GL}}_{\mathbf {C}}(V^{\mathbf {C}})$ the complexification of the monodromy representation of ${\underline V}$ around x. There is a holomorphic vector bundle $\mathscr {V}$ over $\Sigma $ such that ${\underline V}^{\mathbf {C}} \cong \mathscr {V}_{\nu ,\rho ^{\mathbf {C}}}$ . Equip E with the holomorphic structure $\bar {\partial }$ satisfying ${\mathfrak d} = \bar {\partial } + {\mathfrak n}$ with ${\mathfrak n} \in \Omega ^{0,1}(\Sigma ,\overline {\mathrm {End}}_{\mathbf {C}}(E))$ .

By Theorem 2.B.6 and the classical Riemann–Roch formula,

$$ \begin{align*} \operatorname{\mathrm{index}} {\mathfrak d}_{\nu}^{\underline V} &= 2\chi(\mathscr{E}_{\nu} \otimes_{\,{\mathbf{C}}} {\underline V}^{\mathbf{C}}) \\ & = 2\chi(\mathscr{E}_{\nu} \otimes_{\,{\mathbf{C}}} \mathscr{V}) - 2\operatorname{\mathrm{rk}}_{{\mathbf{C}}} E \sum_{z \in Z_{\nu}} \dim(V/V^{\rho_x}) \\ &= \dim V \operatorname{\mathrm{index}} {\mathfrak d} + 2\operatorname{\mathrm{rk}}_{{\mathbf{C}}} E \,\Big({\deg \mathscr{V} - \sum_{z \in Z_{\nu}} \dim(V/V^{\rho_x})}\Big). \end{align*} $$

Therefore, it remains prove that

$$ \begin{align*} \deg \mathscr{V} = \frac12\sum_{z \in Z_{\nu}} \dim(V/V^{\rho_x}). \end{align*} $$

Let $k \in {\mathbf {N}}$ . The complexification of the trivial representation $\rho _0\mskip 0.5mu\colon \thinspace \mu _k \to \operatorname {\mathrm {GL}}(\mathbf {R})$ is the trivial representation $\rho _0^{\mathbf {C}}\mskip 0.5mu\colon \thinspace \mu _k\to \operatorname {\mathrm {GL}}_{\mathbf {C}}({\mathbf {C}})$ . Therefore, the corresponding weight in $(-k,0]$ is $0$ . For $w \in \mathbf {Z}/k\mathbf {Z}$ the complexification of the representation $\rho _w\mskip 0.5mu\colon \thinspace \mu _k \to \operatorname {\mathrm {GL}}({\mathbf {C}})$ defined by $\rho _w(\zeta ) \mapsto \zeta ^w$ is the representation $\rho _w^{\mathbf {C}}\mskip 0.5mu\colon \thinspace \mu _k \to \operatorname {\mathrm {GL}}_{\mathbf {C}}({\mathbf {C}}^2)$ defined by

Therefore, the corresponding weights in $(-k,0]$ are of the form $\tilde w$ and $-(\tilde w+k)$ . It follows from this discussion that for every representation $\mu _k \to \operatorname {\mathrm {GL}}(V)$ the sum of the weights of the complexification is $-\frac {k}{2}\dim (V/V^{\rho })$ . This combined with equation (2.B.7) proves the desired identity for $\deg \mathscr {V}$ .