ON THE GEOMETRY OF THE PAPPAS–RAPOPORT MODELS FOR PEL SHIMURA VARIETIES

Abstract

In this article we study integral models of Shimura varieties, called Pappas–Rapoport splitting model, for ramified P.E.L. Shimira data. We study the special fiber and some stratification of these models, in particular we show that these are smooth and the Rapoport locus and the $\mu $-ordinary locus are dense, under some condition on the ramification.

1 Introduction

Shimura varieties have been at the heart of arithmetic since their introduction by Goro Shimura and later generalization by Pierre Deligne [5]. Nowadays they are a powerful geometric tool for the Langlands program. As they are algebraic varieties over a number field E, their étale cohomology is endowed with an action both of $G_{E} = \operatorname {\mathrm {Gal}}\left (\overline E/E\right )$ and of the adelic points of the underlying reductive group G: understanding the relations between the two actions is the way to realize geometrically (cases of) the association of Galois representations to automorphic representations. This strategy was first realized by Eichler and Shimura and by Deligne for the modular curves, and was later generalized in broader directions, for higher-dimensional Shimura varieties [4, 8, 12, 1, 23], where the previous arithmetic and analytic relations have revealed very complex issues.

One of the ideas to realize this correspondence is the Langlands–Kottwitz method, for which we need to relate the number of points (modulo p) of our Shimura variety (itself related to the étale cohomology of the variety) to some orbital integrals of G, itself related in a somewhat indirect way, but now classical, to automorphic representations. Thus, to make sense of the number of points, we need to find a way to reduce the given Shimura variety modulo p – that is, we need to find a good integral model of it. When the Shimura variety is of PEL type, meaning more or less that it is a moduli space of abelian varieties with some extra structure, the simplest idea is to extend this modular description from E to $\mathcal O_{E}$ , or at least to $\mathcal O_{E_{\mathfrak p}}$ , a p-adic completion of $\mathcal O_{E}$ . In the first case of the modular curve, this has been extensively studied – for example in [6] or [11], in which very satisfying integral models are introduced – for all the interesting levels at p, for example $\Gamma _{0}(p^{n}),\Gamma _{1}(p^{n}),\Gamma (p^{n}) $ -levels. A remark regarding the definition of integral models is that the level away from p is easy to deal with. Also, by satisfying integral models here we mean with as few singularities as possible. For example, when the level at p is maximal, the integral model of the modular curves is smooth, and in general level they are regular. Kottwitz then deeply generalized this in the case of PEL Shimura varieties, provided that the Shimura datum was unramified at p, meaning that both the group is unramified at p (and has a suitable integral model) and the level is hyperspecial at p.

The problem of defining good integral models both with deeper level at p, or for ramified Shimura datum, has since been extensively studied. For a selection we mention the work of Harris and Taylor [8], who study specific Shimura varieties for which the method of Katz and Mazur still applies for deeper level; work of Pappas and Rapoport [19] for cases where the Shimura datum is ramified; and almost any paper of Lan (for example [14, 15, 16]) for generalization in both directions. In this article we study a specific class of PEL integral models with ‘maximal’ level at p (in a specific sense) and for which the group is ramified at p. We take the definition given in [19], also referred to as splitting models in the literature, and study the local and global geometry of these models. Our results depend on the ramification of p on the Shimura datum. Precisely, as explained in §2.5, there is a finite set $\mathcal P$ of ‘primes’ $\pi $ above p. These primes fall into one of the following categories: (C), (AL), (AU), (AR), where the first is the category of primes of symplectic type and the last three are of unitary type, the last one ramified; we exclude all type D factors in our PEL Shimura varieties (see Hypothesis 2.2). The last category, (AR), roughly corresponds to a unitary group over a CM extension $F/F^{+}$ and a prime $\pi $ above p in $F^{+}$ such that $\pi $ ramifies in F. Denote by X the Pappas–Rapoport model at p of a Shimura variety, as in §2. It lives over the ring of integers of K, a finite, well-chosen extension of $\mathbb {Q}_{p}$ . Our first result is the following (see Theorem 2.30):

Theorem 1.1. If no prime in $\mathcal P$ falls in case (AR), then X is smooth over $\operatorname {\mathrm {Spec}}(\mathcal O_{K})$ .

Such a result was clearly expected in [19], and was already proved in the case of the Hilbert modular varieties in [21, 22]. Our proof is very similar, using the definitions of [19] and the local study we make in §2. Also, it is clear that the assumption that no prime falls into case (AR) is necessary, as explained in Appendix A.

The main result of this article is a study of the special fiber $X_{\kappa }$ of X. Recalling that for a (PEL) Shimura variety S associated with data unramified at p, we can look at the Newton stratification of the special fiber of S, which we now know has all the expected properties – in particular, the $\mu $ -ordinary locus is open and dense [7, 26]. In this article we study a similar question in our situation, and we investigate another natural stratification, which we call the Hodge stratification on $X_{\kappa }$ , encoding the position of the Hodge polygon (defined in [3]). Even if we show that this stratification does not behave as well as expected (except in case of very small ramification $e=2$ and even only away from case (AR)), we prove that the open stratum, the generalized Rapoport locus, is dense (again except in case (AR)). This locus coincides with the usual Rapoport locus in the Hilbert case, hence the denomination.

Theorem 1.2. If no prime in $\mathcal P$ falls into case (AR), the generalized Rapoport locus is open and dense.

We actually prove this result by hand by explicitly constructing a deformation of a p-divisible group to the generalized Rapoport locus (see Theorem 3.3). Then we investigate the similar result for the Newton stratification. Because of our earlier results on Pappas–Rapoport data [3], we know that the $\mu $ -ordinary locus, which coincides with the (big) open stratum of the Newton stratification, lies inside the generalized Rapoport locus. Here we prove that it is dense, generalizing work of Wedhorn [26] in the case of a ramified Shimura variety.

Theorem 1.3. If no prime in $\mathcal P$ falls into case (AR), the $\mu $ -ordinary locus in $X_{\kappa }$ is open and dense.

This result actually implies the previous one, but the proof uses the density of the generalized Rapoport locus, together with the methods of deformation of p-divisible groups introduced in [26], and relies on calculations on displays. Here we slightly simplify some arguments of [26], constructing ‘by hand’ deformations when we can. This density result extends the work of [26] (which deals with the unramified case). Note also the work of Wortmann [27] for Hodge-type Shimura varieties with good reduction at p and the work of He and Rapoport [10] and He and Nie [9] to compare the $\mu $ -ordinary locus to EKOR strata and to reformulate this density in terms of Weyl groups.

Finally, we give an equivalent condition, similar to the unramified case, for the existence of an ordinary locus. Namely, we prove that the ordinary locus is nonempty if and only if the prime p is totally split in the reflex field, extending the result in [26] in the unramified case.

2 Shimura datum, Pappas–Rapoport condition, and stratifications

2.1 Shimura datum

Let $(B,\star )$ be a finite-dimensional central semisimple $\mathbb {Q}$ -algebra endowed with a positive involution, with center F, and $(V,\langle .,.\rangle )$ be a nondegenerate skew Hermitian B-module, and let G be the algebraic group over $\mathbb {Q}$ of (similitude) automorphisms of $(V,\langle .,.\rangle )$ – that is, representing the functor

$$ \begin{align*} G(R) = \left\{ (g,c) \in \operatorname{\mathrm{GL}}\left(V\otimes_{\mathbb{Q}} R\right) \times \mathbb G_{m}(R) \mid \langle gz,gz^{\prime}\rangle = c\langle z,z^{\prime}\rangle, \forall z \in V\otimes_{\mathbb{Q}} R\right\} \end{align*} $$

on $\mathbb {Q}$ -algebras R.

Let $h : \mathbb {C} \longrightarrow \operatorname {\mathrm {End}}_{B}(V_{\mathbb {R}})$ be an $\mathbb {R}$ -algebra homomorphism such that $h\left (\overline z\right ) = h(z)^{\star }$ and the bilinear form $(\cdot ,h(i)\cdot )$ on $V_{\mathbb {R}}$ is positive definite. This induces a Shimura datum $(G,h)$ .

2.2 Characteristic $0$ moduli space

Let us denote by E the reflex field of the (Shimura) datum $(G,h)$ ; it is a number field. Fix $K \subset G\left (\mathbb A_{f}\right )$ a neat (for simplicity) compact open subgroup.

Following [14, Definition 1.4.2.1], let $\mathcal S_{K}$ be the moduli problem over $\operatorname {\mathrm {Spec}}(E)$ that associates to S the quasi-isogeny classes of quadruples $(A,\lambda ,\iota ,\eta )$ , where $A \longrightarrow S$ is an abelian scheme, $\lambda $ is a $\mathbb {Q}^{\times }$ -polarization of A, $\iota :B \longrightarrow \operatorname {\mathrm {End}}(A)\otimes (\mathbb {Q})_{S}$ is a morphism compatible with $\star $ and the Rosati involution, and $\eta $ is a rational level structure of type K of A (see [14, Definition 1.4.1.2] for a precise formulation). We moreover require that this quadruple satisfy the determinant condition (see [12, §5] or [14, Definition 1.3.4.1]).

Then $\mathcal S_{K}$ is representable by a scheme over $\operatorname {\mathrm {Spec}}(E)$ . This is, for example, [14, Corollaries 1.4.3.7 and 7.2.3.10].

Remark 2.1. If p is a good prime for $G,K$ , we could give an analogous definition by $\mathbb {Z}_{(p)}$ -isogeny instead of quasi-isogeny (that is, $\mathbb {Q}^{\times }$ -isogeny), as we will do later, but we will need to introduce integral data to give a meaning to good primes (see our definition in §2.4, and [14, §§1.4.2 and 1.4.3].

From now on, we fix a prime p. Let us be more specific about the determinant condition when S is over $\mathbb {Q}_{p}$ . First let us assume that the following hypothesis on p and B is satisfied:

Hypothesis 2.2. We assume that $B_{\mathbb {Q}_{p}}$ is isomorphic to a product of matrix algebras over finite extensions of $\mathbb {Q}_{p}$ , such that factors are either stable by $\star $ or exchanged two by two by $\star $ . Up to isomorphism, the possibilities for each simple involutive factor $B_{i}$ of $B_{\mathbb {Q}_{p}}$ are then (see, for example, [18, Proposition 8.3])

• (D): $B_{i} = M_{n}(L)$ with $\star (A) = J^{t}A J^{-1}$ and J skew-symmetric;

• (C): $B_{i} = M_{n}(L)$ with $\star (A) = M^{t}A M^{-1}$ and M symmetric;

• (AL): $B_{i} = M_{n}(L) \times M_{n}(L)$ and $\star (A,B) = (B,A)$ ;

• (AU) or (AR): $B_{i} = M_{n}(L)$ , $\star (A) = M^{t}\overline A M^{-1}$ , with $\star $ inducing an order $2$ automorphism $\overline {(\cdot )}$ of L.

In the first two cases (resp., the last two cases) the involution is said to be of the first kind (resp., the second kind) – that is, $\star $ induces the identity (resp., an order $2$ automorphism) of L. In the last two cases, if we denote by $L^{+}$ the subfield fixed by $\star $ , then we are in case (AL) when $L = L^{+} \times L^{+}$ , case (AU) when $L/L^{+}$ is unramified, and case (AR) when $L/L^{+}$ is ramified. We assume moreover in this article that each simple involutive factor is of type (C), (AL), (AU), or (AR) – that is, we exclude factors of type (D). In particular we exclude all factors in our Shimura datum of type (D) in the usual sense ([12, §5], [14, Definition 1.2.1.15]).

Example 2.3. If $B = F$ , with $F/F^{+}$ a CM field with totally real field $F^{+}$ and $\star $ the complex conjugation, Hypothesis 2.2 is satisfied, as $B_{\mathbb {Q}_{p}} = \prod _{\pi \mid p} F \otimes _{F^{+}} F^{+}_{\pi }$ , where $\pi $ ranges over places over p in $\mathcal O_{F^{+}}$ and $F^{+}_{\pi }$ is the $\pi $ -adic completion of $F^{+}$ .

By hypothesis, we can decompose $B_{\mathbb {Q}_{p}} = \bigoplus _{i =1}^{r} M_{n_{i}}(F_{i})$ , where $F_{i}/\mathbb {Q}_{p}$ is a finite, possibly ramified extension. We remark that the involution $\star $ on B acts on the set $\{1,\dotsc ,r\}$ ; we denote $s(i)$ the image of i by this involution. Denote by $E_{\nu }$ a p-adic completion of E; thus $E_{\nu }$ is a finite extension of $\mathbb {Q}_{p}$ . If $(A,\lambda ,\iota ,\eta )$ is an object over S in $\mathcal S_{K} \otimes _{E} {E_{\nu }}$ , then $\omega _{A} = \operatorname {\mathrm {Lie}}(A)^{\vee }$ is an $\mathcal O_{S} \otimes _{\mathbb {Q}} B$ -module, but as S is over $\mathbb {Q}_{p}$ , it is an $\mathcal O_{S} \otimes _{\mathbb {Q}_{p}} B_{\mathbb {Q}_{p}}$ -module, and we can thus decompose it as

$$ \begin{align*} \omega_{A} = \bigoplus_{i=1}^{r} \omega_{A,i}, \end{align*} $$

where $\omega _{A,i}$ is an $\mathcal O_{S} \otimes _{\mathbb {Q}_{p}} M_{n_{i}}(F_{i})$ -module. Using Morita equivalence, we decompose $\omega _{A,i} = \mathcal O_{S}^{n_{i}}\otimes _{\mathcal O_{S}} \omega _{i}$ , where $\omega _{i} = e_{i}\omega _{A,i}$ is endowed with an action of $\mathcal O_{F_{i}}$ and $e_{i}$ is the Morita projector associated to the matrix $E_{1,1}$ seen as an element of $M_{n_{i}}(F_{i})$ . Up to an extension of scalars for S, we can further decompose $\omega _{i} = \bigoplus _{\tau \in \mathcal T^{\prime }_{i}}\omega _{i,\tau }$ , as locally free $\mathcal O_{S}$ -modules, where $\mathcal T^{\prime }_{i} = \operatorname {\mathrm {Hom}}\left (F_{i},\mathbb {C}_{p}\right )$ . Then the determinant condition is equivalent to asking the locally free $\left (\omega _{i,\tau ^{\prime }}\right )$ to have fixed dimension $\left (d_{i,\tau ^{\prime }}\right )_{i,\tau ^{\prime }}$ , where the integers $\left (d_{i,\tau ^{\prime }}\right )_{i,\tau ^{\prime }}$ are fixed by h as follows. Denote $V_{\mathbb {C}} = V_{1} \oplus V_{2}$ the decomposition where $h(z)$ acts as z (resp., $\overline z$ ) on $V_{1}$ (resp., $V_{2}$ ). Then the reflex field $E \subset \mathbb {C}$ is the number field where the isomorphism class of the complex B-representation $V_{1}$ is defined. It thus makes sense to consider $V_{\overline {\mathbb {Q}_{p}}} = V_{1,\overline {\mathbb {Q}_{p}}} \oplus V_{2,\overline {\mathbb {Q}_{p}}}$ as a $B_{\overline {\mathbb {Q}_{p}}}$ -representation. Using the hypothesis on B, decompose

$$ \begin{align*} V_{1,\overline{\mathbb{Q}_{p}}} = \prod_{i=1}^{r} V_{1}^{i} \otimes_{F_{i}\otimes_{\mathbb{Q}_{p}}\overline{\mathbb{Q}_{p}}} \left(F_{i}\otimes_{\mathbb{Q}_{p}} \overline{\mathbb{Q}_{p}}\right)^{n_{i}} \end{align*} $$

by Morita, where $V_{1}^{i}$ is an $F_{i}\otimes _{\mathbb {Q}_{p}} \overline {\mathbb {Q}_{p}}$ -module that we can further decompose as

$$ \begin{align*} V_{1}^{i} = \prod_{\tau^{\prime} \in \operatorname{\mathrm{Hom}}\left(F_{i},\overline{\mathbb{Q}_{p}}\right)} \left(V_{1}^{i}\right)_{\tau^{\prime}}. \end{align*} $$

Then $d_{i,\tau ^{\prime }}$ is the dimension of $\left (V_{1}^{i}\right )_{\tau ^{\prime }}$ .

Remark 2.4. As our Shimura datum comes from an object over $\mathbb {Q}$ , we can check that for all i and for all $\tau ^{\prime },\tau ^{\prime \prime }$ ,

$$ \begin{align*} d_{i,\tau^{\prime\prime}} + d_{s\left(i,\tau^{\prime\prime}\right)} = d_{i,\tau^{\prime}} + d_{s\left(i,\tau^{\prime}\right)} = h_{i} \end{align*} $$

is independent of $\tau ^{\prime }$ , where s is the action induced by $\star $ (in the case where two factors $i,j$ are exchanged by $\star $ , recall that we set $j = s(i)$ ). This is, for example, [14, p. 59].

2.3 Pappas–Rapoport data

The goal of this section is to define a Pappas–Rapoport datum in order to define an integral model for the variety $\mathcal S_{K}$ which is analogous to the Kottwitz determinantial condition but better behaved in ramified characteristics. Such a datum, introduced in [19], is referred to there as a splitting datum. We define such a datum in this section, and explain its behavior with duality.

2.3.1 Definition

Let $L/\mathbb {Q}_{p}$ be a finite extension, K be an extension of $\mathbb {Q}_{p}$ containing the Galois closure of L, and S be an $\mathcal O_{K}$ -scheme. Denote $L^{ur}$ the maximal unramified subfield of L and $\mathcal T = \operatorname {\mathrm {Hom}}\left (L^{ur},\mathbb {C}_{p}\right )$ the set of unramified embeddings, and fix $\pi $ a uniformizer of L, with Eisenstein polynomial Q. In particular, sending T to $\pi $ , we can identify

$$ \begin{align*} \mathcal O_{L^{ur}}[T]/(Q(T)) \simeq \mathcal O_{F}. \end{align*} $$

Let us fix an embedding $\tau $ of $L^{ur}$ into K, and define $\Sigma $ as the set of embeddings of L into K extending $\tau $ . It is a set of cardiality e, and we choose an ordering $\Sigma = \{\sigma _{1}, \dotsc , \sigma _{e}\}$ for this set.

Let $\mathcal {N} \to S$ be a locally free sheaf with an action $\mathcal O_{L}$ , such that $\mathcal O_{L^{ur}}$ acts on $\mathcal {N}$ by $\tau $ . We will denote by $[\pi ]$ the action of $\pi $ on $\mathcal {N}$ . Let $(d_{1}, \dotsc , d_{e})$ be a collection of integers. We recall the definition of a Pappas–Rapoport datum:

Definition 2.5. A Pappas–Rapoport datum for $\mathcal {N}$ with respect to the collection $(\sigma _{i}, d_{i})_{i=1,\dotsc ,e}$ consists of a filtration

$$ \begin{align*} 0 = \mathcal{N}^{[0]} \subset \mathcal{N}^{[1]} \subset \dotsb \subset \mathcal{N}^{[e]} = \mathcal{N} \end{align*} $$

such that:

1. 1. The $\mathcal {N}^{[j]}$ are $\mathcal O_{S}$ -locally direct factors stable by $\mathcal O_{L}$ .

2. 2. $\left ([\pi ] - \sigma _{j} (\pi )\right ) \cdot \mathcal {N}^{[j]} \subset \mathcal {N}^{[j-1]}$ , for all $1 \leq j \leq e$ .

3. 3. $\mathcal {N}^{[j]}/\mathcal {N}^{[j-1]}$ is locally free of rank $d_{j}$ for all $1 \leq j \leq e$ .

2.3.2 Duality

Next we want to explain the compatibility with duality for this datum. Assume that there exists a sheaf $\mathcal {E}$ , locally free of rank h as an $\mathcal O_{S} \otimes _{\mathcal O_{F^{ur}}, \tau } \mathcal O_{F}$ -module, such that $\mathcal {N}$ is locally a direct factor of $\mathcal {E}$ . Define $\mathcal {M} := (\mathcal {E} / \mathcal {N})^{\vee }$ ; it is a locally free sheaf over S, and has an action of $\mathcal O_{L}$ (with $\mathcal O_{L^{ur}}$ acting by $\tau $ ). One thus has an exact sequence

$$ \begin{align*} 0 \to \mathcal{N} \to \mathcal{E} \to \mathcal{M}^{\vee} \to 0.\\[-24pt] \end{align*} $$

Let us introduce some more notation. Define $\pi _{i} := \sigma _{i}(\pi )$ for $1 \leq i \leq e$ , and let us introduce for $1 \leq \ell \leq e$ the polynomials

$$ \begin{align*} Q_{\ell} := \prod_{i=1}^{\ell} (T - \pi_{i}) \quad \text{and} \quad Q^{\ell} := \prod_{i=\ell+1}^{e} (T - \pi_{i}). \end{align*} $$

Note that the hypothesis made on $\mathcal E$ means that it is locally free as an $\mathcal O_{S} [T] / Q(T)$ -module (with T acting by $\pi $ ).

Definition 2.6. Let us define a complete filtration on $\mathcal {E}$

$$ \begin{align*} 0 = \mathcal{N}^{[0]} \subset \mathcal{N}^{[1]} \subset \dotsb \subset \mathcal{N}^{[e]} = \mathcal{N} \subset \mathcal{N}^{[e+1]} \subset \dotsb \subset \mathcal{N}^{[2e]} = \mathcal{E} \end{align*} $$

by the formulas

$$ \begin{align*} \mathcal{N}^{[2e-\ell]} = \left( Q^{\ell} (\pi) \right)^{-1} \left(\mathcal{N}^{[\ell]}\right) \end{align*} $$

for every $1 \leq \ell \leq e-1$ .

A full Pappas–Rapoport datum for $(\mathcal E,\mathcal N)$ with respect to $(\sigma _{i},d_{i})_{i=1,\dotsc ,e}$ is a complete filtration of the previous form, where $\left (\mathcal N^{[i]}\right )_{i=1,\dotsc ,e}$ is a Pappas–Rapoport datum for $\mathcal N$ with respect to the same data.

The conditions imposed by the Pappas–Rapoport datum imply that the inclusions $\mathcal {N}^{[e+j]} \subset \mathcal {N}^{[e+j+1]}$ are satisfied for every $0 \leq j \leq e-1$ .

Lemma 2.7. Let $1 \leq j \leq e-1$ be an integer. The sheaf $\mathcal {N}^{[e+j]}$ is locally free of rank

$$ \begin{align*} jh + d_{1} + \dotsb + d_{e-j} = \dim_{\mathcal O_{S}}\mathcal N + h-d_{e} + \dotsb + h-d_{e-j+1}. \end{align*} $$

Moreover, one has

$$ \begin{align*} \left([\pi] - \sigma_{e-j+1}(\pi)\right) \mathcal{N}^{[e+j]} \subset \mathcal{N}^{[e+j-1]}. \end{align*} $$

Proof. This is an easy computation.

One deduces from this lemma that one has a Pappas–Rapoport datum for $\mathcal {M}$ .

Proposition 2.8. The complete filtration on $\mathcal {E}$ induces a Pappas–Rapoport datum for $\mathcal {M}$ with respect to the collection $((\sigma _{1},\dotsc ,\sigma _{e}),(h-d_{1}, \dotsc , h-d_{e}))$ .

Remark 2.9. In special fiber, the situation is much simpler. Indeed, one simply has $Q_{\ell } (\pi ) = \pi ^{\ell }$ , $Q^{\ell } (\pi ) = \pi ^{e-\ell }$ , and

$$ \begin{align*} \mathcal{N}^{[2e-\ell]} = \left( \pi^{e-\ell} \right)^{-1} \left(\mathcal{N}^{[\ell]}\right) \end{align*} $$

for every $1 \leq \ell \leq e-1$ .

2.3.3 Pairing

Assume in this section that the sheaf $\mathcal {E}$ has a perfect alternating pairing $\langle , \rangle : \mathcal {E} \times \mathcal {E} \to \mathcal {O}_{S}$ . Assume also that this pairing is compatible with the action of $\mathcal {O}_{L}$ – that is, that $\langle a \cdot x, y\rangle = \langle x, a \cdot y\rangle $ for $a \in \mathcal {O}_{F}$ and $x,y \in \mathcal {E}$ . This forces the integer h to be even; let g be such that $h = 2g$ . Assume moreover that $\mathcal N$ is maximally isotropic – that is, $\mathcal N = \mathcal N^{\bot }$ , the latter notation referring to the orthogonal of $\mathcal N$ for the pairing considered. This implies that $\mathcal N$ is locally free of rank $eg$ .

Proposition 2.10. Fix a Pappas–Rapoport datum for $\mathcal N$ with respect to the collection $(\sigma _{i},d_{i})_{i=1,\dotsc ,e}$ . There exists a complete filtration of $\mathcal E$ given by

$$ \begin{align*} 0 = \mathcal{N}^{[0]} \subset \mathcal{N}^{[1]} \subset \dotsb \subset \mathcal{N}^{[e]} = \mathcal{N} \subset {\mathcal{N}^{[e-1]}}^{\bot} \subset \dotsb \subset {\mathcal{N}^{[1]}}^{\bot} \subset \mathcal E. \end{align*} $$

This filtration induces a Pappas–Rapoport datum for $\mathcal {M}$ with respect to the collection $((\sigma _{1},\dotsc ,\sigma _{e}),(d_{1}, \dotsc , d_{e}))$ .

Proof. Let us consider the sheaf ${\mathcal {N}^{[e-1]}}^{\bot }$ . It is locally a direct factor of rank $2eg - (d_{1} + \dotsb + d_{e-1})=eg+d_{e}$ , since $d_{1} + \dotsb + d_{e} = eg$ . Set $x \in {\mathcal {N}^{[e-1]}}^{\bot }$ and $y \in \mathcal {N}$ . Then $[\pi ] y - \sigma _{e}(\pi ) y \in \mathcal {N}^{[e-1]}$ , and thus

$$ \begin{align*} 0 = \langle x, [\pi]y - \sigma_{e}(\pi)y\rangle = \langle[\pi] x - \sigma_{e}(\pi) x, y\rangle. \end{align*} $$

One then gets $[\pi ] x - \sigma _{e}(\pi )x \in \mathcal {N}^{\bot } = \mathcal {N}$ . The results for the other sheaves are similar.

One would of course want this filtration to coincide with the previous one. This is possible only if $d_{i}=g$ for all $1 \leq i \leq e$ , which we will assume in the rest of the section.

Definition 2.11. One says that the filtration $\mathcal N^{[\bullet ]}$ is compatible with the pairing if

$$ \begin{align*} \mathcal{N}^{[2e-\ell]} = {\mathcal{N}^{[\ell]}}^{\bot} \end{align*} $$

for all $1 \leq \ell \leq e-1$ .

Let us be a little more explicit about this condition. If R is a polynomial, we denote by $\mathcal {E} [R]$ the kernel of $R(\pi )$ acting on $\mathcal {E}$ . One sees in particular that $\mathcal {N}^{[\ell ]} \subset \mathcal {E} [Q_{\ell }]$ for $1 \leq \ell \leq e$ .

Proposition 2.12. One has, for $1 \leq \ell \leq e$ ,

$$ \begin{align*} \mathcal{E} [Q_{\ell}]^{\bot} = \mathcal{E} \left[Q^{\ell}\right]. \end{align*} $$

Proof. Note that one has $\mathcal {E} [Q_{\ell }] = Q^{\ell } (\pi ) \mathcal {E}$ . The fact that x belongs to $\mathcal {E} [Q_{\ell }]^{\bot }$ is thus equivalent to the fact that $\langle x, Q^{\ell }(\pi ) y\rangle = 0$ for all $y \in \mathcal {E}$ . This is equivalent to the relation $Q^{\ell } (\pi ) x = 0$ .

Since multiplication by $Q^{\ell } (\pi )$ induces an isomorphism $\mathcal {E} / \mathcal {E} \left [Q^{\ell }\right ] \simeq \mathcal {E} [Q_{\ell }]$ , one has an induced perfect pairing

$$ \begin{align*} h_{\ell} : \mathcal{E} [Q_{\ell}] \times \mathcal{E} [Q_{\ell}] \to \mathcal{O}_{S}. \end{align*} $$

Explicitly, since $\mathcal {E} [Q_{\ell }] = Q^{\ell }(\pi ) \mathcal {E}$ , one has

$$ \begin{align*} h_{\ell} (Q_{\ell} (\pi) x, Q_{\ell} (\pi) y) = \langle x, Q_{\ell}(\pi) y\rangle = \langle Q_{\ell} (\pi) x , y\rangle. \end{align*} $$

Corollary 2.13. The filtration $\mathcal N^{[\bullet ]}$ is compatible with the pairing if and only if $\mathcal {N}^{[\ell ]}$ is totally isotropic in $\mathcal {E} [Q_{\ell }]$ for the pairing $h_{\ell }$ , for every $1 \leq \ell \leq e$ .

Proof. To say that the filtration is compatible with the pairing amounts to saying that for every $1 \leq \ell \leq e$ , one has ${\mathcal {N}^{[\ell ]}}^{\bot } = \left (Q^{\ell }(\pi )\right )^{-1} \mathcal {N}^{[\ell ]}$ . Since the orthogonal of $\mathcal {N}^{[\ell ]}$ for $h_{\ell }$ is $Q^{\ell }(\pi ) {\mathcal {N}^{[\ell ]}}^{\bot }$ , the result follows.

Remark 2.14. In special fiber, the situation is again much simpler. In this case, one has simply $\mathcal E [Q_{\ell }] = \mathcal E\left [\pi ^{\ell }\right ] = \pi ^{e-\ell } \mathcal E $ . The pairing $h_{\ell }$ on this sheaf is given by

$$ \begin{align*} h_{ \ell} \left(\pi^{e - \ell} x, \pi^{e - \ell} y\right) = \left\langle\pi^{e - \ell} x , y\right\rangle = \left\langle x, \pi^{e - \ell} y\right\rangle. \end{align*} $$

If $\mathcal F \subset \mathcal E\left [\pi ^{\ell }\right ]$ is totally maximally isotropic for $h_{ \ell }$ , then its orthogonal in $\mathcal E$ is equal to

$$ \begin{align*} \mathcal F^{\bot} = \left(\pi^{e-\ell}\right)^{-1} \mathcal F. \end{align*} $$

2.3.4 Application to p-divisible groups

Let $G \longrightarrow S$ be a p-divisible group of height $h \left [L : \mathbb {Q}_{p}\right ]$ , endowed with an $\mathcal O_{L}$ -action. Thus, we can decompose $\omega _{G}$ , a locally free $\mathcal O_{S}$ -module, into

$$ \begin{align*} \omega_{G} = \bigoplus_{ \tau \in \mathcal T} \omega_{G,\tau}. \end{align*} $$

Assume that $\omega _{G,\tau }$ is locally free of rank $p_{\tau }$ , and suppose, for all $\tau $ , integers

$$ \begin{align*} d_{\tau}^{1},\dotsc,d_{\tau}^{e} \end{align*} $$

such that $d_{\tau }^{i} \leq h$ for all $\tau ,i$ , and $d_{\tau }^{1} + \dotsb + d_{\tau }^{e} = p_{\tau }$ for all $\tau $ . Denote $f = \left [L^{ur}:\mathbb {Q}_{p}\right ]$ , so that $\operatorname {\mathrm {ht}}(G) = efh$ . Define $\mathcal H := \mathcal H(G) := H^{1}_{dR}(G/S) := \mathbb D(G)_{S \longrightarrow S}$ the evaluation of the crystal of G [2] on S. This is a locally free $\mathcal O_{S} \otimes _{\mathbb {Z}_{p}} \mathcal O_{L}$ -module of rank h, which moreover splits as

$$ \begin{align*} \mathcal H = \bigoplus_{\tau \in \mathcal T} \mathcal H_{\tau}, \end{align*} $$

and for each piece, there is an exact sequence given by the Hodge filtration:

$$ \begin{align*} 0 \longrightarrow \omega_{G,\tau} \longrightarrow \mathcal H_{\tau} \longrightarrow \omega_{G^{D},\tau}^{\vee} \longrightarrow 0. \end{align*} $$

Definition 2.15. A Pappas–Rapoport datum for G, with respect to $L,\left (\sigma _{\tau ,j}\right ),\left (d_{\tau }^{j}\right )_{\tau \in \mathcal T,j}$ , is the datum, for all $\tau $ , of a full Pappas–Rapoport datum for $\left (\mathcal H_{\tau },\omega _{G,\tau }\right )$ – that is, a filtration by locally direct $\mathcal O_{S}$ -factors

$$ \begin{align*} 0 = \omega_{G,\tau}^{[0]} \subset \omega_{G,\tau}^{[1]} \subset \dotsb \subset \omega_{G,\tau}^{[e-1]} \subset \omega_{G,\tau}^{[e]} = \omega_{G,\tau} \subset \omega_{G,\tau}^{[e+1]} \subset \dotsb \subset \omega_{G,\tau}^{[2e-1]} \subset\omega_{G,\tau}^{[2e]} = \mathcal H_{\tau} \end{align*} $$

satisfying the following properties:

1. 1. For $j = 1,\dotsc ,e$ , $\dim _{\mathcal O_{S}} \left (\omega _{G,\tau }^{[j]}/\omega _{G,\tau }^{[j-1]}\right ) = d_{\tau }^{j}$ .

2. 2. For $j = 1,\dotsc ,\ e \dim _{\mathcal O_{S}} \omega _{G,\tau }^{[e+j]}/\omega _{G,\tau }^{[e+j-1]} = h- d_{\tau }^{e-j+1}$ .

3. 3. For $j = 1,\dotsc ,e$ , $\left ([\pi ] - \sigma _{\tau ,j}(\pi )\right. \left (\omega _{G,\tau }^{[j]}\right ) \subset \omega _{G,\tau }^{[j-1]}$ .

4. 4. For $j = 1,\dotsc ,e$ , $ \left ([\pi ]-\sigma _{\tau ,e}(\pi )\right )\dotsm \left ([\pi ] - \sigma _{\tau ,e+j-1}(\pi )\right ) \omega _{G,\tau }^{[e+j]} \subset \omega _{G,\tau }^{[e-j]}.$

5. 5. For $j = 1,\dotsc ,e$ , $ \omega _{G,\tau }^{[e+j]} =\left (Q^{e-j} (\pi ) \right )^{-1} \omega _{G,\tau }^{[e-j]}.$

Definition 2.16. Let H be a locally free R-module (of finite rank) and R a ring, and denote

$$ \begin{align*} H \otimes H^{\vee} \longrightarrow R, \end{align*} $$

the perfect pairing between H and $H^{\vee } = \operatorname {\mathrm {Hom}}_{R}(H,R)$ . Let $W \subset H$ be a locally direct factor. The association

$$ \begin{align*} W \longmapsto W^{\perp} := \operatorname{\mathrm{Ker}}\left(H^{\vee} \twoheadrightarrow W^{\vee}\right) \end{align*} $$

is an inclusion reversing involution between locally direct factors of H and $H^{\vee }$ .

Thus by Proposition 2.8 we have the following:

Proposition 2.17. Let $\left (\omega _{G,\tau }^{[i]}\right )_{i=0,\dotsc ,2e}$ be a full Pappas–Rapoport datum for G with respect to $L,\left (\sigma _{\tau ,j}\right )_{\left (\tau ,j\right )},\left (d_{\tau }^{j}\right )_{\left (\tau ,j\right )}$ . Then $\left (\left (\omega _{G,\tau }^{[2e-i]}\right )^{\perp }\right )_{i=0,\dotsc ,2e}$ induces a full Pappas–Rapoport datum for $G^{D}$ with respect to $L,\left (\sigma _{\tau ,j}\right )_{\tau ,j},\left (h-d_{\tau }^{j}\right )_{\left (\tau ,j\right )}$ . Here $(.)^{\perp }$ denotes the previous involution under the identification $\mathcal H\left (G^{D}\right ) = \mathcal H(G)^{\vee }$ (which satisfies $\omega _{G}^{\perp } = \omega _{G^{D}}$ ).

Definition 2.18. Suppose we are given a ring extension $L/L^{+}$ of degree $\leq 2$ , such that L is a field or isomorphic to $L^{+}\times L^{+}$ , and set $s \in \operatorname {\mathrm {Gal}}\left (L/L^{+}\right )$ . Suppose we are given a polarization $\lambda : G \overset {\sim }{\longrightarrow } \left (G^{D}\right )^{(s)}$ . Then we say that a Pappas–Rapoport datum $\mathcal R$ for G is compatible with $\lambda $ if under the isomorphism $\lambda : \mathcal H(G) \overset {\sim }{\longrightarrow } \mathcal H\left (G^{D}\right )^{(s)}$ , the datum $\mathcal R$ and $\left (\mathcal R^{\perp }\right )^{(s)}$ (given by Proposition 2.17 and twisted by s) coincide. In particular, this implies that

$$ \begin{align*} d_{\tau,i}(\mathcal R) = h-d_{s(\tau),i}(\mathcal R). \end{align*} $$

Remark 2.19. In the situation when primes ramify further in $L/L^{+}$ , the previous compatibility is unfortunately impossible to achieve (except possibly in reduced special fiber). For example, let $L^{+} = \mathbb {Q}_{p}$ and $L = \mathbb {Q}_{p}[T]/(E(T))$ a quadratic extension in which $(p) = (\pi )^{2}$ ramifies. Denote by $c(\pi ) = \overline \pi $ the conjugate uniformizer. Thus a Pappas–Rapoport datum for $G/S$ , L, $(\pi ,\overline \pi )$ , and $d \leq h$ is the datum of

$$ \begin{align*} 0 \subset \omega^{[1]} \subset \omega_{G} \subset \mathcal F^{[1]} \subset \mathcal H,\\[-9pt] \end{align*} $$

such that $\mathcal F^{[1]} = (T-\overline {\pi })^{-1}\omega ^{[1]}$ . The associated datum of $\left (G^{D}\right )^{(s)}$ is

$$ \begin{align*} 0 \subset \left(\mathcal F^{[1]}\right)^{\perp,s} \subset \omega_{G^{D}}^{(s)} \subset \left(\omega^{[1]}\right)^{\perp,(s)} \subset H^{\vee,(s)}.\\[-9pt] \end{align*} $$

But T acts on $\mathcal H/\mathcal F^{[1]}$ as $\pi $ , and thus on $\left (\mathcal F^{[1]}\right )^{\perp ,s}$ as $\overline \pi $ . Moreover, $\left (\mathcal F^{[1]}\right )^{\perp ,c}$ is of rank $h-d_{1}$ when $\omega ^{[1]}$ is of rank $d_{1}$ . There is thus no chance that given an isomorphism

$$ \begin{align*} \lambda : G \overset{\sim}{\longrightarrow} G^{D,(s)}, \end{align*} $$

our Pappas–Rapoport datum is $\lambda $ -compatible (except if $\pi = \overline \pi $ on S, for example in special fiber, and $2d_{1} = h$ ). We will refer to this as case (AR) in the rest of the text.

2.4 Pappas–Rapoport models

Let $\mathcal O_{B}$ be a $\mathbb {Z}_{(p)}$ -order in B, preserved by $\star $ , such that its completion is a maximal $\mathbb {Z}_{p}$ -order in $B\otimes _{\mathbb {Q}} \mathbb {Q}_{p}$ , and let $(\Lambda ,\langle .,.\rangle )$ be a PEL $\mathcal O_{B}$ -lattice [14, Definition 1.2.1.3] such that $(\Lambda ,\langle .,.\rangle )\otimes _{\mathbb Z} \mathbb {Q} = (V,\langle .,.\rangle )$ . Moreover, assume Hypothesis 2.2 – that is, say $B_{\mathbb {Q}_{p}}$ is isomorphic to a product of matrix algebras over (necessarily finite) extensions of $\mathbb {Q}_{p}$ . Note that we do not assume that the extensions are unramified. Now assume that p is a good prime, in the sense that $p \nmid \left [\Lambda ^{\sharp }:\Lambda \right ]$ , where $\Lambda ^{\sharp } = \{ x \in V | \langle x,y\rangle \in \mathbb {Z}, \forall y \in \Lambda \}$ . ¹ This assumption will remain in force during all this article.

Let K be an extension of $E_{\nu }$ (with E the reflex field) which contains $F_{i}^{gal}$ for all i. We will want to consider a moduli problem X over $\mathcal O_{K}$ of associating to S quintuples $\left (A,\lambda ,\iota ,\eta ,\omega ^{[\cdot ]}\right )$ up to $\mathbb {Z}_{(p)}^{\times }$ -isogenies, where $\omega ^{[\cdot ]}$ will be a Pappas–Rapoport datum with respect to a combinatorial datum $\mathcal C$ which we now explain.

First, suppose we are given an abelian scheme A over S (itself over $\mathcal O_{K}$ ) endowed with an action $\iota : \mathcal O_{B} \longrightarrow \operatorname {\mathrm {End}}(A) \otimes _{\mathbb {Z}} \left (\mathbb {Z}_{(p)}\right )_{S}$ . Decompose $\omega _{A}$ as before so we get a collection $(\omega _{i})_{i=1,\dotsc ,r}$ , and now the action of $\mathcal O_{F_{i}^{ur}}$ on $\omega _{i}$ can be split (as S is over $\mathcal O_{K}$ ) as

$$ \begin{align*} \omega_{i} = \bigoplus_{\tau \in \mathcal T_{i}} \omega_{i,\tau}, \end{align*} $$

where $\mathcal T_{i} = \operatorname {\mathrm {Hom}}\left (F_{i}^{ur},\mathbb {C}_{p}\right )$ . Unfortunately, if $F_{i}$ is ramified, we cannot further decompose the $\omega _{i,\tau }$ as we did over $\mathbb {Q}_{p}$ .

Denote by $\Sigma _{i,\tau }$ the subset of $\mathcal T^{\prime }_{i} := \operatorname {\mathrm {Hom}}\left (F_{i},\mathbb {C}_{p}\right )$ of embedding $\tau ^{\prime }$ that induces $\tau $ when restricted to $F^{ur}_{i}$ . Let us denote, for all $i \in \{1,\dotsc ,r\}$ , $\pi _{i}$ a chosen uniformizer of $F_{i}/F_{i}^{ur}$ and $Q_{i}$ a corresponding Eisenstein polynomial, and let us choose an ordering $\Sigma _{i,\tau } = \left \{\sigma _{i,\tau ,1},\dotsc ,\sigma _{i,\tau ,e_{i}}\right \}$ for the elements of $\Sigma _{i,\tau }$ , for all $i,\tau $ , where $e_{i} = \left [F_{i}:F_{i}^{ur}\right ]$ (it corresponds to an ordering of the conjugate roots of $\tau (Q_{i})$ ). This induces a bijection

$$ \begin{align*} \begin{array}{ccc} \{1,\dotsc,e_{i}\} & \overset{\sigma_{\bullet}}{\longrightarrow} & \Sigma_{i,\tau}, \\ j & \longmapsto & \sigma_{i,\tau,j} \end{array} \end{align*} $$

and a numbering $\left (d_{i,\tau ,j}\right )_{j = 1,\dotsc ,e_{i}}$ such that $\left \{ d_{i,\tau ,j} : j = 1,\dotsc ,e_{i}\right \} = \left \{ d_{i,\tau ^{\prime }} : \tau ^{\prime } \in \Sigma _{i,\tau }\right \}$ , by setting

$$ \begin{align*} d_{i,\tau,j} = d_{i,\sigma_{i,\tau,j}}, \end{align*} $$

where $d_{i,\sigma _{i,\tau ,j}}$ is defined in §2.2. We assume, moreover, to reflect Remark 2.4, that the choice of these bijections implies that, for all $i,\tau ,j$ ,

$$ \begin{align*} d_{i,\tau,j} = h_{i}-d_{s(i,\tau),j}, \end{align*} $$

where, as before, s is the action induced by $\star $ , the involution on B, on $\prod _{i} \operatorname {\mathrm {Hom}}\left (F_{i},\mathbb {C}_{p}\right )$ . To ease the notation, we will write

$$ \begin{align*} \mathcal{C} = \left(\left(\sigma_{i,\tau,j}\right)_{i,\tau,j},\left(d_{i,\tau,j}\right)_{i,\tau,j}\right) \end{align*} $$

and $\mathcal {C}_{i}= \left (\left (\sigma _{i,\tau ,j}\right )_{\tau ,j},\left (d_{i,\tau ,j}\right )_{\tau ,j}\right )$ for every i. We will also write $\mathcal {C}^{D} = \left (\left (\sigma _{i,\tau ,j} \circ s\right )_{i,\tau ,j},\left (h_{i} - d_{i,\tau ,j}\right )_{i,\tau ,j}\right )$ and $\mathcal {C}^{D}_{i}= \left (\left (\sigma _{i,\tau ,j} \circ s \right )_{\tau ,j},h_{i} -\left (d_{i,\tau ,j}\right )_{\tau ,j}\right )$ . Moreover, if we denote $\mathcal H = H^{1}_{dR}(A/S)$ , we can decompose

$$ \begin{align*} \mathcal H = \bigoplus_{i} \mathcal H_{i}, \end{align*} $$

with $\mathcal H_{i}$ a $M_{n_{i}}\left (\mathcal O_{F_{i}}\right )$ -module, corresponding by Morita equivalence to $e_{i}\mathcal H_{i}$ , which we can further decompose as

$$ \begin{align*} e_{i}\mathcal H_{i} = \bigoplus_{\tau : F_{i}^{ur} \longrightarrow \overline{\mathbb{Q}}_{p}} \mathcal H_{i,\tau}. \end{align*} $$

Note that if S is actually over $\mathbb {Q}_{p}$ , then

$$ \begin{align*} \omega_{i,\tau} = \bigoplus_{\tau^{\prime} \in \Sigma_{i,\tau}} \omega_{i,\tau^{\prime}}, \end{align*} $$

and using the previous decomposition for $\omega $ , we get $\omega _{i,\tau }^{[\cdot ]}$ a filtration of $\omega _{i}$ :

$$ \begin{align*} \omega_{i,\tau}^{[j]} = \bigoplus_{r = 1}^{j} \omega_{i,\sigma_{i,\tau,r}}. \end{align*} $$

Example 2.20. As explained in [12], when C is a semisimple algebra over a field (say of characteristic p), then a C-module V is determined by its determinant $\det _{V}$ associated by Kottwitz. Unfortunately, in the ramified case $\mathcal O_{B}\otimes \mathbb {F}_{p}$ is no longer semisimple, and the determinant fails to determine V. For example, say $B_{\mathbb {Q}_{p}} = L$ a totally ramified extension of $\mathbb {Q}_{p}$ of degree e, and thus $\mathcal O_{B} \otimes \mathbb {F}_{p} = \mathbb {F}_{p}[X]/(X^{e})$ . Let $V = \mathbb {F}_{p}[X]/(X^{e})$ and $W = \mathbb {F}_{p}^{e}$ . Then $\det _{V}(T_{1},\dotsc ,T_{e}) = \det _{W}(T_{1},\dotsc ,T_{e}) = T_{1}^{e}$ , but obviously V and W are not isomorphic as $\mathbb {F}_{p}[X]/(X^{e})$ -modules.

Now assume that $\lambda $ is a $\mathbb {Z}_{p}^{\times }$ -polarization on A and $\iota $ is an $\mathcal O_{B}$ structure for $(A,\lambda )$ (that is, it satisfies the Rosati condition; compare [14, Definition 1.3.3.1]). Using this $\lambda $ , we deduce an isomorphism for all indices i,

$$ \begin{align*} e_{i}\mathcal H_{i} \simeq e_{s(i)}\mathcal H_{s(i)}^{\vee}, \end{align*} $$

coming from an isomorphism of the associated p-divisible group when decomposing $\mathcal A[p^{\infty }]$ , and we can thus apply Definition 2.18 for $F_{i}/F_{i}^{+} = F_{i}^{\star = 1}$ (when $s(i) = i$ ) or $F_{i} \times F_{s(i)}$ (when $i \neq s(i)$ ). As suggested in Remark 2.19, we say that an index i falls into case (AR) if $F_{i}$ is stable by $\star $ and the extension $F_{i}/F_{i}^{\star = 1}$ is ramified. This notation will be explained in more detail in §2.5.

Definition 2.21 [19, §§9 and 14]

Let S be an $\mathcal O_{K}$ -scheme. Consider a Pappas–Rapoport datum for $(A,\lambda ,\iota )/S$ with respect to $\mathcal C$ , for every $(i,\tau )$ .

• – If i does not fall into case AR, it is the datum for a full Pappas–Rapoport datum for $\left (\mathcal H_{i,\tau },\omega _{i,\tau }\right )$ associated to $\mathcal {C}_{i}$ , as in Definition 2.15, that is moreover $\lambda $ -compatible with $\mathcal C_{s(i)}^{D}$ (see Definition 2.18).

• – If i falls into case AR, we ask for a full Pappas–Rapoport datum $\mathcal R$ for $\left (\mathcal H_{i,\tau },\omega _{i,\tau }\right )$ with respect to $\mathcal {C}_{i}$ . This automatically induces a full Pappas–Rapoport datum $\mathcal R^{\perp ,(s)}$ for $\left (\mathcal H_{i,\tau }^{D,s},\omega _{A^{D},i,\tau )}^{(s)}\right ) = \left (\mathcal H_{i,\tau },\omega _{A,i,\tau )}\right )$ with respect to $\mathcal {C}_{i}^{D}$ .

As explained in Remark 2.19, we cannot ask for $\lambda $ -compatibility in case (AR). Unfortunately, in this situation the moduli space will not be studied in much detail in this article, but we will show that our two main theorems fail in this case.

Consider the moduli problem X over $\mathcal O_{K}$ , associating to S quintuples $\left (A,\lambda ,\iota ,\eta ,\omega ^{[\cdot ]}\right )$ up to $\mathbb {Z}_{(p)}^{\times }$ -isogenies, where

• – $A \longrightarrow S$ is an abelian scheme,

• – $\lambda : A \longrightarrow {^{t}}A$ is a $\mathbb {Z}_{(p)}^{\times }$ -polarization,

• – $\iota : \mathcal O_{B} \longrightarrow \operatorname {\mathrm {End}}(A) \otimes _{\mathbb {Z}} \left (\mathbb {Z}_{(p)}\right )_{S}$ is an $\mathcal O_{B}$ -structure of $(A,\lambda )$ (in particular, $\star $ induces the Rosati involution),

• – $\eta $ is a rational level structure outside p (see [14, §1.4.1]),

• – $\omega ^{[\cdot ]}$ is a Pappas–Rapoport datum for $\mathcal C$ , which is defined in Definition 2.21.

Proposition 2.22. The moduli space X associating to each S over $\operatorname {\mathrm {Spec}}(\mathcal O_{K})$ the set of isomorphism classes of quintuples $\left (A,\lambda ,i,\eta ,\omega ^{[\cdot ]}\right )$ is representable by a quasiprojective scheme.

Proof. This is shown in a local context in [19]: the morphism that forgets the Pappas–Rapoport datum is relatively representable over the (PEL) moduli space (Kottwitz’s model). Thus X is representable, locally fibered over Kottwitz’s model as a closed subset of a product of Grassmannians (equivalently, it is a closed subset of some partial flag variety for $\omega _{A^{univ}}$ over Kottwitz’s model).

Proposition 2.23. Let $K^{p} \subset G\left (\mathbb A_{f}^{p}\right )$ be a compact open subgroup as before (neat). Let $\mathfrak C \subset G\left (\mathbb {Q}_{p}\right )$ be the stabilizer of $\Lambda $ , and consider the compact open $\mathfrak CK^{p}$ . Let us choose K a p-adic completion of the Galois closure of the $F_{i}$ , as before. Then the Pappas–Rapoport model $X/\operatorname {\mathrm {Spec}}(\mathcal O_{K})$ coincides with $\mathcal S_{\mathfrak CK^{p}}$ over K – that is, X is an integral model of $\mathcal S_{\mathfrak CK^{p}}$ over $\mathcal O_{K}$ .

Proof. Obviously if $(A,i,\lambda ,\eta )$ is a quadruple in $\mathcal S_{\mathfrak CK^{p}}(S)$ , where S is over K, there is a canonical filtration of $\omega _{A}$ , as explained before, as we have fixed the bijections $\sigma _{\bullet }$ . Moreover, each quintuple $\left (A,i,\lambda ,\eta ,\omega ^{[\cdot ]}\right )$ satisfies Kottwitz’s determinant condition, as the filtration given by $\omega ^{[\cdot ]}$ on $\omega _{i,\tau }$ is split, and the dimensions are fixed by the Pappas–Rapoport condition. The equivalence between definition by $\mathbb {Z}_{(p)}^{\times }$ -isogeny classes and quasi-isogeny classes (in characteristic $0$ ) is then [14, Proposition 1.4.3.4].

From now on, fix a level $K^{p} \subset G\left (\mathbb A_{f}^{p}\right )$ outside p and $\mathfrak C$ as before at p (‘without level at p, or rather maximal level at p), and call X the Pappas–Rapoport model over $\mathcal O_{K}$ of the Shimura variety $\mathcal S_{\mathfrak CK^{p}}$ . It thus make sense to reduce X over $\kappa $ , the residue field of K. The goal of this article is to study the geometry of $X_{\kappa } := X \times \operatorname {\mathrm {Spec}}(\kappa )$ .

2.5 Polygons

As explained in the previous section, we can decompose the Lie algebra $\omega _{A}$ of the universal abelian scheme A over X through the action of $\mathcal O_{B}\otimes \mathbb {Z}_{p}$ . Actually, we can also decompose the p-divisible group $A[p^{\infty }]$ . According to Hypothesis 2.2, we write

$$ \begin{align*} \mathcal O_{B} \otimes \mathbb{Z}_{p} = \prod_{\pi \in \mathcal P} M_{n_{\pi}}(R_{\pi}), \end{align*} $$

where $\pi \in \mathcal P$ is a new indexation for $i \in \{1,\dotsc ,r\}$ – where the two factors $i, s(i)$ , exchanged by $\star $ , share the same index $\pi $ – and $R_{\pi } = \mathcal O_{F_{\pi }}$ if $i = s(i)$ or $R_{\pi } = \mathcal O_{F_{\pi }} \times \mathcal O_{F_{\pi }}$ if there are two factors (that is, $R_{\pi } = \mathcal O_{F_{i}} \times \mathcal O_{F_{s(i)}}$ if $\pi = [i]$ when $i \neq s(i)$ ). Thus $\mathcal P = \{1,\dotsc ,r\}_{/\sim _{s}}$ . We refer to the $\pi \in \mathcal P$ as places over p in B. We can thus decompose

$$ \begin{align*} A[p^{\infty}] = \prod_{\pi \in \mathcal P} A[\pi^{\infty}], \end{align*} $$

where $A[\pi ^{\infty }]$ is an $M_{n_{\pi }}(R_{\pi })$ -module that is p-divisible, and by Morita equivalence,

$$ \begin{align*} A[\pi^{\infty}] = \mathcal O_{R_{\pi}}^{n_{\pi}} \otimes_{\mathcal O_{R_{\pi}}} G_{\pi}. \end{align*} $$

Note that because the (universal) polarization $\lambda $ of A is compatible with $\star $ , each factor $A[\pi ^{\infty }]$ is still endowed with a polarization $\lambda _{\pi }$ and thus also $G_{\pi }$ . We will use this decomposition of $A[p^{\infty }]$ all the time, since if we know the $n_{\pi }$ , it is equivalent to knowing $A[p^{\infty }]$ (and $\lambda $ ) or the collection $(G_{\pi })_{\pi }$ (and the $\lambda _{\pi }$ ).

For all $\pi $ , $G_{\pi }$ is a polarized p-divisible group over X, the Pappas–Rapoport model, endowed with an action of $R_{\pi }$ .

Moreover, if $R_{\pi } = \mathcal O_{F_{\pi }} \times \mathcal O_{F_{\pi }}$ and $\star $ exchanges the two factors, we can further decompose $G_{\pi } = H_{\pi } \times H_{\pi }^{D}$ , and $\lambda _{\pi }$ exchanges the two factors. In this case, called (Split) or (AL), the datum of $(G_{\pi },\lambda _{\pi })$ is equivalent to $H_{\pi }$ .

Otherwise, $G_{\pi }$ is a p-divisible $\mathcal O_{F_{\pi }}$ -module with a polarization $\lambda _{\pi }$ , such that either

• – $\star $ induces the identity on $\mathcal O_{F_{\pi }}$ , which is equivalent to $\lambda _{\pi }$ being compatible with the $\mathcal O_{F_{\pi }}$ -action, which we refer to as case (C); or

• – $\star $ is an automorphism of order $2$ of $\mathcal O_{F_{\pi }}$ , and we denote $\mathcal O_{F^{+}_{\pi }}$ the subfield fixed by $\star $ . If $\mathcal O_{F_{\pi }}$ is unramified over $\mathcal O_{F_{\pi }^{+}}$ , we refer to this case as (Inert) or (AU), and if the extension is ramified, as (Ram) or (AR). In these two cases, $G_{\pi }$ satisfies the symmetry

  $$ \begin{align*} G_{\pi}^{D} \overset{\lambda_{\pi}}{\simeq} G^{(c)}_{\pi}, \end{align*} $$
  where c is the order $2$ automorphism of $F_{\pi }$ induced by $\star $ and $G^{(c)}_{\pi }$ is the p-divisible $\mathcal O_{F_{\pi }}$ -module where the endomorphism structure is $\iota ^{(c)} := \iota \circ c$ if $\iota $ denotes the endomorphism structure of $G_{\pi }$ .

Remark 2.24. The previous denomination comes from the possible decompositions of the p-divisible group of an abelian variety endowed with an action of the ring of integers of a totally real field (C) or a CM-field $F/F^{+}$ in which a place $\pi $ of $F^{+}$ splits in F (AL) where the underlying group at p is a linear group, is inert in F (AU) where the underlying group is an unramified unitary group, or is ramified in F (AR), which itself is related to the classification of Lie algebras as symplectic (C) or unitary (A) (as we have excluded orthogonal factors (D)).

From now on, we will fix an element $\pi \in \mathcal P$ . For the rest of this subsection, assume our base scheme S is a field k over $\kappa $ (thus of characteristic p). Let us be more explicit about the different cases.

2.5.1 Case (C)

In case (C), we will denote the p-divisible group $G_{\pi }$ simply by G, and $F_{\pi }$ by L. The p-divisible group G has an action of $\mathcal O_{L}$ and a polarization. It has height $2dg$ and dimension $dg$ , where d is the degree of L over $\mathbb {Q}_{p}$ . The sheaf $\omega _{G}$ decomposes as

$$ \begin{align*} \omega_{G} = \bigoplus_{\tau \in \mathcal{T}} \omega_{\tau}, \end{align*} $$

where $\mathcal T$ is the set of embeddings of $L^{ur}$ . Recall the Hodge filtration for G:

$$ \begin{align*} 0 \to \omega_{G} \to H^{1}_{dR} \to \omega_{G^{D}}^{\vee} \to 0, \end{align*} $$

where $G^{D}$ is the Cartier dual of G. This exact sequence splits according to the elements of $\mathcal {T}$ . The Pappas–Rapoport condition for G is then as follows.

For each $\tau \in \mathcal {T}$ , one has a filtration

$$ \begin{align*} \omega_{\tau}^{[0]} = 0 \subset \omega_{\tau}^{[1]} \subset \dotsb \subset \omega_{\tau}^{[e]} = \omega_{\tau}, \end{align*} $$

where $\omega _{\tau }^{[i]}$ is locally a direct factor of rank $gi$ . Moreover, one has the following compatibility with the polarization:

$$ \begin{align*} {\omega_{\tau}^{[i]}}^{\bot} = (\pi^{e-i})^{-1} \omega_{\tau}^{[i]}, \end{align*} $$

taken in $H^{1}_{dR,\tau }$ for $1 \leq i \leq e$ (recall $\pi _{i} = 0$ in S here).

For each $\tau \in \mathcal T$ , one can define the polygon $Hdg_{\tau }(G)$ ; it is defined thanks to $\omega _{\tau }$ as in [3, Definition 1.1.7]. It starts at $(0,0)$ and ends at $(2g,g)$ . Since G has a polarization, this polygon is symmetric: its slopes are $\lambda _{1}, \dotsc , \lambda _{g}, 1-\lambda _{g}, \dotsc , 1 - \lambda _{1}$ . We define the polygon $Hdg(G)$ as the mean of the polygons $Hdg_{\tau }(G)$ .

The polygons $PR_{\tau }$ and $PR$ are all equal: they have slope $0$ and $1$ , each of them with multiplicity g.

We define the Newton polygon of G as in [3, Definition 1.1.8] and denote it $Newt(G)$ ; it is also symmetric.

2.5.2 Case (AL)

In this case, one has $G_{\pi } = H_{\pi } \times H_{\pi }^{D}$ . We will consider the p-divisible group $G = H_{\pi }$ . It is endowed with an action of $\mathcal {O}_{L}$ but has no polarization. The sheaf $\omega _{G}$ decomposes as

$$ \begin{align*} \omega_{G} = \bigoplus_{\tau \in \mathcal{T}} \omega_{\tau}. \end{align*} $$

Fix $\left (a_{\tau ,j}\right ) \in \mathbb {Z}^{\mathcal T \times \{1,\dotsc ,e\}}$ , where e is the ramification index of L. Denote $a_{\tau } = \dim \omega _{\tau }$ and $b_{\tau } = \dim \omega _{G^{D},\tau }$ . Then $h^{\prime } = a_{\tau } + b_{\tau }$ is independent of $\tau $ . In the global setting, $\left (a_{\tau ,j}\right )$ will coincide with the part of the integers $d_{i,\tau ,j}$ corresponding to $H_{\pi }$ . The Pappas–Rapoport datum (for $(a_{\tau ,j})$ ) is then as follows. For each $\tau \in \mathcal {T}$ , one has a filtration

$$ \begin{align*} \omega_{\tau}^{[0]} = 0 \subset \omega_{\tau}^{[1]} \subset \dotsb \subset \omega_{\tau}^{[e]} = \omega_{\tau}, \end{align*} $$

where $\omega _{\tau }^{[i]}$ is locally a direct factor of rank $a_{\tau ,1} + \dotsb + a_{\tau ,i}$ .

Note that this Pappas–Rapoport datum induces a Pappas–Rapoport datum for $G^{D}$ (see Definition 2.16). For each $\tau \in \mathcal T$ , one can define the polygon $Hdg_{\tau }(G)$ ; it is defined thanks to $\omega _{\tau }$ [3, Definition 1.1.7]. It starts at $(0,0)$ and ends at $\left (a_{\tau }+b_{\tau },a_{\tau ,1} + \dotsb + a_{\tau ,e}\right ) = (h^{\prime },a_{\tau })$ . We define the polygon $Hdg(G)$ as the mean of the polygons $Hdg_{\tau }(G)$ .

The polygons $PR_{\tau }$ are defined in [3, §1]. The polygon PR is the mean of the polygons $PR_{\tau }$ .

We define the Newton polygon of G using [3, Definition 1.1.8] and denote it $Newt(G)$ . As we have used $H_{\pi }$ instead of $H_{\pi } \times H_{\pi }^{D}$ , these polygons do not need to be polarized (that is, symmetric in any sense).

2.5.3 Case (AU)

In this case, we define $G = G_{\pi }$ and we denote $F_{\pi }$ by L and by $L^{+}$ the subfield of elements fixed by $\star $ , which we denote by $\overline {.}$ as a conjugation. It is endowed with an action of $\mathcal O_{L}$ but has also a polarization. The sheaf $\omega _{G}$ decomposes as

$$ \begin{align*} \omega_{G} = \bigoplus_{\tau \in \mathcal{T^{0}}} \omega_{\tau} \oplus \omega_{\overline{\tau}}, \end{align*} $$

where $\mathcal {T}$ is the set of embedings of L and $\mathcal {T^{0}}$ the embeddings of L modulo conjugation. We define $\left (a_{\tau ,j}\right ) \in \mathbb {Z}^{\mathcal T \times \{1,\dotsc ,e\}}$ as before, and $a_{\tau } = \dim \omega _{\tau }$ , $b_{\tau } = \dim \omega _{G^{D},\tau } = \dim \omega _{\overline \tau } = a_{\overline \tau }$ , and $h^{\prime } = a_{\tau } + b_{\tau }$ . The Pappas–Rapoport condition is then as follows. For each $\tau \in \mathcal {T}$ , one has a filtration

$$ \begin{align*} \omega_{\tau}^{[0]} = 0 \subset \omega_{\tau}^{[1]} \subset \dotsb \subset \omega_{\tau}^{[e]} = \omega_{\tau}, \end{align*} $$

where $\omega _{\tau }^{[j]}$ is locally a direct factor of rank $a_{\tau ,1} + \dotsb + a_{\tau ,j}$ .

Note that this Pappas–Rapoport condition coincides with the induced Pappas–Rapoport condition for $\omega _{\overline {\tau }}$ thanks to the compatibility with the polarization. For each $\tau \in \mathcal T$ , one can define the polygon $Hdg_{\tau }(G)$ ; it is defined using $\omega _{\tau }$ . It starts at $(0,0)$ and ends at $\left (a_{\tau }+b_{\tau },a_{\tau ,1} + \dotsb + a_{\tau ,e})=(h^{\prime },a_{\tau }\right )$ . We define the polygon $Hdg(G)$ as the mean of the polygons $Hdg_{\tau }(G)$ for all $\tau \in \mathcal T$ .

The polygons $PR_{\tau }$ are defined in [3]. The polygon PR is the mean of the polygons $PR_{\tau }$ .

We define the Newton polygon of G as $Newt(G)$ .

2.5.4 Case (AR)

In this case also we still define $G = G_{\pi }$ , which is still polarized and carries an action as in case (C) or (AU), but we no longer have the $\lambda $ -compatibility for the Pappas–Rapoport datum. This does not change anything regarding the polygons: we can still define Hodge and Newton polygons using the action as in [3, §1] (this does not use the Pappas–Rapoport datum), and the polarization implies that ${\mathcal {N}\mathrm {ewt}}(G) = {\mathcal {N}\mathrm {ewt}}\left (G^{D}\right )$ and $\operatorname {\mathrm {Hdg}}(G) = \operatorname {\mathrm {Hdg}}\left (G^{D}\right )$ (equalities between $\operatorname {\mathrm {Hdg}}_{\tau }(G) = \operatorname {\mathrm {Hdg}}_{\overline \tau }(G^{D})$ ). Moreover, the Pappas–Rapoport polygon depends only on the integers $\left (d_{\tau ,j}\right )_{\tau ,j}$ , and these are symmetric for G and $G^{D}$ (see Remark 2.4).

In all the previous cases, thanks to [3, Théorème 1.3.1], one has the following result:

Proposition 2.25 [3, Théorème 1.3.1]

One has the inequalities

$$ \begin{align*} Newt(G) \geq Hdg(G) \geq PR, \end{align*} $$

and these polygons are all symmetric (except in case (AL)).

2.6 Stratifications

Using the previous polygons, we can define subsets of the reduction of X modulo p, $\lvert X_{\kappa }\rvert $ . Denote by $\mathrm {Pol}$ the set of polygonal convex functions on $[0,\dotsc ,h]$ with break points at abscissas in $\frac {1}{e}\mathbb {Z}$ . The previous polygons define two maps

$$ \begin{align*} \lvert X_{\kappa}\rvert \overset{{\mathcal{N}\mathrm{ewt}}_{\pi}}{\longrightarrow} \mathrm{Pol} \quad \text{and} \quad \lvert X_{\kappa}\rvert \overset{\operatorname{\mathrm{Hdg}}_{\pi}}{\longrightarrow} \mathrm{Pol}. \end{align*} $$

Proposition 2.26. The maps ${\mathcal {N}\mathrm {ewt}}_{\pi }$ and $\operatorname {\mathrm {Hdg}}_{\pi ,\tau }$ are semicontinuous, in the sense that polygons can only descend by generization. Moreover, they have same beginning and ending points (which are always locally constant and constant in our global situation).

Proof. The result on the Newton polygon is well known (see, for example, [20, Theorem 3.6]. For the Hodge polygon, note that locally on $X_{\kappa }$ we can trivialize $\omega _{G_{\pi },\tau }$ and the action of $\pi $ on it is nilpotent (as $\pi ^{e} = p = 0$ on $\mathcal O_{X_{\kappa }}$ ). Thus there is (Zariski locally) a continuous map

$$ \begin{align*} X_{\kappa} \longrightarrow Nilp_{p_{\tau}} \end{align*} $$

to the nilpotent cone of $\operatorname {\mathrm {GL}}_{p_{\tau }}$ , sending a point to the matrix of $\pi $ . We can check that the Hodge strata are exactly the pullback of the stratification on the nilpotent cone. But now the analogous result is known for $Nilp_{p_{\tau }}$ .

There is moreover a (constant) map $PR_{\pi } : \lvert X_{\kappa }\rvert \longrightarrow \mathrm {Pol}$ . If $\pi $ is understood from the context, we will drop it from the previous notations. Recall the following:

Definition 2.27. Define the Newton stratification of the reduction mod p of the Pappas–Rapoport model $X \otimes _{\mathcal O_{K}} \overline {\mathbb {F}_{p}} = \coprod _{\nu } X^{\nu }$ by

$$ \begin{align*} X^{\nu} = \{ x \in X_{\kappa} : {\mathcal{N}\mathrm{ewt}}(x) = \nu\}. \end{align*} $$

The locus $X^{\nu =PR}$ of points $x \in \lvert X_{\kappa }\rvert $ such that ${\mathcal {N}\mathrm {ewt}}(x) = \operatorname {\mathrm {PR}}(x)$ is called the $\mu $ -ordinary locus (for $\pi $ ).

In particular, the $\mu $ -ordinary locus is an open stratum, by Proposition 2.25. There is another natural stratification with another natural open stratum:

Definition 2.28. The locus $X_{\nu = PR}$ where $\operatorname {\mathrm {Hdg}}_{\pi }(x) = \operatorname {\mathrm {PR}}(x)$ is called the generalized Rapoport locus (for $\pi $ ). It contains the $\mu $ -ordinary locus because of the inequalities recalled in the previous section. More generally, we can define the Hodge stratification $X_{\kappa } = \coprod _{\nu } X_{\nu }$ by

$$ \begin{align*} X_{\left(\nu_{\tau}\right)} = \left\{ x \in X_{\kappa} : \operatorname{\mathrm{Hdg}}_{\pi,\tau}(x) = \nu_{\tau}, \forall \tau\right\}. \end{align*} $$

Since for every $\tau $ we have $\operatorname {\mathrm {Hdg}}_{\pi ,\tau } \geq PR_{\tau }$ , we have $\operatorname {\mathrm {Hdg}}_{\pi } = \operatorname {\mathrm {PR}}$ if and only if $\operatorname {\mathrm {Hdg}}_{\pi ,\tau } = \operatorname {\mathrm {PR}}_{\tau }$ for all $\tau $ . Note also that in cases (C), (AU), and (AR), because of the polarization there is a symmetry between $\operatorname {\mathrm {Hdg}}_{\pi ,\tau }$ and $\operatorname {\mathrm {Hdg}}_{\pi ,\overline \tau }$ . In particular, we should only consider symmetric data $(\nu _{\tau })$ in these cases, in order to have nonempty strata.

Example 2.29.

1. 1. If the PEL datum is unramified (or if $\pi $ is unramified), the Hodge polygon $\operatorname {\mathrm {Hdg}}_{\pi }$ is constant on $X_{\kappa }$ , and thus the generalized Rapoport locus consists of all the varieties.

2. 2. In the Hilbert–Siegel case, the generalized Rapoport locus is the Rapoport locus – that is, the locus where the conormal sheaf $\omega _{G_{\pi }}$ is locally free as an $\mathcal O_{X_{\kappa }} \otimes _{\mathbb {Z}_{p}} \mathcal O_{F}$ -module. It thus contains the $\mu $ -ordinary locus (which is just the ordinary locus in this case).

2.7 Smoothness of the integral model

In this section, we prove that the Pappas–Rapoport model X is smooth if all the primes above p fall into types (AL), (AU), or (C) (that is, do not fall into case (AR)). We will reduce to working locally; thus let $G/S$ be a p-divisible group over a scheme S, endowed with an action of $\mathcal O_{L}$ , where $L/\mathbb {Q}_{p}$ is a finite extension, ² possibly a polarization, and a Pappas–Rapoport datum. We call such a p-divisible group with action, eventual polarization, and Pappas–Rapoport datum a p-divisible $\mathcal D$ -module, with $\mathcal D$ referring to the type of the extra-datum (including the Pappas–Rapoport datum). Denote by $\mathcal H = H^{1}_{dR}(G/S)$ the locally free $\mathcal O_{S}$ -module associated to G. It is actually a locally free $\mathcal O_{S} \otimes _{\mathbb {Z}_{p}}\mathcal O_{F}$ -module of rank h. It is endowed, except in case (AL), with a polarization (that is, a perfect pairing)

$$ \begin{align*} \langle. ,.\rangle : \mathcal H \times \mathcal H^{s} \longrightarrow \mathcal O_{S}, \end{align*} $$

which is alternating, and such that

$$ \begin{align*} \langle x,ay\rangle = \langle s(a)x,y\rangle, \quad \forall x,y \in \mathcal H, \forall a \in \mathcal O_{F}. \end{align*} $$

Theorem 2.30. Assume that for every prime $\pi $ above p, $\pi $ falls into cases (AL), (AU), or (C). Then X is smooth.

Proof. As X is of finite presentation and thus Noetherian, it is enough to show that it is formally smooth. Let $S \twoheadrightarrow R$ be a surjective morphism of Noetherian rings with ideal I such that $I^{2} = 0$ . In particular, I is endowed with nilpotent divided powers, and thus we can use Grothendieck and Messing’s theory. Thus set $x \in X(R)$ . If p is invertible on R, as we know that X is smooth in generic fiber, there is y in $X(S)$ above x and we are done. Otherwise, by Serre and Tate, it is enough to lift the p-divisible group, and we can divide the task between the primes above p in $\mathcal O_{B}$ . Thus fix one such prime and $G/R$ the associated p-divisible group (with extra structures). By Grothendieck and Messing, it is enough to lift $\omega _{G}$ together with its Pappas–Rapoport datum as a locally direct factor (stable by $\mathcal O_{F}$ and totally isotropic) in

$$ \begin{align*} \mathcal H \otimes_{R} S. \end{align*} $$

We will successively lift $\omega _{G}^{[1]}, \dotsc , \omega _{G}^{[e]}$ . Thus fix $\tau $ an embedding of $L^{ur}$ . We will work separately for each $\tau $ . Recall that in cases (C) and (AU) we have a polarization

$$ \begin{align*} \mathcal H_{\tau} \times \mathcal H_{\overline\tau} \longrightarrow R \end{align*} $$

that lifts to S (as it is defined on the crystalline site of R). In particular, in case (AU) it will be sufficient to choose one element in $[\tau ] = \left \{\tau ,\overline \tau \right \}$ , say $\tau $ , for each embedding $\tau $ , lift the Pappas–Rapoport datum in $\mathcal H_{\tau }$ , and take its orthogonal, which will be a lift of the Pappas–Rapoport datum for $\overline \tau $ . Thus this is similar to case (AL). In case (C), we will moreover need the Pappas–Rapoport datum to be totally isotropic.

Recall that the sheaf $\mathcal H_{\tau }$ is locally free as an $R [T] / Q(T)$ -module, where Q is the Eisenstein polynomial of an uniformizer. We refer to §§2.3.2 and 2.5 for the notations. For example, for all $\ell \in \{1,\dotsc ,e\}$ , we write $Q = Q_{\ell } Q^{\ell }$ with $Q_{\ell }(T) = \prod _{i=1}^{\ell } (T-\pi _{i})$ . We have a submodule

$$ \begin{align*} \omega_{\tau}^{[1]} \subset \mathcal H_{\tau}[T-\pi_{1}], \end{align*} $$

which is moreover totally isotropic for $h_{\tau ,1}$ in case (C) by Corollary 2.13. Thus, there exists a lift (totally isotropic in case (C)) of this module to

$$ \begin{align*} \mathcal H_{\tau} \otimes_{R} S[T-\pi_{1}], \end{align*} $$

which we denote by $\widetilde {\omega }_{\tau }^{[1]}$ . Now let us consider

$$ \begin{align*} E^{1}_{\tau} = \left(\mathcal H_{\tau} \otimes_{R} S /\widetilde{\omega}_{\tau}^{[1]} \right) \left[Q^{1}\right]. \end{align*} $$

It is a locally free $S[T]/\left (Q^{1}(T)\right )$ -module. Indeed, locally one can find a basis $e_{1}, \dotsc , e_{h}$ of $H_{\tau } \otimes _{R} S$ over $S [T] / Q(T)$ such that $\widetilde {\omega }_{\tau }^{[1]}$ is generated by $Q^{1}(T) e_{1}, \dotsc , Q^{1} (T) e_{d}$ . One then sees that $E^{1}_{\tau }$ is locally free over $S[T]/\left (Q^{1}(T)\right )$ , with basis $e_{1}, \dotsc , e_{d}, (T-\pi _{1})e_{d+1}, \dotsc , (T-\pi _{1}) e_{h}$ . In particular, $E^{1}_{\tau } [T - \pi _{2}]$ is locally free over S, and contains modulo I the image of $\omega ^{2}_{\tau }$ as a locally direct factor. In case (C), the fact that $\widetilde \omega _{\tau }^{[1]}$ is totally isotropic for $h_{1, \tau }$ implies that $E^{1}_{\tau }$ inherits the pairing from $\mathcal {H}_{\tau }$ . Thus, it is enough to lift the image of $\omega ^{2}_{\tau }$ as a locally direct factor in

$$ \begin{align*} E_{\tau}^{1}[T-\pi_{2}], \end{align*} $$

which is moreover in case (C) totally isotropic for $h_{\tau ,2}$ (by Corollary 2.13). But such a lift exists (by smoothness of the appropriate partial flag variety), and thus there exists $\widetilde {\omega }_{\tau }^{[2]} \subset \mathcal H_{\tau } \otimes S$ lifting ${\omega }_{\tau }^{[2]}$ .

Suppose that we have constructed for $1 \leq \ell \leq e-1$ locally direct factors

$$ \begin{align*} \widetilde\omega_{\tau}^{[1]} \subset \dotsb \subset \widetilde\omega_{\tau}^{[\ell]} \subset \mathcal H_{\tau}[Q_{\ell}]\otimes_{R} S, \end{align*} $$

which are moreover isotropic for $h_{\tau ,\ell }$ in case (C), lifting the previous datum over R to S. Denote

$$ \begin{align*} E^{\ell}_{\tau} = \left(\mathcal H_{\tau} \otimes_{R} S /\widetilde{\omega}_{\tau}^{[\ell]} \right) \left[Q^{\ell}\right]. \end{align*} $$

As before, it is a locally free $S[T]/\left (Q^{\ell }(T)\right )$ -module, which contains modulo I the image of $\omega ^{\ell + 1}_{\tau }$ as a locally direct factor. In case (C), the fact that $\widetilde \omega _{\tau }^{[\ell ]}$ is totally isotropic for $h_{\ell , \tau }$ implies that $E^{\ell }_{\tau }$ inherits the pairing from $\mathcal {H}_{\tau }$ . With the same argument as before, one lifts the image of $\omega ^{\ell + 1}_{\tau }$ as a locally direct factor in the locally free S-module

$$ \begin{align*} E_{\tau}^{\ell}[T-\pi_{\ell+1}], \end{align*} $$

which is moreover in case (C) totally isotropic for $h_{\tau ,\ell +1}$ (by Corollary 2.13). By induction, we can thus find a lift of the filtration

$$ \begin{align*} 0 \subset \omega^{[1]}_{\tau} \subset \dotsb \subset \omega^{[e]}_{\tau} \end{align*} $$

to S satisfying all the assumptions of the Pappas–Rapoport datum. Thus by Grothendieck and Messing (as $I^{2} = 0$ ) there exists a point $y \in X(S)$ lifting x, and X is smooth.

Remark 2.31. As shown in the proof, we could have argued slightly differently using a local model for X in the spirit of [19].

Remark 2.32. Unfortunately, the analogous result is not true in case (AR). In [13, Theorem 4.5], a local splitting model for $U(1,n-1)$ is constructed for a ramified quadratic extension (but the global construction of [19] would lead to the same singularities) $\mathcal M$ which is regular (thus flat) and whose special fiber is the union of two smooth irreducible varieties of dimension $n-1$ crossing along a smooth irreducible variety of dimension $n-2$ . (See the calculation in Appendix A.)

2.8 Deformations and displays

In order to construct deformations of a point x in $X_{\kappa }$ , associated to a datum $\left (A,\lambda ,i,\eta ,\left (\omega ^{[\cdot ]}_{\bullet }\right )\right )$ , we will deform the p-divisible group $A[p^{\infty }]$ , the action of $\mathcal O_{B}$ , the polarization, and the Pappas–Rapoport condition, and use Serre–Tate theory (because $\eta $ is a level structure outside p, we can deform it trivially).

Moreover, using the previous simplification of $A[p^{\infty }]$ using $\mathcal O_{B}\otimes \mathbb {Z}_{p}$ and Morita equivalence, it is enough to deform the polarized p-divisible $\mathcal O_{F_{\pi }}$ -modules $G_{\pi }$ together with their Pappas–Rapoport filtration – that is, the p-divisible $\mathcal D_{\pi }$ -module for all $\pi $ . We thus remove $\pi $ from the notation, and we have G a (possibly) polarized p-divisible $\mathcal O$ -module. To such a p-divisible group over a perfect field k of characteristic p is associated a Dieudonné module over $W(k)$ (more precisely, its Dieudonné crystal), and we want to deform it over $k[[X]]$ such that the special fiber at $X=0$ corresponds to G, and the generic fiber satisfies better properties, like being $\mu $ -ordinary for example. In order to do this, we will use the theory of displays (compare [28] and [17] for equivalence with the étale part). We will be interested mainly in the tools developed in [26, §3.2] (particularly §3.2.7 and Theorem 3.2.8). In particular, we have the following:

Proposition 2.33 Zink–Wedhorn; [26, Theorem 3.2.8]

Let k be a perfect field of characteristic p, $G/k$ (with additional structures $\iota _{0},\lambda _{0}$ ), and denote by $P_{0}$ the associated display (with additional structures $\iota _{0},\lambda _{0}$ ). Let N be a $W(k)$ -linear endomorphism of $P_{0}$ satisfying the following properties:

1. 1. $N^{2} = 0$ .

2. 2. N is skew-symmetric with respect to $\lambda _{0}$ .

3. 3. N is $\mathcal O$ -linear.

Then there exists a deformation $(P,\iota ,\lambda )$ of $(P_{0},\iota _{0},\lambda _{0})$ (of display with additional structures) over $k[[t]]$ whose associated p-divisible group (with additionnal structure) $(X_{N},\iota ,\lambda )$ lifts $(X,\iota _{0},\lambda _{0})$ and such that if $P_{0}$ is bi-infinitesimal,

$$ \begin{align*} (X_{N},\iota,\lambda) \otimes_{k[[t]]} k((t))^{perf} \simeq \mathcal{BT^{\prime}}\left((P,\iota,\lambda)\otimes_{k[[t]]} k((t))^{perf}\right), \end{align*} $$

where $\mathcal {BT}^{\prime }$ associates to a crystal (over a perfect field) its p-divisible group.

Remark 2.34. Conditions $1$ and $2$ in Proposition 2.33 are only needed to lift the polarization. In particular, they will not be needed in case (AL). We will use this kind of deformation only to modify the Newton polygon; in particular, we will be able to choose any lift of the Pappas–Rapoport datum, which is why we do not make any reference to it in the proposition.

3 The Hodge stratification

As explained before, we have fixed $\pi \in \mathcal P$ , and we assume that $\pi $ is in case (AL), (AU), or (C). We have also defined Hodge polygons, and the generalized Rapoport locus is by definition the locus where this polygon is minimal.

We will now prove that this locus is dense.

3.1 Lifting a module with filtration

This is an intermediary section which contains some results concerning the existence of lifts of modules satisfying certain properties.

Lemma 3.1. Let M be a free $k[[X]]$ -module of rank h, and $N_{1} \subset \dotsb \subset N_{r} \subset M$ be direct factors with $N_{i}$ of rank $d_{i}$ . Let $\overline {Fil}$ be a k-vector subspace of $M \otimes _{k[[X]]} k$ of dimension l. There exists a lift $Fil$ of $\overline {L}$ such that in generic fiber, the dimension of $L \cap N_{i}$ is $\max (0, l+d_{i} -h )$ .

Proof. We prove this result by induction on the integer r. Let us consider the case $r=1$ .

Define $\overline {M} = M \otimes _{k[[X]]} k$ , and let s be the dimension of $\overline {N_{1}} \cap \overline {Fil}$ . Then there exists a basis $e_{1}, \dotsc , e_{d_{1}}$ of $N_{1}$ such that the reduction of $e_{1}, \dotsc , e_{s}$ is a basis for $\overline {N_{1}} \cap \overline {Fil}$ . One can then complete in a basis $e_{1}, \dotsc , e_{h}$ of M such that the reduction of $e_{1}, \dotsc , e_{s}, e_{d_{1}+1}, \dotsc , e_{d_{1}+l-s}$ forms a basis for $\overline {Fil}$ .

If $l+d_{1} \leq h$ , one defines $Fil$ to be generated by $e_{1} + X e_{d_{1}+l-s +1}, \dotsc , e_{s} + X e_{d_{1}+l}, e_{d_{1}+1}, \dotsc , e_{d_{1}+l-s}$ .

If $l+d_{1}>h$ , one defines $Fil$ to be generated by

$$ \begin{align*} e_{1} + X e_{d_{1}+l-s +1}, \dotsc, e_{h+s-d_{1}-l} + X e_{h}, e_{h+s-d_{1}-l+1}, \dotsc, e_{s}, e_{d_{1}+1}, \dotsc, e_{d_{1}+l-s}. \end{align*} $$

Now let us turn to the general case. Let $\overline {L_{0}}$ be a complementary subspace of $\overline {Fil} \cap \overline {N_{k}}$ inside $\overline {Fil}$ . One will lift the direct sum $\overline {L_{0}} \oplus \overline {Fil} \cap \overline {N_{k}}$ . One will take an arbitrary lift of $\overline {L_{0}}$ ; by doing so, one reduces to the case where $\overline {Fil} \subset \overline {N_{k}}$ . Let s be the dimension of $\overline {L_{1}} := \overline {Fil} \cap \overline {N_{1}}$ . One will then distinguish two cases.

If $s \leq h - d_{k}$ , one defines a lift $L_{1}$ of $\overline {L_{1}} $ such that $L_{1} \cap N_{k} = \{ 0 \}$ in the generic fiber. Considering a complementary subspace $\overline {Fil}^{\prime }$ of $\overline {L_{1}} $ in $\overline {L}$ , one can use the induction hypothesis by considering the modules $N_{2} \subset \dotsb \subset N_{k} \subset M$ .

If $s> h - d_{k}$ , let $e_{1}, \dotsc , e_{h}$ be a basis of M adapted to the filtration $N_{1} \subset \dotsb \subset N_{k} \subset M$ . Assume also that the reduction of $e_{1}, \dotsc , e_{s}$ is a basis for $\overline {L_{1}} $ . Let $\overline {L_{0}} $ be the vector subspace of $\overline {L_{1}}$ spanned by the reduction of $e_{1}, \dotsc , e_{h-d_{k}}$ . Let us consider the lift $L_{0}$ of $\overline {L_{0}} $ spanned by $e_{1} + X e_{d_{k}+1}, \dotsc , e_{h-d_{k}} + X e_{h}$ . Let $\overline {Fil^{\prime }} $ be a complementary subspace of $\overline {L_{0}} $ in $\overline {Fil}$ . To lift $\overline {Fil^{\prime }}$ , we are thus reduced to lifting it in $M^{\prime } = \operatorname {\mathrm {Vect}}\left (e_{s+1},\dotsc ,e_{d_{k}}\right )$ , endowed with the filtration $N_{1} \cap M^{\prime } \subset N_{k-1} \cap M^{\prime } \subset M^{\prime }$ . We are thus reduced to the case of a smaller k, and by induction we are done.

In the polarized case, one will use the following lemma:

Lemma 3.2. Let M be a free $k[[X]]$ -module of rank $2g$ with a perfect pairing, and $N \subset M$ be a totally isotropic direct factor rank g. Let $\overline {Fil}$ be a totally isotropic k-vector subspace of $M \otimes _{k[[X]]} k$ of dimension g. There exists a lift $Fil$ of $\overline {Fil}$ such that in generic fiber, the dimension of $Fil \cap N$ is $0$ .

Proof. Let us first consider the case where $\overline {Fil} = N \otimes _{k[[x]]} k$ . Let $e_{1}, \dotsc , e_{g}$ be a basis of N, completed in a basis $e_{1}, \dotsc , e_{2g}$ of M, such that the pairing $\left \langle e_{i} , e_{j} \right \rangle $ is $1$ if $j = g+i$ and $0$ otherwise. One will define the module $Fil$ to be generated by the columns of the matrix $\left ( \begin {array}{c} I_{g} \\ XA \end {array} \right ) $ , where $I_{g}$ is the identity matrix and A is any invertible symmetric matrix of size g.

Let us now turn to the general case. Define $\overline {M} := M \otimes _{k[[x]]} k$ , $\overline {N} := N \otimes _{k[[x]]} k$ , and $\overline {L_{0}} = \overline {Fil} \cap \overline {N}$ . Let $\overline {L_{1}}$ be a complementary subspace of $\overline {L_{0}}$ in $\overline {Fil}$ . Let $L_{1}$ be a totally isotropic lifting of $\overline {L_{1}}$ in M. One will look for a lift $Fil$ inside $L_{1}^{\bot } $ and containing $L_{1}$ . One is then led to consider the module $M_{1} := L_{1}^{\bot } / L_{1}$ , which is free of rank $2(g-s)$ , where s is the dimension of $\overline {L_{1}}$ . The module $N \cap L_{1}^{\bot }$ is free of rank $g-s$ , and so is its projection onto $M_{1}$ . By doing so, one reduces to the previous case.

3.2 Density of the generalized Rapoport locus

In this section, we use the previous lemmas to prove the following:

Theorem 3.3. The generalized Rapoport locus is (open and) dense.

To prove this theorem, we do it for one $\pi $ at a time and find a lift ‘locally’ – that is, for $G_{\pi }$ . We will thus consider the possible cases: (AL), (AU), and (C).

Remark 3.4. Again, the analogous result in case (AR) is false, as shown in the examples in Appendix A.

3.2.1 Cases (AL) and (AU)

Proof. Let x be a point of $X_{\kappa } := X\otimes \kappa $ and $G = G_{\pi }$ the associated p-divisible group. We want to prove that there exists a deformation of x which lies in the generalized Rapoport locus. Thanks to Grothendieck and Messing, we deform the Hodge filtration of G. For each $\tau \in \mathcal {T}$ , one uses Lemma 3.1 to deform $\omega _{\tau }$ . By duality in case (AU), one automatically has a deformation of $\omega _{\overline {\tau }}$ , hence of the whole of $\omega _{G}$ .

Let us now describe the way to lift $\omega _{\tau }$ . Let $D=H^{1}_{dR, \tau }$ ; it is free as a $k[X]/(X^{e})$ -module. Let M be the ( $\tau $ -part of the) evaluation of the crystal at $k[[t]]$ ; it is free as a $k[[t]][X]/(X^{e})$ , and reduces to D modulo t. To lift $\omega _{\tau }$ , one successively lifts $\omega _{\tau }^{[1]}, \dotsc , \omega _{\tau }^{[e]}$ . First, one lifts $\omega _{\tau }^{[1]}$ in $M[X]$ , the X-torsion of M. Let $L_{1}$ be any such lift. Then in order to lift $\omega _{\tau }^{[2]}$ , one works in $M_{1} := X^{-1} L_{1} / L_{1}$ . One has the submodule $N_{1} = M[X]/ L_{1}$ . One uses Lemma 3.1 to lift $\omega _{\tau }^{[2]}$ to a module $L_{2}$ in such a way that the dimension of $L_{2} \cap M[X]$ in generic fiber is $\max (d_{1},d_{2})$ . Then one considers $M_{2} := X^{-1} L_{2} / L_{2}$ . One has the submodules $N_{1}^{\prime } = (M[X] + L_{2})/L_{2}$ and $N_{2}^{\prime } = \left (M\left [X^{2}\right ]\cap X^{-1}L_{2} + L_{2}\right ) / L_{2}$ . Again, one uses Lemma 3.1, and gets a lift $L_{3}$ of $\omega _{\tau }^{[3]}$ such that in generic fiber the dimensions of $L_{3} \cap M[X]$ and $L_{3} \cap M\left [X^{2}\right ]$ are, respectively, $\max (d_{1},d_{2}, d_{3})$ and $\max (d_{1}+d_{2}, d_{1} + d_{3}, d_{2}+d_{3})$ .

First we prove two auxiliary lemmas.

Lemma 3.5. There exists a lift $\left (\widetilde {\omega _{\tau }^{[i]}}\right )$ of $\omega _{\tau }^{[1]} \subset \dotsb \subset \omega _{\tau }^{[e]} \subset D$ in $D\otimes k[[t]]$ satisfying the Pappas–Rapoport conditions and such that

$$ \begin{align*} \dim \left(\widetilde{\omega_{\tau}^{[i]}}\right)\left[X^{j}\right] = \max_{ 0 <k_{1} < \dotsb < k_{j} \leq i} d_{k_{1}} + \dotsb + d_{k_{j}}. \end{align*} $$

Proof. Indeed, by induction we have proven the result for $i = 1,2,3$ . Suppose it is true for $i \geq 1$ , and denote

$$ \begin{align*} M_{i} = X^{-1}L_{i}/L_{i}, \qquad N_{j} = \left(M\left[X^{j}\right]\cap X^{-1}L_{i} + L_{i}\right)/L_{i}, \quad \forall j \leq i, \qquad \overline L = \omega_{\tau}^{[i+1]}/\omega_{\tau}^{[i]}. \end{align*} $$

A direct calculation shows that $\dim _{k[[t]]} M_{i} = h$ , $\dim _{k} \overline L = d_{i+1}$ , and $\dim _{k[[t]]} N_{j} = h - \dim \left (L_{i}\left [X^{j}\right ]\backslash L_{i}\left [X^{j-1}\right ]\right )$ . We use Lemma 3.1 with these data to find L such that

$$ \begin{align*} \dim_{k[[t]]} L\cap N^{j} = \max\left(0,d_{i+1} - \dim \left(L_{i}\right.\left[X^{j}\right]\backslash L_{i}\left[X^{j-1}\right]\right). \end{align*} $$

Let $\widetilde {\omega _{\tau }^{[i]}}$ be the preimage of L via $X^{-1}L_{i} \longrightarrow M_{i}$ ; we thus have

$$ \begin{align*} \dim \widetilde{\omega_{\tau}^{[i]}}\left[X^{j}\right] = \dim L_{i}\cap N^{j} + \dim L_{i}\left[X^{j}\right] &= \max\left(L_{i}\left[X^{j}\right], d_{i+1} + \dim L_{i}\left[X^{j-1}\right]\right) \\ &= \max_{ 0 <k_{1} < \dotsb < k_{j} \leq i+1} d_{k_{1}} + \dotsb + d_{k_{j}}. \end{align*} $$

Thus the lemma is proved.

Lemma 3.6. Let S be a $\kappa $ -scheme and $\omega ^{[e]} \subset \left (\mathcal O_{S} \otimes _{\mathcal O_{F^{ur}}} \mathcal O_{F}\right )^{h}$ be a sub- $\mathcal O_{S} \otimes \mathcal O_{F}$ -module. Then $\omega ^{[e]}$ is in the generalized Rapoport locus for the datum $(d_{i})_{1 \leq i \leq e}$ (that is, $\operatorname {\mathrm {Hdg}}(\omega ^{e}) = \operatorname {\mathrm {PR}}(d_{i})$ ) if and only if for all j,

$$ \begin{align*} \dim \omega^{[e]}\left[X^{j}\right] = \max_{ 0 <k_{1} < \dotsb < k_{j} \leq e} d_{k_{1}} + \dotsb + d_{k_{j}}. \end{align*} $$

Proof. As the two properties are independent of the ordering of the values $(d_{i})$ , simplify $d_{1} \geq d_{2} \geq \dotsb \geq d_{e}$ . Then the last proposition means $\dim \omega ^{[e]}\left [X^{j}\right ] = d_{1} + \dotsb + d_{j}$ , which means exactly $\operatorname {\mathrm {Hdg}}(i) = \operatorname {\mathrm {PR}}(i)$ .

Reducing inductively the datum given in Lemma 3.5 to $k[[t]]/(t^{n})$ , one gets a map by Grothendieck and Messing:

$$ \begin{align*} \operatorname{\mathrm{Spf}}(k[[t]]) \longrightarrow X_{\kappa}. \end{align*} $$

(Note that $k[[t]]/(t^{n}) \longrightarrow k[[t]]\left (t^{n-1}\right )$ is endowed with nilpotent divided powers.) But as $X_{\kappa }$ is a scheme, this induces a map $\widetilde {x} : \operatorname {\mathrm {Spec}}(k[[t]]) \longrightarrow X_{\kappa }$ , generizing our point x, and such that in generic fiber the module $\widetilde {x}^{*}\omega _{\tau }[1/t]$ is given by $\widetilde {\omega _{\tau }}[1/t]$ and thus lies in the generalized Rapoport locus, by Lemma 3.6.

3.2.2 Case (C)

Proof. One keeps the same notations as in the previous section. One lifts the module $\omega _{\tau }$ in M. But in this case, the module M has a perfect pairing, and one needs to consider totally isotropic lifts.

One starts by considering the module $M_{1} := M[X]$ . This module has a perfect pairing $h_{1}$ (induced by the one on M; see §2.3.3 and Remark 2.14), and we can lift the module $\omega _{\tau }^{[1]}$ to a module $L_{1} \subset M_{1}$ , which is still totally isotropic for $h_{1}$ . Then one considers the module $M_{2} := X^{-1} L_{1} / L_{1}$ . Since $L_{1}$ is totally isotropic in $M_{1}$ for $h_{1}$ , the pairing $h_{2}$ induces a pairing on $M_{2}$ . Using Lemma 3.2, one takes a lift $L_{2}$ of $\omega _{\tau }^{[2]}$ , such that $L_{2} / L_{1}$ is totally isotropic in $M_{2}$ for $h_{2}$ and disjoint from $M[X]/L_{1}$ in generic fiber.

One repeats this process and gets lifts $L_{1} \subset L_{2} \subset \dotsb \subset L_{e}$ . In generic fiber, the multiplication by X is an isomorphism between $L_{i+1} / L_{i}$ and $L_{i} / L_{i-1}$ for every $1 \leq i \leq e-1$ , and the lift $L_{e}$ of $\omega _{\tau }$ is thus generically free as an $\mathcal O_{F} \otimes _{\tau } k[[t]] = k[[t]][X]/(X^{e})$ -module. As before, by Grothendieck and Messing this leads to an algebraizable map

$$ \begin{align*} \operatorname{\mathrm{Spf}}(k[[t]]) \longrightarrow X_{\kappa} \end{align*} $$

generizing x and whose generic fiber lies in the Rapoport locus.

3.3 Futher strata

As the Hodge stratification is constructed using the nilpotent cone of some $\operatorname {\mathrm {GL}}_{n}$ , for which the stratification is a strong stratification, we can investigate the same question for X. First recall the definition of a strong stratification:

Definition 3.7. Let X be a topological space. A (weak) stratification of X with respect to a partially ordered set $(I, \leq )$ is a decomposition

$$ \begin{align*} X = \coprod_{i \in I} X_{i} \end{align*} $$

such that $\overline {X_{i}} \subset \coprod _{j \leq i} X_{j}$ . A (weak) stratification is a strong stratification if, moreover,

$$ \begin{align*} \overline{X_{i}} = \coprod_{j \leq i} X_{j}. \end{align*} $$

Example 3.8.

1. 1. In the case of an unramified PEL datum, the Hodge stratification of $X_{\kappa }$ is a strong stratification (this is trivial, as there is only one stratum).

2. 2. Still in the case of an unramified PEL datum, the Newton stratification and the Ekedahl–Oort stratification of $X_{\kappa }$ are strong stratifications (see [25, Theorem 2] and [7, Theorem 1.1]).

Proposition 3.9. In general, the Hodge stratification is not a strong stratification.

Proof. This has nothing to with do with abelian varieties but rather with the space of partial flags on a fixed space, together with a nilpotent operator (called Spaltenstein varieties). Let K be any field, $V = K^{6}$ , and suppose we are given a full flag

$$ \begin{align*} 0 \subset V_{1} \subset V_{2} \subset V_{3} = V, \end{align*} $$

with $\dim _{K} V_{i} = 2i$ . Suppose moreover that there is $\pi \in \operatorname {\mathrm {End}}_{K}(V)$ such that $\pi (V_{i}) \subset V_{i-1}$ for all $i \geq 1$ . This corresponds to (the local model of) a Pappas–Rapoport datum (AL) with $e = 3$ and $d_{1} = d_{2} = d_{3} = 2$ . Fix a basis $e_{1},\dotsc ,e_{6}$ of V such that $e_{1},e_{2}$ is a basis of $V_{1}$ and $e_{1},\dotsc ,e_{4}$ a basis of $V_{2}$ . Then set the following choices for $\pi $ in this basis:

$$ \begin{align*} \pi_{1} = \left( \begin{array}{ccc} 0 & I_{2} & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right) \quad \text{and} \quad \pi_{2} = \left( \begin{array}{cc} 0 & \left( \begin{array}{ccc} 1 &0 & 0 \\ 0 & 1 & 0\\ 0 & 0 & 1 \end{array} \right) \\ 0 & 0 \end{array} \right). \end{align*} $$

In both cases, $\pi $ satisfies the condition of a Pappas–Rapoport datum, with $\pi _{j}^{2} = 0$ . The Hodge polygons are associated with the partitions $(4,2)$ and $(3,3)$ of $6$ , and $(3,3) \leq (4,2)$ , but there exists no deformation from $((V_{i}),\pi _{1})$ to a space with Hodge polygons $(3,3)$ which satisfies the Pappas–Rapoport condition. Indeed, if it were the case, then the Hodge polygon of $(\pi _{1})_{\rvert V_{2}}$ would descend by generization, and thus would remain the same, and thus the $\pi $ -torsion of the deformation should intersect $V_{2}$ only along $V_{1}$ , and $\pi $ would send $V_{2}$ surjectively to $V_{1}$ . Now that means, as $\pi $ sends $V_{3}$ to $V_{2}$ and $\pi ^{2}$ sends $V_{3}$ to $0$ (as it is of Hodge polygon $(3,3)$ ), $\pi $ sends V to $V_{1}$ , and thus the kernel of $\pi $ is of rank $4$ – a contradiction.

Denote X the moduli space of all possible $((V_{i}),\pi )$ with $d_{i} = 2$ and $\pi (V_{i}) \subset V_{i-1}$ . Denote by $X_{(3,3)}$ the Hodge stratum corresponding to the Hodge polygon of partition $(3,3)$ and by $X_{(4,2)}$ the analogous one. If $\overline {X_{(3,3)}} \supset X_{(4,2)} \ni x = ((V_{i}),\pi _{1})$ , take C an irreducible component of $X_{(3,3)}$ passing through x and look at the local ring of C at x, $\mathcal O_{C,x}$ . This induces a generization of x such that the Hodge polygon is above $(3,3)$ and under $(4,2)$ , and by the previous calculation cannot be equal to $(3,3)$ . Thus it is generically $(4,2)$ too. This is true for all components C, and thus locally at x, $X_{(4,2)}$ is an open component of $X_{(3,3)}$ , and thus $x \not \in \overline {X_{(3,3)}}$ .

We still hope to construct a strong stratification on $X_{\kappa }$ , by ‘cutting’ in parts the Hodge strata. Unfortunately, the situation gets very complicated when the ramification index e grows. One has, however, the following result when $e=2$ . Recall that in polarized cases – (C), (AU), and (AR) – we consider only symmetric data $(\nu _{\tau })_{\tau }$ for the Hodge strata.

Proposition 3.10. If $e \leq 2$ and every $\pi $ falls into case (AL), (AU), or (C), then the Hodge stratification is a strong stratification.

Proof. If $e =1$ , there is only one Hodge stratum and everything is trivial. If $e=2$ , we can assume $d_{\tau ,1} \geq d_{\tau ,2}$ . Indeed, in case (C) there is an equality, and in cases (AL) or (AU) considering the dual group (which coincides with $G^{(s)}$ in case (AU), and thus we can in case (AU) consider only half of the embeddings $\tau $ , as we did in the proof of Theorem 2.30), we can reduce to this case. The $\tau $ -Hodge polygon is given by two integers $a_{\tau ,1} \geq a_{\tau ,2}$ such that if $x \in X_{\kappa }(k)$ , then $\omega _{\tau ,x}[\pi ]$ is a k-vector space of dimension $a_{\tau ,1}$ and $\omega _{\tau ,x}$ is of dimension $a_{\tau ,1}+a_{\tau ,2}$ . As $\omega _{\tau ,x}^{[1]}$ is of dimension $d_{\tau ,1}$ and of $\pi $ -torsion by the Pappas–Rapoport condition, we have that $d_{\tau ,1} \leq a_{\tau ,1}$ . Thus all the possible $\tau $ -Hodge polygons are classified by couples $\left (a_{\tau ,1},a_{\tau ,2}\right )$ with $a_{\tau ,1} \geq d_{\tau ,1}$ and $a_{\tau ,1} + a_{\tau ,2} = d_{\tau ,1} + d_{\tau ,2}$ . Moreover, if $\left (a_{\tau ,1},a_{\tau ,2}\right )$ and $\left (b_{\tau ,1},b_{\tau ,2}\right )$ are two $\tau $ -Hodge polygons, the former is above the latter if and only if $a_{\tau ,1} \geq b_{\tau ,1}$ . The generalized Rapoport locus corresponds to $\left (a_{\tau ,1},a_{\tau ,2}\right ) = \left (d_{\tau ,1},d_{\tau ,2}\right )$ . Thus we will prove that given any point $x \in X(k)$ with $\tau $ -Hodge polygon $\left (a_{\tau ,1},a_{\tau ,2}\right )$ not in the generalized Rapoport locus, there is a deformation to $k[[t]]$ with $\tau $ -Hodge polygon $\left (a_{\tau ,1}-1,a_{\tau ,2}+1\right )$ . Fix such a point, and fix a $k[\pi ]/\pi ^{2}$ basis of $H^{1}_{dR,\tau }$ , $e_{1},\dotsc ,e_{h}$ such that $\pi e_{1},\dotsc , \pi e_{d_{1}}$ is a basis of $\omega _{\tau }^{[1]}$ over k and $e_{1},\dotsc ,e_{r},\pi e_{d_{1}+1},\dotsc ,\pi e_{d_{1}+s}$ (necessarily with $r \leq d_{1}$ ) induces a basis of $\omega _{\tau }/\omega _{\tau }^{[1]}$ . Then $r +s = d_{2}$ and $a_{\tau ,1} = d_{1} + s$ . As this point is not in the generalized Rapoport locus, we have $s> 0$ , and thus $r < d_{1}$ . Then set in $H^{1}_{dR,\tau } \otimes _{k[\pi ]/\pi ^{2}} \left (k[\pi ]/\pi ^{2}\right )[[t]]$

$$ \begin{align*} \widetilde{\omega_{\tau}}^{[1]} &= k[[t]](\pi e_{1},\dotsc,\pi e_{d_{1}}) \quad \text{and} \quad \widetilde{\omega_{\tau}} = \widetilde{\omega_{\tau}}^{[1]}\\&\quad + k[[t]]\left(e_{1},\dotsc,e_{r},e_{d_{1}+s} + te_{r+1},\pi e_{d_{1}+1},\dotsc,\pi e_{d_{1}+s-1}\right). \end{align*} $$

In case (C), we need to choose a lift of $\operatorname {\mathrm {Vect}}\left (e_{1},\dotsc ,e_{r},\pi e_{d_{1}+1},\dotsc ,\pi e_{d_{1}+s}\right )$ in $\pi ^{-1}\widetilde {\omega _{\tau }}^{[1]}/\widetilde {\omega _{\tau }}^{[1]}$ (which has a perfect pairing), whose intersection with $H^{1}_{dR,\tau }[\pi ]/\widetilde {\omega _{\tau }}^{[1]}$ is of dimension $s-1$ . But we can quotient further by $\operatorname {\mathrm {Vect}}(e_{1},\dotsc ,e_{r})$ (which is totally isotropic), and we are reduced to the case $r =0$ . In this case, this is as in the proof of the first part of Lemma 3.2, taking the matrix A to be symmetric of rank $1$ .

Then, as in the proof of §3.2.1, there is a lift of x whose Hodge filtration is given by $\widetilde {\omega _{\tau }}$ . Moreover,

$$ \begin{align*} \dim_{k((t))^{perf}} \left(\widetilde{\omega_{\tau}}\otimes k((t))^{perf}\right)[\pi] = d_{1}+s-1.\\[-36pt] \end{align*} $$

Remark 3.11. The calculation of Appendix A still shows that even when $e =2$ , the analogous result in case (AR) is false.

4 The $\mu $ -ordinary locus

4.1 Density of the $\mu $ -ordinary locus

The goal of this section is to show the following theorem:

Theorem 4.1. Let $\pi $ be a prime as in §2.5, and assume that it falls into case (AL), (AU), or (C). Then the $\mu $ -ordinary locus (for $\pi $ ) $X^{\nu = PR}$ inside X is dense.

To prove the theorem, we will once again follow the strategy of deformations of the p-divisible group (by Serre and Tate’s theorem), one prime at a time in each case. Moreover, by Theorem 3.3, we only need to deform p-divisible groups that are already in the generalized Rapoport locus. Our main tool is proposition 2.33. From now on we will consider only lifts of a crystal in the sense of [26, §3.2.3]. We thus call a deformation of a crystal over k a display over $k[[t]]$ of the form $P_{\alpha }$ for some $\alpha \in \operatorname {\mathrm {Hom}}_{W(k[[t]])}(P, W(tk[[t]])P)$ in the notation of [26] (with P the base change to $k[[t]]$ of our crystal). A generization will then be the generic fiber of a deformation.

Remark 4.2. In case (AR), our argument breaks down. It is likely that in general the $\mu $ -ordinary locus is not dense in every irreducible component, similar to the Hilbert case at Iwahori level described in [24]. This is confirmed by the case of $U(1,n-1)$ , calculated by [13], whose calculation is shown in Appendix A.

Moreover, for cases (AU) and (C) we will have to use slightly more adapted polarizations. Indeed, the objects considered have a natural polarization, compatible in a certain sense to the additional action of a ring $\mathcal O$ , but not $\mathcal O$ -Hermitian (or bilinear). As the methods used in this section are of purely local nature, we can forget about the PEL datum used to define the variety X, and thus we will freely reuse notation; in particular, F will denote Frobeniuses in this section. Denote by $L/L^{+}$ the extension of (local) fields at the prime considered (thus $L = L^{+}$ in case (C)), and denote by $s \in \operatorname {\mathrm {Gal}}\left (L/L^{+}\right )$ the nontrivial automorphism (if it exists; $s=\operatorname {\mathrm {id}}$ otherwise). We denote by $e,f$ the ramification index and residual degree of $L^{+}$ , and by $\kappa $ the residue field of L. Denote by $\operatorname {\mathrm {Diff}}^{-1} = \operatorname {\mathrm {Diff}}^{-1}_{L}$ the inverse different of L, and $\mathcal O = \mathcal O_{L}$ , and $\operatorname {\mathrm {tr}} : \operatorname {\mathrm {Diff}}^{-1} \longrightarrow \mathbb {Z}_{p}$ . We call a p-divisible $\mathcal D$ -module (resp., a $\mathcal D$ -crystal, a $\mathcal D$ -display) a p-divisible group (resp., a crystal, a display) with extra structure depending on the situation (that is, an action of $\mathcal O$ , together with a polarization in case (AU) or (C) satisfying certain properties). In this section we can forget a bit about the Pappas–Rapoport data, as explained later, thanks to Theorem 3.3, and thus we will not make it appear in the datum $\mathcal D$ . In cases (AU) and (C), we have the following:

Proposition 4.3. Let k be a perfect field of characteristic p and G be a p-divisible $\mathcal D$ -module. Its Dieudonné module M, a free $W_{}(k)\otimes _{\mathbb {Z}_{p}}\mathcal O$ -module – together with two applications

$$ \begin{align*} V : M \longrightarrow M \quad \text{and} \quad F : M \longrightarrow M, \end{align*} $$

which are $\sigma ^{-1}$ (resp., $\sigma $ )-linear, satisfying $FV = VF = p\operatorname {\mathrm {id}}$ – is endowed with an s-anti-Hermitian $W_{\mathcal O}(k)$ -pairing

$$ \begin{align*} h : M \times M \longrightarrow W(k)\otimes \operatorname{\mathrm{Diff}}^{-1} \end{align*} $$

satisfying

$$ \begin{align*} h(x,Fy) = h(Vx,y)^{\sigma},\quad \forall x,y \in M. \end{align*} $$

Moreover, if $\langle .,.\rangle $ denotes the original alternating pairing on M, we have $\operatorname {\mathrm {tr}}_{F/\mathbb {Q}_{p}} h = \langle. ,.\rangle $ . Such an h is unique. In particular, in case (C), h is alternating. If $\mathcal P = \left (P,Q,F,V^{-1}\right )$ is a $\mathcal D$ -display over $k[[t]]$ with pairing $\langle ,\rangle $ , then there exists an s-anti-Hermitian $W(k[[t]])\otimes _{\mathbb {Z}_{p}}\mathcal O$ -pairing

$$ \begin{align*} h : P \times P \longrightarrow W(k[[t]])\otimes_{\mathbb{Z}_{p}} \operatorname{\mathrm{Diff}}^{-1} \end{align*} $$

satisfiying $\operatorname {\mathrm {tr}} h = \langle ,\rangle $ and $^{v}h\left (V^{-1}x,V^{-1}y\right )= h(x,y)$ for all $x,y \in Q$ , and which is moreover compatible with the previous construction over k and $k((t))^{perf}$ .

Proof. The existence of h, the s-anti-Hermitian satisfying $\operatorname {\mathrm {tr}} h = \langle , \rangle $ , is [14, Lemma 1.1.4.5] for $R = W(k)$ or $R=W(k[[t]])$ and $R_{0} = \mathbb {Z}_{p}$ . Thus it suffices to prove the compatibility with F and V. But for all $x,y \in M, o \in \mathcal O$ ,

$$ \begin{align*} \operatorname{\mathrm{tr}} o h(x,Fy) = \operatorname{\mathrm{tr}} h(ox,Fy) = \langle ox,Fy\rangle = \langle Vox,y\rangle^{\sigma}& = \langle oVx,y\rangle^{\sigma} \\ &\qquad= \operatorname{\mathrm{tr}}(h(oVx,y)^{\sigma}) = \operatorname{\mathrm{tr}}(o h(Vx,y)^{\sigma}). \end{align*} $$

Thus, by [14, Corollary 1.1.4.1] we have $h(x,Fy) = h(Vx,y)^{\sigma }$ , for all $x,y \in M$ . Similarly, for a display $\mathcal P$ ,

$$ \begin{align*} \operatorname{\mathrm{tr}} \left(o \cdot^{v}h\left(V^{-1}x,V^{-1}y\right)\right) &= \operatorname{\mathrm{tr}}\left({}^{v}h\left(V^{-1}(ox),V^{-1}y\right)\right) ={}^{v}\operatorname{\mathrm{tr}}\left(h\left(V^{-1}(ox),V^{-1}y\right)\right) \\ &\qquad={}^{v}\left\langle V^{-1}(ox),V^{-1}y\right\rangle = \langle ox,y\rangle = \operatorname{\mathrm{tr}}(h(ox,y))=\operatorname{\mathrm{tr}}(o h(x,y)), \end{align*} $$

for all $o \in \mathcal O$ , and thus $^{v}h\left (V^{-1}x,V^{-1}y\right )= h(x,y)$ . The compatibility is obvious when looking at the display associated to a Dieudonné module (and unicity).

Unfortunately, it will not always be possible to assume h alternating, even if $\langle , \rangle = \operatorname {\mathrm {tr}} h$ is, as shown by the following example:

Example 4.4. Let $\mathbb {C} = \mathbb {R}^{2}$ endowed with the $\mathbb {R}$ -linear alternating pairing $\mathbb {C} \times \mathbb {C} \longrightarrow \mathbb {R}$ given in basis $(1,i)$ of $\mathbb {C}$ by

$$ \begin{align*} \left( \begin{array}{cc} 0 & -1 \\ 1 & 0 \end{array} \right), \end{align*} $$

whose h is given by $\frac {i}{2} z_{1} \overline { z_{2}}$ .

But in case (C), as $s \in \operatorname {\mathrm {Gal}}\left (L/L^{+}\right )$ satisfies $s = \operatorname {\mathrm {id}}$ , we have that h is antisymmetric, and thus alternating (as $\operatorname {\mathrm {Char}}(W_{\mathcal O}(k)) \neq 2$ ).

4.2 Case (AL)

Let $x_{G}$ be a point in the Rapoport locus, with values in a perfect field k, corresponding to a group G, and fix $\tau _{0}$ . Note that any generization of $x_{G}$ still lives in the Rapoport locus. Let M be the Dieudonné crystal of G over k.

Lemma 4.5. In order to prove the density result, one can assume that for all $\tau $ , the first slope of the Hodge polygon is $0$ . In particular, for all $\tau $ there exists $x_{\tau } \in M_{\tau }$ such that

$$ \begin{align*} F_{\tau}(x_{\tau}) \not\equiv 0 \pmod{\pi}. \end{align*} $$

Proof. Indeed, otherwise denote by $a_{\tau }$ the first slope for all $\tau $ ; this means that $F_{\tau } : M_{\sigma ^{-1}\tau }\longrightarrow M_{\tau }$ is divisible by $\pi ^{a_{\tau }}$ on the Dieudonné module of G. Denote $F^{0}_{\tau } = \frac {1}{\pi ^{a_{\tau }}}F_{\tau }$ and $V^{0}_{\tau } = \pi ^{a_{\tau }}V_{\tau }$ . Denote by $G^{\prime }$ the p-divisible group associated to $\left (M,F^{0},V^{0}\right )$ . Then it is easily checked that the association $G \mapsto G^{\prime }$ is bijective on a p-divisible group with $\mathcal O$ -action with fixed $\tau $ -Hodge polygons on the source to fixed $\tau $ -Hodge polygons where each $\tau $ -slope is decreased by $a_{\tau }$ . Moreover, this is compatible with display deformations to $k[[t]]$ and specialization to $k((t))^{perf}$ in Proposition 2.33. Thus we only need to deform $G^{\prime }$ , whose first Hodge slope is $0$ for all $\tau $ .

Thus, assume the first slope of $\operatorname {\mathrm {Hdg}}_{\tau }(G)$ is $0$ for all $\tau $ . If the first slope for $Newt(x_{G})$ is also $0$ , then there is a splitting

$$ \begin{align*} G = G^{0} \times G^{\prime}, \end{align*} $$

where the Newton polygon of $G^{\prime }$ does not have a slope $0$ [3, Théorème 1.3.2]. If we can find a deformation of $G^{\prime }$ which is $\mu $ -ordinary, then we are finished. Thus up to exchanging G by $G^{\prime }$ , we can suppose that the first slope of $Newt(G)$ is nonzero and proceed by induction on the height of G.

Set $x \in M_{\tau _{0}}$ such that $F(x) \not \equiv 0 \pmod {\pi }$ (this is possible by the preceding lemma). Let $i(x)$ be the minimal integer such that

$$ \begin{align*} F^{i(x)}(x) \equiv 0 \pmod \pi. \end{align*} $$

As the Newton polygon of G does not have a zero slope, there exists such an $i(x)$ . Denote by $i(G)$ (or $i(x_{G})$ ) the maximum of $i(x)$ , for $x \in M_{\tau _{0}}$ . By the preceding, we know that $i(G) \geq 2$ .

Lemma 4.6. There exists a generization $x_{G}^{\prime }$ of $x_{G}$ such that $i\left (x_{G}^{\prime }\right )> f$ .

Proof. Suppose $i=i(x_{G}) \leq f$ . Thus $F^{i}(x) \equiv 0 \pmod \pi $ . By the previous lemma, there is $x_{i-1} \in M_{\sigma ^{i-1}\tau }$ such that $F(x_{i-1}) \not \equiv 0 \pmod \pi $ . As $y = F^{i-1}(x) \not \equiv 0 \pmod \pi $ but $F(y) \equiv 0 \pmod \pi $ , $(x_{i-1},y)$ is a linearly independent family in $M_{\sigma ^{i-1}\tau }/\pi M_{\sigma ^{i-1}\tau }$ . Define a homomorphism of $M/\pi M$ by

$$ \begin{align*} N_{\tau^{\prime}} = 0, \quad \forall \tau^{\prime} \neq \sigma^{i-1}\tau, \quad \text{and} \quad N_{\sigma^{i-1}\tau}y = x_{i-1}, \qquad N_{\sigma^{i-1}\tau}x_{i-1} = 0, \end{align*} $$

and extend N by zero on a complementary basis of $M_{\sigma ^{i-1}\tau }$ . Denote by N any nilpotent lift of N to M. Define $D_{N}$ as the extension of M to $W(k[[u]])$ given by N as in [26] (see also [28]). We can calculate

$$ \begin{align*} F_{N}^{i}(x) {\,=\,} F_{N}\left(F_{N}\left(F_{N}^{i-2}(x)\right)\right) {\,=\,} F_{N}\left(F_{N}\left(F^{i-2}(x)\right)\right) &= F_{N}(y\otimes 1 + x_{i-1} \otimes u) {\,\equiv\,} F_{N}(x_{i-1}\otimes u) \\ &\qquad\equiv F(x_{i-1})\otimes u \not\equiv 0 \pmod \pi. \end{align*} $$

Thus over $W\left (k((u))^{perf}\right )$ , the display $D_{N}$ corresponds to a p-divisible group $G^{\prime }$ such that $i(G^{\prime })> i$ , by Proposition 2.33. By induction, we get the result.

Lemma 4.7. There exists a deformation of $x_{G}$ such that the generic fiber is not infinitesimal.

Proof. By the previous lemma, we can assume $F^{f}(x) \not \equiv 0 \pmod \pi $ . If G is not itself infinitesimal, let $r_{0}$ be the minimal integer such that

$$ \begin{align*} F^{r_{0}f}(x) \equiv 0 \pmod \pi. \end{align*} $$

The family $\left (x,F^{f}(x),\dotsc ,F^{\left (r_{0}-1\right )f}(x)\right )$ is linearly independent mod $\pi $ . Indeed, suppose we are given

$$ \begin{align*} \lambda_{0}x + \lambda_{1} F^{f}(x) + \dotsb + \lambda_{r_{0}-1}F^{\left(r_{0}-1\right)f}(x) \equiv 0 \pmod \pi, \end{align*} $$

and denote by i the smallest integer such that $\lambda _{i} \neq 0 \pmod \pi $ . Then

$$ \begin{align*} F^{if}(x) = \lambda_{i}^{-1}\left(\lambda_{i+1} F^{(i+1)f}(x) + \dotsb + \lambda_{r_{0}-1}F^{\left(r_{0}-1\right)f}(x)\right), \end{align*} $$

and thus $F^{\left (r_{0}-i-1\right )f}\left (F^{if}(x)\right ) = F^{\left (r_{0}-1\right )f}(x) \equiv 0 \pmod \pi $ , which is impossible. Set N such that

$$ \begin{align*} N_{\tau^{\prime}} = 0, \quad \forall \tau^{\prime} \neq \tau, \qquad Nx = NF^{if}(x) = 0, \quad \forall i \neq 1, \qquad\text{and}\qquad NF^{f}(x) = x. \end{align*} $$

Set $D_{N},F_{N}$ the associated display over $W(k[[u]])$ which reduces to $(M,F)$ . We calculate

$$ \begin{align*} F_{N}^{f}(x) = F^{f}(x) + uNF^{f}(x) = F^{f}(x) + ux, \end{align*} $$

and more generally,

$$ \begin{align*} F_{N}^{if}(x) = F^{if}(x) + uF^{(i-1)f}(x_{0}) + \dotsb + u^{i}x_{0}. \end{align*} $$

In particular,

$$ \begin{align*} F_{N}^{r_{0}f}(x) = 0 + uF^{\left(r_{0}-1\right)f}(x) + \dotsb + u^{r_{0}}x \neq 0 \pmod \pi. \end{align*} $$

Thus, the base change to $W\left (k((u))^{perf}\right )$ of $D_{N}$ satisfies $r_{0}(D_{N})> r_{0}(M)$ . By induction, we can assume that $r_{0}> \dim D$ , and thus that D is not infinitesimal.

By induction on the number of Newton and Hodge slopes of G that are not equal (see Lemma 4.5), we get a chain of generizations starting to $x_{G}$ and ending at a $\mu $ -ordinary point.

Corollary 4.8. In case (AL), the $\mu $ -ordinary locus $X_{\mu -ord}^{PR}$ is dense.

4.3 Case (AU)

In the linear case, as our deformation by N will be polarized – which means that we deform at the same time $F_{\tau }$ and $F_{\overline \tau }$ – these two deformations might cancel out. Thus, we will work almost as in [26], by finding a deformation sequence (which ensures that there will be no cancellations in calculating $F^{2f}_{N}$ ). We denote $\overline M = M\pmod \pi $ .

Definition 4.9. $(x_{\tau }) \in \overline M = \bigoplus _{\tau } \overline M_{\tau }$ is a deformation sequence if

1. 1. $x_{\tau } \in \overline M_{\tau }$ for all $\tau $ ,

2. 2. $Fx_{\tau } \not \equiv 0 \pmod \pi $ for all $\tau $ ,

3. 3. if $F^{2}x_{\tau } \not \equiv 0 \pmod \pi $ , then $Fx_{\tau } = x_{\sigma \tau }$ .

We will use Proposition 2.33 to deform a given p-divisible $\mathcal O$ -module to a $\mu $ -ordinary one. By Theorem 3.3 we can suppose that the p-divisible group G associated to a point $x_{G} \in X^{PR}$ we start with is in the Rapoport locus – that is, $\operatorname {\mathrm {Hdg}}(x_{G}) = \operatorname {\mathrm {PR}}(x_{G})$ . In particular, the first slope of $\operatorname {\mathrm {Hdg}}(x_{G})$ coincides with that of $\operatorname {\mathrm {PR}}(x_{G})$ . We will construct the deformation by induction, using the fact that if G is not bi-infinitesimal, we can decompose

$$ \begin{align*} G = G^{et} \times G^{bi} \times G^{m}, \end{align*} $$

where $G^{et}$ is étale, $G^{m}$ is multiplicative, and $G^{bi}$ is bi-infinitesimal. If $G^{bi}$ is $\mu $ -ordinary, then so is G, and thus we only need to deform $G^{bi}$ . From now on, suppose G is bi-infinitesimal; we will prove that there exists a deformation of (some modification of) G that is polarized but not necessarily bi-infinitesimal generically. By induction on the Newton polygon of the bi-infinitesimal part, we will then deduce that we can deform G to a $\mu $ -ordinary p-divisible group. Just like in the beginning of §4.2, we can suppose that the first slope of $\operatorname {\mathrm {Hdg}}_{\tau }(x_{G})$ is $0$ for all $\tau $ , but here there will be some minor complications, and we need to introduce the notion of a not necessarily parallel (NNP) $\mathcal O$ -crystal, and the version for displays.

Definition 4.10. Let $(a_{\tau })_{\tau \in \mathcal T}$ be a collection of nonnegative integers such that $a_{\tau } + a_{\overline \tau } \leq e$ for all $\tau $ . An NNP polarized $\mathcal O$ -crystal (of type (AU)) of amplitude $(a_{\tau })_{\tau }$ over a perfect field k is a tuple $(M,V,F,\iota ,h(\cdot ,\cdot ))$ , where the following are true:

1. 1. M is a free $W(k)$ module.

2. 2. F is $\sigma $ -linear and V is $\sigma ^{-1}$ -linear.

3. 3. $\iota : \mathcal O \longrightarrow \operatorname {\mathrm {End}}_{W(k)}M$ such that $F(\iota (x)m) = \iota (x)F(m)$ and $V(\iota (x)m) = \iota (x)V(m)$ for all $x \in \mathcal O, m \in M$ .

4. 4. The pairing

   $$ \begin{align*} h( \cdot, \cdot ) : M \times M \longrightarrow W(k)\otimes \operatorname{\mathrm{Diff}}^{-1} \end{align*} $$
   is perfect and anti-Hermitian, and satisfies
   $$ \begin{align*} \iota(x) h(m,n) = h( \iota(x)m,n) = h\left( m,\iota\left(\overline x\right)n\right), \quad \forall x \in \mathcal O,m,n \in M. \end{align*} $$

Thus we can decompose

$$ \begin{align*} M = \bigoplus_{\tau : L^{ur}\hookrightarrow \mathbb{C}_{p}} M_{\tau}, \end{align*} $$

and accordingly we have maps

$$ \begin{align*} F_{\tau} : M_{\sigma^{-1}\tau} \longrightarrow M_{\tau} \qquad \text{and} \quad V_{\tau} : M_{\tau} \longrightarrow M_{\sigma^{-1}\tau} \end{align*} $$

such that for all $\tau $ , we have $V_{\tau } F_{\tau } = p\pi ^{-a_{\tau }}\overline {\pi }^{-a_{\overline \tau }} \operatorname {\mathrm {Id}}_{M_{\sigma ^{-1}\tau }}$ , $F_{\tau } V_{\tau } = p\pi ^{-a_{\tau }}\overline {\pi }^{-a_{\overline \tau }} \operatorname {\mathrm {Id}}_{M_{\tau }}$ , and

$$ \begin{align*} h\left(F_{\tau} x_{\tau}, x_{\sigma\overline\tau}\right) = h\left(x_{\tau},V_{\overline\tau}x_{\sigma\overline\tau}\right)^{\sigma} \qquad \text{and}\qquad h\left(V_{\tau} x_{\sigma\tau}, x_{\overline\tau}\right) = h\left(x_{\sigma\tau},F_{\overline\tau}x_{\overline\tau}\right)^{\sigma^{-1}}. \end{align*} $$

The tuple of integers $(a_{\tau })_{\tau \in \mathcal T}$ is called the amplitude of $(M,F,V,\iota )$ .

Similarly, we can make the following analogous definition:

Definition 4.11. Assume $R = k[[t]]$ for a perfect field $k \supset \kappa $ . An NNP polarized $\mathcal O$ -display $\mathcal P$ over R of amplitude $(a_{\tau })_{\tau }$ is a quintuple $\left (P,Q,F,V^{-1},\iota ,h\right )$ such that the following are true:

1. 1. P is a locally free $W(R)\otimes _{\mathbb {Z}_{p}} \mathcal O$ -module via $\iota : \mathcal O \longrightarrow \operatorname {\mathrm {End}}_{W(R)} P$ .

2. 2. $Q \subset P$ is a $W(R) \otimes \mathcal O$ -submodule.

3. 3. $F : P \longrightarrow P$ and $V^{-1} : Q \longrightarrow P$ are $\sigma $ -linear and $V^{-1}$ is an epimorphism.

4. 4. $\iota : \mathcal O \longrightarrow \operatorname {\mathrm {End}}_{W(k)}M$ such that $F(\iota (x)m) = \iota (x)F(m)$ and $V^{-1}(\iota (x)n) = \iota (x)V(n)$ for all $x \in \mathcal O, m \in P, n \in Q$ .

5. 5. The pairing

   $$ \begin{align*} h( \cdot, \cdot ) : P \times P \longrightarrow W(R)\otimes \operatorname{\mathrm{Diff}}^{-1} \end{align*} $$
   is perfect and anti-Hermitian, and satisfies
   $$ \begin{align*} \iota(x) h(m,n) = h( \iota(x)m,n) = h\left( m,\iota\left(\overline x\right)n\right), \quad \forall x \in \mathcal O,m,n \in P. \end{align*} $$

Thus, we can decompose the action of $O^{ur} := \mathcal O_{L^{ur}}$ ,

$$ \begin{align*} P = \bigoplus_{\tau} P_{\tau} \qquad \text{and}\qquad Q = \bigoplus_{\tau} Q_{\tau} \end{align*} $$

with

$$ \begin{align*} F_{\tau} : P_{\sigma^{-1}\tau} \longrightarrow P_{\tau} \quad \text{and} \quad V^{-1} : Q_{\sigma^{-1}\tau} \longrightarrow P_{\tau}, \end{align*} $$

such that for all $\tau $ ,

$$ \begin{align*} I_{R} P_{\sigma^{-1}\tau} \subset \overline\pi^{a_{\overline \tau}} Q_{\sigma^{-1}\tau}, \end{align*} $$

$Q_{\tau }/I_{R} P_{\tau } \subset P_{\tau }/I_{R}P_{\tau }$ is locally a direct $W(R)$ -factor, and

$$ \begin{align*} V_{\tau}^{-1}(^{v}(w)x) = \overline\pi^{a_{\overline\tau}}\pi^{a_{\tau}}wF_{\tau}(x), \quad \forall x \in P_{\sigma^{-1}\tau}, w \in W(R). \end{align*} $$

Moreover, we ask that

$$ \begin{align*} {}^{v}h\left(V^{-1}x,V^{-1}y\right) = \overline{\pi}^{a_{\tau} + a_{\overline\tau}} h(x,y), \quad x \in Q_{\sigma^{-1}\overline\tau}, y \in Q_{\sigma^{-1}\tau}. \end{align*} $$

Example 4.12. A polarized $\mathcal O$ -crystal is a particular case of an NNP polarized $\mathcal O$ -crystal for which the amplitude is constant equal to $0$ – that is, $a_{\tau } =0$ for all $\tau $ – and similarly for a display. The Dieudonné module of a p-divisible group over a perfect field k with $\mathcal O$ -action, such that the $\tau $ -Hodge polygon of G [3] has first slope $a_{\tau }$ for all $\tau $ , can be modified to get an NNP crystal of amplitude $(a_{\tau })_{\tau }$ (see Proposition 4.14). The base change to $k[[t]]$ of (the display associated to) this NNP crystal is then an NNP display of amplitude $(a_{\tau })_{\tau }$ .

Definition 4.13. The $\tau $ -Hodge polygon of an NNP $\mathcal O$ -crystal $(M,F,V,\iota )$ is defined by

$$ \begin{align*} \operatorname{\mathrm{Hdg}}_{\tau}(M,F,V,\iota)(i) = \frac{c_{1} + \dotsb + c_{i}}{e}, \end{align*} $$

where $d = \dim _{\mathcal O} M_{\tau }$ , $M_{\tau }/FM_{\sigma ^{-1}\tau } \simeq \bigoplus _{i=1}^{d} W_{\mathcal O}(k)/\pi ^{c_{i}}W_{\mathcal O}(k)$ , and $c_{1} \leq c_{2} \leq \dotsb \leq c_{d}$ .

Proposition 4.14. Assume k is perfect and contains $\kappa $ , the residue field of L. To any polarized $\mathcal O$ -crystal $(M,F,V,\iota ,h)$ , we can associate an NNP polarized $\mathcal O$ -crystal

$$ \begin{align*} \left(M^{0},F^{0},V^{0},\iota^{0},h^{0}\right), \end{align*} $$

for which the first slope of $\operatorname {\mathrm {Hdg}}_{\tau }\left (M^{0},F^{0},V^{0},\iota ^{0}\right )$ is $0$ . Moreover, if $a_{\tau }$ denotes for all $\tau $ the first slope of $\operatorname {\mathrm {Hdg}}_{\tau }(M,F,V,\iota )$ , then $\left (M^{0},F^{0},V^{0},\iota ^{0},h^{0}\right )$ is of amplitude $(a_{\tau })_{\tau }$ . Moreover, we have $\left (M^{0},\iota ^{0},h^{0}\right ) = (M,\iota ,h)$ , and for all $\tau $ ,

$$ \begin{align*} F_{\tau} = \pi^{a_{\tau}}F_{\tau}^{0} \qquad \text{and}\qquad V_{\tau} = \overline{\pi}^{a_{\overline\tau}}V_{\tau}^{0}. \end{align*} $$

If $\left (P,Q,V^{-1},F,\iota ,h\right )$ is the base change of $(M,F,V,\iota ,h)$ to $k[[t]]$ (that is, the polarized $\mathcal O$ -deformation with $N = 0$ in Proposition 2.33), we can also associate to it another (NNP) display by the same operation, such that the Dieudonné modules of the base change over k and $k((t))^{perf}$ coincide with the previous construction for crystals. In particular, the association $M \longrightarrow \mathcal P \longrightarrow D := \mathcal P \otimes k((t))^{perf}$ does not change the Hodge polygons.

Proof. Indeed, M is a module over $W(k) \otimes _{\mathbb {Z}_{p}}\mathcal O$ , which we can split as $M = \bigoplus _{\tau } M_{\tau }$ over $\tau : \mathcal O^{ur} \hookrightarrow \mathcal O_{C}$ . As F is $\sigma $ -linear, $F_{\tau } : M_{\sigma ^{-1}\tau } \longrightarrow M_{\tau }$ , and $F_{\tau }$ is generically inversible, thus M is free as a $W(k)\otimes _{\mathbb {Z}_{p}}\mathcal O$ -module. By hypothesis, $F_{\tau }$ is divisible by $\pi ^{a_{\tau }}$ ; so set $F^{0}_{\tau } = \pi ^{-a_{\tau }} F_{\tau }$ . As $F_{\overline \tau }^{\vee } = V_{\tau }$ , we have that $V_{\tau }$ is also divisible by $\pi ^{a_{\overline \tau }}$ , thus by $\overline {\pi }^{a_{\overline \tau }}$ , and we can set $V_{\tau }^{0} = \overline {\pi }^{-a_{\overline \tau }}V_{\tau } : M_{\tau } \longrightarrow M_{\sigma ^{-1}\tau }$ . By Proposition 4.3, M is endowed with an Hermitian pairing

$$ \begin{align*} h : M \times M \longrightarrow W(k) \otimes_{\mathbb{Z}_{p}} \operatorname{\mathrm{Diff}}^{-1}, \end{align*} $$

such that $\operatorname {\mathrm {tr}} h = \langle .,.\rangle $ and $h(x,Fy) = h(Vx,y)^{\sigma }$ . Thus, for $x \in M_{\sigma ^{-1}\tau },y \in M_{\overline \tau }$ ,

$$ \begin{align*} h\left(\pi^{a_{\tau}}F_{\tau}^{0}x,y\right) =\overline{\pi}^{a_{\tau}}h\left(F_{\tau}^{0}x,y\right) \quad \text{and} \quad h\left(x,\overline{\pi}^{a_{\tau}}V_{\overline\tau}^{0}y\right) = \overline{\pi}^{a_{\tau}}h\left(x,V_{\overline\tau}^{0}y\right). \end{align*} $$

Thus,

$$ \begin{align*} h\left(F_{\tau}^{0}x,y\right) = h\left(x,V_{\overline\tau}^{0}y\right)^{\sigma}. \end{align*} $$

M is free over $W(k)\otimes \mathcal O$ , and we have $V^{0} M_{\tau } = \frac {1}{\pi ^{a_{\overline \tau }}} VM_{\tau }$ , and thus as a display over $W(k)$ ,

$$ \begin{align*} \left(V^{0}\right)^{-1} : V^{0} M_{\tau} \overset{\overline{\pi}^{a_{\overline\tau}}}{\longrightarrow} VM_{\tau} \overset{V^{-1}}{\longrightarrow} M_{\tau}. \end{align*} $$

Now if $\left (P,Q,V^{-1},F\right )$ is the base change of $(M,F,V,\iota ,h)$ , we have that P is free over $W(k[[t]])\otimes _{\mathbb {Z}_{p}}\mathcal O$ . We can thus decompose P and Q over $W(R) \otimes _{\mathbb {Z}_{p}}\mathcal O = \prod _{\tau } W(R)_{\tau }\otimes _{\mathcal O_{\tau }^{ur}}\mathcal O = \prod _{\tau } W(R)_{\tau }[\pi ]/(E_{\tau }(\pi )).$

We have the $\sigma $ -linear morphism

$$ \begin{align*} V_{\tau}^{-1} : Q_{\sigma^{-1}\tau} \longrightarrow P_{\tau}, \end{align*} $$

and we set $Q^{0} = VM^{0} \otimes _{W(k)}W(k[[t]])$ , with $\left (V_{\tau }^{0}\right )^{-1} = \left (1\otimes \overline \pi ^{a_{\overline \tau }}\right )V_{\tau }^{-1} : Q^{0}_{\sigma ^{-1}\tau } \overset {\overline \pi ^{a_{\overline \tau }}\otimes 1}{\longrightarrow } Q_{\sigma ^{-1}\tau } \overset {V_{\tau }^{-1}}{\longrightarrow } P_{\tau }$ . As $P_{\tau }$ is a free $W(R)_{\tau }[\pi ]/(E_{\tau }(\pi ))$ -module, we can also divide $F_{\tau ,P} = F_{\tau ,M} \otimes 1$ by $\pi ^{a_{\tau }}$ , and the rest is a simple verification. As $Q_{\sigma ^{-1}\tau } = \pi ^{a_{\overline \tau }} Q^{0}_{\sigma ^{-1}\tau }$ , we have that both the $\tau $ -Hodge polygons of M and $M^{\prime }$ are those of $Q^{0}$ – that is, of $M^{0}$ – with $\tau $ -slopes increased by $a_{\tau }$ .

Proposition 4.15. Denote by M a polarized $\mathcal O$ -crystal and by $M^{0}$ the nonparallel crystal associated to it above. Suppose we have a (polarized $\mathcal O$ -)deformation $\mathcal P^{0}$ of $M^{0}$ by an NNP display over $k[[t]]$ , given as in Proposition 2.33; then we have an associated (polarized $\mathcal O$ -)deformation $\mathcal P$ of M (with $\mathcal P$ a parallel display) such that $\left (\mathcal P \otimes _{k[[t]]} k((t))^{perf}\right )^{0} = \mathcal P^{0} \otimes _{k[[t]]} k((t))^{perf}$ . If, moreover, the crystal $D^{0} = \mathcal P^{0} \otimes k((t))^{perf}$ is not bi-infinitesimal, then we can decompose

$$ \begin{align*} D^{0} = D^{0,et} \times D^{0-bi} \times D^{0,mult}, \end{align*} $$

and we have an associated decomposition of $D = \mathcal P \otimes k((t))^{perf}$

$$ \begin{align*} D = D^{a-et} \times D^{bi} \times D^{a-mult}, \end{align*} $$

where $D^{a-et}$ is isoclinic of slope $\frac {1}{ef}\sum _{\tau } a_{\tau }$ and $D^{a-mult} = \left (D^{a-et}\right )^{\vee }$ . Moreover, the association $M \mapsto D$ does not change the Hodge polygons.

Proof. Let $\mathcal P^{0} = \left (P^{0},Q^{0},F^{0},\left (V^{0}\right )^{-1}\right )$ be the display associated to the base change of $M^{0}$ to $k[[t]]$ , and $\mathcal P$ the analogous display for M. Suppose we are given N a $W(k)$ -linear morphism of $P_{0} \otimes _{k[[t]]} k$ of square $0$ , which is $\mathcal O$ -linear and skew-symmetric. We thus have a deformation $\mathcal P^{0}_{N}$ by setting $F^{0}_{N} = (\operatorname {\mathrm {id}} + [t]N)F^{0}$ and $\left (V^{0}_{N}\right )^{-1} = (\operatorname {\mathrm {id}} + [t]N) \left (V^{0}\right )^{-1}$ . Let us set $\mathcal P_{N}$ the analogous deformation for $\mathcal P$ .

We claim that the crystal $\mathcal P_{N} \otimes _{k[[t]]} k((t))^{perf}$ satisfies $\left (\mathcal P_{N} \otimes _{k[[t]]} k((t))^{perf}\right )^{0} = \mathcal P^{0}_{N} \otimes _{k[[t]]} k((t))^{perf}$ . Indeed, as $F_{N} = (\operatorname {\mathrm {id}} + [t]N)F$ on $\mathcal P$ , and F is the pullback of F on $\underline M$ , we have that $F_{\tau }$ is divisible by $\pi ^{a_{\tau }}$ for all $\tau $ ; thus on $\mathcal P \otimes k((t))^{perf}$ , $(F_{N})^{0}$ is $\frac {1}{\pi ^{a_{\tau }}}F_{N} = (\operatorname {\mathrm {id}} + [t]N) \frac {1}{\pi ^{a_{\tau }}}F$ , as N is $\mathcal O$ -linear, and this is $\left (F^{0}\right )_{N}$ . The same is true for $V^{-1}$ , and thus we have the claim. Now if $D^{0}$ is not bi-infinitesimal, we have a decomposition

$$ \begin{align*} D^{0} = D^{0-et} \times D^{0-bi} \times D^{0,mult}, \end{align*} $$

where $D^{0,mult} \simeq \left (D^{0,et}\right )^{\vee }$ by the polarization on $D^{0}$ . But as $D^{0} = (D)^{0}$ , we have the asserted decomposition of D, and a direct calculation gives the slope of $D^{a-et}$ . As passing from $\mathcal P$ to $\mathcal P_{N}$ does not change the Hodge filtration, we have the assertion on Hodge polygons.

With this proposition we can explain our strategy. We start with a point in the Rapoport locus. As any deformation of it is still in the Rapoport locus – by Proposition 4.15, for example – we will be able to lift the Pappas–Rapoport filtration canonically (compare Theorem 3.3) and the deformation will still be in the Rapoport locus; thus we can forget about the Pappas–Rapoport datum for now. Then we will modify the crystal M of our p-divisible group by Proposition 4.15, deform this crystal inductively by a display, and ultimately the NNP crystal generization of $M^{0}$ will not be bi-infinitesimal anymore. Thus the associated deformation of M will split, and by induction on the Newton polygon we will be able to conclude. More precisely, we can always write

$$ \begin{align*} G = G^{1} \times G^{00} \times \left(G^{1}\right)^{D}, \end{align*} $$

by Hodge–Newton decomposition [3, Théorème 1.3.2], where $G^{1}$ is the biggest subgroup of G such that the Hodge and Newton polygons of $G^{1}$ are equal. Note that $G^{00}$ is still polarized (and in the Rapoport locus, if G is). The induction is on the height of $G^{00}$ .

There will be a slight issue in the case where $a_{\tau } + a_{\overline \tau } = e$ for all $\tau $ in our method. Fortunately, we have the following proposition:

Proposition 4.16. Let M be a polarized $\mathcal O$ -crystal such that for all $\tau $ , if $a_{\tau }$ denotes the first slope of $\operatorname {\mathrm {Hdg}}_{\tau }$ , we have $a_{\tau } + a_{\overline \tau } =e$ . Then the p-divisible group associated to M is $\mu $ -ordinary.

Proof. As $F_{\tau } V_{\tau } = p\operatorname {\mathrm {Id}}_{M_{\tau }}$ for all $\tau $ , this implies that $F^{0}_{\tau }$ and $V_{\tau }^{0}$ are, respectively, invertible $\sigma $ - and invertible $\sigma ^{-1}$ - $W(k)\otimes \mathcal O$ -morphisms. Thus the Newton slopes of $F^{0}_{\sigma ^{f-1}\tau }\circ \dotsb \circ F_{\tau }^{0}$ are all equal to $1$ , and thus $F_{\sigma ^{f-1}\tau }\circ \dotsb \circ F_{\tau }$ (which we write usually $F^{f}$ ) is isoclinic of slope $\frac {1}{e}\sum _{\tau } a_{\tau }$ . Thus G is isoclinic, and thus $\mu $ -ordinary.

Thus let $x_{G} \in X^{PR}$ be a point in the Rapoport locus, and denote by M the bi-infinitesimal part of its crystal. From now on, by Proposition 4.14, we can suppose that M is an NNP crystal, whose first slope for $\operatorname {\mathrm {Hdg}}_{\tau }$ is $0$ for all $\tau $ . In particular, this means that for all $\tau $ there exists $x \in M_{\tau }$ such that

$$ \begin{align*} F(x) \not\equiv 0 \pmod{\pi}. \end{align*} $$

By Proposition 4.16, we can moreover assume that there exists $\tau _{0}$ such that $a_{\tau _{0}} + a_{\overline {\tau _{0}}} < e$ (otherwise $x_{G}$ is in the $\mu $ -ordinary locus and we are done). Thus we have that $F_{\tau _{0}}V_{\tau _{0}} \equiv 0 \pmod \pi $ .

Lemma 4.17. Denote $f = 2d$ where $\left [L:\mathbb {Q}_{p}\right ] = ef$ . Let $M/W(k)$ be an NNP polarized $\mathcal O$ -crystal as before (bi-infinitesimal with first slope of $Hdg_{\tau }$ being $0$ ). Then there exist a deformation of M to a (NNP polarized $\mathcal O$ -)display $\mathcal P = \left (P,Q,F,V^{-1},h\right )$ over $k[[X]]$ and $x \in P_{\tau _{0}}$ such that $F^{d}(x) \neq 0 \pmod \pi $ .

Proof. Suppose that it is not already the case for M – that is, for all $x \in M_{\tau _{0}} F^{d}(x) \equiv 0 \pmod \pi $ . Take $x \in M_{\tau _{0}}$ such that $F(x) \not \equiv 0 \pmod \pi $ . Let r be the maximal integer such that $F^{r}(x) \not \equiv 0 \pmod \pi $ . Set $F^{r}(x) \in M_{\sigma ^{r}\tau }$ and take $y \in M_{\sigma ^{r}\tau }$ such that $F(y) \not \equiv 0 \pmod \pi $ . Thus $F^{r}(x)$ and y are not collinear and are nonzero modulo $\pi $ ; then we can construct an endomorphism $N_{r}$ of $M_{\sigma ^{r}\tau }$ that is $\mathcal O_{\sigma ^{r}\tau }$ -linear and such that $N_{r}(F^{r}(x)) = y$ and $N_{r}(y) =0$ , and $N^{2}_{r} = 0$ . Then set $N_{\overline {r}} = - N_{r}^{*} \in \operatorname {\mathrm {End}}\left (M_{\sigma ^{r}\overline {\tau }}\right )$ , and for every embedding $\chi \neq \sigma ^{r}\tau , \sigma ^{r}\overline \tau $ , set $N_{\chi } = 0$ . N is $\mathcal O$ -linear and polarized, and $N^{2} = 0$ . Now in $P_{N} = M\otimes _{W(k)} W(k[[X]])$ , we can calculate

$$ \begin{align*} F_{N}^{r+1}(x \otimes 1) = F_{N}^{2}\left(F_{N}^{r-1}(x\otimes 1)\right) &= F_{N}^{2}\left(F^{r-1}(x)\otimes 1\right) = F_{N}(F^{r}(x)\otimes 1 + y \otimes X) \\ &\qquad= F^{r+1}(x) \otimes 1 + F(y) \otimes X \equiv XF(y) \not\equiv 0 \pmod \pi. \end{align*} $$

By induction, we can thus assume that $F^{d}(x) \not \equiv 0 \pmod \pi $ up to deforming M.

Lemma 4.18. Let M be as in the conclusion of the previous lemma. There exists a generization D of M such that there exists $y \in D_{\overline {\tau _{0}}}$ satisfying

$$ \begin{align*} F^{d}(y) \not\equiv 0 \pmod \pi. \end{align*} $$

Moreover, there is still $x \in D_{\tau _{0}}$ such that $F^{d}(x) \not \equiv 0 \pmod \pi $ .

Proof. If it is not already the case for M, set $y \in M_{\overline {\tau _{0}}}$ such that $F(y) \not \equiv 0 \pmod \pi $ and denote by r the maximal integer such that $F^{r}(y) \not \equiv 0 \pmod \pi $ . We will construct a deformation such that $F^{r+1}(y) \not \equiv 0 \pmod \pi $ . Choose $z \in M_{\sigma ^{r}\overline {\tau }}$ such that $F(z) \not \equiv 0 \pmod \pi $ . We then set N as in the previous lemma:

$$ \begin{align*} N_{\sigma^{r}\overline\tau}(F^{r}(y)) = z, \qquad N_{\sigma^{r}\overline\tau}(z) = 0, \qquad N_{\sigma^{r}\tau} = - N_{\sigma^{r}\overline\tau}^{*}, \qquad N_{\chi} = 0,\quad \forall \chi \neq \sigma^{r}\tau,\sigma^{r}\overline\tau. \end{align*} $$

The same calculation shows that $F_{N}^{r+1}(y \otimes 1) \not \equiv 0 \pmod \pi $ . Moreover, $F_{N}^{d}(x)$ reduces to $F^{d}(x)$ modulo X, and thus $F_{N}^{d}(x)$ is still nonzero modulo $\pi $ as $F^{d}(x)$ is.

Lemma 4.19. Let M be as in the conclusion of Lemma 4.17. Then there exists a generization D such that $F^{2d}(x) \not \equiv 0 \pmod \pi $ for some $x \in D_{\tau _{0}}$ .

Proof. If it is not already the case, let x be the element given in Lemma 4.17, and up to deforming M, we can also have an element $y \in M_{\overline \tau }$ as in Lemma 4.18. Then we can construct an $\mathcal O$ -linear N such that $N^{2} = 0$ and

$$ \begin{align*} N_{\overline\tau}F^{d}(x) = y, \qquad N_{\overline\tau}(y) = 0, \qquad N_{\tau} = - N_{\overline\tau}^{*}, \qquad N_{\chi} = 0 \quad, \forall \chi \neq \tau,\overline\tau. \end{align*} $$

Set $\left (P,Q,F_{N},V^{-1}_{N}\right )$ as in [26]. Then we can calculate

$$ \begin{align*} F_{N}^{2d}(x) = F^{2d}(x) + XNF^{2d}(x) + XF^{d}(y) + X^{2}NF^{d}(y). \end{align*} $$

But $F^{2d}(x) \equiv 0 \pmod \pi $ , and thus $NF^{2d}(x)\equiv 0 \pmod \pi $ as well by the linearity of N. Moreover, $F^{d}(y) \not \equiv 0 \pmod \pi $ , thus

$$ \begin{align*} F^{d}(y) + XNF^{d}(y) \not\equiv 0 \pmod \pi, \end{align*} $$

as it is the case modulo X, and thus $F_{N}^{2d}(x) \not \equiv 0 \pmod \pi $ .

Proposition 4.20. Set $x \in X^{PR}(k)$ . Then there exists a sequence of deformations $x_{i}$ , $i = 0,\dotsc ,n$ , such that $x_{i} \in X^{PR}(k_{i}[[X_{i}]])$ for all $i = 1,\dotsc ,n$ and some perfect field $k_{i}$ above $k_{i-1}((X_{i-1}))$ , with $k_{1} = k$ and $x_{0} = x$ , $x_{i} \pmod X = x_{i-1} \otimes _{k_{i-1}[[X]]} k_{i}$ , and $x_{n} \otimes _{k_{n}[[X_{n}]]} k_{n}((X_{n}))^{perf}$ is $\mu $ -ordinary.

Proof. By Theorem 3.3, we can assume that we have constructed $x_{1}$ and that $x_{1} \otimes k((X_{1}))$ is in the Pappas–Rapoport locus. We will proceed by induction on the number of slopes of the bi-infinitesimal part of $x_{i}$ already constructed. If $x_{i}$ has no or only one slope for its Newton polygon, we are done, as it is $\mu $ -ordinary. Otherwise we can always assume that $G_{x_{i}}$ is split,

$$ \begin{align*} G_{x_{i}} = G_{x_{i}}^{1} \times G_{x_{i}}^{00} \times G_{x_{i}}^{2}, \end{align*} $$

with the Hodge and Newton polygons of $G_{x_{1}}^{1}$ being equal, $G_{x_{i}}^{2} = \left (G_{x_{i}}^{1}\right )^{D}$ , and the first slopes of the Newton and Hodge polygons of $G_{x_{i}}^{00}$ differing. By the previous results, we can moreover assume that there exists $r> 0$ and that we have constructed $x_{1},\dotsc ,x_{r}$ such that $M^{0} := M\left (\left (G_{x_{r}}\otimes k_{r}((X_{r}))^{perf}\right )^{00}\right )^{0}$ satisfies the conclusion of Lemma 4.19. Denote $x \in M^{0}_{\tau _{0}}$ such that $\left (F^{0}\right )^{2d}(x) \not \equiv 0 \pmod \pi $ . We can always assume that

(1) $$ \begin{align} h \left(x, \left(F^{0}\right)^{d}(x)\right) \equiv 0 \pmod \pi).\end{align} $$

Indeed, if it is not the case, then for all $m \in M_{\sigma \tau _{0}}$ ,

$$ \begin{align*} h\left(x + V^{0}m,\left(F^{0}\right)^{d}(x)\right) &= h\left(x,\left(F^{0}\right)^{d}(x)\right) + h\left(V^{0}m,\left(F^{0}\right)^{d}(x)\right)\\ &= h\left(x,\left(F^{0}\right)^{d}(x)\right) +h\left(m,\left(F^{0}\right)^{d+1}(x)\right)^{\sigma^{-1}}. \end{align*} $$

As $\left (F^{0}\right )^{d+1}(x) \neq 0 \pmod \pi $ , there exists m such that this expression vanishes. Replacing x by $x+V^{0}m$ – which does not change $F^{0}(x)$ , as $F_{\tau _{0}}^{0}V_{\tau _{0}}^{0} \equiv 0 \pmod \pi $ ³ – we are done. Denote by s the maximal integer such that $\left (F^{0}\right )^{s}(x) \not \equiv 0 \pmod \pi $ , and write

$$ \begin{align*} s = 2dq + j, \quad 0 \leq j < 2d.\\[-10pt] \end{align*} $$

Set $m_{0} = F^{2(q-1)d + j}(x)$ . Then the sequence $\left (m_{0},F^{0}(m_{0}),\dotsc , \left (F^{0}\right )^{2d-1}(m_{0})\right )$ is a deformation sequence. Moreover, by applying $\left (F^{0}\right )^{2(q-1)d+j+i}$ to formula (1) and by the semilinearity of h on the left, we have

$$ \begin{align*} h\left(\left(F^{0}\right)^{i}(m_{0}),\left(F^{0}\right)^{i+d}(m_{0})\right) = 0, \quad \forall i \in\{0,\dotsc,d\}. \end{align*} $$

We can now follow [26] to construct N and thus a deformation of $M^{0}$ which is not infinitesimal anymore. As $h\left (F^{2d}(m_{0}),F^{d}(m_{0})\right ) = 0$ , the subspace $\operatorname {\mathrm {Vect}}\left (F^{2d}(m_{0}),m_{0},F^{d}(m_{0})\right ) \pmod \pi \subset M_{\sigma ^{j}\tau }/\pi \oplus M_{\sigma ^{j}\overline \tau }/\pi $ is totally isotropic. We can thus find a totally isotropic complement $U = U_{\sigma ^{j}\tau }\oplus U_{\sigma ^{j}\overline \tau }$ such that $h(. ,. )$ induces a perfect pairing between

$$ \begin{align*} M_{0} := \operatorname{\mathrm{Vect}}\left(F^{2d}(m_{0}),m_{0},F^{d}(m_{0})\right) \pmod \pi \quad \text{and} \quad U. \end{align*} $$

We can then set $N\left (F^{2d}(m_{0})\right ) = m_{0}, Nm_{0} = NF^{d}(m_{0}) = 0$ , and extend N uniquely to U such that N is skew-symmetric. Then $N^{2} = 0 \pmod \pi $ and we can extend N by zero on $(M_{0} \oplus U)^{\perp }$ and lift to M so that $N^{2} = 0$ and is still skew-symmetric and $\mathcal O$ -linear. Then we can calculate that for the deformation $P_{N}$ ,

$$ \begin{align*} \left(F_{N}^{0}\right)^{2d}(m_{0}\otimes 1) &= F_{N}^{0} \left(\left(F^{0}\right)^{2d-1}(m_{0}) \otimes 1\right) \\&= \left(F^{0}\right)^{2d}(m_{0})\otimes 1 + m_{0}\otimes X \equiv Xm_{0}\pmod{\operatorname{\mathrm{Ker}}(F \pmod\pi)}. \end{align*} $$

In particular, $F_{N}^{0}$ is not nilpotent, and thus $N^{0} = \mathcal P_{N}^{0} \otimes k((X))^{perf}$ is not bi-infinitesimal. The p-divisible group associated to $P_{N}$ gives a $k[[X]]$ -point $x_{r+1}$ of $X^{PR}$ such that $G_{x_{r+1}} \otimes _{k[[X]]} k = G_{x_{r}}\otimes _{k_{r}[[X]]} k$ and $G_{x_{r+1}} \otimes k((X))$ is split,

$$ \begin{align*} G_{x_{r+1}} = G_{x_{r+1}}^{1} \times G_{x_{r+1}}^{00} \times G_{x_{r+1}}^{1,D}, \end{align*} $$

with $G_{x_{r+1}}^{00}$ having height less than $G_{x_{i}}^{00}$ . By induction on this height, we get the result.

Corollary 4.21. In case (AU), the $\mu $ -ordinary locus $X^{PR}_{\mu -ord}$ is Zariski dense.

4.4 Case (C)

In this case, $M = \oplus _{\tau } M_{\tau }$ and

$$ \begin{align*} h : M \times M \longrightarrow W(k) \otimes \operatorname{\mathrm{Diff}}^{-1}_{F} \end{align*} $$

is $\mathcal O_{F}$ -linear and alternating. Let $x_{G} \in X(k)$ be a point corresponding to a group G (with k perfect). We suppose that $x_{G}$ is in the Rapoport locus (by Theorem 3.3, as in this case the Rapoport and generalized Rapoport loci coincide).

Lemma 4.22. If G is bi-infinitesimal and in the generalized Rapoport locus, there exists a deformation sequence (as in Definition 4.10).

Proof. As G is bi-infinitesimal, denote (for $x \in M$ ) $w(x) = \sup \{ n \mid F^{n}(x) \not \equiv 0 \pmod {\pi }\}$ . As G is in the generalized Rapoport locus (and in case (C) we have $d_{\tau } = \frac {h}{2}$ , for all $\tau $ ), we have for all $\tau $ that a $x_{\tau } \in M_{\tau }$ such that $w(x_{\tau }) \geq 1$ . This is proved exactly as in [26, Proposition 4.1.4].

Lemma 4.23. There exists a deformation $G^{\prime }$ of G such that $G^{\prime }$ is not bi-infinitesimal.

Proof. If G is not bi-infinitesimal, any deformation will do. Otherwise, let $(x_{\tau })_{\tau }$ be the deformation sequence given by Lemma 4.22. We can construct a deformation endomorphism N such that $NFx_{\tau } = 0$ if $Fx_{\tau } = x_{\sigma \tau }$ and $NFx_{\tau } = x_{\sigma \tau }$ otherwise. Indeed, as h is $\mathcal O_{F}$ -linear, this is done exactly as in [26, Proposition 4.4.3]. The calculation of $F_{N}$ using this deformation shows that $G^{\prime }$ , the deformation of G associated to N, is not bi-infinitesimal in generic fiber.

Proposition 4.24. In case (C), the ordinary locus is dense.

Proof. By Theorem 3.3, it suffices to prove that each $x \in X^{PR}(k)$ , with k a perfect field, can be deformed into an ordinary p-divisible group. We can split

$$ \begin{align*} G = G^{m} \times G^{00} \times G^{et}, \end{align*} $$

where $G^{00}$ is bi-infinitesimal. We will argue on the height of $G^{00}$ . If $G^{00}$ is trivial, then G is ordinary and we are done. Otherwise, by Lemma 4.23, there is a deformation H of $G^{00}$ which is not bi-infinitesimal, and thus $\widetilde G = G^{m} \times H \times G^{et}$ is a deformation of G and

$$ \begin{align*} \operatorname{\mathrm{ht}}_{\mathcal O} \widetilde G^{00} = \operatorname{\mathrm{ht}}_{\mathcal O} H^{00} < \operatorname{\mathrm{ht}}_{\mathcal O} G^{00}. \end{align*} $$

Thus by induction there exists a deformation of G which is ordinary.

4.5 Ordinary locus

Definition 4.25. A p-divisible group over a base of characteristic p is said to be ordinary if it is an extension of an étale group by a multiplicative one. Equivalently, it is ordinary if its Hasse invariant is invertible. Denote by $X^{ord}$ the (open) subset of $X_{\kappa }$ of ordinary p-divisible groups.

Proposition 4.26. We have the following properties;

1. 1. If the ordinary locus is nonempty, it is equal to the $\mu $ -ordinary locus and is thus dense.

2. 2. The ordinary locus is nonempty if and only if $\left (d_{\tau }^{[i]}\right )$ is constant for all $\tau ,i$ .

3. 3. The ordinary locus is nonempty if and only if the local reflex field E is equal to $\mathbb {Q}_{p}$ .

Proof. If $\left (d_{\tau }^{[i]}\right )$ is constant, say equal to d, then the Pappas–Rapoport polygon has slopes $0$ (d times) and $1$ ( $(h-d)$ times). In particular,

$$ \begin{align*} X^{\mu-ord} = X^{ord}. \end{align*} $$

If the ordinary locus is nonempty, then a point x corresponding to an ordinary p-divisible group has a Newton polygon with only slopes $0$ and $1$ , and the same ending point as $\operatorname {\mathrm {PR}}$ ; and thus as ${\mathcal {N}\mathrm {ewt}}(x) \geq \operatorname {\mathrm {PR}}$ , this means that $\operatorname {\mathrm {PR}}\left (d_{\tau }^{[i]}\right )$ has only slopes $0$ and $1$ , and thus, as the breaking points are at the abscissa $d_{\tau }^{[i]}$ , the collection $\left (d_{\tau }^{[i]}\right )$ is constant. This proves 1 and 2. E is the (finite) extension of $\mathbb {Q}_{p}$ , inside K, fixing the collection $\left (d_{\tau }^{[i]}\right )$ . Thus if the ordinary locus is nonempty, $E = \mathbb {Q}_{p}$ . For every $\sigma \in \operatorname {\mathrm {Gal}}\left (K/K^{0}\right )$ , we have $\sigma \cdot d_{\tau }^{[i]} = d_{\tau }^{[\sigma \cdot i]}$ , where i corresponds to a conjugate $\pi _{i}$ of $\pi $ and $\pi _{\sigma \cdot i} = \sigma (\pi _{i})$ . Thus $\operatorname {\mathrm {Gal}}\left (K/K^{0}\right )$ is transitive on the collection $\left (d_{\tau }^{[i]}\right )_{i}$ . Thus if $E = \mathbb {Q}_{p}$ , then $d_{\tau }^{[i]} = d_{\tau }$ for all i. But $\operatorname {\mathrm {Gal}}\left (K^{0}/\mathbb {Q}_{p}\right )$ is transitive on the set $\mathcal T$ , and thus if $E = \mathbb {Q}_{p}$ , $d_{\tau }^{[i]} = d$ for all $\tau ,i$ . Another way to say it is that using the characteristic $0$ description $d_{i,\tau ^{\prime }}$ , $\tau ^{\prime } \in \operatorname {\mathrm {Hom}}\left (K,\overline {\mathbb {Q}}_{p}\right )$ , we have that $\operatorname {\mathrm {Gal}}\left (K/\mathbb {Q}_{p}\right )$ acts transitively on $\operatorname {\mathrm {Hom}}\left (K,\overline {\mathbb {Q}}_{p}\right )$ .