Local terms for transversal intersections

Abstract

The goal of this note is to show that in the case of ‘transversal intersections’ the ‘true local terms’ appearing in the Lefschetz trace formula are equal to the ‘naive local terms’. To prove the result, we extend the strategy used in our previous work, where the case of contracting correspondences is treated. Our new ingredients are the observation of Verdier that specialization of an étale sheaf to the normal cone is monodromic and the assertion that local terms are ‘constant in families’. As an application, we get a generalization of the Deligne–Lusztig trace formula.

Introduction

Let $f:X\to X$ be a morphism of schemes of finite type over an algebraically closed field $k$, let $\ell$ be a prime number different from the characteristic of $k$, and let $\mathcal{F} \in D_c^b(X,\overline {\mathbb{Q} }_{\ell })$ be equipped with a morphism $u:f^*\mathcal{F} \to \mathcal{F}$. Then for every fixed point $x\in \operatorname {Fix}(f)\subseteq X$, one can consider the restriction $u_x:\mathcal{F} _x\to \mathcal{F} _x$. Hence, one can consider its trace $\operatorname {Tr}(u_x)\in \overline {\mathbb{Q} }_{\ell }$, called the ‘naive local term’ of $u$ at $x$.

On the other hand, if $x\in \operatorname {Fix}(f)\subseteq X$ is an isolated fixed point, one can also consider the ‘true local term’ $LT_x(u)\in \overline {\mathbb{Q} }_{\ell }$, appearing in the Lefschetz–Verdier trace formula, so the natural question is when these two locals terms are equal.

Motivated by work of many people, including Illusie [SGA5], Pink [Pin92], and Fujiwara [Fuj97], it was shown in [Var07] that this is the case when $f$ is ‘contracting near $x$’, by which we mean that the induced map of normal cones $N_x(f):N_x(X)\to N_x(X)$ maps $N_x(X)$ to the zero section. In particular, this happens when the induced map of Zariski tangent spaces $d_x(f):T_x(X)\to T_x(X)$ is zero.

A natural question is whether the equality $LT_x(u)=\operatorname {Tr}(u_x)$ holds for a more general class of morphisms. For example, Deligne asked whether the equality holds when $x$ is the only fixed point of $d_x(f):T_x(X)\to T_x(X)$, or, equivalently, when the linear map $d_x(f)-\operatorname {Id}:T_x(X)\to T_x(X)$ is invertible. Note that when $X$ is smooth at $x$, this condition is equivalent to the fact that the graph of $f$ intersects transversally with the diagonal at $x$.

The main result of this note gives an affirmative answer to Deligne's question. Moreover, in order to get an equality $LT_x(u)=\operatorname {Tr}(u_x)$ it suffices to assume a weaker condition that $x$ is the only fixed point of $N_x(f):N_x(X)\to N_x(X)$ (see Corollary 4.11). In particular, we show this in the case when $f$ is an automorphism of $X$ of finite order, prime to the characteristic of $k$, or, more generally, a ‘semisimple’ automorphism (see Corollary 5.6).

Actually, as in [Var07], we show a more general result (see Theorem 4.10) in which a morphism $f$ is replaced by a correspondence, and a fixed point $x$ is replaced by a $c$-invariant closed subscheme $Z\subseteq X$. Moreover, instead of showing the equality of local terms we show a more general ‘local’ assertion that in some cases the so-called ‘trace maps’ commute with restrictions. Namely, we show it in the case when $c$ has ‘no almost fixed points in the punctured tubular neighborhood of $Z$’ (see Definition 4.4).

As an easy application, we prove a generalization of the Deligne–Lusztig trace formula (see Theorem 5.9).

To prove our result, we follow the strategy of [Var07]. First, using additivity of traces, we reduce to the case when $\mathcal{F} _x\simeq 0$. In this case, $\operatorname {Tr}(u_x)=0$, thus we have to show that $LT_x(u)=0$. Next, using specialization to the normal cone, we reduce to the case when $f:X\to X$ is replaced by $N_x(f):N_x(X)\to N_x(X)$ and $\mathcal{F}$ by its specialization $sp_x(\mathcal{F} )$. In other words, we can assume that $X$ is a cone with vertex $x$, and $f$ is $\mathbb{G} _m$-equivariant.

In the contracting case, treated in [Var07], the argument stops there. Indeed, after passing to normal cones we can assume that $f$ is the constant map with image $x$. In this case, our assumption $\mathcal{F} _x\simeq 0$ implies that $f^*\mathcal{F} \simeq 0$, thus $u=0$, hence $LT_x(u)=0$.

In general, by a theorem of Verdier [Ver83], we can assume that $\mathcal{F}$ is monodromic. As it is enough to show an analogous assertion for sheaves with finite coefficients, we can thus assume that $\mathcal{F}$ is $\mathbb{G} _m$-equivariant with respect to the action $(t,y)\mapsto t^n(y)$ for some $n$.

As $f$ is homotopic to the constant map with image $x$ (via the homotopy $f_t(y):=t^n f(y)$) it suffices to show that local terms are ‘constant in families’. We deduce the latter assertion from the fact that local terms commute with nearby cycles.

The paper is organized as follows. In § 1 we introduce correspondences, trace maps, and local terms. In § 2 we define relative correspondences and formulate Proposition 2.5 asserting that in some cases trace maps are ‘constant in families’. In § 3 we study a particular case of relative correspondences, obtained from schemes with an action of an algebraic monoid $(\mathbb {A}^1,\cdot )$. In § 4 we formulate our main result (Theorem 4.10), asserting that in some cases trace maps commute with restrictions to closed subschemes. We also deduce an affirmative answer to Deligne's question, discussed earlier. In § 5 we apply the results of § 4 to the case of an automorphism and deduce a generalization of the Deligne–Lusztig trace formula. Finally, we prove Theorem 4.10 in § 6 and prove Proposition 2.5 in § 7.

Notation

For a scheme $X$, we denote by $X_{\operatorname {red}}$ the corresponding reduced scheme. For a morphism of schemes $f:Y\to X$ and a closed subscheme $Z\subseteq X$, we denote by $f^{-1}(Z)\subseteq Y$ the schematic inverse image of $Z$.

Throughout most of the paper, all schemes will be of finite type over a fixed algebraically closed field $k$. The only exception is § 7, where all schemes will be of finite type over a spectrum of a discrete valuation ring over $k$ with residue field $k$.

We fix a prime $\ell$, invertible in $k$, and a commutative ring with identity $\Lambda$, which is either finite and is annihilated by some power of $\ell$, or a finite extension of $\mathbb{Z} _{\ell }$ or $\mathbb{Q} _{\ell }$.

To each scheme $X$ as above, we associate a category $D_{\rm ctf}^b(X,\Lambda )$ of ‘complexes of finite tor-dimension with constructible cohomology’ (see [SGA4$\frac{1}{2}$, Rapport 4.6] when $\Lambda$ is finite and [Del80, §§ 1.1.2–3] in other cases). This category is known to be stable under the six operations $f^*, f^!, f_*, f_!, \otimes$, and $\mathcal{R}\textit{Hom}$ (see [SGA4$\frac{1}{2}$, Théorème finitude 1.7]).

For each scheme $X$ as above, we denote by $\pi _X:X\to \operatorname {pt}:=\operatorname {Spec} k$ the structure morphism, by $\Lambda _X\in D_{\rm ctf}^b(X,\Lambda )$ the constant sheaf with fiber $\Lambda$, and by $K_X=\pi _X^!(\Lambda _{\operatorname {pt}})$ the dualizing complex of $X$. We also write $R\Gamma (X,\cdot )$ (respectively, $R\Gamma _c(X,\cdot )$) instead of $\pi _{X*}$ (respectively, $\pi _{X!}$).

For an embedding $i:Y\hookrightarrow X$ and $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, we often write $\mathcal{F} |_Y$ instead of $i^*\mathcal{F}$.

We freely use various base change morphisms (see, for example, [SGA4, Exposé  XVII, § 2.1.3 and Exposé XVIII, §§ 3.1.12.3, 3.1.13.2, 3.1.14.2]), which we denote by $BC$.

1. Correspondences and trace maps

1.1 Correspondences

1. (a) By a correspondence, we mean a morphism of schemes of the form $c=(c_l,c_r):C\to X\times X$, which can be also viewed as a diagram $X\overset {c_l}{\longleftarrow }C \overset {c_r}{\longrightarrow }X$.

2. (b) Let $c:C\to X\times X$ and $b:B\to Y\times Y$ be correspondences. By a morphism from $c$ to $b$, we mean a pair of morphisms $[f]=(f,g)$, making the following diagram commutative.

   (1.1)

   

3. (c) A correspondence $c:C\to X\times X$ gives rise to a Cartesian diagram

   
   where $\Delta :X\to X\times X$ is the diagonal map. We call $\operatorname {Fix}(c)$ the scheme of fixed points of $c$.

4. (d) We call a morphism $[f]$ from part (b) Cartesian, if the right inner square of diagram (1.1) is Cartesian.

1.2 Restriction of correspondences

Let $c:C\to X\times X$ be a correspondence, let $W\subseteq C$ be an open subscheme, and let $Z\subseteq X$ be a locally closed subscheme.

1. (a) We denote by $c|_W:W\to X\times X$ the restriction of $c$.

2. (b) Let $c|_Z:c^{-1}(Z\times Z)\to Z\times Z$ be the restriction of $c$. By definition, the inclusion maps $Z\hookrightarrow X$ and $c^{-1}(Z\times Z)\hookrightarrow C$ define a morphism $c|_Z\to c$ of correspondences.

3. (c) We say that a subscheme $Z$ is (schematically) $c$-invariant, if $c_r^{-1}(Z)\subseteq c_l^{-1}(Z)$. This happens if and only if we have $c^{-1}(Z\times Z)=c_r^{-1}(Z)$ or, equivalently, the natural morphism of correspondences $c|_Z\to c$ from part (b) is Cartesian.

Remark 1.3 Our conventions slightly differ from those of [Var07, § 1.5.6]. For example, we do not assume that a subscheme $Z$ is closed, our notion of $c$-invariance is stronger than that of [Var07, § 1.5.1], and when the subscheme $Z$ is $c$-invariant, then the restriction $c|_Z$ in the sense of [Var07] is the correspondence $c^{-1}(Z\times Z)_{\operatorname {red}}\to Z\times Z$.

1.4 Cohomological correspondences

Let $c:C\to X\times X$ be a correspondence, and let $\mathcal{F} \in D^b_{\rm ctf}(X,\Lambda )$.

1. (a) By $c$-morphism or a cohomological correspondence lifting $c$, we mean an element of

   \[ \operatorname{Hom}_c(\mathcal{F},\mathcal{F}):=\operatorname{Hom}(c_l^*\mathcal{F},c_r^!\mathcal{F})\simeq\operatorname{Hom}(c_{r!}c_l^*\mathcal{F},\mathcal{F}). \]

2. (b) Let $[f]:c\to b$ be a Cartesian morphism of correspondences (see § 1.1(d)). Then every $b$-morphism $u:b_l^*\mathcal{F} \to b_r^!\mathcal{F}$ gives rise to a $c$-morphism $[f]^*(u):c_l^*(f^*\mathcal{F} )\to c_r^!(f^*\mathcal{F} )$ defined as a composition

   \[ c_l^*(f^*\mathcal{F})\simeq g^*(b_l^*\mathcal{F})\overset{u}{\longrightarrow}g^*(b_r^!\mathcal{F})\overset{BC}{\longrightarrow}c_r^!(f^*\mathcal{F}), \]
   where the base change morphism $BC$ exists, because $[f]$ is Cartesian.

3. (c) As in [Var07, § 1.1.9], for an open subset $W\subseteq C$, every $c$-morphism $u$ gives rise to a $c|_W$-morphism $u|_W:(c_l^*\mathcal{F} )|_W\to (c_r^!\mathcal{F} )|_W$.

4. (d) It follows from part (b) and § 1.2(c) that for a $c$-invariant subscheme $Z\subseteq X$, every $c$-morphism $u$ gives rise to a $c|_Z$-morphism $u|_Z$ (compare [Var07, § 1.5.6(a)]).

1.5 Trace maps and local terms

Fix a correspondence $c:C\to X\times X$.

1. (a) As in [Var07, § 1.2.2], to every $\mathcal{F} \in D^b_{\rm ctf}(X,\Lambda )$ we associate the trace map

   \[ \mathcal{T}r_c:\operatorname{Hom}_c(\mathcal{F},\mathcal{F})\to H^0(\operatorname{Fix}(c),K_{\operatorname{Fix}(c)}). \]

2. (b) For an open subset $\beta$ of $\operatorname {Fix}(c)$,¹ we denote by

   \[ \mathcal{T}r_{\beta}: \operatorname{Hom}_c(\mathcal{F},\mathcal{F})\to H^0(\beta, K_{\beta}) \]
   the composition of $\mathcal{T}r _c$ and the restriction map $H^0(\operatorname {Fix}(c),K_{\operatorname {Fix}(c)})\to H^0(\beta, K_{\beta })$.

3. (c) If, in addition, $\beta$ is proper over $k$, we denote by

   \[ LT_{\beta}:\operatorname{Hom}_c(\mathcal{F},\mathcal{F})\to\Lambda \]
   the composition of $\mathcal{T}r _{\beta }$ and the integration map $\pi _{\beta !}:H^0(\beta, K_{\beta })\to \Lambda$.

4. (d) In the case when $\beta$ is a connected component of $\operatorname {Fix}(c)$,² which is proper over $k$, $LT_{\beta }(u)$ is usually called the (true) local term of $u$ at $\beta$.

2. Relative correspondences

Notation 2.1 Let $S$ be a scheme over $k$.

By a relative correspondence over $S$, we mean a morphism ${c}=({c}_l,{c}_r):{C}\to {X}\times _S{X}$ of schemes over $S$, or, equivalently, a correspondence ${c}=({c}_l,{c}_r):{C}\to {X}\times {X}$ such that ${c}_l$ and ${c}_r$ are morphisms over $S$.

1. (a) For a correspondence $c$ as above and a morphism $g:S'\to S$ of schemes over $k$, one can form a relative correspondence $g^*({c}):={c}\times _S S'$ over $S'$. Moreover, it follows from § 1.4(b) that every $c$-morphism ${u}\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$ gives rise to the $g^*({c})$-morphism

   \[ g^*({u})\in \operatorname{Hom}_{g^*(c)}(g^*\mathcal{F},g^*\mathcal{F}), \]
   where $g^*\mathcal{F} \in D^b_{\rm ctf}(X\times _S S',\Lambda )$ denotes the $*$-pullback of $\mathcal{F}$.

2. (b) For a geometric point $s$ of $S$, we denote by $i_s:\{s\}\to S$ the canonical map, and set ${c}_s:=i_s^*({c})$. Then, by part (a), every ${c}$-morphism ${u}\in \operatorname {Hom}_{{c}}(\mathcal{F},\mathcal{F} )$ gives rise to a ${c}_s$-morphism ${u}_s:=i_s^*({u})\in \operatorname {Hom}_{{c}_s}(\mathcal{F} _s,\mathcal{F} _s)$. Thus, we can form the trace map

   \[ \mathcal{T}r_{{c}_s}({u}_{s})\in H^0(\operatorname{Fix}({c}_s), K_{\operatorname{Fix}({c}_s)}). \]

Remark 2.2 In other words, a relative correspondence ${c}$ over $S$ gives rise to a family of correspondences ${c}_s:{C}_s\to {X}_s\times {X}_s$, parameterized by a collection of geometric points $s$ of $S$. Moreover, every ${c}$-morphism ${u}$ gives rise to a family of ${c}_s$-morphisms ${u}_s\in \operatorname {Hom}_{{c}_s}(\mathcal{F} _s,\mathcal{F} _s)$, thus a family of trace maps $\mathcal{T}r _{{c}_s}({u}_s)\in H^0(\operatorname {Fix}({c}_s), K_{\operatorname {Fix}({c}_s}))$.

Proposition 2.5, whose proof is given in § 7, asserts that in some cases the assignment $s\mapsto \mathcal{T}r _{{c}_s}({u}_s)$ is ‘constant’.

Notation 2.3 We say that a morphism $f:X\to S$ is a topologically constant family, if the reduced scheme ${X}_{\operatorname {red}}$ is isomorphic to a product $Y\times S_{\operatorname {red}}$ over $S_{\operatorname {red}}$.

Claim 2.4 Assume that $f:X\to S$ is a topologically constant family, and that $S$ is connected. Then for every two geometric points $s,t$ of $S$, we have a canonical identification

\[ R\Gamma({X}_{s},K_{{X}_{s}})\simeq R\Gamma({X}_{t},K_{X_t}),\text{ hence }H^0({X}_{s},K_{{X}_{s}})\simeq H^0({X}_{t},K_{X_t}). \]

Proof. Set $K_{X/S}:=f^!(\Lambda _S)\in D_{\rm ctf}^b(X,\Lambda )$ and $\mathcal{F} :=f_*(K_{X/S})\in D_{\rm ctf}^b(S,\Lambda )$. Our assumption on $f$ implies that for every geometric point $s$ of $S$, the base change morphisms

\[ \mathcal{F}_s=R\Gamma(s,\mathcal{F}_s)\to R\Gamma({X}_{s},i_s^*(K_{X/S}))\to R\Gamma({X}_{s},K_{{X}_{s}}) \]

are isomorphisms. Furthermore, the assumption also implies that $\mathcal{F}$ is constant, that is, isomorphic to a pullback of an object in $D_{\rm ctf}^b(\operatorname {pt},\Lambda )$. Then, for every specialization arrow $\alpha :t\to s$, the specialization map $\alpha ^*:\mathcal{F} _s\to \mathcal{F} _t$ (see [SGA4, Exposé VIII, § 7]) is an isomorphism (because $\mathcal{F}$ is locally constant), and does not depend on the specialization arrow $\alpha$ (only on $s$ and $t$). Thus, the assertion follows from the assumption that $S$ is connected.

Proposition 2.5 Let ${c}:{C}\to {X}\times {X}$ be a relative correspondence over $S$ such that $S$ is connected, and that $\operatorname {Fix}({c})\to S$ is a topologically constant family.

Then, for every ${c}$-morphism ${u}\in \operatorname {Hom}_{{c}}(\mathcal{F},\mathcal{F} )$ such that $\mathcal{F}$ is universally locally acyclic (ULA) over $S$, the assignment $s\mapsto \mathcal{T}r _{{c}_s}({u}_s)$ is ‘constant’, that is, for every two geometric points $s,t$ of $S$, the identification

\[ H^0({X}_{s},K_{{X}_{s}})\simeq H^0({X}_{t},K_{{X}_{t}}) \]

from Claim 2.4 identifies $\mathcal{T}r _{{c}_{s}}({u}_{s})$ with $\mathcal{T}r _{{c}_{t}}({u}_{t})$.

In particular, we have $\mathcal{T}r _{{c}_{s}}({u}_{s})=0$ if and only if $\mathcal{T}r _{{c}_{t}}({u}_{t})=0$.

3. An $(\mathbb {A}^1,\cdot )$-equivariant case

3.1 Construction

Fix a scheme $S$ over $k$ and a morphism $\mu :X\times S\to X$.

1. (a) A correspondence $c:C\to X\times X$ gives rise to the correspondence

   \[ c_S={c}_S^{\mu}:C_S\to X_S\times_{S} X_S \]
   over $S$, where $C_S:=C\times S$ and $X_S:=X\times S$, while $c_{Sl},c_{Sr}:C\times S\to X\times S$ are given by
   \[ c_{Sr}:=c_r\times\operatorname{Id}_S \text{ and } c_{Sl}:=(\mu,\operatorname{pr}_S)\circ(c_l\times\operatorname{Id}_S), \]
   that is, $c_{Sl}(y,s)= (\mu (c_l(y),s),s)$ and $c_{Sr}(y,s)=(c_r(y),s)$ for all $y\in C$ and $s\in S$.

2. (b) For every geometric point $s$ of $S$, we get an endomorphism $\mu _s:=\mu (-,s):X_s\to X_s$. Then $c_s:=i_s^*(c_S)$ is the correspondence

   \[ c_s=(\mu_s\circ c_l,c_r):C_s\to X_s\times X_s. \]
   In particular, for every $s\in S(k)$ we get a correspondence $c_s:C\to X\times X$.

3. (c) Suppose we are given an object $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, a $c$-morphism $u\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$ and a morphism $v:\mu ^*\mathcal{F} \to \mathcal{F} _S$ in $D_{\rm ctf}^b(X_S,\Lambda )$, where we set

   \[ \mathcal{F}_S:=\mathcal{F}\boxtimes\Lambda_S\in D_{\rm ctf}^b(X_S,\Lambda). \]
   To this data we associate a $c_S$-morphism $u_S\in \operatorname {Hom}_{c_S}(\mathcal{F} _S,\mathcal{F} _S)$, defined as a composition
   \[ c_{Sl}^*(\mathcal{F}_S)\simeq (c_l\times\operatorname{Id}_S)^*(\mu^*\mathcal{F})\overset{v}{\to}(c_l\times\operatorname{Id}_S)^*(\mathcal{F}_S)\simeq (c_l^*\mathcal{F})\boxtimes\Lambda_S\overset{u}{\to} (c_r^!\mathcal{F})\boxtimes\Lambda_S\simeq c_{Sr}^!(\mathcal{F}_S). \]

4. (d) For every geometric point $s$ of $S$, morphism $v$ restricts to a morphism $v_s=i_s^*(v):\mu _s^*\mathcal{F} \to \mathcal{F}$, and the $c_s$-morphism $u_s:=i_s^*(u_S):c_l^*\mu _s^*\mathcal{F} \to c_r^!\mathcal{F}$ decomposes as

   \[ u_s:c_l^*\mu_s^*\mathcal{F}\overset{v_s}{\longrightarrow}c_l^*\mathcal{F}\overset{u}{\longrightarrow}c_r^!\mathcal{F}. \]

Remark 2.4 For a morphism $\mu :X\times S\to X$ and a closed point $a\in S$, we set $S^a:=S\smallsetminus \{a\}$, and $\mu ^a:=\mu |_{X\times S^a}:X\times S^a\to X$. Let $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$ be such that $\mu _a^*\mathcal{F} \simeq 0$.

1. (a) Every morphism $v^a:(\mu ^a)^*\mathcal{F} \to \mathcal{F} _{S^a}$ uniquely extends to a morphism $v:\mu ^*\mathcal{F} \to \mathcal{F} _S$:

   Indeed, let $j:X\times S^a\hookrightarrow X\times S$ and $i:X\times \{a\}\hookrightarrow X\times S$ be the inclusions. Using distinguished triangle $j_!j^*\mu ^*\mathcal{F} \to \mu ^*\mathcal{F} \to i_*i^*\mu ^*\mathcal{F}$ and the assumption that $i^*\mu ^*\mathcal{F} \simeq \mu _a^*\mathcal{F} \simeq 0$, we conclude that the map $j_!j^*\mu ^*\mathcal{F} \to \mu ^*\mathcal{F}$ is an isomorphism. Therefore, the restriction map

   \[ j^*:\operatorname{Hom}(\mu^*\mathcal{F},\mathcal{F}_S)\to \operatorname{Hom}(j^*\mu^*\mathcal{F},j^*\mathcal{F}_S)\simeq \operatorname{Hom}(j_!j^*\mu^*\mathcal{F},\mathcal{F}_S) \]
   is an isomorphism, as claimed.

2. (b) Our assumption $\mu _a^*\mathcal{F} \simeq 0$ implies that $\operatorname {Hom}_{c_a}(\mathcal{F},\mathcal{F} )=\operatorname {Hom}(c_l^*\mu _a^*\mathcal{F},c_r^!\mathcal{F} )\simeq 0$.

3.3 Equivariant case

Let $S$ be an algebraic monoid, acting on $X$, and let $\mu :X\times S\to X$ be the action map.

1. (a) We say that an object $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$ is weakly $S$-equivariant, if we are given a morphism $v:\mu ^*\mathcal{F} \to \mathcal{F} _S$ such that $v_1:\mathcal{F} =\mu _1^*\mathcal{F} \to \mathcal{F}$ is the identity map. In particular, the construction of § 3.1 applies, so to every $c$-morphism $u\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$ we associate a $c_S$-morphism $u_S\in \operatorname {Hom}_{c_S}(\mathcal{F} _S,\mathcal{F} _S)$.

2. (b) In the situation of part (a), the correspondence $c_1$ equals $c$, and the assumption on $v_1$ implies that the $c$-morphism $u_1$ equals $u$.

3.4 Basic example

1. (a) Let $X$ be a scheme, equipped with an action $\mu :X\times \mathbb {A}^1\to X$ of the algebraic monoid $(\mathbb {A}^1,\cdot )$, let $\mu _0:X\to X$ be the induced (idempotent) endomorphism, and let $Z=Z_X\subseteq X$ be the scheme of $\mu _0$-fixed points, also called the zero section. Then $Z_X\subseteq X$ is a locally closed subscheme, whereas $\mu _0:X\to X$ factors as $X\to Z_X\hookrightarrow X$, thus inducing a projection $\operatorname {pr}_X:X\to Z_X$, whose restriction to $Z_X$ is the identity.

2. (b) The correspondence $X\mapsto (Z_X\subseteq X\overset {\operatorname {pr}_X}{\longrightarrow } Z_X\!)$ is functorial. Namely, every $(\mathbb {A}^1,\cdot )$-equivariant morphism $f:X'\to X$ induced a morphism $Z_f:Z_{X'}\to Z_X$ between zero sections, and we have an equality $Z_f\circ \operatorname {pr}_{X'}=\operatorname {pr}_X\circ f$ of morphisms $X'\to Z_X$.

3. (c) Let $c:C\to X\times X$ be any correspondence. Then the construction of § 3.1 gives rise to a relative correspondence $c_{\mathbb {A}^1}:C_{\mathbb {A}^1}\to C_{\mathbb {A}^1}\times C_{\mathbb {A}^1}$ over $\mathbb {A}^1$, hence a family of correspondences $c_t:C\to X\times X$, parameterized by $t\in \mathbb {A}^1(k)$.

4. (d) For every $t\in \mathbb {A}^1(k)$, the zero section $Z\subseteq X$ is $\mu _t$-invariant, and the induced map $\mu _t|_Z$ is the identity. Therefore, we have an inclusion $\operatorname {Fix}(c|_Z)\subseteq \operatorname {Fix}(c_t|_Z)$ of schemes of fixed points.

5. (e) For every $t\in \mathbb{G} _m(k)$, we have an equality $\operatorname {Fix}(c_t|_Z)=\operatorname {Fix}(c|_Z)$. Indeed, one inclusion was shown in part (d), whereas the opposite inclusion follows from the first together with identity $(c_t)_{t^{-1}}=c$.

6. (f) As $\mu _0$ factors through $Z\subseteq X$, we have an equality $\operatorname {Fix}(c_0|_Z)=\operatorname {Fix}(c_0)$. Moreover, if $Z$ is $c$-invariant, we have an equality $\operatorname {Fix}(c_0|_Z)=\operatorname {Fix}(c|_Z)$. Indeed, one inclusion was shown in part (d), whereas the opposite follows from the inclusion

   \[ \operatorname{Fix}(c_0|_Z)\subseteq c_r^{-1}(Z)=c^{-1}(Z\times Z). \]

3.5 Twisted action

Assume that we are in the situation of § 3.4. For every $n\in \mathbb{N}$, we can consider the $n$-twisted action $\mu (n):X\times \mathbb {A}^1\to X$ of $(\mathbb {A}^1,\cdot )$ on $X$ given by formula $\mu (n)(x,t)=\mu (x,t^n)$. It gives rise to the family of correspondences $c^{\mu (n)}_{t}:C\to X\times X$ such that $c^{\mu (n)}_{t}=c_{t^n}$. Clearly, $\mu (n)$ restricts to an $n$-twisted action of $\mathbb{G} _m$ on $X$.

Proposition 3.6 Let $X$ be an $(\mathbb {A}^1,\cdot )$-equivariant scheme, and let $c:C\to X\times X$ be a correspondence such that:

• • a subscheme $Z=Z_X\subseteq X$ is closed and $c$-invariant;

• • we have $\operatorname {Fix}(c)\smallsetminus \operatorname {Fix}(c|_Z)=\emptyset$;

• • the set $\{t\in \mathbb {A}^1(k)\,|\,\operatorname {Fix}(c^{\mu }_t)\smallsetminus \operatorname {Fix}(c^{\mu }_t|_Z)\neq \emptyset \}$ is finite.

Then for every weakly $\mathbb{G} _m$-equivariant object $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$ (see § 3.3(a)) with respect to the $n$-twisted action (see § 3.5) such that $\mathcal{F} |_Z=0$ and every $c$-morphism $u\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$, we have $\mathcal{T}r _c(u)=0$.

Proof. Consider the $n$-twisted action $\mu (n):X\times \mathbb {A}^1\to X$, and let $\mu (n)^0:X\times \mathbb{G} _m\to X$ be the induced $n$-twisted action of $\mathbb{G} _m$. The weakly $\mathbb{G} _m$-equivariant structure on $\mathcal{F}$ gives rise to the morphism $v^0:(\mu (n)^0)^*\mathcal{F} \to \mathcal{F} _{\mathbb{G} _m}$ (see § 3.3(a)).

Next, because $\mu (n)_0=\mu _0:X\to X$ factors through $Z$, whereas $\mathcal{F} |_Z=0$, we conclude that $(\mu (n)_0)^*\mathcal{F} \simeq 0$. Therefore, morphism $v^0$ extends uniquely to the morphism $v:\mu (n)^*\mathcal{F} \to \mathcal{F} _{\mathbb {A}^1}$ (see § 3.2(a)). Thus, by construction § 3.1(c), our $c$-morphism $u$ gives rise to the $c^{\mu (n)}_{\mathbb {A}^1}$-morphism $u_{\mathbb {A}^1}\in \operatorname {Hom}_{c^{\mu (n)}_{\mathbb {A}^1}}(\mathcal{F} _{\mathbb {A}^1},\mathcal{F} _{\mathbb {A}^1})$ such that $u_1=u$ (see § 3.3(b)).

Note that because $u_0\in \operatorname {Hom}_{c_0}(\mathcal{F},\mathcal{F} )=0$ (see § 3.2(b)), we have $\mathcal{T}r _{c_0}(u_0)=0$. We would like to apply Proposition 2.5 to deduce that $\mathcal{T}r _c(u)=\mathcal{T}r _{c_1}(u_1)=0$.

Consider the set

\[ T:=\{t\in\mathbb{A}^1(k)\,|\,\operatorname{Fix}(c^{\mu}_{t^n})\smallsetminus\operatorname{Fix}(c^{\mu}_{t^n}|_Z)\neq\emptyset\}. \]

Then $0\notin T$ (by § 3.4(f)), and our assumption says that $T$ is finite, and $1\notin T$. Then the complement $S:=\mathbb {A}^1\smallsetminus T\subseteq \mathbb {A}^1$ is an open subscheme, and $0,1\in S$. Let $c^{\mu (n)}_S$ be the restriction of $c^{\mu (n)}_{\mathbb {A}^1}$ to $S$, and it suffices to show that $\operatorname {Fix}(c^{\mu (n)}_S)\to S$ is a topologically constant family, thus Proposition 2.5 applies.

We claim that we have the equality

\[ \operatorname{Fix}(c^{\mu(n)}_S)_{\operatorname{red}}=\operatorname{Fix}(c^{\mu(n)}_S|_{Z\times S})_{\operatorname{red}}=\operatorname{Fix}(c|_Z)_{\operatorname{red}}\times S \]

of locally closed subschemes of $C\times S$. For this it suffices to show that for every $t\in S(k)$ we have equalities

\[ \operatorname{Fix}(c_{t^n})_{\operatorname{red}}=\operatorname{Fix}(c_{t^n}|_{Z})_{\operatorname{red}}=\operatorname{Fix}(c|_Z)_{\operatorname{red}}. \]

The left equality follows from the identity $\operatorname {Fix}(c^{\mu }_{t^n})\smallsetminus \operatorname {Fix}(c^{\mu }_{t^n}|_Z)=\emptyset$ used to define $S$. As $Z$ is $c$-invariant, the right equality follows from §§ 3.4(e) and (f).

3.7 Equivariant correspondences

Let $c:C\to X\times X$ be an $(\mathbb {A}^1,\cdot )$-equivariant correspondence, by which we mean that both $C$ and $X$ are equipped with an action of a monoid $(\mathbb {A}^1,\cdot )$, and both projections $c_l,c_r:C\to X$ are $(\mathbb {A}^1,\cdot )$-equivariant.

1. (a) Note that the subscheme of fixed points $\operatorname {Fix}(c)\subseteq C$ is $(\mathbb {A}^1,\cdot )$-invariant, correspondence $c$ induces a correspondence $Z_c:Z_C\to Z_X\times Z_X$ between zero sections, and we have an equality $\operatorname {Fix}(Z_c)=Z_{\operatorname {Fix}(c)}$ of locally closed subschemes of $C$.

2. (b) By § 3.1(a), correspondence $c$ gives rise to a relative correspondence $c_{\mathbb {A}^1}:C_{\mathbb {A}^1}\to X_{\mathbb {A}^1}\times X_{\mathbb {A}^1}$ over $\mathbb {A}^1$. Let the monoid $(\mathbb {A}^1,\cdot )$ act on $X_{\mathbb {A}^1}$ and $C_{\mathbb {A}^1}$ by the product of its actions on $X$ and $C$ and the trivial action on $\mathbb {A}^1$. Then $c_{\mathbb {A}^1}$ is an $(\mathbb {A}^1,\cdot )$-equivariant correspondence, and the induced correspondence $Z_{c_{\mathbb {A}^1}}$ between zero sections is the product of $Z_c$ (see part (a)) and $\operatorname {Id}_{\mathbb {A}^1}:\mathbb {A}^1\to \mathbb {A}^1\times \mathbb {A}^1$.

3. (c) Using part (b), for every $t\in \mathbb {A}^1(k)$, we get an $(\mathbb {A}^1,\cdot )$-equivariant correspondence $c_t:C\to X\times X$, which satisfy $Z_{c_t}=Z_c$ and $Z_{\operatorname {Fix}(c_t)}=\operatorname {Fix}(Z_c)$ (use part (a)).

3.8 Cones

Recall (see § 3.4(a)) that for every $(\mathbb {A}^1,\cdot )$-equivariant scheme $X$, there is a natural projection $\operatorname {pr}_X:X\to Z_X$.

1. (a) We say that $X$ is a cone, if the projection $\operatorname {pr}_X:X\to Z$ is affine. In concrete terms this means that $X\simeq \mathcal{S}\textit{pec} (\mathcal{A} )$, where $\mathcal{A} =\bigoplus _{n=0}^{\infty } \mathcal{A} _n$ is a graded quasi-coherent $\mathcal{O} _Z$-algebra, where $\mathcal{A} _0=\mathcal{O} _Z$, and each $\mathcal{A} _n$ is a coherent $\mathcal{O} _Z$-module. In this case, the zero section $Z_X\subseteq X$ is automatically closed.

2. (b) In the situation of part (a), the open subscheme $X\smallsetminus Z_X\subseteq X$ is $\mathbb{G} _m$-invariant, and the quotient $(X\smallsetminus Z_X)/\mathbb{G} _m$ is isomorphic to $Proj(\mathcal{A} )$ over $Z_X$, hence is proper over $Z_X$.

3. (c) Note that if $c:C\to X\times X$ is an $(\mathbb {A}^1,\cdot )$-equivariant correspondence such that $C$ and $X$ are cones, then $\operatorname {Fix}(c)$ is a cone as well (compare § 3.7(a)).

Our next goal is to show that in some cases the finiteness assumption in Proposition 3.6 is automatic.

Lemma 3.9 Let $c:C\to X\times X$ be an $(\mathbb {A}^1,\cdot )$-equivariant correspondence over $k$ such that:

• • $X$ is a cone with zero section $Z$;

• • $C$ is a cone with zero section $c_r^{-1}(Z)$;

• • $\operatorname {Fix}(c|_Z)$ is proper over $k$.

Then the set $\{t\in \mathbb {A}^1(k)\,|\,\operatorname {Fix}(c_t)\smallsetminus \operatorname {Fix}(c_t|_Z)\neq \emptyset \}$ is finite.

Proof. We let $c_{\mathbb {A}^1}$ be as in § 3.7(b), and set

\[ \operatorname{Fix}(c_{\mathbb{A}^1})':=\operatorname{Fix}(c_{\mathbb{A}^1})\smallsetminus \operatorname{Fix}(c_{\mathbb{A}^1}|_{Z_{\mathbb{A}^1}})\subseteq \operatorname{Fix}(c_{\mathbb{A}^1}). \]

We have to show that the image of the projection $\pi :\operatorname {Fix}(c_{\mathbb {A}^1})'\to \mathbb {A}^1$ is a finite set.

Note that the fiber of $\operatorname {Fix}(c_{\mathbb {A}^1})'$ over $0\in \mathbb {A}^1$ is $\operatorname {Fix}(c_0)\smallsetminus \operatorname {Fix}(c_0|_Z)=\emptyset$ (by § 3.4(f)). It thus suffices to show that the image of $\pi$ is closed. By § 3.8(c), we conclude that $\operatorname {Fix}(c_{\mathbb {A}^1})$ is a cone, whereas using §§ 3.7(a) and (b) we conclude that

\[ Z_{\operatorname{Fix}(c_{\mathbb{A}^1})}=\operatorname{Fix}(c_{\mathbb{A}^1}|_{Z_{\mathbb{A}^1}})=\operatorname{Fix}(c|_Z)\times\mathbb{A}^1. \]

It now follows from § 3.8(b) that the open subscheme $\operatorname {Fix}(c_{\mathbb {A}^1})'\subseteq \operatorname {Fix}(c_{\mathbb {A}^1})$ is $\mathbb{G} _m$-invariant, and that $\pi$ factors through the quotient $\operatorname {Fix}(c_{\mathbb {A}^1})'/\mathbb{G} _m$, which is proper over $\operatorname {Fix}(c|_Z)\times \mathbb {A}^1$. As $\operatorname {Fix}(c|_Z)$ is proper over $k$ by assumption, the projection $\overline {\pi }:\operatorname {Fix}(c_{\mathbb {A}^1})'/\mathbb{G} _m\to \mathbb {A}^1$ is therefore proper. Hence, the image of $\overline {\pi }$ is closed, completing the proof.

4. Main result

4.1 Normal cones

Compare [Var07, § 1.4.1 and Lemma 1.4.3].

1. (a) Recall that to a pair $(X,Z)$, where $X$ is a scheme and $Z\subseteq X$ a closed subscheme, one associates the normal cone $N_Z(X)$ defined to be $N_Z(X)=\mathcal{S}\textit{pec} (\bigoplus _{n=0}^{\infty }(\mathcal{I} _Z)^n/(\mathcal{I} _Z)^{n+1})$, where $\mathcal{I} _Z\subseteq \mathcal{O} _X$ is the sheaf of ideals of $Z$. By definition, $N_Z(X)$ is a cone in the sense of § 3.8, and $Z\subseteq N_Z(X)$ is the zero section.

2. (b) The assignment $(X,Z)\mapsto (N_Z(X),Z)$ is functorial. Namely, every morphism $f:X'\to X$ such that $Z'\subseteq f^{-1}(Z)$ gives rise to an $(\mathbb {A}^1,\cdot )$-equivariant morphism $N_{Z'}(X')\to N_Z(X)$, whose induced morphism between zero sections is $f|_{Z'}:Z'\to Z$.

3. (c) By part (b), every morphism $f:X'\to X$ induces a morphism

   \[ N_Z(f):N_{f^{-1}(Z)}(X')\to N_Z(X), \]
   lifting $f|_Z:f^{-1}(Z)\to Z$. Moreover, the induced map $N_{f^{-1}(Z)}(X')\to N_Z(X)\times _Z f^{-1}(Z)$ is a closed embedding, and we have an equality $N_Z(f)^{-1}(Z)=f^{-1}(Z)\subseteq N_{f^{-1}(Z)}(X')$.

The following standard assertion will be important later.

Lemma 4.2 Assume that $N_Z(X)$ is set-theoretically supported on the zero section, that is, $N_Z(X)_{\operatorname {red}}=Z_{\operatorname {red}}$. Then $Z_{\operatorname {red}}\subseteq X_{\operatorname {red}}$ is open.

Proof. As the assertion is local on $X$, we can assume that $X$ is affine. Moreover, replacing $X$ by $X_{\operatorname {red}}$, we can assume that $X$ is reduced. Then our assumption implies that there exists $n$ such that $I_Z^n=I_Z^{n+1}$. Using the Nakayama lemma, we conclude that the localization of $I_Z^n$ at every $x\in Z$ is zero. Thus, the localization of $I_Z$ at every point $z\in Z$ is zero, which implies that $Z\subseteq X$ is open, as claimed.

4.3 Application to correspondences

1. (a) Let $c:C\to X\times X$ be a correspondence, and $Z\subseteq X$ a closed subscheme. Then, by § 4.1, correspondence $c$ gives rise to an $(\mathbb {A}^1,\cdot )$-equivariant correspondence

   \[ N_Z(c): N_{c^{-1}(Z\times Z)}(C)\to N_Z(X)\times N_Z(X) \]
   such that the induced correspondence between zero sections is $c|_Z:c^{-1}(Z\times Z)\to Z\times Z$.

2. (b) Combining §§ 3.8(c) and 3.7(a), we get that $\operatorname {Fix}(N_Z(c))$ is a cone with zero section $\operatorname {Fix}(c|_Z)$. Moreover, $N_{\operatorname {Fix}(c|_Z)}(\operatorname {Fix}(c))$ is closed subscheme of $\operatorname {Fix}(N_Z(c))$ (see [Var07, Corollary 1.4.5]).

3. (c) By § 3.7(b), for every $t\in \mathbb {A}^1(k)$ we get a correspondence

   \[ N_Z(c)_t: N_{c^{-1}(Z\times Z)}(C)\to N_Z(X)\times N_Z(X). \]
   Moreover, every $\operatorname {Fix}(N_Z(c)_t)$ is a cone with zero section $\operatorname {Fix}(c|_Z)$ (use §§ 3.8(c) and 3.7(c)).

Definition 4.4 Let $c:C\to X\times X$ be a correspondence, and let $Z\subseteq X$ be a closed subscheme.

1. (a) We say that $c$ has no fixed points in the punctured tubular neighborhood of $Z$, if correspondence $N_Z(c)$ satisfies $\operatorname {Fix}(N_Z(c))\smallsetminus \operatorname {Fix}(c|_Z)=\emptyset$.

2. (b) We say that $c$ has no almost fixed points in the punctured tubular neighborhood of $Z$, if $\operatorname {Fix}(N_Z(c))\smallsetminus \operatorname {Fix}(c|_Z)=\emptyset$, and the set $\{t\in \mathbb {A}^1(k)\,|\,\operatorname {Fix}(N_Z(c)_t)\smallsetminus \operatorname {Fix}(c|_Z)\neq \emptyset \}$ is finite.

Remark 4.5

1. (a) The difference $N_{c^{-1}(Z\times Z)}(C)\smallsetminus c^{-1}(Z\times Z)$ can be thought as the punctured tubular neighborhood of $c^{-1}(Z\times Z)\subseteq C$. Therefore, our condition 4.4(a) means that any point $y\in N_{c^{-1}(Z\times Z)}(C)\smallsetminus c^{-1}(Z\times Z)$ is not a fixed point of $N_Z(c)$, that is,

   \[ N_Z(c)_l(y)\neq N_Z(c)_r(y). \]

2. (b) Condition 4.4(b) means that there exists an open neighbourhood $U$ of $1\in \mathbb {A}^1$ such that for every $y\in N_{c^{-1}(Z\times Z)}(C)\smallsetminus c^{-1}(Z\times Z)$ we have $\mu _t(N_Z(c)_l(y))\neq N_Z(c)_r(y)$ for every $t\in U$. In other words, $y$ is not an almost fixed point of $N_Z(c)$.

4.6 The case of a morphism

Let $f:X\to X$ be a morphism, and let $x\in \operatorname {Fix}(f)$ be a fixed point. We take $c$ be the graph $\operatorname {Gr}_f=(f,\operatorname {Id}_X)$ of $f$, and set $Z:=\{x\}$.

1. (a) Then $N_x(X):=N_Z(X)$ is a closed conical subset of the tangent space $T_x(X)$, the morphism $N_x(f):N_x(X)\to N_x(X)$ is $(\mathbb {A}^1,\cdot )$-equivariant, thus $\operatorname {Fix}(N_x(c))=\operatorname {Fix}(N_x(f))$ is a conical subset of $N_x(X)\subseteq T_x(X)$. Hence, $\operatorname {Gr}_f$ has no fixed points in the punctured tubular neighborhood of $x$ if and only if set-theoretically we have $\operatorname {Fix} N_x(f)=\{x\}$.

2. (b) Let $T_x(f):T_x(X)\to T_x(X)$ be the differential of $f$ at $x$. Then $\operatorname {Fix} T_x(f)=\{x\}$ if and only if the linear map $T_x(f)-\operatorname {Id}:T_x(X)\to T_x(X)$ is invertible, that is, $\operatorname {Gr}_f$ intersects with $\Delta _X$ at $x$ transversally in the strongest possible sense. In this case, $\operatorname {Gr}_f$ has no fixed points in the punctured tubular neighborhood of $x$ (by part (a)).

3. (c) Assume now that $X$ is smooth at $x$. Then, by parts (a) and (b), $\operatorname {Gr}_f$ has no fixed points in the punctured tubular neighborhood of $x$ if and only if $\operatorname {Gr}_f$ intersects with $\Delta _X$ at $x$ transversally.

Though the next result is not needed for what follows, it shows that our setting generalizes that studied in [Var07].

Lemma 4.7 Assume that $c$ is contracting near $Z$ in the neighborhood of fixed points in the sense of [Var07, § 2.1.1(c)]. Then $c$ has no almost fixed points in the punctured tubular neighborhood of $Z$. Moreover, the subset of $\mathbb {A}^1(k)$, defined in Definition 4.4(b), is empty.

Proof. Choose an open neighborhood $W\subseteq C$ of $\operatorname {Fix}(c)$ such that $c|_W$ is contracting near $Z$ (see [Var07, § 2.1.1(b)]). Then $\operatorname {Fix}(c|_W)=\operatorname {Fix}(c)$, hence we can replace $c$ by $c|_W$, thus assuming that $c$ is contracting near $Z$. In this case, the set-theoretic image of the morphism

\[ N_Z(c)_l:N_{c^{-1}(Z\times Z)}(C)\to N_Z(X) \]

lies in the zero section. Therefore, for every $t\in \mathbb {A}^1(k)$ the set-theoretic image of the map $\operatorname {Fix}(N_Z(c)_t)\to N_Z(X)$ lies in the zero section, implying the assertion.

By Lemma 4.7, the following result is a generalization of [Var07, Theorem 2.1.3(a)].

Lemma 4.8 Let $c:C\to X\times X$ be a correspondence, which has no fixed points in the punctured tubular neighborhood of $Z\subseteq X$. Then the closed subscheme $\operatorname {Fix}(c|_Z)_{\operatorname {red}}\subseteq \operatorname {Fix}(c)_{\operatorname {red}}$ is open.

Proof. Using § 4.3(b), we have inclusions

\[ \operatorname{Fix}(c|_Z)_{\operatorname{red}}\subseteq N_{\operatorname{Fix}(c|_Z)}(\operatorname{Fix}(c))_{\operatorname{red}}\subseteq \operatorname{Fix}(N_Z(c))_{\operatorname{red}}, \]

whereas our assumption implies an equality $\operatorname {Fix}(c|_Z)_{\operatorname {red}}= \operatorname {Fix}(N_Z(c))_{\operatorname {red}}$. Therefore, we have an equality $\operatorname {Fix}(c|_Z)_{\operatorname {red}}=N_{\operatorname {Fix}(c|_Z)}(\operatorname {Fix}(c))_{\operatorname {red}}$, from which our assertion follows by Lemma 4.2.

Notation 4.9 Let $c:C\to X\times X$ be a correspondence, which has no fixed points in the punctured tubular neighborhood of $Z\subseteq X$. Then, by Lemma 4.8, $\operatorname {Fix}(c|_Z)\subseteq \operatorname {Fix}(c)$ is an open subset, thus (see § 1.5(b)) to every $c$-morphism $u\in \operatorname {Hom}_c(F,F)$ one can associate an element

\[ \mathcal{T}r_{\operatorname{Fix}(c|_Z)}(u)\in H^0(\operatorname{Fix}(c|_Z),K_{\operatorname{Fix}(c|_Z)}). \]

Now we are ready to formulate the main result of this note, which by Lemma 4.7 generalizes [Var07, Theorem 2.1.3(b)].

Theorem 4.10 Let $c:C\to X\times X$ be a correspondence, and let $Z\subseteq X$ be a $c$-invariant closed subscheme such that $c$ has no fixed points in the punctured tubular neighborhood of $Z$.

1. (a) Assume that $c$ has no almost fixed points in the punctured tubular neighborhood of $Z$. Then for every $c$-morphism $u\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$, we have an equality

   \[ \mathcal{T}r_{\operatorname{Fix}(c|_Z)}(u)=\mathcal{T}r_{c|_Z}(u|_Z)\in H^0(\operatorname{Fix}(c|_Z),K_{\operatorname{Fix}(c|_Z)}). \]

2. (b) Every connected component $\beta$ of $\operatorname {Fix}(c|_Z)$, which is proper over $k$, is also a connected component of $\operatorname {Fix}(c)$. Moreover, for every $c$-morphism $u\in \operatorname {Hom}_c(\mathcal{F},\mathcal{F} )$, we have equalities

   \[ \mathcal{T}r_{\beta}(u)=\mathcal{T}r_{\beta}(u|_Z)\in H^0(\operatorname{Fix}(\beta),K_{\operatorname{Fix}(\beta)}) \quad\text{and}\quad LT_{\beta}(u)=LT_{\beta}(u|_Z)\in\Lambda. \]

As an application, we now deduce the result, stated in the introduction.

Corollary 4.11 Let $f:X\to X$ be a morphism, and let $x\in \operatorname {Fix}(f)$ be a fixed point such that the induced map of normal cones $N_x(f):N_x(X)\to N_x(X)$ has no non-zero fixed points. Then:

1. (a) point $x$ is an isolated fixed point of $f$;

2. (b) for every morphism $u:f^*\mathcal{F} \to \mathcal{F}$ with $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, we have $LT_{x}(u)=\operatorname {Tr}(u_x)$. In particular, if $\mathcal{F} =\Lambda$ and $u$ is the identity, then $LT_{x}(u)=1$.

Proof. As it was observed in § 4.6(a), our assumption implies that $\{x\}\subseteq X$ is a closed $\operatorname {Gr}_f$-invariant subscheme, and correspondence $\operatorname {Gr}_f$ has no fixed points in the punctured tubular neighborhood of $\{x\}$. Therefore part (a) follows from Lemma 4.8, while the first assertion of part (b) is an immediate corollary of Theorem 4.10. The second assertion of part (b) now follows from the obvious observation that $\operatorname {Tr}(u_x)=1$.

5. The case of group actions

Lemma 5.1 Let $D$ be a reduced diagonalizable algebraic group acting on a scheme $X$ such that either $D$ is finite or $X$ is separated, and let $Z\subseteq X$ be a $D$-invariant closed subscheme.

Then $D$ acts on the normal cone $N_Z(X)$, and the induced morphism $N_{Z^D}(X^D)\to N_Z(X)^D$ on $D$-fixed points is an isomorphism.

Proof. By the functoriality of the normal cone (see § 4.1(b)), $D$ acts on the normal cone $N_Z(X)$, so it remains to show that the map $N_{Z^D}(X^D)\to N_Z(X)^D$ is an isomorphism.

Assume first that $D$ is finite. Then every $z\in Z^D$ has a $D$-invariant open affine neighbourhood $U\subseteq X$. Thus, replacing $X$ by $U$ and $Z$ by $Z\cap U$, we can assume that $X$ and $Z$ are affine. Then we have to show that the map

(5.1)\begin{equation} k[N_Z(X)]_D\cong k[N_Z(X)^D]\to k[N_{Z^D}(X^D)] \end{equation}

is an isomorphism.

As the group $D$ is diagonalizable, its order is prime to the characteristic of $k$. Thus, the functor of coinvariants $M\mapsto M_D$ is exact on $k[D]$-modules, hence the isomorphism $k[X]_D\overset {\thicksim }{\to } k[X^D]$ induces an isomorphism between $((I_Z)^n)_D\subseteq k[X]_D$ and $(I_{Z^D})^n\subseteq k[X^D]$ for every $n$. From this the fact that the map (5.1) is an isomorphism follows.

To show the case for a general $D$, note that the set of torsion elements $D_{\operatorname {tor}}\subseteq D$ is Zariski dense. As $X$ is separated, whereas $X,Z$ and $N_Z(X)$ are Noetherian, therefore there exists a finite subgroup $D'\subseteq D$ such that $X^D=X^{D'}$ and similarly for $Z$ and $N_Z(X)$. Hence, the assertion for $D$ follows from that for $D'$, shown previously.

Corollary 5.2 Let $D$ and $X$ be as in Lemma 5.1, let $g\in D$, and let $Z\subseteq X$ be a $g$-invariant closed subscheme. Then $g$ induces an endomorphism of the normal cone $N_Z(X)$, and the induced morphism $N_{Z^g}(X^g)\to N_Z(X)^g$ between $g$-fixed points is an isomorphism.

Proof. Let $D':=\overline {\langle g\rangle }\subseteq D$ be the Zariski closure of the cyclic group $\langle g\rangle \subseteq D$. Then $D'$ is a diagonalizable group, and we have an equality $X^g=X^{D'}$ and similarly for $Z^g$ and $N_Z(X)^g$. Thus, the assertion follows from Lemma 5.1 for $D'$.

Example 5.3 Let $g:X\to X$ be an automorphism of finite order, which is prime to the characteristic of $k$. Then the cyclic group $\langle g\rangle \subseteq \operatorname {Aut}(X)$ is a diagonalizable group, thus Corollary 5.2 applies in this case. Thus, for every $g$-invariant closed subscheme $Z\subseteq X$, the natural morphism $N_{Z^g}(X^g)\to N_Z(X)^g$ is an isomorphism.

As a consequence, we get a class of examples, when the condition of Definition 4.4(a) is satisfied.

Corollary 5.4 Let $G$ be a linear algebraic group acting on a scheme $X$.

1. (a) Let $g\in G$, let $\overline {\langle g\rangle }$ be the Zariski closure of the cyclic group generated by $g$, let $s\in \overline {\langle g\rangle }$ be a semisimple element such that either $s$ is of finite order or $X$ is separated, and let $Z\subseteq X$ be an $s$-invariant closed subscheme such that $(X\smallsetminus Z)^s=\emptyset$. Then $g$ has no fixed points in the punctured tubular neighborhood of $Z$.

2. (b) Let $g\in G$ be semisimple such that either $g$ is of finite order or $X$ is separated, and let $Z\subseteq X$ be a $g$-invariant closed subscheme such that $(X\smallsetminus Z)^g=\emptyset$. Then $g$ has no fixed points in the punctured tubular neighborhood of $Z$.

Proof. (a) We have to show that $N_Z(X)^g\smallsetminus Z=\emptyset$. By assumption, we have $N_Z(X)^g\subseteq N_Z(X)^s$. Therefore, it suffices to show that $N_Z(X)^s\smallsetminus Z=N_Z(X)^s\smallsetminus Z^s=\emptyset$. As $s$ is semisimple, we conclude from Corollary 5.2 that $N_Z(X)^s=N_{Z^s}(X^s)$. Since $(X^s)_{\operatorname {red}}=(Z^s)_{\operatorname {red}}$, by assumption, we conclude that $N_{Z^s}(X^s)_{\operatorname {red}}=(Z^s)_{\operatorname {red}}$, implying the assertion.

(b) Part (b) is a particular case of part (a).

Example 5.5 An important particular case of Corollary 5.4(a) is when $s=g_s$ is the semisimple part of $g$, that is, $g=g_sg_u$ is the Jordan decomposition.

The following result gives a version of Corollary 4.11, whose assumptions are easier to check.

Corollary 5.6 Let $G$, $X$, and $g$ be as in Corollary 5.4(b), and let $x\in X^g$ be an isolated fixed point of $g$. Then the induced map of normal cones $g:N_x(X)\to N_x(X)$ has no non-zero fixed points. Therefore, for every morphism $u:g^*\mathcal{F} \to \mathcal{F}$ with $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, we have an equality

\[ LT_{x}(u)=\operatorname{Tr}(u_x). \]

Proof. The first assertion follows from Corollary 5.4(b), whereas the second follows from Corollary 4.11(b).

5.7 An application

Corollary 5.6 is used in the work of Hansen, Kaletha, and Weinstein (see [HKW22, Proposition 5.6.2]).

As a further application, we get a slight generalization of the Deligne–Lusztig trace formula.

Notation 5.8 To every proper endomorphism $f:X\to X$ and a morphism $u:f^*\mathcal{F} \to \mathcal{F}$ with $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, one associates an endomorphism $R\Gamma _c(u): R\Gamma _c(X,\mathcal{F} )\to R\Gamma _c(X,\mathcal{F} )$ (compare [Var07, § 1.1.7]).

Moreover, for an $f$-invariant closed subscheme $Z\subseteq X$, we set $U:=X\smallsetminus Z$ and form endomorphisms $R\Gamma _c(u|_Z): R\Gamma _c(Z,\mathcal{F} |_Z)\to R\Gamma _c(Z,\mathcal{F} |_Z)$ and $R\Gamma _c(u|_U): R\Gamma _c(U,\mathcal{F} |_U)\to R\Gamma _c(U,\mathcal{F} |_U)$ (compare § 1.4(d)).

Theorem 5.9 Let $G$ be a linear algebraic group acting on a separated scheme $X$, let $g\in G$ be such that $X$ has a $g$-equivariant compactification, and let $s\in \overline {\langle g\rangle }$ be a semisimple element.

Then $X^s\subseteq X$ is a closed $g$-invariant subscheme, and for every morphism $u:g^*\mathcal{F} \to \mathcal{F}$ with $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, we have an equality of traces $\operatorname {Tr}(R\Gamma _c(u))=\operatorname {Tr}(R\Gamma _c(u|_{X^s}))$ (see § 5.8).

Proof. Using the equality

\[ \operatorname{Tr}(R\Gamma_c(u))=\operatorname{Tr}(R\Gamma_c(u|_{X^s}))+\operatorname{Tr}(R\Gamma_c(u|_{X\smallsetminus X^s})), \]

it remains to show that $\operatorname {Tr}(R\Gamma _c(u|_{X\smallsetminus X^s}))=0$. Thus, replacing $X$ by $X\smallsetminus X^s$ and $u$ by $u|_{X\smallsetminus X^s}$, we may assume that $X^s=\emptyset$, and we have to show that $\operatorname {Tr}(R\Gamma _c(u))=0$.

Choose a $g$-equivariant compactification $\overline {X}$ of $X$, and set $Z:=(\overline {X}\smallsetminus X)_{\operatorname {red}}$. Let $j:X\hookrightarrow \overline {X}$ be the open inclusion, and set $\overline {\mathcal{F} }:=j_!\mathcal{F} \in D_c^b(\overline {X},\overline {\mathbb{Q} }_{\ell })$. As $X\subseteq \overline {X}$ is $g$-invariant, our morphism $u$ extends to a morphism $\overline {u}=j_!(u):g^*\overline {\mathcal{F} }\to \overline {\mathcal{F} }$, and we have an equality $\operatorname {Tr}(R\Gamma _c(u))=\operatorname {Tr}(R\Gamma _c(\overline {u}))$ (compare [Var07, § 1.1.7]). Thus, because $\overline {X}$ is proper, the Lefschetz–Verdier trace formula says that

\[ \operatorname{Tr}(R\Gamma_c(u))=\operatorname{Tr}(R\Gamma_c(\overline{u}))=\sum_{\beta\in\pi_0(\overline{X}^g)}LT_{\beta}(\overline{u}), \]

so it suffices to show that each local term $LT_{\beta }(\overline {u})$ vanishes.

As $X^g\subseteq X^s=\emptyset$, we have $(\overline {X}^g)_{\operatorname {red}}=(Z^g)_{\operatorname {red}}$. Thus, every $\beta$ is a connected component of $Z^g$. In addition, $g$ has no fixed points in the punctured neighborhood of $Z$ (by Corollary 5.4(a)). Therefore, by Theorem 4.10, we have an equality $LT_{\beta }(\overline {u})=LT_{\beta }(\overline {u}|_Z)$. However, the latter expression vanishes, because $\overline {\mathcal{F} }|_Z=0$, therefore $\overline {u}|_Z=0$. This completes the proof.

Corollary 5.10 Let $X$ be a scheme over $k$, let $g:X\to X$ be an automorphism of finite order, and let $s$ be a power of $g$ such that $s$ is of order prime to the characteristic of $k$. Then for every morphism $u:g^*\mathcal{F} \to \mathcal{F}$ with $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, we have an equality of traces

\[ \operatorname{Tr}(R\Gamma_c(u))=\operatorname{Tr}(R\Gamma_c(u|_{X^s})). \]

Proof. Note that because $g$ is an automorphism of finite order, $X$ has a $g$-invariant open dense affine subscheme $U$. Using additivity of traces

\[ \operatorname{Tr}(R\Gamma_c(u))=\operatorname{Tr}(R\Gamma_c(u|_{U}))+\operatorname{Tr}(R\Gamma_c(u|_{X\smallsetminus U})) \]

and Noetherian induction on $X$, we can therefore assume that $X$ is affine. Then $X$ has a $g$-equivariant compactification, so the assertion follows from Theorem 5.9.

Example 5.11 Applying Corollary 5.10 in the case when $\mathcal{F} =\overline {\mathbb{Q} }_{\ell }$ and $u$ is the identity, we recover the identity

\[ \operatorname{Tr}(g,R\Gamma_c(X,\mathbb{Q}_{\ell}))=\operatorname{Tr}(g,R\Gamma_c(X^s,\mathbb{Q}_{\ell})), \]

proven in [DL76, Theorem 3.2].

6. Proof of Theorem 4.10

6.1 Deformation to the normal cone

See [Var07, § 1.4.1 and Lemma 1.4.3]. Let $R=k[t]_{(t)}$ be the localization of $k[t]$ at $(t)$, set $\mathcal{D} :=\operatorname {Spec} R$, and let $\eta$ and $s$ be the generic and the special points of $\mathcal{D}$, respectively.

1. (a) Let $X$ be a scheme over $k$, and let $Z\subseteq X$ be a closed subscheme. Recall [Var07, § 1.4.1] that to these data one can associate a scheme $\widetilde {X}_{Z}$ over $X_{\mathcal{D} }:=X\times \mathcal{D}$, whose generic fiber (that is, fiber over $\eta \in \mathcal{D}$) is $X_{\eta }:=X\times \eta$, and special fiber is the normal cone $N_Z(X)$.

2. (b) We have a canonical closed embedding $Z_{\mathcal{D} }\hookrightarrow \widetilde {X}_Z$, whose generic fiber is the embedding $Z_{\eta }\hookrightarrow X_{\eta }$, and special fiber is $Z\hookrightarrow N_Z(X)$.

3. (c) The assignment $(X,Z)\mapsto \widetilde {X}_Z$ is functorial, that is, for every morphism $f:(X',Z')\to (X,Z)$ there exists a unique morphism $\widetilde {X'}_{Z'}\to \widetilde {X}_{Z}$ lifting $f_{\mathcal{D} }$ (see [Var07, Lemma 1.4.3]). In particular, $f$ gives rise to a canonical morphism $N_{Z'}(X')\to N_Z(X)$ from § 4.1(b).

4. (d) Let $c:C\to X\times X$ be a correspondence, and let $Z\subseteq X$ be a closed subscheme. Then, by part (c), one gets the correspondence $\widetilde {c}_Z: \widetilde {C}_{c^{-1}(Z\times Z)}\to \widetilde {X}_Z\times \widetilde {X}_Z$ over $\mathcal{D}$, whose generic fiber is $c_{\eta }$, and special fiber is the correspondence

   \[ N_Z(c): N_{c^{-1}(Z\times Z)}(C)\to N_Z(X)\times N_Z(X) \]
   from § 4.3(a).

5. (e) By part (b), we have a canonical closed embedding $\operatorname {Fix}(c|_Z)_{\mathcal{D} }\hookrightarrow \operatorname {Fix}(\widetilde {c}_Z)$ over $\mathcal{D}$, whose generic fiber is the embedding $\operatorname {Fix}(c|_Z)_{\eta }\hookrightarrow \operatorname {Fix}(c)_{\eta }$, and special fiber is $\operatorname {Fix}(c|_Z)\hookrightarrow \operatorname {Fix}(N_Z(c))$.

6.2 Specialization to the normal cone

Assume that we are in the situation of § 6.1.

1. (a) As in [Var07, § 1.3.2], we have a canonical functor $sp_{\widetilde {X}_Z}:D^b_{\rm ctf}(X,\Lambda )\to D^b_{\rm ctf}(N_Z(X),\Lambda )$. Moreover, for every object $\mathcal{F} \in D^b_{\rm ctf}(X,\Lambda )$, we have a canonical morphism

   \[ sp_{\widetilde{c}_Z}:\operatorname{Hom}_c (\mathcal{F},\mathcal{F})\to \operatorname{Hom}_{N_Z(c)}(sp_{\widetilde{X}_Z}(\mathcal{F}), sp_{\widetilde{X}_Z}(\mathcal{F})). \]

2. (b) As in [Var07, § 1.3.3(b)], we have a canonical specialization map

   \[ sp_{\operatorname{Fix}(\widetilde{c}_Z)}:H^0(\operatorname{Fix}(c), K_{\operatorname{Fix}(c)})\to H^0(\operatorname{Fix}(N_Z(c)), K_{\operatorname{Fix}(N_Z(c))}), \]
   which is an isomorphism when $\operatorname {Fix}(\widetilde {c}_Z)\to \mathcal{D}$ is a topologically constant family.

3. (c) Applying [Var07, Proposition 1.3.5] in this case, we conclude that for every $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$, the following diagram is commutative.

   (6.1)

   

Now we are ready to prove Theorem 4.10, mostly repeating the argument of [Var07, Theorem 2.1.3(b)].

6.3 Proof of Theorem 4.10(a)

Step 1. We may assume that $\operatorname {Fix}(c)_{\operatorname {red}}=\operatorname {Fix}(c|_Z)_{\operatorname {red}}$.

Proof. By Lemma 4.8, there exists an open subscheme $W\subseteq C$ such that

\[ W\cap\operatorname{Fix}(c)_{\operatorname{red}}=\operatorname{Fix}(c|_Z)_{\operatorname{red}}. \]

Replacing $c$ by $c|_W$ and $u$ by $u|_W$, we can assume that $\operatorname {Fix}(c)_{\operatorname {red}}=\operatorname {Fix}(c|_Z)_{\operatorname {red}}$.

Step 2. We may assume that $\mathcal{F} |_Z\simeq 0$, and it suffices to show that in this case $\mathcal{T}r _c(u)=0$.

Proof. Set $U:=X\smallsetminus Z$, and let $i:Z\hookrightarrow X$ and $j:U\hookrightarrow X$ be the embeddings. As $Z$ is $c$-invariant, one can associate to $u$ two $c$-morphisms

\[ [i_Z]_!(u|_Z)\in\operatorname{Hom}_c(i_!(\mathcal{F}|_Z),i_!(\mathcal{F}|_Z)) \quad\text{and}\quad [j_U]_!(u|_U)\in\operatorname{Hom}_c(j_!(\mathcal{F}|_U),j_!(\mathcal{F}|_U)) \]

(see [Var07, § 1.5.9]). Then, by the additivity of the trace map [Var07, Proposition 1.5.10], we conclude that

\[ \mathcal{T}r_c(u)=\mathcal{T}r_c([i_Z]_!(u|_Z))+\mathcal{T}r_c([j_U]_!(u|_U)). \]

Moreover, using the assumption $\operatorname {Fix}(c|_Z)_{\operatorname {red}}=\operatorname {Fix}(c)_{\operatorname {red}}$ and the commutativity of the trace map with closed embeddings [Var07, Proposition 1.2.5], we conclude that

\[ \mathcal{T}r_c([i_Z]_!(u|_Z))=\mathcal{T}r_{c|_Z}(u|_Z). \]

Thus, it remains to show that $\mathcal{T}r _c([j_U]_!(u|_U))=0$. For this we can replace $\mathcal{F}$ by $j_!(\mathcal{F} |_U)$ and $u$ by $[j_U]_!(u|_U)$. In this case, $\mathcal{F} |_Z\simeq 0$, and it remains to show that $\mathcal{T}r _c(u)=0$ as claimed.

Step 3: specialization to the normal cone. By the commutative diagram (6.1), we have an equality

\[ \mathcal{T}r_{N_Z(c)}(sp_{\widetilde{c}_Z}(u))=sp_{\operatorname{Fix}(\widetilde{c}_Z)}(\mathcal{T}r_c(u)). \]

Thus, to show the vanishing of $\mathcal{T}r _c(u)$, it suffices to show that:

1. (i) the map $sp_{\operatorname {Fix}(\widetilde {c}_Z)}$ is an isomorphism;

2. (ii) we have $\mathcal{T}r _{N_Z(c)}(sp_{\widetilde {c}_Z}(u))=0$.

Step 4: proof of Step 3(i). By § 6.2(b), it suffices to show that the closed embedding $\operatorname {Fix}(c|_Z)_{\mathcal{D},\operatorname {red}}\hookrightarrow \operatorname {Fix}(\widetilde {c}_Z)_{\operatorname {red}}$ (see § 6.1(b)) is an isomorphism. Moreover, we can check separately the corresponding assertions for the generic and the special fibers.

For generic fibers, the assertions follows from our assumption $\operatorname {Fix}(c)_{\operatorname {red}}=\operatorname {Fix}(c|_Z)_{\operatorname {red}}$ (see Step 1), whereas the assertion for special fibers $\operatorname {Fix}(c|_Z)_{\operatorname {red}}=\operatorname {Fix}(N_Z(c))_{\operatorname {red}}$ follows from our assumption that $c$ has no fixed points in the punctured tubular neighborhood of $Z$.

Step 5: proof of Step 3(ii). By a standard reduction, one can assume that $\Lambda$ is finite. We are going to deduce the assertion from Proposition 3.6 applied to the correspondence $N_Z(c)$ and a weakly $\mathbb{G} _m$-equivariant $sp_{\widetilde {X}_Z}(\mathcal{F} )\in D_{\rm ctf}(N_Z(X),\Lambda )$.

Note that the zero section $Z\subseteq N_Z(X)$ is closed (by § 4.1(a)). Next, because $Z$ is $c$-invariant, we have $c^{-1}(Z\times Z)=c_r^{-1}(Z)$. Therefore, it follows from § 4.1(c) that $Z\subseteq N_Z(X)$ is $N_Z(c)$-invariant, and the correspondence $N_Z(c)_t|_Z$ is identified with $Z_{N_Z(c)}=c|_Z$.

As $c$ has no almost fixed points in the punctured tubular neighborhood of $Z$, we conclude that $N_Z(c)$ satisfies the assumptions of Proposition 3.6. Thus, it remains to show that $sp_{\widetilde {X}_Z}(\mathcal{F} )|_Z\simeq 0$ and that $sp_{\widetilde {X}_Z}(\mathcal{F} )$ is weakly $\mathbb{G} _m$-equivariant with respect to the $n$-twisted action for some $n$.

Both assertions follow from results of Verdier [Ver83]. Namely, the vanishing assertion follows from isomorphism $sp_{\widetilde {X}_Z}(\mathcal{F} )|_Z\simeq \mathcal{F} |_Z$ (see [Ver83, $\S$8, (SP5)] or [Var07, Proposition 1.4.2]) and our assumption $\mathcal{F} |_Z\simeq 0$ (see Step 2). The equivariance assertion follows from the fact that $sp_{\widetilde {X}_Z}(\mathcal{F} )$ is monodromic (see [Ver83, $\S$8, (SP1)]), because $\Lambda$ is finite (use [Ver83, Proposition 5.1]).

6.4 Proof of Theorem 4.10(b)

The first assertion follows from Lemma 4.8. To show the second, choose an open subscheme $W\subseteq C$ such that $W\cap \operatorname {Fix}(c)_{\operatorname {red}}=\beta _{\operatorname {red}}$. Replacing $c$ by $c|_W$, we can assume that $\beta _{\operatorname {red}}=\operatorname {Fix}(c)_{\operatorname {red}}=\operatorname {Fix}(c|_Z)_{\operatorname {red}}$, thus $\operatorname {Fix}(c|_Z)$ is proper over $k$.

As it was already observed in Step 5 of § 6.3, the correspondence $N_Z(c)|_Z$ is identified with $c|_Z$. Thus $\operatorname {Fix}(N_Z(c)|_Z)=\operatorname {Fix}(c|_Z)$ is proper over $k$. It now follows from Lemma 3.9 that the finiteness condition in Definition 4.4(b) is satisfied automatically, therefore $c$ has no almost fixed points in the tubular neighborhood of $Z$ (see § 4.5(c)). Now the equality $LT_{\beta }(u)=LT_{\beta }(u|_Z)$ follows from obvious equalities $\mathcal{T}r _{\beta }(u)=\mathcal{T}r _{\operatorname {Fix}(c|_Z)}(u), \mathcal{T}r _{\beta }(u|_Z)=\operatorname {Tr}_{c|_Z}(u|_Z)$ and part (a).

7. Proof of Proposition 2.5

We are going to deduce the result from the assertion that trace maps commute with nearby cycles.

7.1 Set up

Let $\mathcal{D}$ be a spectrum of a discrete valuation ring over $k$ with residue field $k$, and let $f:X\to \mathcal{D}$ be a morphism of schemes of finite type.

1. (a) Let $\eta$, $\overline {\eta }$, and $s$ be the generic, the geometrically generic, and the special point of $\mathcal{D}$, respectively. We denote by $X_{\eta }$, $X_{\overline {\eta }}$, and $X_s$ the generic, the geometric generic, and the special fiber of $X$, respectively, and let $i_{{\eta }}:X_{{\eta }}\to X$, $i_{\overline {\eta }}:X_{\overline {\eta }}\to X$, $i_s:X_s\to X$, and $\pi _{\eta }:X_{\overline {\eta }}\to X_{\eta }$ be the canonical morphisms.

2. (b) For every object $\mathcal{F} \in D(X,\Lambda )$, we set $\mathcal{F} _{\eta }:=i^*_{\eta }(\mathcal{F} )$, $\mathcal{F} _{\overline {\eta }}:=i^*_{\overline {\eta }}(\mathcal{F} )$, and $\mathcal{F} _{s}:=i^*_{s}(\mathcal{F} )$. For every object $\mathcal{F} _{\eta }\in D(X_{\eta },\Lambda )$, we set $\mathcal{F} _{\overline {\eta }}:=\pi _{\eta }^*(\mathcal{F} _{\eta })$.

3. (c) Let $\Psi =\Psi _X:D_{\rm ctf}^b(X_{\eta },\Lambda )\to D_{\rm ctf}^b(X_{s},\Lambda )$ be the nearby cycle functor. By definition, it is defined by the formula $\Psi _X(\mathcal{F} _{\eta }):=i^*_s i_{\overline {\eta }*}(\mathcal{F} _{\overline {\eta }})$.

4. (d) Consider functor $\overline {\Psi }_X:=i^*_s\circ i_{\overline {\eta }*}:D(X_{\overline {\eta }},\Lambda )\to D(X_{s},\Lambda )$. Then we have an equality $\Psi _X(\mathcal{F} _{\eta })=\overline {\Psi }_X(\mathcal{F} _{\overline {\eta }})$ for all $\mathcal{F} _{\eta }\in D_{\rm ctf}^b(X_{\eta },\Lambda )$.

7.2 ULA sheaves

Assume that we are in the situation of § 7.1.

1. (a) We have a canonical isomorphism $\Psi _{{X}}\circ i^*_{\eta }\simeq i_s^*\circ i_{\overline {\eta }*}\circ i^*_{\overline {\eta }}$ of functors $D_{\rm ctf}^b(X,\Lambda )\to D_{\rm ctf}^b(X_s,\Lambda )$. In particular, the unit map $\operatorname {Id}\to i_{\overline {\eta }*}\circ i^*_{\overline {\eta }}$ induces a morphism of functors $i_s^*\to \Psi _{{X}}\circ i^*_{\eta }=\overline {\Psi }_{{X}}\circ i^*_{\overline {\eta }}$.

2. (b) Note that if $\mathcal{F} \in D_{\rm ctf}^b(X,\Lambda )$ is ULA over $\mathcal{D}$, then the induced morphism

   \[ \mathcal{F}_s=i_s^*(\mathcal{F})\to (\Psi_{{X}}\circ i^*_{\eta})(\mathcal{F})=\Psi_X(\mathcal{F}_{\eta})=\overline{\Psi}_X(\mathcal{F}_{\overline{\eta}}) \]
   is an isomorphism. In particular, we have a canonical isomorphism $\Lambda _{s}\simeq \overline {\Psi }_{\mathcal{D} }(\Lambda _{\overline {\eta }})$.

7.3 Construction

Assume that we are in the situation of § 7.1.

1. (a) For every $\mathcal{F} _{\overline {\eta }}\in D(X_{\overline {\eta }},\Lambda )$, consider composition

   \[ R\Gamma(X_{\overline{\eta}}, \mathcal{F}_{\overline{\eta}})\simeq R\Gamma(X, i_{\overline{\eta}*}(\mathcal{F}_{\overline{\eta}}))\overset{i_s^*}{\longrightarrow} R\Gamma(X_s, i_s^*i_{\overline{\eta}*}(\mathcal{F}_{\overline{\eta}}))=R\Gamma(X_s, \overline{\Psi}_X(\mathcal{F}_{\overline{\eta}})). \]

2. (b) Consider canonical morphism $\overline {\Psi }_X(K_{X_{\overline {\eta }}})\to K_{X_s}$, defined as a composition

   \[ \overline{\Psi}_X(K_{X_{\overline{\eta}}})=\overline{\Psi}_X(f_{\overline{\eta}}^!(\Lambda_{\overline{\eta}}))\overset{BC}{\longrightarrow}f_{s}^!(\overline{\Psi}_{\mathcal{D}}(\Lambda_{\overline{\eta}}))\simeq f_{s}^!(\Lambda_s)=K_{X_s}. \]

3. (c) Denote by $\overline {\operatorname {Sp}}_X$ the composition

   \[ R\Gamma(X_{\overline{\eta}}, K_{X_{\overline{\eta}}})\overset{(a)}{\longrightarrow} R\Gamma(X_s, \overline{\Psi}_X(K_{X_{\overline{\eta}}}))\overset{(b)}{\longrightarrow} R\Gamma(X_s,K_{X_s}). \]

4. (d) Using the observation $K_{X_{\overline {\eta }}}\simeq \pi _{\eta }^*(K_{X_{{\eta }}})$, we denote by $\operatorname {Sp}_X$ the composition

   \[ R\Gamma(X_{{\eta}}, K_{X_{{\eta}}})\overset{\pi_{\eta}^*}{\longrightarrow} R\Gamma(X_{\overline{\eta}}, K_{X_{\overline{\eta}}})\overset{\overline{\operatorname{Sp}}_X}{\longrightarrow} R\Gamma(X_s, K_{X_s}). \]

Lemma 7.4 Assume that $f:X\to \mathcal{D}$ is a topologically constant family (see § 2.3). Then the specialization map $\overline {\operatorname {Sp}}_X:R\Gamma (X_{\overline {\eta }}, K_{X_{\overline {\eta }}})\to R\Gamma (X_s, K_{X_s})$ of § 7.3(c) coincides with the canonical identification of Claim 2.4.

Proof. Though the assertion follows by straightforward unwinding the definitions, we sketch the argument for the convenience of the reader.

As in the proof of Claim 2.4, we set $K_{X/\mathcal{D} }:=f^!(\Lambda _{\mathcal{D} })$ and $\mathcal{F} :=f_*(K_{X/\mathcal{D} })$. Consider the diagram

(7.1)

where:

• • maps denoted by $BC_*$ are induced by the (base change) isomorphisms $\mathcal{F} _{\overline {\eta }}\overset {\thicksim }{\to } f_{\overline {\eta }*}((K_{X/\mathcal{D} })_{\overline {\eta }})$, $\mathcal{F} _{s}\overset {\thicksim }{\to } f_{s*}((K_{X/\mathcal{D} })_s)$ and base change morphisms; whereas

• • maps denoted by $BC^*$ are induced by the (base change) isomorphisms $(K_{X/\mathcal{D} })_{\overline {\eta }}\overset {\thicksim }{\to } K_{X_{\overline {\eta }}}$ and $(K_{X/\mathcal{D} })_{s}\overset {\thicksim }{\to } K_{X_s}$.

We claim that the diagram (7.1) is commutative. As the top left, the top right, and the bottom left inner squares are commutative by functoriality, it remain to show the commutativity of the right bottom inner square. In other words, it suffices to show the commutativity of the following diagram.

Moreover, using identity $K_{X/\mathcal{D} }=f^!(\Lambda _{\mathcal{D} })$, it suffices to show the commutativity of the following diagram, which is standard.

By the commutativity of (7.1), it remains to show that the top arrow

\[ \mathcal{F}_{\overline{\eta}}=R\Gamma(\overline{\eta},\mathcal{F}_{\overline{\eta}})\to R\Gamma(s, \overline{\Psi}_{\mathcal{D}}(\mathcal{F}_{\overline{\eta}}))\simeq R\Gamma(s,\mathcal{F}_s)=\mathcal{F}_s \]

of (7.1) equals the inverse of the specialization map

\[ \mathcal{F}_s=R\Gamma(s,\mathcal{F}_s)\simeq R\Gamma(\mathcal{D},\mathcal{F})\overset{i_{\overline{\eta}}^*}{\longrightarrow}R\Gamma(\overline{\eta},\mathcal{F}_{\overline{\eta}})=\mathcal{F}_{\overline{\eta}}. \]

But this follows from the commutativity of the following diagram.

7.5 Specialization of cohomological correspondences

Let ${c}:{C}\to {X}\times {X}$ be a correspondence over $\mathcal{D}$, let ${c}_{\eta }:{C}_{\eta }\to {X}_{\eta }\times {X}_{\eta }$, ${c}_{\overline {\eta }}:{C}_{\overline {\eta }}\to {X}_{\overline {\eta }}\times {X}_{\overline {\eta }}$, and ${c}_{s}:{C}_{s}\to {X}_{s}\times {X}_{s}$ be the generic, the geometric generic, and the special fibers of ${c}$, respectively. Fix $\mathcal{F} _{\eta }\in D_{\rm ctf}^b({X}_{\eta },\Lambda )$.

1. (a) Using the fact that the projection $\pi _{\eta }:\overline {\eta }\to \eta$ is pro-étale, we have the following commutative diagram.

   

2. (b) Consider the map

   \[ \Psi_{{c}}:\operatorname{Hom}_{{c}_{\eta}}(\mathcal{F}_{\eta},\mathcal{F}_{\eta})\to \operatorname{Hom}_{{c}_s}(\Psi_{{X}}(\mathcal{F}_{\eta}),\Psi_{{X}}(\mathcal{F}_{\eta})), \]
   which sends morphism $u_{\eta }:{c}_{\eta l}^*(\mathcal{F} _{\eta })\to {c}_{\eta r}^!(\mathcal{F} _{\eta })$ to the composition
   \[ {c}_{sl}^*(\Psi_{{X}}(\mathcal{F}_{\eta}))\overset{BC}{\longrightarrow}\Psi_{C}({c}_{\eta l}^*(\mathcal{F}_{\eta}))\overset{\Psi_{{C}}(u_{\eta})}{\longrightarrow} \Psi_{{C}}({c}_{\eta r}^!(\mathcal{F}_{\eta}))\overset{BC}{\longrightarrow} {c}_{sr}^!(\Psi_{{X}}(\mathcal{F}_{\eta})). \]

Proposition 7.6 In the situation § 7.5, the following diagram is commutative.

Proof. The assertion and its proof is a small modification [Var07, Proposition 1.3.5]. Alternatively, the assertion can be deduced from the general criterion of [Var07, § 4]. Namely, repeating the argument of [Var07, § 4.1.4(b)] word-by-word, one shows that the nearby cycle functors $\Psi _{\cdot }$ together with base change morphisms define a compactifiable cohomological morphism in the sense of [Var07, § 4.1.3]. Therefore, the assertion follows from (a small modification of) [Var07, Corollary 4.3.2].

Lemma 7.7 Let ${c}:{C}\to {X}\times {X}$ be a correspondence over $\mathcal{D}$. Then for every $\mathcal{F} \in D_{\rm ctf}^b({X},\Lambda )$ and $u\in \operatorname {Hom}_{{c}}(\mathcal{F},\mathcal{F} )$, the following diagram is commutative.

Proof. The assertion is a rather straightforward diagram chase. Indeed, it suffices to show the commutativity of the following diagram.

(7.2)

We claim that all inner squares of (

7.2

) are commutative. Namely, the middle inner square is commutative by functoriality, whereas the commutativity of the left and the right inner squares follows by formulas $\overline {\Psi }_{\cdot }=i^*_s\circ i_{\overline {\eta }*}$ and definitions of the base change morphisms.

Now we are ready to show Proposition 2.5.

7.8 Proof of Proposition 2.5

Without loss of generality, we can assume that $s$ is a specialization of $t$ of codimension one. Then there exists a spectrum of a discrete valuation ring $\mathcal{D}$ and a morphism $f:\mathcal{D} \to S$ whose image contains $s$ and $t$. Taking base change with respect to $f$ we can assume that $S=\mathcal{D}$, $t=\overline {\eta }$ is the geometric generic point, whereas $s$ is the special point.

Then we have equalities

\begin{align*} \mathcal{T}r_{{c}_s}(u_s)&=\mathcal{T}r_{c_s}(\Psi_{c}(u_{\eta}))=\operatorname{Sp}_{\operatorname{Fix}(c)}(\mathcal{T}r_{{c}_{\eta}}(u_{\eta}))\\&= \overline{\operatorname{Sp}}_{\operatorname{Fix}(c)}(\pi_{\eta}^*(\mathcal{T}r_{{c}_{\eta}}(u_{\eta}))) =\overline{\operatorname{Sp}}_{\operatorname{Fix}(c)}(\mathcal{T}r_{{c}_{\overline{\eta}}}(u_{\overline{\eta}})), \end{align*}

where:

• • the first equality follows from the fact that the isomorphism $\mathcal{F} _s\to \Psi _{X}(\mathcal{F} _{\eta })$ from § 7.2(b) identifies $u_{s}$ with $\Psi _{{c}}(u_{\eta })$ (by Lemma 7.7);

• • the second equality follows from the commutative diagram of Proposition 7.6;

• • the third equality follows from definition of $\operatorname {Sp}_X$ in § 7.3(d);

• • the last equality follows from the commutative diagram of § 7.5(a).

Now the assertion follows from Lemma 7.4.