Global index of real polynomials

Abstract

We develop two methods for expressing the global index of the gradient of a 2 variable polynomial function $f$: in terms of the atypical fibres of $f$, and in terms of the clusters of Milnor arcs at infinity. These allow us to derive upper bounds for the global index, in particular refining the one that was found by Durfee in terms of the degree of $f$.

1. Introduction

The index of a vector field with isolated zeroes enters in the celebrated Poincaré–Hopf theorem which holds for compact manifolds and has been extended in various directions, either in real or in complex geometry. The index of the gradient vector field $\mathrm {{grad\ }} f$ along the boundary of a large disk which includes all the singularities of a polynomial function $f : {\mathbb {R}}^{2} \to {\mathbb {R}}$ of degree $d\ge 2$ is the ‘global index,’ or the ‘index at infinity’ of $f$, denoted ${\rm {ind}}_\infty f$. The study of this index at infinity originates, as far as we know, in Durfee's paper [12], and is addressed in several other papers, see e.g. [15, 33, 34].

In the local setting, Arnold's index theorem [5] asserts that the index at an isolated singular point $p\in C$ of a real plane curve $C :=\{g=0\}$ with $r$ branches satisfies the equality ${\rm {ind}}_{p} (\mathrm {{grad\ }} g )= 1-r$. In the complex setting, for a holomorphic function germ $h$ of $n\ge 2$ variables and with isolated singularity, it is well-known that the local index ${\rm {ind}}_{p} (\mathrm {{grad\ }} h )$ equals the Minor number $\mu _{h}$, which has several topological and algebraic interpretations.

We address here the problem of computing the index at infinity of $f$ by using the topological behaviour of $f$ at infinity, and more precisely its atypical fibres. Let us briefly recall (cf. definition 2.1) that a fibre $f^{-1}(\lambda )$ of a polynomial function $f:{\mathbb {K}}^{n} \to {\mathbb {K}}$, where ${\mathbb {K}} ={\mathbb {R}}$ or ${\mathbb {C}}$, is typical if $f$ is a trivial C$^{\infty }$-fibration over some neighbourhood of $\lambda$, and that the bifurcation locus of $f$ is the minimal subset ${\rm {Atyp}\ } f\subset {\mathbb {K}}$ such that the restriction $f_{|}: {\mathbb {K}}^{n}\setminus {\rm {Atyp}\ } f \to {\mathbb {K}}$ is a locally trivial ${\rm C}^{\infty }$-fibration. The case of 2 variables is the only one where a complete characterization of ${\rm {Atyp}\ } f$ is known, e.g. by Suzuki's study [36]. Motivated by the Jacobian Conjecture, [36] treated the complex setting showing that the variation of the Euler characteristic detects precisely which regular fibre is atypical or not; see also the subsequent contribution [17], and in more general settings [19, 25]. The real setting is more delicate, and a complete characterization in 2 variables occurred more than two decades later [40], see also the subsequent contributions, e.g. [7, 9, 10, 18].

Our study uses a method which permits to approach Arnold's local index formula as well as the more complex computation of the index at infinity, namely the Milnor locus of $f$ (definition 3.1). This has already been used by several authors in the study of the topology of function germs, probably starting with Milnor's lecture notes [27], and in the study of the change of topology of fibres of polynomials at infinity, cf [9, 11, 31, 38]. We introduce Milnor arcs at infinity (in § 3), define the clusters of such arcs, and show how these clusters detect the bifurcation locus of $f$ via the phenomena of splitting and vanishing at infinity. Our first main result, theorem 4.4, tells that ${\rm {ind}}_\infty f$ can be expressed in terms of the numbers of vanishing and splitting components of the fibres of $f$.

The Milnor arcs at some point $p$ on the line at infinity of the projective compactification ${\mathbb {P}}^{2} \supset {\mathbb {R}}^{2}$ come with an index, which may be $+\frac 12$, $0$ or $-\frac 12$, and their sum defines the local index at infinity $i_{p}$. Our key lemma 5.2 highlights the inequality $i_{p}\le d_{p}-1$ observed by Durfee in [12], where $d_{p}$ is the order at the point $p$ of the top degree part $f_{d}$ of $f$. We establish the origin of the ‘gaps’ which produce the difference between the two sides, we classify these gaps and we explain how to track down their occurrence.

It is not difficult to show that $| {\rm {ind}}_\infty f | \le d-1$ (by Bezout's theorem), and that the lower bound $1-d$ is realized for instance by a generic arrangement of lines.¹ Durfee [12] proved the inequality:

(1.1)\begin{equation} {\rm{ind}}_\infty f \le \max\{1, d-3\} \end{equation}

and raised the problem of estimating a better upper bound, since many examples show that the index at infinity may be quite far from $d-3$.

In order to get a grip on sharper upper bounds one has to consider more invariants than just the degree of the polynomial. To this aim, we use the knowledge on the intrinsically defined atypical values and atypical points at infinity. Theorem 5.1 exploits the classification provided by lemma 5.2 and casts new invariants into a formula which lowers bound (1.1) found by Durfee in [12], as follows:

\begin{align*} & {\rm{ind}}_{\infty}(f) \le 1 + d_{Re} - 2|{\mathcal{L}}_{f}|\\ & \quad - \sum_{p\in L^{\infty}\cap \{f_{d} =0\}} \left(\frac12 \left\lfloor \frac{\deg(R_{p}^{{\mathrm{red}}})-1}{2} \right\rfloor + \deg(S_{p}) + \deg(K_{p}) \right), \end{align*}

where $d_{Re}$ is the real degree of $f$, ${\mathcal {L}}_{f}$ is the set of points at infinity of $f$, $L^{\infty }$ is the line at infinity and $R_{p}, S_{p}, K_{p}$ are certain subvarieties of the tangent cone to the Milnor set at $p\in L^{\infty }$ (cf. § 5 for all these definitions and notations). Corollary 5.4 offers two simpler upper bound estimations.

It appears that the case of a single point at infinity is responsible for the indices closest to Durfee's bound (1.1). In § 5.2 we revisit and complete Durfee's proof of (1.1) in [12], and we clarify a couple of shadow points in it. Our theorem 5.5 is a slight improvement, in particular we show: If $f$ has a single point at infinity, then ${\rm {ind}}_{\infty }(f) \le d-3$ for $d \ge 4$, and ${\rm {ind}}_{\infty }(f) \le 0$ for $d\le 3$.

Section 2 concerns properties of fibres of polynomials and their connected components. We review some of the Durfee results [12] and solve some unclear points in his paper, see for instance lemma 2.2, the remark and the corollary following it.

In § 3 we provide the necessary preparation for the statements and proofs of theorem 4.4, lemma 5.2 and theorem 5.1. In particular we show the key propositions 3.15 and 3.16. Section 6 contains examples with explicit computations of the index at infinity and of the ingredients of our main formulas.

2. Fibres of polynomials

Let $f : {\mathbb {R}}^{2} \to {\mathbb {R}}$ be a polynomial function of degree $d\ge 2$, and let $f_{d}$ denote its degree $d$ homogeneous part.

Definition 2.1 We say that $\lambda \in {\mathbb {R}}$ is a typical value, or that $f^{-1}(\lambda )$ is a typical fibre, of $f$ if the restriction $f_{|}: f^{-1}(D) \to D$ is a trivial C$^{\infty }$-fibration, for some small enough disk $D\subset {\mathbb {R}}$ centred at $\lambda$. We also say that $\lambda \in {\mathbb {R}}$ is a typical value of $f$ at infinity if there is a disk $D\subset {\mathbb {R}}$ centred at $\lambda$ and a large enough ball $B\subset {\mathbb {R}}^{2}$ centred at the origin, such that the restriction $f_{|}: f^{-1}(D) \cap ({\mathbb {R}}^{2}\setminus B) \to D$ is a trivial fibration.

The set ${\rm {Atyp}\ } f$ of atypical values of $f$ (also called the bifurcation set of $f$) is the minimal subset of ${\mathbb {R}}^{2}$ which contains the critical set $f({\rm {Sing}\ } f)$ and such that the restriction $f_{|}: {\mathbb {R}}^{2}\setminus {\rm {Atyp}\ } f \to {\mathbb {R}}$ is a locally trivial ${\rm C}^{\infty }$-fibration.

In more than 2 variables there is no complete characterization of atypical values and the studies focused on finding effective approximations of the bifurcation locus, in particular finding upper bounds for the number of atypical values in terms of the degree (and possibly of some other data), cf. [11, 20, 21, 23]. The problem of finding such upper bounds is equally important in 2 variables, see e.g. [14, 16, 22, 26].

Let $X:= \{ \tilde f(x,y, z) -tz^{d} =0 \} \subset {\mathbb {P}}^{2}\times {\mathbb {R}}$, where $\tilde f$ denotes the homogenization of $f$ of degree $d$. Let $\tau : X \to {\mathbb {R}}$ be the projection on the second factor, and let $X_{t}:= \tau ^{-1}(t)$ be the fibre of $\tau$ over $t$. Let $L^{\infty } := \{z=0\}\simeq {\mathbb {P}}^{1}$ be the line at infinity of ${\mathbb {P}}^{2}$. The part at infinity $X^{\infty }:= X \cap (L^{\infty }\times {\mathbb {R}}) = \{ f_{d} =0\} \times {\mathbb {R}}$ consists of finitely many lines. The algebraic space $X$ may be endowed with a Whitney stratification such that $X^{\infty }$ is a union of strata. This consists of the open stratum ${\mathbb {R}}^{2}\subset X$ of dimension 2, and the finitely many strata of $X^{\infty }$ which are either of dimension 1 or of dimension 0. Let us denote by ${\mathcal {S}}_{0}$ the finite set which is the union of these strata of dimension 0.

Then $X_{t}\subset {\mathbb {P}}^{2}$ is a closed set and contains the closure $\overline {F_{t}}$ of the fibre $F_{t} := f^{-1}(t)$. The part at infinity $X_{t}\cap L^{\infty } =\{ f_{d} =0\}\subset {\mathbb {P}}^{1}$ is a finite set and it is independent of $t\in {\mathbb {R}}$. We have the inclusion $\{ f_{d} =0\} \supset \overline {F_{t}}\cap L^{\infty }$, which may be strict, like in the example $f = x^{2}+y^{4}$ where we have $\overline {F_{t}} \cap L^{\infty } = \emptyset$, but $X_{t} \cap L^{\infty }=[1:0]$ for all $t\in {\mathbb {R}}$.

The map $\tau : X \to {\mathbb {R}}$ has as singular locus ${\rm {Sing}\ } \tau = {\mathcal {S}}_{0} \cup {\rm {Sing}\ } f$. In particular the set $\tau ({\rm {Sing}\ } \tau )$ of critical values of $\tau$ is finite, and it is well-known, see e.g. [38–40], that we have the inclusion ${\rm {Atyp}\ } f \subset \tau ({\rm {Sing}\ } \tau )$.

The set ${\mathbb {R}}\setminus {\rm {Atyp}\ } f$ is therefore a union of intervals, two of which are unbounded (and they coincide with ${\mathbb {R}}$ in case ${\rm {Atyp}\ } f = \emptyset$). Let us denote the infinite intervals by $I_+$ (the one towards $+\infty$), and by $I_{-}$ (the one towards $-\infty$). Consequently $f_{|} : f^{-1}(I) \to I$ is a trivial fibration, where $I$ is either $I_+$ or $I_{-}$.

Let then $F_{-}$ and $F_+$ denote the fibre of $f$ over some point of $I_{-}$, and of $I_+$, respectively.

Lemma 2.2 Assume that the fibre $F_+$ contains a compact component $C$. Then $F_+ =C$ and $F_{-}$ is empty, and moreover, all the non-empty fibres of $f$ are compact.

The same statement holds if we switch the roles of $F_{-}$ and $F_+$.

Remark 2.3 In [12, Prop 4.3, point 2 of the conclusion] it is stated that if all fibres of $f$ are compact or empty, then $f_{d}$ has no linear factors. A simple polynomial $f=x^4+y^2$ shows that this conclusion is not true: we have $d=4$, all fibres are compact or empty, and $f_4=x^4$ has four linear factors $x$.

Proof Proof of lemma 2.2

Let $F_+ = f^{-1}(a)$ for some $a\in I_+$. Consider the surface $S_+ :=f^{-1}\left ( [a, +\infty )\right ) \subset {\mathbb {R}}^{2}$.

Let us denote by $C_+$ the connected component of $S_+$ which contains $C$. The family of ovals $C_+$ has been considered and studied in [24], to which we refer the reader for more details. We claim that $C_+$ equals the exterior of the oval $C$. Indeed, along each direction outside the oval $C$, the value of the function $f$ tends to infinity. Therefore the fibres of $f$ over $[a, +\infty )$ must fill in the region of ${\mathbb {R}}^{2}$ outside $C$. Since this region coincides with $S_+$ and it is connected, and since it is the total space of the trivial fibration $f_{|} : S_+\to [a, +\infty )$, we deduce the equality $S_+=C_+$. The hypothesis that $f$ is a trivial fibration over the interval $I_+$ then implies that the fibre $F_+$ coincides with the oval $C$. In particular the fibres of $f$ over $I_+$ are compact.

Moreover, this also shows that $f$ takes values less than $a$ inside the oval $C$, and since $f$ is bounded inside the compact oval $C$, it follows that the fibres $f^{-1}(t)$ are compact for $t\le a$, and that they are empty for all $t < b$ for some value $b\in {\mathbb {R}}$.

Definition 2.4 Let ${\mathcal {L}}_{f } :=\{\overline {F_{t}} \cap L^{\infty } \mid t\in {\mathbb {R}} \}$, which is a finite set of points included in the part at infinity $\{f_{d}=0 \}$, and let $|{\mathcal {L}}_{f}| := \# {\mathcal {L}}_{f }$.

By definition ${\mathcal {L}}_{f }$ collects all the ‘points at infinity’ of the fibres of $f$. Unlike the complex setting where the image of $f_{{\mathbb {C}}}$ is ${\mathbb {C}}$ and all fibres have the same points at infinity, in our real setting some fibres may be empty (example 6.4), some fibres may be compact and some others not (example 6.3), or those fibres which contain non-compact components may not have all the same points at infinity (example 2.7).

Corollary 2.5 [12, Prop 4.4(1) and Prop 4.3]

If $p\in {\mathcal {L}}_{f }$ then $p\in \overline {F_+}\cap L^{\infty }$ or $p\in \overline {F_{-}}\cap L^{\infty }$.

Proof. We consider $D := X\setminus (\{p\}\times {\mathbb {R}})$, which is a connected set, and we apply the proof of lemma 2.2 to $D$ instead of ${\mathbb {R}}^{2}$, and to the fibres $X_{t}$ restricted to $D$. Namely, by contradiction, if $p\not \in \overline {F_+}\cap L^{\infty }$ then $\overline {F_+}$ is compact in $D$ and so, by lemma 2.2, all the fibres $X_{t}\cap D$ are compact in $D$ or empty. This shows that $p\not \in X_{t}$ for any $t\in {\mathbb {R}}$, which contradicts our hypothesis.

We give an account of Durfee's proof of the following result, for the reader's convenience.

Proposition 2.6 [12, Cor 4.2 and Prop. 4.4(2)]

The number of connected components of $F_+\cup F_{-}$ is at least $2|{\mathcal {L}}_{f}|$.

Proof. For the reader's convenience, we recall here Durfee's proof. One considers a sequence of blow-ups at points at infinity in ${\mathbb {P}}^{2}$ which yields a resolution at infinity of $f$. This produces a space $M$ and a proper map $\hat f : M\to {\mathbb {R}}$ which extends $f$. One may regard the space $M$ as the disjoint union of ${\mathbb {R}}^{2}$ and a finite number of divisors at infinity. Some of these divisors (denoted by $E$) are ‘horizontal,’ i.e. the restriction $\hat f_{|E} : E\to {\mathbb {R}}$ is a non-constant polynomial of one variable, and the others are ‘vertical,’ i.e. the restriction $\hat f_{|E} : E\to {\mathbb {R}}$ is constant.

For each point of ${\mathcal {L}}_{f}$ there is at least one horizontal divisor. Considering a horizontal divisor $E$, if $\hat f_{|E}$ is a polynomial of odd degree, then it is injective outside a compact interval $[-A, A]$, and therefore $(\hat f)^{-1}(a)$ has one solution for every $a$ such that $|a| > A >0$. If the degree is even, then we have two solutions towards $-\infty$ and no solution towards $+\infty$, or the other way around. To each such solution corresponds a local branch of $(\hat f)^{-1}(a)$, and for each half-branch we count one connected component of the fibre $f^{-1}(a)$. In this way each connected component of $f^{-1}(a)$ is counted twice, whether or not its two intersection points with the horizontal divisors coincide.

We therefore obtain that $\# F_+\cup F_{-}$ equals the double of the number of the horizontal divisors. In particular we get the claimed inequality $\# F_+\cup F_{-} \ge 2| {\mathcal {L}}_{f}|$.

The following example by Durfee shows that the inequality of proposition 2.6 may be very far from an equality.

Example 2.7 [12, p. 1347] Let $f=x(y+1) \cdots (y+k)$, $k\ge 2$. The fibre $f^{-1}(0)$ produces a partition of the plane into $2(k+1)$ horizontal strips between parallel lines delimited by the vertical axis. Each horizontal strip contains a connected component of $F_+\cup F_{-}$, whereas $|{\mathcal {L}}_{f}|=2$.

3. Milnor arcs and clusters at infinity

3.1. Milnor arcs at infinity

The idea of the Milnor set was introduced by Milnor [27] and has been used in many papers ever since, either locally for germs of functions [1, 2, 4, 6, 32, 37], or globally for polynomial functions [3, 8, 9, 29, 31, 38, 39].

Let $\rho _a : {\mathbb {R}}^2 \rightarrow {\mathbb {R}} _{\geq 0}$, $\rho _a(x,y)=(x-a_1)^2+(y-a_2)^2$ be the square of the Euclidean distance to $a:= (a_1 , a_2) \in {\mathbb {R}}^2$.

Definition 3.1 The Milnor set of $f : {\mathbb {R}} ^2 \rightarrow {\mathbb {R}}$ relative to $\rho _{a}$ is the set of $\rho _a$-nonregular points of $f$, namely

\[ M_a(f) := \{ (x,y) \in {\mathbb{R}}^2 \mid \rho_a \not\pitchfork_{(x,y)} f \}. \]

Equivalently, the Milnor set $M_a(f)$ is the zero set $\{{\rm {Jac}\ } F_{a} =0\}$ considered with reduced structure, where ${\rm {Jac}\ } F_{a}$ denotes the determinant of the Jacobian matrix of the map $F_a:= (f, \rho _a): {\mathbb {R}}^2 \rightarrow {\mathbb {R}}^2$.

Proposition/Definition 3.2 Milnor arcs at infinity. [29,30]

Let $f : {\mathbb {R}}^2 \to {\mathbb {R}}$ be a non-constant polynomial. There exists a dense subset $\Omega (f)\subset {\mathbb {R}}^{2}$ of points such that $M_{a}(f)$ is of dimension 1 for any $a\in \Omega (f)$. For each such point $a \in \Omega (f)$, there exists a radius $R_{a}\gg 1$ such that for any $R\ge R_{a}$, and denoting by $D_{R}(a)\subset {\mathbb {R}}^{2}$ the closed disk centred at $a$ of radius $R$, one has:

1. (a) The set $M_{a}(f) \setminus [ D_{R}(a) \cup {\rm {Sing}\ } f ]$ is a disjoint union of finitely many 1-dimensional connected manifolds, that we denote by $\gamma _{1}, \ldots, \gamma _{s}$.

2. (b) One may endow each $\gamma _{i}$ with a parametrization $\gamma _{i} : ]R, +\infty [ \to {\mathbb {R}}^2$ such that the restriction $(\rho _{a})_{|\gamma _{i}}$ is strictly monotonous and tends to infinity as the parameter $t$ tends to infinity; we call Milnor arc at infinity the parametrized curve $\gamma _{i}$.

3. (c) for every Milnor arc at infinity, the restriction $f_{|\gamma _{i}(t)}$ is either:

   • • strictly increasing as $t\to +\infty$, and if $\lim _{t \to \infty } f(\gamma (t)) = \lambda \in {\mathbb {R}}\cup \{+\infty \}$, then we say that $\gamma$ is an increasing Milnor arc at infinity of $f$ associated to $\lambda$, and abbreviate this by $f_{|\gamma }{\nearrow } \lambda$,or

   • • strictly decreasing as $t\to +\infty$, and if $\lim _{t \to \infty } f(\gamma (t)) = \lambda \in {\mathbb {R}}\cup \{-\infty \}$, then we say that $\gamma$ is a decreasing Milnor arc at infinity of $f$ associated to $\lambda$, and abbreviate this by $f_{|\gamma }{\searrow } \lambda$.

Definition 3.3 Any Milnor arc $\gamma$ has a unique point at infinity $p\in L^{\infty }\cap \overline {\gamma }$; we shall say that ‘the Milnor arc $\gamma$ has the point $p$ at infinity.’

We may and will assume from now on that, modulo a translation of coordinates, the origin 0 is a point of $\Omega (f)$, and for this point we will use the simplified notations without lower index, such as $M(f)$ etc.

Remark 3.4 Unlike the setting of complex polynomials of 2 variables where the existence of the Milnor set at a point at infinity is a precise indicator of the existence of an atypical fibre (see e.g. [35, 39]), in the real setting this is no more true. For instance in [40, example 3.2] $0\not \in {\rm {Atyp}\ } f$ but there are Milnor arcs at the point $[1:0:0] \in L^\infty$ and $f$ tends to the value 0 along each of these arcs.

3.2. Clusters of Milnor arcs

By proposition 3.2, the Milnor arcs at infinity do not intersect mutually. It follows that if $C\subset {\mathbb {R}}^{2}$ is some large enough circle centred at the origin, then $M(f) \cap C$ is a finite set of points $\{p_{1}, \ldots, p_{s}\}$. We define the following counterclockwise relation between these points²: we say that ‘$p_{j}$ is the successor of $p_{k}$,’ or that ‘$p_{k}$ is the antecedent of $p_{j}$,’ if starting from the point $p_{k}$ and moving counterclockwise along the circle $C$ one arrives at the point $p_{j}$ without meeting any other point of the set $M(f) \cap C$.

We also say that $\{p_{1}, \ldots, p_{k}\}$ is a sequence of consecutive points of the set $M(f) \cap C$ if $p_{i+1}$ is the successor of $p_{i}$ for all $i=1, \ldots, k-1$. This relation between the points $M(f) \cap C$ on the circle $C$ allows us to define a similar one among the Milnor arcs at infinity, as follows:

Definition 3.5 Counterclockwise ordering of Milnor arcs at infinity

We say that ‘$\gamma _{j}$ is the successor of $\gamma _{k}$,’ or that ‘$\gamma _{k}$ is the antecedent of $\gamma _{j}$,’ if the point $p_{j} := \gamma _{j}\cap C$ is the successor of the point $p_{k} := \gamma _{k}\cap C$. This relation is independent on the size of the circle $C$, provided large enough. We also say that $\{\gamma _{1}, \ldots, \gamma _{k}\}$ is a sequence of consecutive Milnor arcs at infinity if $\{p_{1}, \ldots, p_{k}\}$, where $p_{i}:= \gamma _{i}\cap C$, is a sequence of consecutive points of the set $M(f) \cap C$.

Definition 3.6 Clusters of Milnor arcs at infinity

We call increasing cluster at $\lambda \in {\mathbb {R}} \cup \{+\infty \}$ a sequence of consecutive Milnor arcs at infinity $\gamma _k, \ldots, \gamma _{k+l}$, $l\ge 0$, such that the condition $f_{ | \gamma _i} \nearrow \lambda$ holds precisely for all $i= k, \ldots, k+l$ and does not hold for the antecedent of $\gamma _k$ nor for the successor of $\gamma _{k+l}$.

Similarly, we define a decreasing cluster at $\lambda \in {\mathbb {R}} \cup \{-\infty \}$ by replacing $\searrow$ instead of $\nearrow$ in the above definition. We will use the generic name ‘Milnor cluster,’ or simply ‘cluster,’ for any increasing or decreasing cluster.

A similar definition of Milnor clusters was given in [18] in the setting of surfaces in ${\mathbb {R}}^{n}$ instead of ${\mathbb {R}}^{2}$. Earlier, polar clusters have been defined in [7]. In [7, 18], clusters are used for detecting atypical fibres. An effective detection of atypical values via Milnor clusters can be found in [29]. Let us point out that [9] develops an algorithmic detection of atypical fibres without using Milnor clusters.

Theorem 3.7 [18, 29]

Let $f:{\mathbb {R}}^{2}\to {\mathbb {R}}$ be a non-constant polynomial function, and let $\lambda \in {\mathbb {R}}$ such that the fibre $f^{-1}(\lambda )$ has at most isolated singularities. Then $\lambda$ is an atypical value of $f$ at infinity³ if and only if there exists a cluster at $\lambda$ (either increasing or decreasing) having an odd number of arcs.

Remark 3.8 It was shown in [29] that for any cluster ${\mathcal {C}}$ at $\lambda \in {\mathbb {R}} \cup \{\pm \infty \}$ there is a unique connected component of the fibre $f^{-1}(t)\setminus D_{R}$, denoted by $\alpha _{t}({\mathcal {C}})$, which intersects all the Milnor arcs of the cluster ${\mathcal {C}}$, for $t$ close enough to $\lambda$. It was proved in [29] that the correspondence ${\mathcal {C}}\mapsto \alpha _{t}({\mathcal {C}})$, for a large enough disk $D_{R}$, is a well-defined map from the set of clusters to the set of fibre components in ${\mathbb {R}}^{2}\setminus D_{R}$, which is moreover injective.

Let us also point out that two different components $\alpha _{t}({\mathcal {C}})$ of $f^{-1}(t)\setminus D_{R}$ may belong to the same connected component of the affine fibre $f^{-1}(t)$, and see example 6.3 for such a situation.

Convention. In the rest of this paper we shall designate the Milnor arcs at infinity simply as ‘Milnor arcs.’

3.3. The Splitting (Sp) and the Vanishing (Va) at infinity

The splitting (denoted Sp) and the vanishing (denoted Va) of fibre components are phenomena which may happen, at some point $(p, \lambda )\in {\mathcal {L}}_{f}\times {\mathbb {R}}$ where $\lambda$ denotes a value of $f$. These have been defined⁴ in [40]. They are related to Milnor arcs at infinity. More precisely, we will see in the following that any odd cluster is either a splitting cluster or a vanishing cluster.

After [9] (see theorem 3.13 below), the existence of atypical fibres is equivalent to the existence of atypical points at infinity in ${\mathcal {L}}_f \times {\mathbb {R}}$, which are defined with respect to the local splitting and local vanishing (which are the localizations of the Sp and Va phenomena).

In order to display the definitions, let us recall a few preliminaries following [9]. Let $\{ M_t \}_{t\in {\mathbb {R}}}$ be a family of sets in ${\mathbb {R}}^2$. We say that the limit set of the family $\{ M_t \}_{t\in {\mathbb {R}}}$ when $t \to \lambda$, and we denote it by $\lim _{t\to \lambda } M_t$, is the set of points $x \in {\mathbb {R}}^2$ such that there exists a sequence $t_k \in {\mathbb {R}}$ with $t_k \to \lambda$ and a sequence of points $x_{k} \in M_{t_k}$ such that $x_{k} \to x$.

Definition 3.9 [9]

Let $\lambda \in {\mathbb {R}}$ such that $\text {Sing} f^{-1}(\lambda )$ is a compact set.

1. (i) One says that $f$ has a vanishing at infinity at $\lambda$, if $\lim _{t \to \lambda ^{-}} {\text {max}_j} { \text {inf}_{q \in F_{t,j}}} \| q\|= \infty$, or $\lim _{t \to \lambda ^+} {\text {max}_j} {\text {inf}_{q \in F_{t,j}}} \| q\|= \infty$, where $j$ runs over all connected components $F_{t,j}$ of the fibre $f^{-1}(t)$.

2. (ii) One says that $f$ has a splitting at infinity at $\lambda$, if there exists $\eta >0$ and a continuous family of analytic paths $\phi _{t} : [ 0,1 ] \to f^{-1}(t)$ for $t \in ( \lambda - \eta, \lambda )$, or for $t \in (\lambda, \lambda + \eta )$, such that:

   1. (1) ${\rm {Im}\ }\phi _t \cap M(f) \neq \emptyset$, and $\lim _{t \to \lambda }\|q_t\| = \infty$ for any $q_t \in {\rm {Im}\ }\phi _t \cap M(f)$,and

   2. (2) the limit set $\lim _{t \to \lambda ^{-}} {\rm {Im}\ } \phi _t$, or $\lim _{t \to \lambda ^+} {\rm {Im}\ }\phi _t$, respectively, is not connected.

Definition 3.10 We say that a cluster ${\mathcal {C}}$ is odd (or even) if it has an odd number of Milnor arcs (or an even number of Milnor arcs, respectively).

By remark 3.8, there is an injective correspondence between the clusters ${\mathcal {C}}$ at $\lambda \in {\mathbb {R}} \cup \{\pm \infty \}$ and the connected components of the fibre $f^{-1}(t) \setminus D_{R}$ for $t\to \lambda$ and large enough radius $R$. We have denoted by $\alpha _t({\mathcal {C}})$ the connected component defined by ${\mathcal {C}}$ in this correspondence. It was proved in [29, Proof of theorem 6.5] that ${\mathcal {C}}$ is an odd cluster if and only if $\alpha _t({\mathcal {C}})$ is either vanishing or splitting at infinity, in the sense of definition 3.9(i-ii), where one replaces $f^{-1}(t)$ by $\alpha _t({\mathcal {C}})$.

In the following we will therefore call such an odd cluster ${\mathcal {C}}$ at $\lambda$ either a vanishing cluster, or a splitting cluster, accordingly.

The paper [9] shows that one can localize at some points $(p,\lambda ) \in L^{\infty } \times {\mathbb {R}}$ the vanishing and the splitting at infinity of $f$ at $\lambda$ (cf. definition 3.9). To explain this result that we will need here, let us recall some notations and definitions.

By a linear change of coordinates we may assume, without loss of generality, that $p \in L^\infty$ is the point $[0:1:0]$. Recall that $\tilde f (x,y,z)$ denotes the homogenization of degree $d= \deg f$ with respect to the new variable $z$. In some chart $U \simeq {\mathbb {R}}^2 \subset {\mathbb {P}}^2$ at $p$, the family of polynomial functions $g_t:= \tilde f(x,1,z)-tz^d$ defines a family of algebraic curve germs $C_t:=\{g_t =0\}$ at $p$, of parameter $t$.

Definition 3.11 [9] (Localization). One says that $f$ has a splitting at $(p,\lambda ) \in L^\infty \times {\mathbb {R}}$ if there is a small disk $D_\varepsilon$ at $p$ in some chart at infinity ${\mathbb {R}}^2$ such that the representative of the curve $C_t$ in $D_\varepsilon$ has a connected component $C^i_t$ such that $C_t^i \cap \partial D_\varepsilon \neq \emptyset$ for all $t > \lambda$ (or for all $t<\lambda$) close enough to $\lambda$, and that the local Euclidean distance ${\rm {dist}}(C_t^i,p)\neq 0$ tends to $0$ when $t \to \lambda$.

One says that $f$ has a vanishing at $(p,\lambda ) \in L^\infty \times {\mathbb {R}}$ if there is a small disk $D_\varepsilon$ at $p \in U$ such that $C_t \cap D_\varepsilon \setminus \{p\}$ has a non-empty connected component $C_t^i\setminus \{p\}$ with $C_t^i \cap \partial D_\varepsilon = \emptyset$ for all $t<\lambda$ (or for all $t>\lambda$) close enough to $\lambda$, such that $\lim _{t \to \lambda }C_t^i\cap D_\varepsilon =\{p\}$.

Definition 3.12 [9] (Atypical points at infinity). We say that $(p,\lambda )\in L^\infty \times {\mathbb {R}}$ is an atypical point at infinity of $f$ if there is either splitting or vanishing at $(p,\lambda )$.

Theorem 3.13 A value $\lambda \in {\mathbb {R}}$ is an atypical value at infinity of $f$ (cf. definition 2.1) if and only if there exists $p\in {\mathcal {L}}_{f}$ such that $(p,\lambda )$ is an atypical point at infinity.

More precisely, if ${\mathcal {C}}$ is a splitting cluster at $\lambda$ (cf. definition 3.10), then after splitting, the two local fibre components have the same point $p$ at infinity, and if ${\mathcal {C}}$ is a vanishing cluster at $\lambda$, then before vanishing, the fibre component has the unique point $p$ at infinity.

Proof. The first claim was proved in [9, theorem 1.1, theorem 2.10]. To show the second claim, let $\alpha _t({\mathcal {C}})$ be the unique connected component of $f^{-1}(t) \setminus D_R$ (for a radius $R$ large enough, and for $t$ close enough to $\lambda$) which corresponds to the cluster ${\mathcal {C}}$ by remark 3.8.

If ${\mathcal {C}}$ is a vanishing cluster at $\lambda$ then for $t$ close enough to $\lambda$, $\alpha _t({\mathcal {C}})$ is a loop at some point $p\in L^\infty$, and therefore $(p, \lambda ) \in L^\infty \times {\mathbb {R}}$ is the unique vanishing point of $\alpha _t({\mathcal {C}})$ for $t\to \lambda$. Since all the Milnor arc in the cluster ${\mathcal {C}}$ intersect $\alpha _t({\mathcal {C}})$, it follows that all of them have the point $p$ at infinity.

If ${\mathcal {C}}$ is a splitting cluster at $\lambda$ then, as $t\to \lambda$, $\alpha _t({\mathcal {C}})$ splits at least at some point $p\in L^\infty \cap \overline {\alpha _t({\mathcal {C}})}\subset L^\infty \cap \overline {f^{-1}(\lambda )}$. The limit set $Z :=\lim _{t \to \lambda } \alpha _t({\mathcal {C}})$ is then non-connected (see § 3.3 for the definition of the limit set). By contradiction, if $\alpha _t({\mathcal {C}})$ splits at more than one point, then the limit set $Z\subset f^{-1}(\lambda )\setminus D_{R}$ contains at least an arc $A \simeq {\mathbb {R}}$ which has at infinity two such points. Clearly, the arc $A$ cannot be contained in the exterior of the disk $D_{R}$ for any large enough $R$. As $A$ is part of the limit set $Z$, it then follows that the nearby fibre component $\alpha _t({\mathcal {C}})$ has the same property. This means that $\alpha _t({\mathcal {C}})$ has at least two connected components in ${\mathbb {R}}^{2}\setminus D_{R}$ for some large enough $R$, which is a contradiction to its definition. The proof of the unicity of the splitting point $p$ is now complete.

Finally, since all the Milnor arcs in the cluster ${\mathcal {C}}$ intersect $\alpha _t({\mathcal {C}})$, it follows that all of them have this point $p$ at infinity.

By comparing the proof of theorem 3.13 to definition 3.12, we immediately get the following rephrasing⁵:

Corollary 3.14 To an odd cluster ${\mathcal {C}}$ at $\lambda \in {\mathbb {R}}$ there corresponds a unique atypical point $(p,\lambda )\in {\mathcal {L}}_f \times {\mathbb {R}}$, such that all the Milnor arcs in the cluster ${\mathcal {C}}$ have the same point $p$ at infinity.

However, let us note that corollary 3.14 is not anymore true for odd clusters corresponding to connected components of fibres of $f$ which tend to the values $\pm \infty$, see example 6.4.

3.4. Points at infinity, clusters and tangents

We recall that the notation ${\mathcal {L}}_{f}$ stands for the points at infinity of all the fibres of a non-constant polynomial $f: {\mathbb {R}}^{2}\to {\mathbb {R}}$, and $|{\mathcal {L}}_{f}| = \# {\mathcal {L}}_{f}$. Also recall that a vanishing or a splitting cluster at some value $\lambda$ contains an odd number of Milnor arcs, by theorem 3.7, and that all these arcs contain the point $p$ in their closure at infinity, by corollary 3.14.

Proposition 3.15 Let ${\mathcal {C}}$ be an odd cluster and let $(p, \lambda )\in {\mathcal {L}}_{f}\times {\mathbb {R}}$ be its unique atypical point at infinity $($cf. corollary 3.14 $)$. Then$:$

1. (a) If ${\mathcal {C}}$ is a splitting cluster then, after splitting, the resulting two germs at $p$ of fibre components, denoted by $C_{p,1}$ and $C_{p,2}$, have a common tangent semi-line, call it $T$, and all the Milnor arcs of the cluster ${\mathcal {C}}$ are also tangent to the same semi-line $T$ at $p$.

2. (b) If ${\mathcal {C}}$ is a vanishing cluster, let $C_{1}$ and $C_{2}$ be the two local arcs at $p$ of the component $\alpha _{t}({\mathcal {C}})$ which vanishes at $p$ when $t\to \lambda$. Then $C_{1}$ and $C_{2}$ are tangent at $p$ to the same semi-line $T$, and all the Milnor arcs of this cluster are tangent to the same semi-line $T$ at $p$.

Proof. We give the proof for (a) only, as the one for (b) is analogous.

By contradiction, suppose that $T_{p}C_{p,1}=R$ and $T_{p}C_{p,2}=T$, for two semi-lines $R \not = T$ at $p$. Let us then consider some semi-line $L$ at $p$ in the interior of the angle $\delta$ of measure less than $\pi$ at $p$ spanned by the semi-lines $T$ and $R$. For any $t$ close enough to $\lambda$, the component $\alpha _{t}({\mathcal {C}})$ must intersect $L$: if not, then $\alpha _t({\mathcal {C}})$, is contained in one of the two angles spanned by $L^\infty$ and the semi-line $L$, which contradicts our above assumption about two different tangent semi-lines $T$ and $R$.

Now, since $\alpha _{t}({\mathcal {C}})$ intersects $L$ for any $t$ close enough to $\lambda$, then it follows that the restriction $f_{| L}$ of $f$ to the line $L$ is not constant. Therefore $f_{| L}$ must be unbounded, since the restriction $f_{| L}$ is a non-constant polynomial of one variable. More precisely, for any $t$ close to $\lambda$, there is a point of intersection $q(t) \in \alpha _{t}({\mathcal {C}})\cap L$ which tends to $p$ when $t\to \lambda$, and thus the value $f_{| L}(q(t))$ must converge to infinity as $t\to \lambda$. On the other hand, we have $f_{| L}(q(t)) = t$ and the limit $\lim _{t\to \lambda }f_{| L}(q(t))$ is $\lambda$ by construction. This yields a contradiction.

To show the tangency to the semi-line $T$ of the Milnor arcs of the cluster ${\mathcal {C}}$, let us remark that $\alpha _t({\mathcal {C}})$, for $t$ close enough to $\lambda$, is included in the thin region ${\mathcal {A}}$ spanned by the splitting components $C_{p,1}$ and $C_{p,2}$ with common tangent $T$. Any Milnor arc in ${\mathcal {C}}$ intersects $\alpha _t({\mathcal {C}})$ for $t\to \lambda$, so it is constraint by ${\mathcal {A}}$ to have the same tangent $T$ at $p$.

Proposition 3.16 Let $\gamma$ and $\delta$ be two consecutive Milnor arcs $($in the order of arcs, cf definition 3.5) such that they have either different points at infinity, or the same point at infinity $p\in {\mathcal {L}}_{f}$ but are tangent to different semi-lines at $p$. Let ${\mathcal {C}}_{\gamma }$ and ${\mathcal {C}}_{\delta }$ be their respective clusters. Then the corresponding fibre components $\alpha _{t}({\mathcal {C}}_{\gamma })$ and $\alpha _{t}({\mathcal {C}}_{\delta })$ cannot both split.

Proof. First of all, the hypotheses imply, via corollary 3.14 and proposition 3.15, that ${\mathcal {C}}_{\gamma } \not = {\mathcal {C}}_{\delta }$. Suppose then, by contradiction, that both components $\alpha _{t}({\mathcal {C}}_{\gamma })$ and $\alpha _{t}({\mathcal {C}}_{\delta })$ split at $p$. We first assume that $\gamma$ and $\delta$ have the same point $p\in {\mathcal {L}}_{f}$ at infinity but different tangent semi-lines at $p$. The splitting can happen only at atypical values of $f$, so let $\lambda _{\gamma }, \lambda _{\delta } \in {\mathbb {R}}$ be the atypical values where $\alpha _{t}({\mathcal {C}}_{\gamma })$ and $\alpha _{t}({\mathcal {C}}_{\delta })$ split at $p$, respectively.

Let $T$ be the semi-line tangent to $\delta$ at $p$, and let $L$ be the semi-line tangent to $\gamma$ at $p$. By our hypothesis, $L\not = T$.

The component $\alpha _{t}({\mathcal {C}}_{\gamma })$ splits as $t\to \lambda _{\gamma }$ into two branches $C^{1}_{\gamma }$ and $C^{2}_{\gamma }$, and the component $\alpha _{t}({\mathcal {C}}_{\delta })$ splits as $t\to \lambda _{\delta }$ into two branches $C^{1}_{\delta }$ and $C^{2}_{\delta }$. Since there is no other Milnor arc between $\gamma$ and $\delta$ in the counterclockwise ordering, there must be a family of fibre components between $C^{i}_{\gamma }$ and $C^{j}_{\delta }$, for appropriate $i, j\in \{1,2\}$, which is a topologically trivial family at infinity (in the sense employed in definition 2.1). But there cannot be a trivial fibration at infinity since all the fibres in such a trivial fibration must have the same tangent semi-line at $p$. This implies that there exist an atypical point at infinity $(p, \lambda )$, with $\lambda$ in the open interval between $\lambda _{\gamma }$ and $\lambda _{\delta }$. In turn, this implies that there exists a Milnor arc ‘between’ $\gamma$ and $\delta$ in the counterclockwise ordering, which is a contradiction to our assumption.

Let us now assume that the consecutive Milnor arcs $\gamma$ and $\delta$ do not have the same point at infinity, and that the corresponding clusters are splitting like described above. Then, as observed in the preceding paragraph, the region $\mathcal {R}_{ij}$ outside a large enough disk $D$ and between the two corresponding fibre components $C^{i}_{\gamma }$ and $C^{j}_{\delta }$ is either filled with a trivial fibration (defined by the appropriate restriction of $f$), or there is no such trivial fibration containing $C^{i}_{\gamma }$ and $C^{j}_{\delta }$. Since the connected components $C^{i}_{\gamma }$ and $C^{j}_{\delta }$ have different points at infinity, they cannot live in a trivial family of connected fibres. But if there is no fibration between $C^{i}_{\gamma }$ and $C^{j}_{\delta }$, then there exists an atypical fibre at infinity in that region $\mathcal {R}_{ij}$, and thus there exists another Milnor arc ‘between’ $\gamma$ and $\delta$, which is a contradiction to our assumption that $\gamma$ and $\delta$ are consecutive Milnor arcs.

Remark 3.17 Example 6.2 shows two consecutive Milnor arcs $\gamma$ and $\delta$ belonging to different clusters, such that both fibre components $\alpha _t({\mathcal {C}}_\gamma )$ and $\alpha _t({\mathcal {C}}_{\beta })$ split. However they have the same tangent line. This shows that the hypotheses of proposition 3.16 are sharp.

4. Index at infinity via Milnor arcs and clusters

Let $f:{\mathbb {R}}^{2}\to {\mathbb {R}}$ be a non-constant polynomial function with isolated singularities. We assume as in §3.1 that the origin $0\in {\mathbb {R}}^{2}$ is a point in $\Omega (f)$. Let $D$ be a disk centred at the origin of large enough radius such that it contains ${\rm {Sing}\ } f$ in its interior and satisfies proposition 3.2. Let $S^1$ be the unitary circle in ${\mathbb {R}}^2$. The restriction of the Gauss map $\psi :=\frac {\mathrm {{grad\ }} f}{\|\mathrm {{grad\ }} f\|}$ to the circle $C:= \partial D$ defines a $C^\infty$ oriented map $\psi _{|C}: C \to S^1$ between the circles $C$ and $S^1$ endowed with their counterclockwise orientation.

Durfee introduced in [12] the index at infinity of $f$:

(4.1)\begin{equation} {\rm{ind}} _\infty (f) := \deg (\psi_{|C}), \end{equation}

where $\deg$ denotes the Brower degree as in [28].

4.1. Index of a Milnor arc, after [12]

Recall that the Milnor set $M(f)$ is the set of points where the fibres of $f$ are tangent to the level sets of the Euclidean distance function $\rho$, and that, by definition 3.2, the Milnor arcs do not intersect ${\rm {Sing}\ } f$. For any point $q$ of a Milnor arc $\gamma$ outside a disk $D = \{\rho (x,y) \le R \}$ of large enough radius $R$, the fibre of $f$ passing through $q$ may be in only one of the following three situations:

1. (a) locally inside the disk $D$, and then one defines the index $i(\gamma ) := +\frac {1}{2}$

2. (b) locally outside $D$, and then one defines the index $i(\gamma ) := -\frac {1}{2}$,

3. (c) a local half-branch inside $D$ and the other local half-branch outside $D$, in which case one defines the index $i(\gamma ) := 0$. Actually, we will see in the proof of lemma 4.3 that, for a generic choice of the origin, the Milnor set $M(f)$ does not contain Milnor arcs $\gamma$ of index $0$.

It then follows that along a fixed fibre component, outside the disk $D$, the distance function has alternating local maxima and minima in the counterclockwise order, and without counting the inflexion points. Therefore we get:

Lemma 4.1 The consecutive Milnor arcs $($cf. definition 3.5) in the same cluster, without counting the index 0 arcs, must have alternating index signs.

Now by theorem 3.7, remark 3.8, and lemma 4.1, we directly get the following consequence:

Corollary 4.2 Any splitting cluster at $\lambda \in {\mathbb {R}}$ has total index $+\frac 12$. Any vanishing cluster at $\lambda \in {\mathbb {R}} \cup \{\pm \infty \}$ has total index $-\frac 12$. Any even cluster has total index $0$.

Let us set the following notation:

\[ i_{p,c} := \sum_{\gamma} i(\gamma) \]

where $\gamma$ runs over all Milnor arcs $\gamma$ such that $p\in \overline {\gamma }$ and that $\lim f_{|\gamma }= c$.

Lemma 4.3 [12, p. 1356]

(4.2)\begin{equation} {\rm{ind}}_{\infty}(f) = 1 + \sum_{p\in L^{\infty}, \ c\in {\mathbb{R}} \cup \{{\pm} \infty\}} i_{p,c}. \end{equation}

Proof. From definition 3.1, after identifying ${\mathbb {R}}^{2}$ with ${\mathbb {C}}$, we get the defining equality:

(4.3)\begin{equation} M(f) = \left\{ q\in {\mathbb{C}}\ \Big\vert\ \frac{\mathrm{{grad\ }} f(q)}{\|\mathrm{{grad\ }} f(q) \|} ={\pm} \frac{q}{\|q\|} \right\} . \end{equation}

Let us therefore consider the oriented $C^\infty$-map $\phi := \psi _{|C} \cdot \left (\frac {z}{\|z\|}\right )^{-1}: C \to S^1$, where $z$ denotes the variable in ${\mathbb {C}}$, and where both circles $C$ and $S^1$ are endowed with their counterclockwise orientation. The map $\phi$ is by definition the multiplication of $\psi _{|C}$ with the clockwise rotation $\left (\frac {z}{\|z\|}\right )^{-1} =\frac {\overline {z}}{\|z\|} : C \to S^{1}$ of degree $-1$. Using the winding number interpretation of the degree, one obtains the equality:

(4.4)\begin{equation} \deg (\phi) = \deg (\psi_{|C}) -1 = {\rm{ind}}_ \infty (f)-1. \end{equation}

Without loss of generality, we may assume that $1,-1 \in S^1$ are regular values of $\phi$. We obtain:

(4.5)\begin{equation} \deg (\phi) = \frac{1}{2} \left( \sum_{q \in \phi^{{-}1}(1)} {\rm{or}} (T_q \phi) + \sum_{q \in \phi^{{-}1}({-}1)} {\rm{or}} (T_q \phi) \right) = \sum_{q \in M(f) \cap C} \frac{1}{2} {\rm{or}} (T_q \phi), \end{equation}

where ${\rm {or}} (T_q \phi )$ denotes the orientation of the tangent map, and where the last equality follows since we have $\phi ^{-1}(\{-1, 1\}) = M(f)\cap C$ in view of (4.3).

Let us explain here what are the local orientations ${\rm {or}} (T_q \phi )$. Let $q \in \gamma \cap C$, for some Milnor arc $\gamma$. Referring to the definition in the beginning of §4.1, we have the following correspondences (where ‘increasing’ means here counterclockwise, and ‘decreasing’ means clockwise):

• • In the case (a) the Gauss map $\psi$ is increasing relative to the radial map $\frac {z}{\|z\|}$, and thus ${\rm {or}} (T_q\phi ) = +1$.

• • In the case (b) the Gauss map $\psi$ is decreasing relative to the radial map $\frac {z}{\|z\|}$, and therefore ${\rm {or}} (T_q\phi ) = -1$.

• • The case (c) at the point $q$ means that this point is a local maximum or a local minimum for the map $\phi$, thus a critical point. This situation cannot occur because we have assumed that $q$ is a regular point of $\phi$.

From (4.4) and (4.5) we then obtain:

\[ {\rm{ind}}_\infty (f) = 1 + \sum_{\gamma} i(\gamma), \]

where the sum runs over all the Milnor arcs of $f$.

In case $F_+\cup F_{-}$ contains a compact component, by lemma 2.2 we get that $|{\mathcal {L}}_{f}| =0$, and all fibres of $f$ are either compact and connected, or empty. A non-empty fibre of $f$ is then homomorphic to a circle. The winding number of $\mathrm {{grad\ }} f$ over such a circle is 1, and it follows that ${\rm {ind}}_{\infty }(f)=1$. Therefore, in the following we will tacitly consider only polynomials which have at least a non-compact fibre.

Let ${\rm {Sp}}(p, \lambda )$ and ${\rm {Va}}(p, \lambda )$ denote the numbers of connected components of fibres of $f$ outside the large disk $D$ which are splitting or vanishing, at the point $(p, \lambda )$, respectively.

Let ${\rm {Va}}(\pm \infty )$ denote the number of components of $F_+\cup F_{-}$. Note that there are two type of components which are counted in ${\rm {Va}}(\pm \infty )$: those which tend to a nontrivial segment of the line at infinity as the value of $f$ tends to infinity, and those which tend to a point $p\in L^{\infty }$ as the value of $f$ tends to infinity. These two types are illustrated in examples 2.7 and 6.3.

Theorem 4.4

(4.6)\begin{equation} {\rm{ind}}_{\infty}(f) = 1 + \frac12 \sum_{p\in {\mathcal{L}}_{f}, \lambda\in {\mathbb{R}}}{\rm{Sp}}(p, \lambda) - \frac12 \sum_{p\in {\mathcal{L}}_{f}, \lambda\in {\mathbb{R}}}{\rm{Va}}(p, \lambda) - \frac12 {\rm{Va}}({\pm} \infty) \end{equation}

Proof. By gathering the indices of the Milnor arcs of the same cluster, one recasts (4.2) as:

(4.7)\begin{equation} {\rm{ind}} _{\infty} (f) = 1 + \sum_{{\mathcal{C}}} \sum_{\gamma \in {\mathcal{C}}} i(\gamma), \end{equation}

where the sum runs over all Milnor clusters.

Let us compute the total index $\sum _{\gamma \in {{\mathcal {C}}}} i(\gamma )$ for each cluster ${\mathcal {C}}$. By corollary 4.2 the number of odd clusters of total index $+\frac {1}{2}$, or $-\frac {1}{2}$, is equal to the number ${\rm {Sp}}(p,\lambda )$, or ${\rm {Va}}(p,\lambda )$, respectively. Moreover, the number of clusters at $\pm \infty$ is equal to the number ${\rm {Va}}(\pm \infty )$ and the total index of each of these clusters is $-\frac {1}{2}$. Even clusters have total index $0$, thus do not contribute to the formula.

Our formula (4.6) follows by plugging in all these data in (4.7).

4.2. The local degree at infinity

We continue to consider a polynomial $f: {\mathbb {R}}^{2}\to {\mathbb {R}}$ of degree $d\ge 2$, and we prove here a key result that will be used for finding an upper bound of the index at infinity.

Definition 4.5 Let $d_{Re}$ denote the number of real solutions of the equation $f_{d}=0$ counted with multiplicity. We call it the real degree of $f_{d}$.

We denote by $d_{p}$ the order of $f_{d}$ at the point $p\in \{f_{d} =0\}\subset {\mathbb {P}}^{1}$. This is equal to the multiplicity of the linear factor of $f_{d}$ corresponding to $p$.

Remark 4.6 The inequality $d_{p}>0$ does not imply that $p\in {\mathcal {L}}_{f}$, like in the example $f= xy^{2}+x$, where $p:=[1:0:0]\in \{f_{d} =0\}$ with $d_{p}=2$, but $p\not \in {\mathcal {L}}_{f}$.

By this reason, out of the obvious inequalities:

(4.8)\begin{equation} d_{Re} \ge \sum_{p\in \{ f_{d} = 0\} \cap L^{\infty} } d_{p} \ge \sum_{p\in {\mathcal{L}}_{f}}d_{p} \end{equation}

the second may be strict, for instance in the example $f = x^{4}y + y^{3}$ where $d_{Re} =5$, but one has $\sum _{p\in {\mathcal {L}}_{f}}d_{p} =1$ because ${\mathcal {L}}_{f} = \{ [1:0:0]\}$.

Remark 4.7 Let $p\in \{f_{d}=0\}\cap L^{\infty }$. The following equality is displayed in [12, lemma 7.3]:

(4.9)\begin{equation} {\rm{mult}}_{p}(\overline{M_{{\mathbb{C}}}(f)}, L^{\infty}_{{\mathbb{C}}}) =d_p-1. \end{equation}

We provide here an explicit proof of (4.9). One may assume (by an adequate linear change of coordinates) that $p= [1:0:0]$, and thus we have:

\[ f (x,y)= y^{d_{p}} r(x,y) + {\rm{l.o.t.}} \]

where $r$ is a homogeneous polynomial of degree $d-d_{p}$, and not divisible by $y$. In the chart $\{x \neq 0\}$, the Milnor set $\overline {M_{{\mathbb {C}}}(f)}$ has equation:

(4.10)\begin{equation} \hat{h}(y,z) ={-} d_p y^{d_p-1} r(1,y) - y^{d_p} r_{y}(1,y)+ y^{d_p+1} r_{x}(1,y) + z q(1,y,z) =0 \end{equation}

where $r_x$ and $r_y$ denote the partial derivatives of $r$, and $q(x,y,z)$ is a homogeneous polynomial of degree $d-1$. By our assumption, we also have $r(1,y)= c_0 + \cdots +c_ky^k$, where $c_0 \neq 0$, and $k\le d-d_p$. One has by definition:

\[ {\rm{mult}}_p(\overline{M_{{\mathbb{C}}}(f)}, L^\infty_{{\mathbb{C}}}) = {\rm{ord}}_{y}\left( \hat{h}(y,z)_{|L^\infty_{{\mathbb{C}}}} \right) \]

and due to (4.10), the later is precisely $d_p-1$.

5. The ‘index gap,’ and upper bounds for the index at infinity

Durfee showed in [12] the inequality:

(5.1)\begin{equation} {\rm{ind}}_{\infty}(f) \le 1 + d_{Re} - 2|{\mathcal{L}}_{f}|. \end{equation}

We will improve his upper bound by counting in a more refined manner the contributions of the Milnor branches at infinity.

Let $p\in L^{\infty }\cap \{f_{d} =0\}$, let $\overline {M_{{\mathbb {C}}}(f)}\subset {\mathbb {P}}^{2}_{{\mathbb {C}}}$ be the projective closure of the complex Milnor set $M_{{\mathbb {C}}}(f)$. We will consider the germ $\overline {M_{{\mathbb {C}}}(f)}_{p}$ and its complex Milnor branches. Our theorem uses the following sub-varieties of ${\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}$, where the multiplicity of each line is taken into account:

• • $R_{p} := \bigl \{ L\in {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p} \mid$ there is some real Milnor branch tangent to $L$ at $p \bigr \}$.

• • $K_{p} :=\bigl \{ L\in {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p} \mid$ there is some complex non-real Milnor branch tangent to $L$ at $p \bigr \}$.

• • $S_{p} :=\bigl \{ L\in {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p} \mid$$L\in R_{p}$ and either $L$ is tangent to some singular real branch of $\overline {M_{{\mathbb {C}}}(f)}_{p}$, or $L$ is $L^{\infty } \bigr \}$.

Note that $R_{p}^{{\mathrm {red}}} \cup S_{p}^{{\mathrm {red}}} \cup K_{p}^{{\mathrm {red}}}= {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}^{{\mathrm {red}}}$ with reduced structure, but that this union may be not disjoint.

In order to state our index bound theorems, we denote by $\lfloor \cdot \rfloor$ the non-negative floor function, i.e. with the convention that if $\lfloor r \rfloor <0$, then we replace this value by $0$.

Theorem 5.1

(5.2)\begin{align} & {\rm{ind}}_{\infty}(f) \le 1 + d_{Re} - 2|{\mathcal{L}}_{f}|\nonumber\\ & \quad - \sum_{p\in L^{\infty}\cap \{f_{d} =0\}} \left(\frac12 \left\lfloor \frac{\deg(R_{p}^{{\mathrm{red}}})-1}{2} \right\rfloor + \deg(S_{p}) + \deg(K_{p}) \right). \end{align}

The case $|{\mathcal {L}}_{f}|=1$ is studied in detail in the next section. In order to prove theorem 5.1 we need the following key result.

Lemma 5.2 The index gaps

Let $p\in L^{\infty }\cap \{f_{d} =0\}$. Then:

(5.3)\begin{equation} \sum_{c\in {\mathbb{R}}}i_{p,c} \le d_{p }-1. \end{equation}

The following phenomena are producing the difference between the two terms in the inequality (5.3), to which we shall refer as ‘index gap’:

1. (a) A Milnor arc $\gamma$ at $p$ of index $i(\gamma ) = - \frac 12$, yields a gap of at least $1$.

2. (b) A Milnor arc $\gamma$ at $p$ such that $\lim f_{|\gamma } = \pm \infty$, of any index, yields a gap of at least $\frac 12$.

3. (c) A Milnor branch $\beta$ at $p$ such that $\beta _{{\mathbb {C}}}$ is tangent to $L^ {\infty }_{{\mathbb {C}}}$, or that $\beta _{{\mathbb {C}}}$ is singular at $p$, yields a gap of at least $1$.

4. (d) A complex Milnor branch at $p$ that is not the complexified of a real Milnor branch produces a gap of at least $1$. If moreover this branch verifies the hypotheses of (c), then the gap increases to at least $2$.

Moreover, the ‘sign gaps’ (a), as well as the gaps (b), cumulate with the ‘singularity gaps’ (c).

Remark 5.3 Case (a) is illustrated by example 6.4. Case (b) can be seen in example 6.3 in the cluster $\{ \gamma _8,\gamma _1,\gamma _2 \}$. In the same example 6.3, the Milnor arcs $\gamma _3$ and $\gamma _7$ have index $+\frac {1}{2}$ and are tangent to $L^\infty$, which means case (c).

For case (d), let us consider the polynomial $f(x,y) =-\frac 53 y^3+y^2-2y+ 4x^2+x$. Then $p=[1:0:0] \in \overline {M_{{\mathbb {C}}}(f)}$. The tangent cone ${\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}$ is given by the equation $2z^2+6zy+5y^2 =0$, so there are at least two complex non-real branches at $p$.

Proof Proof of lemma 5.2

For a fixed point $p\in \{f_{d}=0\}\cap L^{\infty }$, we have the inequalities:

(5.4)\begin{equation} \sum_{c\in {\mathbb{R}}}i_{p,c} \le \sum_{c\in {\mathbb{R}}}i_{p,c} + \sum_{c\in \{{\pm} \infty\}} |i_{p,c}|\le \sum_{c\in {\mathbb{R}}\cup \{{\pm} \infty\}} |i_{p,c}| \le {\rm{mult}}_{p}(\overline{M_{{\mathbb{C}}}(f)}, L^{\infty}_{{\mathbb{C}}}) = d_{p}-1 \end{equation}

all of which may be strict. The first inequalities are obvious, whereas the last one is implied by the fact that there is a unique complex curve $\gamma _{{\mathbb {C}}}$ which is the complexification of the Milnor branch $\gamma$. The right hand side equality is (4.9).

Each real Milnor branch at $p$ has two Milnor arcs, of indices $- \frac 12$ or $+\frac 12$. The inequality (5.3) compares the indices of the real Milnor arcs with the intersection multiplicities of the corresponding complex Milnor branches. The proof of the index gap cases goes as follows.

1. (a) A Milnor arc $\gamma$ at $p$ with $i(\gamma ) = -\frac 12$ becomes $|i(\gamma )| = \frac 12$ on the right side of (5.4), which produces a gap of 1 in (5.3).

2. (b) If $\gamma$ is a Milnor arc at $p$ such that $f_{|\gamma } \to \pm \infty$ then it does not exist in the sum of the left side of (5.3), whereas it contributes to the right side of (5.4) by $|i(\gamma )| = \frac 12$.

3. (c) A Milnor branch $\beta$ at $p$ such that its complexification $\beta _{{\mathbb {C}}}$ is tangent to $L^ {\infty }_{{\mathbb {C}}}$ or singular at $p$ contributes by at most 1 in the left hand side of (5.3), whereas the multiplicity ${\rm {mult}}_{p}(\beta _{{\mathbb {C}}}, L^{\infty }_{{\mathbb {C}}})$ contributes by at least 2 in the right hand side of (5.3).

4. (d) Any real Milnor branch has a unique complexification. However, not all local complex branches are complexifications of real branches: there may be some purely complex Milnor branches, and all these count in the intersection index ${\rm {mult}}_p(\overline {M_{{\mathbb {C}}}(f)}, L^\infty _{{\mathbb {C}}})$, thus they give positive integer contributions in the right hand side of (5.3) whereas they do not exist in the left hand side of (5.3).

5.1. Proof of theorem 5.1

Lemma 4.3 reads:

(5.5)\begin{equation} {\rm{ind}}_{\infty}(f) =1 + \sum_{p\in {\mathcal{L}}_{f }, c\in {\mathbb{R}}} i_{p,c} + \sum_{q\in L^{\infty}} i_{q,\infty}. \end{equation}

By (5.3), for each $p\in L^{\infty }\cap \{f_{d} =0\}$, in particular for $p\in {\mathcal {L}}_{f }$, we have the inequality $\sum _{c\in {\mathbb {R}}} i_{p,c} \le d_{p}-1$, for which we will evaluate the gaps studied in lemma 5.2.

To any complex non-real line $L\in {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}$ there corresponds a positive number of complex non-real branches having $L$ as tangent at $p$. Each such complex non-real branch generates a gap of at least 1.

To any real line of ${\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}$ there may correspond real and complex non-real tangent branches. The non-real branches contribute, by lemma 5.2, with gaps of at least $1$.

A real tangent branch $\beta$ can be either:

1. (1) non-singular at $p$ and its two arcs are on both sides of $L^{\infty }$, or

2. (2) tangent to $L^{\infty }$ or singular at $p$, and thus yields a gap of at least 1, by lemma 5.2(c). According to lemma 5.2(c), this gap cumulates with the sign gaps of type (a).

We compute a lower bound for the total gap at $p$.

The ‘sign gaps,’ and the compensating exchange. The counterclockwise ordering of the Milnor arcs (cf. definition 3.5) induces an ordering in the subset of arcs at $p$. In turn, this induces an ordering among the real semi-lines of the real tangent cone ${\rm {Cone}\ } \overline {M(f)_{p}}$ which are on the same side of the line at infinity.

Let us assume that on the two sides of $L^{\infty }$ there are $r_{p}\ge 0$, respectively $s_{p}\ge 0$, semi-lines with tangent real Milnor arcs, and therefore $r_{p} + s_{p} \ge \deg (R_{p}^{{\mathrm {red}}})$. Applying proposition 3.16 to the consecutive arcs on each side we obtain that there are at least $\lfloor \frac {r_{p}}{2} \rfloor$ and $\lfloor \frac {s_{p}}{2} \rfloor$ arcs with non-positive index, respectively. According to lemma 5.2, each such arc produces a ‘sign gap’ of at least $\frac 12$. We thus obtain a total ‘sign gap’ of at least $\frac 12 \left (\lfloor \frac {r_{p}}{2}\rfloor + \lfloor \frac {s_{p}}{2} \rfloor \right )$, and notice that we have the inequality: $\lfloor \frac {r_{p}}{2} \rfloor + \lfloor \frac {s_{p}}{2} \rfloor \ge \left \lfloor \frac {\deg (R_{p}^{{\mathrm {red}}})-1}{2} \right \rfloor$.

Let us remark at this point that in the above computation, which starts with the ordered Milnor arcs at $p$, to be able to apply proposition 3.16 we ought to count all the Milnor arcs at $p$, thus not only the Milnor arcs for which $f$ tends to a finite value $c\in {\mathbb {R}}$ but also the Milnor arcs⁶ for which $f$ tends to $\pm \infty$. And since the later Milnor arcs do not occur in the sum of (5.3), we should remove them from the above ‘sign gap’ count. Nevertheless, in the same time each of those contribute with a gap of type (b) of lemma 5.2, and this gap of $\frac 12$ is compensating the necessary removal of the corresponding ‘sign gap’ that has been counted as $\frac 12$ too. With this extra argument of compensating exchange, the above estimation of the ‘sign gap’ still holds.

The ‘singular gaps’ and the ‘complex gaps.’ The Milnor branches which are tangent at $p$ to the lines of the sub-variety $S_{p}$ correspond to case (c) of lemma 5.2 and each one produces a gap of at least 1 which adds up to the total sign gap. The same effect produces a complex non-real branch [by lemma 5.2(d)], while its tangent line at $p$ belongs to $K_{p}$. In both cases, the lines are considered with their multiple structure.

By adding up those gaps, we thus get the total gap at $p$ of at least $\frac 12 \left \lfloor \frac {\deg (R_{p}^{{\mathrm {red}}})-1}{2} \right \rfloor + \deg (S_{p})+ \deg (K_{p})$, i.e. the inequality:

(5.6)\begin{equation} \sum_{p\in {\mathcal{L}}_{f }, c\in {\mathbb{R}}} i_{p,c} \le d_{p}-1 - \frac12 \left\lfloor \frac{\deg(R_{p}^{{\mathrm{red}}})-1}{2} \right\rfloor - \deg(S_{p})- \deg(K_{p}) . \end{equation}

Finally we need to sum this up over all points $p\in {\mathcal {L}}_{f }$. Therefore we get from (4.8) the following inequality:

(5.7)\begin{equation} \sum_{p\in {\mathcal{L}}_{f }} d_{p}-1 \le d_{Re} - |{\mathcal{L}}_{f }|. \end{equation}

We have now to deal with the term $\sum _{q\in L^{\infty }} i_{q,\infty }$ in (5.5). By remark 3.8, to each fibre component counted by ${\rm {Va}}(\pm \infty )$ there corresponds injectively a cluster ${\mathcal {C}} = {\mathcal {C}}(\pm \infty )$, and for this cluster the sum of indices $\sum _{\gamma \in {\mathcal {C}}} i(\gamma )$ is $-1/2$ by corollary 4.2. By proposition 2.6, we have ${\rm {Va}}(\pm \infty ) \ge 2|{\mathcal {L}}_{f}|$. We thus obtain:

(5.8)\begin{equation} \sum_{q\in L^{\infty}} i_{q,\infty}= \sum_{{\mathcal{C}}}\sum_{\gamma \in {\mathcal{C}}} i(\gamma) \le -\frac12 2|{\mathcal{L}}_{f}| ={-}|{\mathcal{L}}_{f}| \end{equation}

where the first sum at the right hand side is taken over all clusters ${\mathcal {C}} ={\mathcal {C}}(\pm \infty )$.

Finally, plugging (5.6), (5.7) and (5.8) into (5.5), we obtain the claimed inequality (5.2).

We show how to compute an upper bound for the total gap in a different manner, by merging the set of singular Milnor branches (which projects onto the subset $S_{p}^{{\mathrm {red}}}$ of $R_{p}^{{\mathrm {red}}}$) into the total set of Milnor branches (which projects onto the set $R_{p}^{{\mathrm {red}}}$). This produces a more handy upper bound for the index at infinity, in particular formula (5.10) is in terms of the local degrees at infinity.

Corollary 5.4

(5.9)\begin{equation} {\rm{ind}}_{\infty}(f) \le 1 + d_{Re} - 2|{\mathcal{L}}_{f}| - \sum_{p\in L^{\infty}\cap \{f_{d} =0\}} \left(\left\lfloor \frac{\deg(R_{p}^{{\mathrm{red}}}) }{2} \right\rfloor + \deg(K_{p}) \right). \end{equation}

In particular$:$

(5.10)\begin{equation} {\rm{ind}}_{\infty}(f) \le 1 + d_{Re} - 2|{\mathcal{L}}_{f}| - \sum_{p\in L^{\infty}\cap \{f_{d} =0\}} \left\lfloor \frac{\delta_{p}}{2} \right\rfloor , \end{equation}

where $\delta _{p} := \deg {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}^{{\mathrm {red}}_{{\mathbb {R}}}}$ is the degree of the cone in which the real line components are taken with reduced structure.

Proof. For some fixed point $p\in L^{\infty }\cap \{f_{d} =0\}$, we first consider the extreme case where each $L\in R_{p}$ has some tangent real Milnor branch with arcs on both sides of the line at infinity. We thus have $r_{p}= s_{p}=\deg (R_{p}^{{\mathrm {red}}})$ and, by applying proposition 3.16 as in the proof of theorem 5.1, we get the total sign gap greater or equal to $\left \lfloor \frac {\deg (R_{p}^{{\mathrm {red}}})}{2} \right \rfloor$.

Next we operate the following change: choose one of the Milnor branches with arcs on both sides of $L^{\infty }$ and replace it by a Milnor branch with arcs on the same side of $L^{\infty }$. Then this new branch is necessarily singular or tangent to $L^{\infty }$, thus in case (c) of lemma 5.2 and therefore, besides its possible sign gap contribution, it gives a singular gap contribution of at least 1. Loosing one arc at one side of $L^{\infty }$ may diminish the total gap by at most 1. Consequently, the effect of this replacement is that the total gap does not diminish.

Finally, we observe that one may obtain any configuration of real Milnor arcs by repeating a finite number of times the above described operation on Milnor branches. Our first statement is proved.

The second statement is a simple consequence of the inequality $\left \lfloor \frac {\delta _{p}}{2} \right \rfloor \le \left \lfloor \frac {\deg (R_{p}^{{\mathrm {red}}}) }{2} \right \rfloor + \deg (K_{p})$, which follows since $\deg (R_{p}^{{\mathrm {red}}}) + \deg (K_{p})$ is (by definition) greater or equal to $\deg {\rm {Cone}\ }\overline {M_{{\mathbb {C}}}(f)}_{p}^{{\mathrm {red}}_{{\mathbb {R}}}}$.

5.2. Revisiting Durfee's upper bound

Durfee [12] has proved an upper bound (1.1) in terms of the degree $d$ only, namely ${\rm {ind}}_\infty f \le \max \{1, d-3\}$. Revisiting and completing Durfee's proof in [12], we show here the following slight improvement:

Theorem 5.5 Let $f: {\mathbb {R}}^{2}\to {\mathbb {R}}$ be a polynomial of degree $d\ge 2$ with isolated singularities. Then$:$

1. (a) If $|{\mathcal {L}}_{f}| \ge 2$, then ${\rm {ind}}_{\infty }(f) \le d-3$.

2. (b) If $|{\mathcal {L}}_{f}| =1$, then ${\rm {ind}}_{\infty }(f) \le d-3$ for $d \ge 4$, and ${\rm {ind}}_{\infty }(f) \le 0$ for $d\le 3$.

3. (c) If $|{\mathcal {L}}_{f}| =0$ then ${\rm {ind}}_{\infty }(f)=1$.

Proof. (a). Follows immediately, either from (5.1) or from (5.2). (c). By lemma 2.2, the set $\{| f(x,y)| =R\}$ for $R\gg 1$ is diffeomorphic to a circle, and the winding number over a circle is $+1$, thus ${\rm {ind}}_{\infty }(f)=1$. This trivial fact was also observed in [12, theorem 7.8].

(b). Let ${\mathcal {L}}_{f} = \{p\}$. Then, by a linear change of coordinates, we may assume that $p=[1:0:0]$. For $|{\mathcal {L}}_{f}| =1$, by theorem 4.4 and by Durfee's inequality (5.1), or by our improvement (5.10), we get:

(5.11)\begin{equation} {\rm{ind}}_{\infty}(f) = 1+ \frac12 \left( \sum_{ \lambda\in {\mathbb{R}}}{\rm{Sp}}(p, \lambda) - \sum_{\lambda\in {\mathbb{R}}\cup \{{\pm} \infty\}} {\rm{Va}}(p, \lambda) \right) \le d_{p} - 1 \end{equation}

If $d_{p} \le d-2$ then theorem 5.5 follows directly from (5.11). We consider in the following the two remaining cases: $d_{p} = d-1$ and $d_{p} = d$.

Case 1. $d_{p} = d-1$.

Remark 5.6 Durfee claims in [12, pag. 1359], this case is not possible. His argument is the following: ‘the roots of $f_d$ other than $p$ are complex, and hence occur in conjugate pairs, thus $d_{p}\le d-2$.’ This seems to be grounded on the same assertion discussed in Remark 2.3: ‘${\mathcal {L}}_{f}= \emptyset \implies d_{Re}=0$,’ which is false, as shown by the simple example $f=x^{4} +y^{2}$, whereas its converse is obviously true.

We think that a slightly different assertion could be nevertheless true:

Conjecture: $d_{p} = d_{Re}-1 \!>0\!\implies {\mathcal {L}}_{f}\not = \emptyset, \text { and if moreover } p\in {\mathcal {L}}_{f} \text { then } |{\mathcal {L}}_{f}| \!=2.$

We will prove by the next lemma a version of this conjecture with $d_{Re}$ replaced by $d$.

Lemma 5.7 If $d_{p} = d-1 >0$ then ${\mathcal {L}}_{f}\not = \emptyset$, and if moreover $p\in {\mathcal {L}}_{f}$ then $|{\mathcal {L}}_{f}| =2$.

Proof. If $d_{p} = d-1 >0$ then one may assume, by an appropriate linear change of variables, that $p = [1:0:0]$ and $f_{d}=xy^{d-1}$. One therefore has$:$

\[ f(x,y)= xy^{d-1}+ xh(x,y) + u(y) \]

with $\deg h(x,y) < d-2$ and $\deg u(y)\le d-1$.

We will show that the projective closure $\overline {f^{-1}(0)}$ contains the point $[0:1:0] \in L^{\infty }$, which is different from $p$. We may assume that $u\not \equiv 0$, since if not, then $\{x=0\} \subset f^{-1}(0)$, thus $[0:1:0]\in {\mathcal {L}}_{f}$ and the claim is proved. Let then $b\in {\mathbb {R}}^{*}$ be the leading coefficient of $u$.

For every fixed $x_0 \in {\mathbb {R}}^{*}$, $x_{0} \not = -b$, the sign of the polynomial $f(x_0,y)$ of variable $y$, for $y\gg 1$, is the sign of its leading coefficient; this is $x_0+b$ in case $\deg u=d-1$, and it is $x_0$ in case $\deg u < d-1$. Let us choose $x_0 \in {\mathbb {R}}^*$ such that $b (x_0 +b)<0$, and thus $b x_0<0$ too. Then $f(x_0,y)f(0,y)<0$ for any $y\gg 1$. This implies that for any $y \gg 1$ there exists some value $t_{y}$ bounded between $0$ and $x_{0}$ such that $(t_{y}, y)\in f^{-1}(0)$. By taking the limit $y\to \infty$, this shows that $[0:1:0]\in \overline {f^{-1}(0)} \cap L^{\infty }$.

Lemma 5.7 shows that the case $d= d_{p} - 1$, with $p\in {\mathcal {L}}_{f}$ and $|{\mathcal {L}}_{f}|=1$ is impossible, confirming Durfee's claim.

Case 2. $d_{p} = d$.

We may assume as above that $p=[1:0:0]$ and since $d_{p} = d$, one may also assume that, modulo some appropriate linear change of coordinates, one has $f_{d} =y^{d}$.

Lemma 5.8 Let $p= [1:0:0] \in {\mathcal {L}}_{f}$ and let $f_{d} = y^{d}$. Then either there exists a Milnor branch at $p$ which is tangent to $L^{\infty }$, or $f(x,y) = y^{d} + v(x) + u(y)$, where $\deg v \le 2 < d$ and $\deg u \le d-1$.

Proof. Let us assume that $f$ contains mixed terms, namely let $f = y^{d}+ xyq(x,y) +v(x) + u(y)$, where $q \not \equiv 0$ is a polynomial of degree $\le d-3$, and $v(x)$ and $u(y)$ are some polynomials of degrees $\le d-1$. We write explicitly the equation $\hat h(1,y,z) =0$ of the closure $\overline {M(f)}$ of the Milnor set in the chart $\{x=1\}$. This is a polynomial of degree at least $d-1$ because it contains the term $dy^{d-1}$. Its order at $(0,0)$ is ${\rm {ord}} \hat h(1,y,z) \le {\rm {ord}} z\hat q(1,y,z) + zy\hat q(1,y,z) \le d-2$, where $\hat q(x,y,z)$ denotes the homogenization of $q(x,y)$ of degree $d-3$ by the variable $z$. Thus all the terms of $\hat h(1,y,z)$ of degree $< d-1$ contain $z$. This implies that $\overline {M(f)}_{p}$ contains a (complex) branch which is tangent to the line at infinity $L^{\infty } = \{z=0\}$.

Let us now treat the complementary case, i.e. whenever $f$ contains no mixed terms, i.e. $f= y^{d} +v(x) + u(y)$, where $v(x)$ and $u(y)$ are some polynomials of degrees $\le d-1$. Then the Milnor set germ $\overline {M(f)}_{p}$ in the chart $\{x =1\}$ is defined by the equation:

(5.12)\begin{equation} \hat{h}(1,y,z)={-} d y^{d-1} + zy\hat{v}_x(1,z) - z \hat{u}_y(y,z), \end{equation}

where $\hat {v}_x(x,z)$ and $\hat {u}_y(y,z)$ denote the homogenization of degree $d-2$ of the derivatives $v_x$ and $u_y$. The 1st and the 3rd terms of (5.12) are homogeneous of degree $d-1$, while the term in the middle is of order $\le d-2$ if and only if $\deg v \ge 3$. We deduce that the tangent cone ${\rm {Cone}\ } _{p}\overline {M(f)}$ contains the line $\{z=0\}$ if and only if $\deg v \ge 3$.

We compute the index in the special case of lemma 5.8.

Lemma 5.9 Let $f=v(x) + u(y)$, where $\deg u =\deg f =d$, and $\deg v \le 2< d$ such that $v_{x} \not \equiv 0$. If $d \geq 3$ then $|{\rm {ind}} _\infty (f)|$ is $0$ or $1$.

Proof. If $\deg v \le 1$ then ${\rm {Sing}\ } f = \emptyset$ and therefore ${\rm {ind}} _\infty (f) =0$.

If $\deg v =2$, then the derivative $v_{x} = ax+b$, where $a\not = 0$ by our assumption, changes sign one time, precisely at $x = -b/a$. Consider a large enough circle $C\subset {\mathbb {R}}^{2}$ centred at the origin. Consider the two points $N, S \in C\cap \{x = -b/a\}$. In the following we will compute the index at infinity ${\rm {ind}}_{\infty } (f)$ as the winding number over $C$.

If $d \geq 3$ then $u_{y}$ is a polynomial of degree $d-1$ and therefore has a constant sign outside a compact subset of ${\mathbb {R}}$. This implies that on each half circle of $C$ cut out by the vertical line $\{x = -b/a\}$, the variation of the vector field $\mathrm {{grad\ }} f$ over the circle $C$ between the two points $N$ and $S$ is either zero, or $\pi$ or $-\pi$.

We continue the proof of Case 2. By lemma 5.8 and by lemma 5.9, if there is no Milnor branch tangent to $L^{\infty }$ at $p$, then we obtain ${\rm {ind}}_{\infty }(f) =0$ when $d =3$, and ${\rm {ind}}_{\infty }(f) \le d-3$ when $d>3$, hence theorem 5.5 is proved in this situation.

In what follows we focus on the last remaining case established by lemma 5.8, namely: there exists at least one (complex) Milnor branch $\beta$ tangent to $L^{\infty }$ at $p$. The study falls into the following 4 situations:

1. (i) There are at least two Milnor branches at $p$ which are tangent to $L^{\infty }$, then by lemma 5.2(c) we get a gap of at least 1 for each of these branches. Therefore ${\rm {ind}}_{\infty }(f) \le d_{p}-1-2= d_{p} - 3$.

2. (ii) There is a single Milnor branch $\beta$ tangent to $L^{\infty }$ at $p$, and such that ${\rm {mult}}_{p}(\beta, L^{\infty }_{{\mathbb {C}}}) \ge 2$. By (4.9), this implies $d_{p}\ge 3$.

   If ${\rm {mult}}_{p}(\beta, L^{\infty }_{{\mathbb {C}}}) >2$ then we have a gap of at least 2, and thus ${\rm {ind}}_{\infty }(f) \le d_{p}-3$. If ${\rm {mult}}_{p}(\beta, L^{\infty }_{{\mathbb {C}}}) = 2$ and the indices of the arcs of $\beta$ are not both $+\frac 12$, then we get a gap of at least $3/2$. Since the index is an integer, the gap is of at least 2, and therefore ${\rm {ind}}_{\infty }(f) \le d_{p}-3$ again.

   Last case, let ${\rm {mult}}_{p}(\beta, L^{\infty }_{{\mathbb {C}}}) = 2$ and such that both arcs of $\beta$ have index $+\frac 12$. Since $\beta$ is also tangent to $L^{\infty }$, it follows that $\beta$ is nonsingular at $p$, more precisely the germ of $\beta$ at $p$ is equivalent, after some linear change of coordinates, with the curve $z=y^{2}$. Thus the two Milnor arcs are in the same half-plane of the chart ${\mathbb {R}}^{2}$ cut by the line $z=0$. According to proposition 3.15, after each of the two splittings we obtain two components of the fibres of $f$ which are tangent to the line $L := \{z=0\}$, and moreover, along the two Milnor arcs the tangency is to different semi-lines, say $L_+$ and $L_{-}$. This implies that, in the absence of other splittings or vanishings at $p$, there should exist a trivial fibration connecting two components tangent to different semi-lines, which is treated by proposition 3.16, and which tells that this situation is impossible.

3. (iii) There is a single Milnor branch tangent to $L^{\infty }$, and there are also non-tangent Milnor branches at $p$, such that at least one of the Milnor arcs has index $-\frac 12$. Then by lemma 5.2(a) and (c) we obtain an index gap of at least 2. Therefore we get ${\rm {ind}}_{\infty }(f) \le d_{p}-3$.

4. (iv) There is a single tangent Milnor branch at $p$, there are one or more transversal Milnor branches, and such that all the Milnor arcs at $p$ have index $+\frac 12$. This means that all arcs are of splitting type, and in particular each arc is a cluster. Our proposition 3.16 shows that this situation is impossible.

6. Examples

We consider here four examples. For three of them we will use pictures to encode information, and in order to draw the frame we will use the following construction employed in [13]. Let ${\mathbb {R}}^2 \hookrightarrow {\mathbb {P}}^2 \simeq {\mathbb {R}}^3\setminus \{0\}/{\mathbb {R}}^{*}$ be the embedding defined by $(x,y)\mapsto [x:y:1]$, and let

\[ S := \{ (a,b,0) \in {\mathbb{R}}^3\setminus \{0\} \}/ {\mathbb{R}}_+, \]

be the circle which is a double covering of the line at infinity $L^\infty \subset {\mathbb {P}}^2$. The compactification ${\mathbb {R}}^2 \sqcup S$ of ${\mathbb {R}}^{2}$ may be represented as a 2-disk $D$ with boundary $S$.

The dashed circle is the boundary $\partial D_{R}$ of the disk $D_{R}$ centred at the origin of radius $R\gg 1$ as in proposition 3.2. The Milnor arcs live in the annulus between $\partial D_{R}$ and $S$. By enumerating the Milnor arcs as $\gamma _1,\ldots, \gamma _k$ we mean that they are consecutive in the counterclockwise ordering (definition 3.5).

The limit $\lambda \in {\mathbb {R}} \cup \{\pm \infty \}$ to which $f_{|\gamma }$ tends along some Milnor arc $\gamma$ is written near the Milnor arc $\gamma$ close to $S$ (see proposition 3.2). The index $i(\gamma )$ is attached to each Milnor arc $\gamma$ at the intersection with the doted circle in the middle of the annulus, and the respective little arrow indicates the direction of the gradient of the Milnor arc. We write ‘${\rm {Sp}}$’ or ‘${\rm {Va}}$’ next to a cluster when the corresponding fibre component is splitting or vanishing, respectively. Whenever a cluster contains more than one Milnor arc, we connect all its arcs by a thicker curve.

Example 6.1 Let $f(x,y)=x^2y+x$. This polynomial has two clusters at the atypical value $\lambda =0$, each of them composed by a single Milnor arc of positive index. Both clusters have the point $p=[0:1:0]$ at infinity, and no other Milnor arcs abut to this point. Thus $d_p=2$, and (5.3) is an equality.

The polynomial $f$ has two clusters having the point $q=[1:0:0]$ at infinity, the corresponding fibre components of which tend to the value $+\infty$. And there are two more clusters with the corresponding fibre components tending to the value $-\infty$. These 4 clusters being vanishing clusters at $\lambda = \pm \infty$, all of them have index $-\frac 12$ by corollary 4.2.

It then follows from theorem 4.4 that ${\rm {ind}}_{\infty }f = 1+2\cdot \frac 12 - 4\cdot \frac 12 =0$. Comparing to lemma 5.2, there are no index gaps of any kind, and this example realizes the maximal index at infinity that a polynomial of degree $3$ may have, cf. theorem 5.5(a).

Example 6.2 Let $f(x,y)=y^5+x^2y^3-y$. The Milnor set $M(f)$ is defined by the equation $x (-1 + 3 x^2 y^2 + 3 y^4)=0$.

We have $d=5$, ${\mathcal {L}}_{f} = \{p\}$ with $p:=[1:0:0]$, and there are two other complex non-real points at infinity due to the factor $y^2+x^2$ of the top homogeneous part $f_{5}$. In the chart $\{x=1\}$ of ${\mathbb {P}}^2$, the germ at $p$ of the Milnor set $\overline {M(f)}$ is defined by the equation $\hat {h}(y,z)=-z^4+3y^2+3y^4=0$, thus ${\rm {Cone}\ } \overline {M_{{\mathbb {C}}}(f)}_{p}=2 \{y=0\}$. There are 4 clusters having the point $p \in {\mathcal {L}}_f$ at infinity, each containing a single Milnor arc, all being splitting clusters at the value $0$, and one pair of clusters is tangent to a semi-line, and the second pair of clusters is tangent to the other semi-line. Compare also to proposition 3.16.

The fibres at infinity $F_+ \cup F_-$ consist of two components; one corresponds to the cluster $\{\gamma _2\}$, see figure 1, and covers the upper semi-circle of $S$, and the other corresponds to the cluster $\{\gamma _5\}$ and covers the lower semi-circle of $S$. By theorem 4.4, one then has ${\rm {ind}}_{\infty }(f)=2$, which is the highest possible index at infinity of a polynomial of degree $d=5$ with $|{\mathcal {L}}_f| =1$, according to theorem 5.5(b). Since $d_{p} = d_{Re} = 3$, $\deg (R_{p}^{{\mathrm {red}}})=1$, $S_{p}= \emptyset$, and $K_{p}=\emptyset$, the inequality (5.2) reads:

\[ {\rm{ind}}_\infty (f) \le 1 + 3 - 2 = 2. \]

This is an equality in our case, and the same are (5.3), (5.9) and (5.10).

Example 6.3 Let $f(x,y)=(x-y^2)\left ( (x-y^2)(y^2+1)-1\right )$.

Figure 1. Milnor arcs of $f = y^5+x^2y^3-y$.

Figure 2. Milnor arcs of $f= y^5+x^2y^3-y$.

Its Milnor set is defined by the equation: $y (1 - 4 x^2 + 2 x^3 + 2 y^2 + 2 x y^2 - 8 x^2 y^2 + 2 y^4 + 6 x y^4)=0.$

One has $d = d_{Re}=6$, and ${\mathcal {L}}_{f} = \{p\}$ where $p:=[1:0:0]$, with $d_{p} =6$ (cf. definition 4.5). In the chart $\{x=1\}$ of ${\mathbb {P}}^2$, the germ at $p$ of $\overline {M(f)}$ is defined by the equation:

\[ \hat{h}(y,z)= y (6 y^4 - 8 y^2 z + 2 y^4 z + 2 z^2 + 2 y^2 z^2 - 4 z^3 + 2 y^2 z^3 + z^5) =0, \]

and therefore ${\rm {Cone}\ } \overline {M_{{\mathbb {C}}}(f)}_{p}= L^\infty \cup \{y=0\}$.

The polynomial $f$ has a global minimum at the point $\left (\frac {1}{2},0\right )\in {\mathbb {R}}^{2}$, with critical value $-\frac 14$. The fibre of $f$ is empty over the interval $]-\infty, -\frac 14[$. Over $[-\frac {1}{4}, 0[$, the fibre of $f$ is compact and connected, having two arcs outside the disk $D_{R}$, one of which is splitting along the cluster $\{\gamma _7\}$, and the other is splitting along the cluster $\{\gamma _3\}$; both Milnor clusters are tangent to $L^\infty$ at the point $p$. Over the interval $]0, +\infty [$, the fibre of $f$ has two connected components: one of them corresponds to the cluster $\{\gamma _8,\gamma _1,\gamma _2\}$, and is vanishing⁷ at the point $p$ with the value of $f$ tending to $+\infty$. The other component corresponds to the vanishing cluster $\{\gamma _4, \gamma _5, \gamma _6\}$ and covers the entire line at infinity $L^\infty$ as $t\to +\infty$.

By direct computations we see that the germ $\overline {M_{{\mathbb {C}}}(f)_{p}}$ has 3 non-singular branches: one is $\{y=0\}$ and contains the Milnor arcs $\gamma _1$ and $\gamma _5$. The other two branches are tangent to the line at infinity⁸ and have both their two arcs on the same side of it, namely $\gamma _8$ with $\gamma _2$, and $\gamma _7$ with $\gamma _3$, respectively. There is a single Milnor arc on the other side of $L^\infty$, which fact may be contrasted to lemma 5.2(b),(c) about index gaps, see also remark 5.3 and the computation of sign gaps in the proof of theorem 5.1.

By theorem 4.4 we get ${\rm {ind}}_{\infty }(f)= 1+ 2\frac 12 -2\frac 12 =1$. We have $\deg S_{p}= 2$ because of the two non-singular real branches of $\overline {M_{{\mathbb {C}}}(f)}_{p}$ which are tangent to $L^{\infty }$, no non-real branches $K_{p}=\emptyset$, and $\deg (R_{p}^{{\mathrm {red}}})=2$. The inequality (5.2) of theorem 5.1 then reads:

\[ {\rm{ind}}_{\infty}(f) \le 1+6 - 2 - 2 = 3, \]

since the ‘sign gaps’ of this formula count for zero in our case. The same inequality ${\rm {ind}}_{\infty }(f) \le 3$ is provided by theorem 5.5(b).

Nevertheless, if we consider the sharper estimation of the sign gap in the proof of theorem 5.1, namely $\frac 12 \left (\lfloor \frac {r_{p}}{2}\rfloor + \lfloor \frac {s_{p}}{2} \rfloor \right )$, and since in our case we have $r_{p}=5$ and $s_{p} =1$, we get a sign gap of 1. With this extra gap, we then obtain:

\[ {\rm{ind}}_{\infty}(f) \le 1 +6-2-2 -1 =2, \]

which is indeed a better estimation, still away by 1 from the actual index ${\rm {ind}}_{\infty }(f) =1$ of this example as computed above via theorem 4.4.

Example 6.4 Let $f(x,y)=x^2 + (xy-1)^2$. One has $d=d_{Re}=4$, and ${\mathcal {L}}_f=\{ p,q\}$ where $p:=[1:0:0]$ and $q:=[0:1:0]$, with $d_p=d_q=2$. Note that $f$ has empty fibres over $]-\infty, 0[$.

The Milnor set $M(f)$ is defined by the equation $x^2 + x y - x^3 y - y^2 + x y^3=0$. At $q \in L^\infty$ there are two clusters with a single Milnor arc, namely $\{\gamma _3\}$ and $\{\gamma _7\}$, and the corresponding fibre components are both vanishing at the value $0$. There are two more clusters, namely $\{\gamma _8,\gamma _1,\gamma _2\}$, and $\{\gamma _4,\gamma _5,\gamma _6\}$, the fibre components of which are both tending to $+\infty$ and cover half the circle $S$ each of them.

By theorem 4.4 we get:

\begin{align*} {\rm{ind}}_{\infty}(f) & = 1+ \frac12 \sum {\rm{Sp}}(p, \lambda) - \frac12 \sum{\rm{Va}}(p, \lambda) - \frac12 {\rm{Va}}({\pm} \infty)=1\\ & \quad + \frac12 \cdot 0 - \frac12 \cdot 2 - \frac12 \cdot 2={-}1, \end{align*}

where the sums are over $\{p\in {\mathcal {L}}_{f}, \lambda \in {\mathbb {R}}\}$.

We have ${\rm {Cone}\ } \overline {M_{{\mathbb {C}}}(f)}_q=\{x=0\}$ and ${\rm {Cone}\ } \overline {M_{{\mathbb {C}}}(f)}_p=\{y=0\}$, with multiplicity 1, and the Milnor set germs at $p$ and $q$ are non-singular and transversal to $L^\infty _{{\mathbb {C}}}$. Therefore all the sets $S_{p}, K_p, S_{q}$ and $K_q$ are empty, and $\deg R^{{\mathrm {red}}}_{q}=\deg R^{{\mathrm {red}}}_{p}=1$. Then theorem 5.1 and corollary 5.4, with (5.9) and (5.10), yield all the same bound:

\[ {\rm{ind}}_{\infty}(f) \le 1+ d_{Re} - |{\mathcal{L}}_{f}| = 1+4-4=1. \]

By considering the genuine sign gaps as in lemma 5.2(a), one actually obtains a gap of 2 at the point $q \in {\mathcal {L}}_f$ due to the two Milnor arcs with index $-\frac 12$ at the value 0 of $f$. We then get

\[ {\rm{ind}}_{\infty}(f) \le 1+4-4-2={-}1, \]

which coincides with the actual index at infinity of $f$ computed above (figure 3).

Figure 3. The Milnor arcs of $f= x^2 + (xy-1)^2$.