Asymptotics and sign patterns of Hecke polynomial coefficients

Abstract

We determine the asymptotic behavior of the coefficients of Hecke polynomials. In particular, this allows us to determine signs of these coefficients when the level or the weight is sufficiently large. In all but finitely many cases, this also verifies a conjecture on the nanvanishing of the coefficients of Hecke polynomials.

1 Introduction

For integers $m \geq 1$ , N coprime to m, and $k\geq 2$ even, let $S_k(\Gamma _0(N))$ denote the space of cuspforms of level N and weight k. Let $T^{\prime }_{m}(N,k) := \frac {1}{m^{(k-1)/2}} T_{m}(N,k)$ denote the normalized mth Hecke operator on $S_k(\Gamma _0(N))$ . For each integer $r\ge 0$ , let $c_{r}(m,N,k)$ denote the rth coefficient of the characteristic polynomial $T^{\prime }_{m}(N,k)(x)$ associated with $T^{\prime }_{m}(N,k)$ as follows:

$$ \begin{align*} T^{\prime}_{m}(N,k)(x)=\sum_{r=0}^{d} c_r(m,N,k) x^{d-r}, \end{align*} $$

where $d=\dim S_k(\Gamma _0(N))$ . Hecke operators are of central importance in the theory of modular forms, and are completely characterized by the Hecke polynomials. We would like to study the coefficients of these Hecke polynomials in order to understand their structure. In particular, for any fixed m and r, the main goal of this article is to determine the asymptotic behavior of $c_{r}(m,N,k)$ as $N+k \to \infty $ . This will also show that $c_{r}(m,N,k)$ is nonvanishing and further determine its sign in all but finitely many cases.

We give an outline of this article. In Section 2, we apply the Girard–Newton formula to the coefficients $c_{r}(m,N,k)$ , and state the asymptotic behavior of $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k)$ . In Section 3, we consider the case when m is a perfect square and prove the following result determining the asymptotic behavior of the $c_r(m,N,k)$ . In the following, all big-O notation is with respect to N and k. Additionally, we use the notation “ $O(N^\varepsilon )$ ”, for example, to mean “ $O(N^\varepsilon )$ for all $\varepsilon> 0$ ”.

Theorem 1.1 Fix an integer $r \ge 0$ and a perfect square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} c_r(m,N,k) = \frac{(-1)^r}{r!} \left( \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N)\right)^r + O(k^{r-1}N^{r-1/2+\varepsilon}). \end{align*} $$

Here, $\psi (N)$ denotes the multiplicative function $\psi (N) = N \prod _{p\mid N} \left (1+\frac 1p\right )$ .

In Section 4, we consider the case when m is not a perfect square and establish the following asymptotics of $c_r(m,N,k)$ . Recall here that $\sigma _1(m)$ denotes the sum of divisors function $\sigma _1(m) := \sum _{d \mid m} d$ , and that $(2r)!!$ denotes the double factorial $(2r)!! := 2r (2r-2)(2r-4) \cdots 2$ .

Theorem 1.2 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} c_{2r}(m,N,k) &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^r + O(k^{r-1} N^{r-1/2+\varepsilon}) \qquad \text{and} \\ c_{2r+1}(m,N,k) &= c_1(m,N,k) \cdot \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi(N)\right)^r + O(k^{r-1} N^{r-1/2+\varepsilon}). \end{align*} $$

In Section 5, we extend Theorems 1.1 and 1.2 to the new subspace $S_k^{\text {new}}(\Gamma _0(N))$ .

Finally, in Sections 6 and 7, we discuss these results. In Section 6, we discuss how the arguments given in Theorems 1.1 and 1.2 for the Hecke polynomials can also be applied to other polynomials. In particular, these arguments reveal a coefficient sign pattern for a wide class of polynomials. Then in Section 7, we discuss a conjecture on the nonvanishing of the Hecke polynomial coefficients and survey its current progress.

2 Preliminary calculations

For simplicity, we write $c_r$ for the coefficients $c_r(m,N,k)$ . Let $d=\dim S_k(\Gamma _0(N))$ and $\lambda _1, \ldots , \lambda _d$ denote the eigenvalues of $T^{\prime }_m(N,k)$ . Observe that $(-1)^r c_r$ is just the rth elementary symmetric polynomial of these eigenvalues:

$$ \begin{align*} c_0=1,\quad -c_1 = \sum_{1\leq i\leq d} \lambda_i \,, \quad c_2 = \sum_{1\leq i < j \leq d} \lambda_i \lambda_j \,, \quad -c_3 = \sum_{1\leq i<j<\ell\leq d} \lambda_i \lambda_j \lambda_\ell\,, \quad \ldots \end{align*} $$

We also write $p_r$ for the sum of rth powers of these eigenvalues:

(1) $$ \begin{align} p_r:=\sum_{i=1}^{d} \lambda_i^r. \end{align} $$

Then the Girard–Newton identities yield the following relation between the $c_r$ and the $p_r$ .

Lemma 2.1 [13, p. 38]

Let $c_r$ and $p_r$ be defined as above. Then for $r\ge 1$ ,

$$ \begin{align*} c_r = \frac{-1}{r} \sum_{j=1}^r c_{r-j} p_j. \end{align*} $$

We also give estimates on the traces of Hecke operators. These estimates will be needed shortly when we express the $p_r$ in terms of traces of certain Hecke operators. In a previous paper [11], we proved the following result by analyzing the various terms of the Eichler–Selberg trace formula.

Lemma 2.2 [11, Lemmas 4.1 and 4.2]

Fix an integer $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} \operatorname{\mathrm{Tr}} T^{\prime}_m(N,k) = \begin{cases} \frac{1}{\sqrt{m}}\frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), & \text{if }m\text{ is a perfect square}, \\ O(N^\varepsilon), & \text{if }m\text{ is not a perfect square}. \end{cases} \end{align*} $$

To gauge the growth of the terms in this formula, note that $\psi (N) \geq N$ and $\psi (N) = O(N^{1+\varepsilon })$ [7, Sections 18.1 and 22.13].

3 When $m$ is a perfect square

In this section, we consider the case when m is a perfect square. We then have the following estimates on the $p_j$ (1).

Lemma 3.1 Fix an integer $r \ge 1$ and a perfect square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} p_1 &= \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), \qquad \text{and} \\ p_{j} &= O(kN^{1+\varepsilon}), \qquad \text{for all }1 \leq j \leq r. \end{align*} $$

Proof The first claim follows immediately from Lemma 2.2.

For the second claim, note from Lemma 2.2 and the fact that $\psi (N)=O(N^{1+\epsilon })$ ,

$$ \begin{align*} d := \dim S_k(\Gamma_0(N)) = \operatorname{\mathrm{Tr}} T^{\prime}_1 = O(kN^{1+\varepsilon}). \end{align*} $$

Then utilizing Deligne’s bound $\left \lvert \lambda _i \right \lvert \leq \sigma _0(m) = \sum _{d\mid m} 1$ , we obtain

$$ \begin{align*}\left\lvert p_j \right\lvert = \left\lvert \sum_{i=1}^d \lambda_i^j \right\lvert \leq \sum_{i=1}^d \sigma_0(m)^j = O(kN^{1+\varepsilon}), \end{align*} $$

as desired.

For m and r fixed, we now determine the asymptotic behavior of $c_r(m,N,k)$ as $N+k \to \infty $ . Note $c_r(m,N,k)$ is not technically defined for $N,k$ such that $\dim S_k(\Gamma _0(N)) < r$ . However, there are only finitely many such pairs $(N,k)$ [10, Theorem 1.1], so it is well-defined here to ask about $c_r(m,N,k)$ as $N+k\to \infty $ .

Theorem 1.1 Fix an integer $r \ge 0$ and a perfect square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} c_r(m,N,k) = \frac{(-1)^r}{r!} \left( \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N)\right)^r + O(k^{r-1}N^{r-1/2+\varepsilon}). \end{align*} $$

Proof We proceed by strong induction on r. The base case of $r=0$ is immediate since $c_0 = 1$ .

For $r \geq 1$ , we have by Lemma 2.1 that

(2) $$ \begin{align} c_{r} = \frac{-1}{r} \sum_{j=1}^{r} c_{r-j} p_j = \frac{-1}{r} \Bigg[ c_{r-1} p_1 + \sum_{j=2}^{r} c_{r-j} p_j \Bigg]. \end{align} $$

Then by the induction hypothesis,

$$ \begin{align*} c_{r-1} &= \frac{(-1)^{r-1}}{(r-1)!} \left( \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N)\right)^{r-1} + O(k^{r-2}N^{r-3/2+\varepsilon}), \\ c_{r-j} &= O(k^{r-2} N^{r-2+\varepsilon}), \qquad\qquad\qquad \text{for }2\le j \le r, \end{align*} $$

and by Lemma 3.1,

$$ \begin{align*} p_1 &= \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), \\ p_j &= O(kN^{1+\varepsilon}), \qquad\qquad\qquad \text{for }2\le j \le r. \end{align*} $$

Applying these estimates to (2), we obtain

$$ \begin{align*} c_{r} &= \frac{-1}{r} \Bigg[ c_{r-1} p_1 + \sum_{j=2}^{r} c_{r-j} p_j \Bigg] \\&= \frac{-1}{r} \Bigg[ \left(\frac{(-1)^{r-1}}{(r-1)!} \left(\frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N)\right)^{r-1} + O(k^{r-2} N^{r-3/2+\varepsilon}) \right) \\& \quad\qquad \times \left( \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}) \right) + \sum_{j=2}^{r} O(k^{r-2} N^{r-3/2+\varepsilon}) \cdot O(k N^{1+\varepsilon}) \Bigg] \\&= \frac{-1}{r} \Bigg[ \frac{(-1)^{r-1}}{(r-1)!} \left(\frac{1}{\sqrt m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}) \Bigg] \\&= \frac{(-1)^r}{r!} \left(\frac{1}{\sqrt m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}). \end{align*} $$

This completes the proof.

Theorem 1.1 allows us in particular to determine the sign of $c_r(m,N,k)$ for all but finitely many pairs $(N,k)$ .

Corollary 3.2 Fix an integer $r \ge 0$ and a perfect square $m \ge 1$ . Then $c_{r}(m,N,k)$ has sign $(-1)^r$ for all but finitely pairs $(N,k)$ .

Proof Since $\psi (N) \ge N$ , we can write the asymptotic formula from Theorem 1.1 as

$$ \begin{align*} c_r(m,N,k) &= \frac{(-1)^r}{r! \sqrt{m}^r} \left(\frac{k-1}{12} \psi(N)\right)^r \left[1 + O\left(k^{r-1}N^{r-1/2+\varepsilon}\right) \left(\frac{k-1}{12} \psi(N)\right)^{-r} \right] \\ &= \frac{(-1)^r}{r! \sqrt{m}^r} \left(\frac{k-1}{12} \psi(N)\right)^r \left[ 1 + O\left(k^{-1}N^{-1/2+\varepsilon}\right) \right]. \end{align*} $$

Then since the $O\left (k^{-1}N^{-1/2+\varepsilon }\right )$ term tends to $0$ as $N \to \infty $ or $k \to \infty $ , it will have magnitude less than $1$ for all but finitely many pairs $(N,k)$ . This then yields the desired result.

4 When $m$ is not a perfect square

In this section, we consider the remaining case when m is not a perfect square. First, we have the following estimates on the $p_j$ (1).

Lemma 4.1 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then

$$ \begin{align*} p_1 &= -c_1 = O(N^\varepsilon), \\ p_2 &= \frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), \\ p_{3} &= O(N^\varepsilon), \\ p_{j} &= O(kN^{1+\varepsilon}), \qquad \text{for all }1 \leq j \leq r. \end{align*} $$

Proof The first claim follows immediately from Lemma 2.2.

For the second claim, observe that $p_2 = \operatorname {\mathrm {Tr}} {T_m'}^{2}$ . Then by the Hecke operator composition formula [6, Theorem 10.2.9] and Lemma 2.2,

$$ \begin{align*} p_2 = \operatorname{\mathrm{Tr}} {T_m'}^2 = \sum_{d \mid m} \operatorname{\mathrm{Tr}} T_{m^2/d^2}' &= \sum_{d \mid m} \frac{d}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}) \\ &= \frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}). \end{align*} $$

For the third claim, we similarly have by the Hecke operator composition formula and Lemma 2.2,

$$ \begin{align*} p_3 = \operatorname{\mathrm{Tr}} {T_m'}^3 = \operatorname{\mathrm{Tr}} \sum_{d \mid m} T_{m^2/d^2}' T_m' &= \sum_{d \mid m} \ \sum_{\delta \mid (m^2/d^2, m)} \operatorname{\mathrm{Tr}} T^{\prime}_{m^3/d^2\delta^2} \\ &= \sum_{d \mid m} \ \sum_{\delta \mid (m^2/d^2, m)} O(N^\varepsilon) \\ &= O(N^{\varepsilon}). \end{align*} $$

Finally, the fourth claim follows from an identical argument as in Lemma 3.1.

For m and r fixed, we now determine the asymptotic behavior of $c_r(m,N,k)$ as $N+k \to \infty $ .

Theorem 1.2 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} c_{2r} &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^r + O(k^{r-1} N^{r-1/2+\varepsilon}) \qquad \text{and} \\ c_{2r+1} &= c_1 \cdot \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi(N)\right)^r + O(k^{r-1} N^{r-1/2+\varepsilon}). \end{align*} $$

Proof We proceed by strong induction on r. The base case of $r=0$ is immediate since $c_0 = 1$ and $c_1 = c_1$ .

Then for $r\ge 1$ , we have from Lemma 2.1 that

(3) $$ \begin{align} c_{2r} &= \frac{-1}{2r} \sum_{j=1}^{2r} c_{2r-j} p_j = \frac{-1}{2r} \Bigg[ c_{2r-1} p_1 + c_{2r-2} p_2 + \sum_{j=3}^{2r} c_{2r-j} p_j \Bigg]. \end{align} $$

Then by the induction hypotheses,

$$ \begin{align*} c_{2r-1} &= O(k^{r-1} N^{r-1+\varepsilon}), \\ c_{2r-2} &= \frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r-1} + O(k^{r-2} N^{r-3/2+\varepsilon}), \\ c_{2r-j} &= O(k^{r-2} N^{r-2+\varepsilon}), \qquad\qquad \text{for }3\le j \le 2r, \end{align*} $$

and by Lemma 4.1,

$$ \begin{align*} p_1 &= O(N^\varepsilon), \\ p_2 &= \frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), \\ p_j &= O(kN^{1+\varepsilon}), \qquad\qquad \text{for }3\le j \le 2r. \end{align*} $$

Applying these estimates to (3), we obtain

$$ \begin{align*} c_{2r} &= \frac{-1}{2r} \Bigg[ c_{2r-1} p_1 + c_{2r-2} p_2 + \sum_{j=3}^{2r} c_{2r-j} p_j \Bigg] \\&= \frac{-1}{2r} \Bigg[ O(k^{r-1} N^{r-1+\varepsilon}) \cdot O(N^\varepsilon) \\&\qquad\ \ + \left(\frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r-1} + O(k^{r-2} N^{r-3/2+\varepsilon}) \right) \\&\qquad\ \ \times \left( \frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}) \right) + \sum_{j=3}^{2r} O(k^{r-2} N^{r-2+\varepsilon}) \cdot O(k N^{1+\varepsilon}) \Bigg] \\&= \frac{-1}{2r} \Bigg[ O(k^{r-1}N^{r-1+\varepsilon}) + \frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} \\&\qquad\ \ + O(k^{r-1} N^{r-1/2+\varepsilon}) + O(k^{r-1} N^{r-1+\varepsilon}) \Bigg] \\&= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}), \end{align*} $$

verifying the first claim of the inductive step.

For the second claim of the inductive step, we similarly have from Lemma 2.1 that

(4) $$ \begin{align} c_{2r+1} &= \frac{-1}{2r+1} \sum_{j=1}^{2r+1} c_{2r+1-j} p_j = \frac{-1}{2r+1} \Bigg[ c_{2r} p_1 + c_{2r-1} p_2 + c_{2r-2} p_3 + \sum_{j=4}^{2r+1} c_{2r+1-j} p_j \Bigg]. \end{align} $$

Then by the induction hypotheses and the proof for $c_{2r}$ ,

$$ \begin{align*} c_{2r} &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}), \\ c_{2r-1} &= c_1 \cdot \frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi(N)\right)^{r-1} + O(k^{r-2} N^{r-3/2+\varepsilon}), \\ c_{2r-2} &= O(k^{r-1} N^{r-1+\varepsilon}), \\ c_{2r+1-j} &= O(k^{r-2} N^{r-2+\varepsilon}), \qquad\qquad \text{for }4\le j \le 2r+1, \end{align*} $$

and by Lemma 4.1,

$$ \begin{align*} p_1 &= -c_1, \\ p_2 &= \frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon}), \\ p_3 &= O(N^\varepsilon), \\ p_j &= O(kN^{1+\varepsilon}), \qquad\qquad \text{for }4\le j \le 2r+1. \end{align*} $$

Applying these estimates to (4), we obtain

$$ \begin{align*} c_{2r+1} &= \frac{-1}{2r+1} \Bigg[ c_{2r} p_1 + c_{2r-1} p_2 + c_{2r-2} p_3 + \sum_{j=4}^{2r+1} c_{2r+1-j} p_j \Bigg] \\&= \frac{-1}{2r+1} \Bigg[\left(\frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}) \right) \cdot (-c_1) \\&\qquad\qquad + \left(c_1 \cdot \frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi(N)\right)^{r-1} + O(k^{r-2} N^{r-3/2+\varepsilon})\right) \\&\qquad\qquad \times \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N) + O(N^{1/2+\varepsilon})\right) \\&\qquad\qquad + O(k^{r-1} N^{r-1+\varepsilon}) \cdot O(N^\varepsilon) + \sum_{j=4}^{2r+1} O(k^{r-2} N^{r-2+\varepsilon}) \cdot O(kN^{1+\varepsilon}) \Bigg] \\&= \frac{-1}{2r+1} \Bigg[ -c_1 \cdot \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}) \\&\qquad\qquad + c_1 \cdot \frac{(-1)^{r-1}}{(2r-2)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}) \\&\qquad\qquad + O(k^{r-1} N^{r-1+\varepsilon}) + O(k^{r-1} N^{r-1+\varepsilon}) \Bigg]\\&= c_1 \cdot \frac{1}{2r+1} \left(\frac{(-1)^r}{(2r)!!} - \frac{(-1)^{r-1}}{(2r-2)!!} \right) \cdot \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}) \\&= c_1 \cdot \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi(N)\right)^{r} + O(k^{r-1} N^{r-1/2+\varepsilon}), \end{align*} $$

verifying the second claim of the inductive step.

This completes the proof.

Theorem 1.2 allows us in particular to determine the sign of the even-indexed coefficients for all but finitely many pairs $(N,k)$ . The following corollary can be shown using an identical argument as in Corollary 3.2.

Corollary 4.2 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then $c_{2r}(m,N,k)$ has sign $(-1)^r$ for all but finitely pairs $(N,k)$ .

The behavior of the odd-indexed coefficients, on the other hand, is determined by the behavior of the trace.

Corollary 4.3 Fix an integer $r \ge 0$ , a non-square $m \ge 1$ , and an even integer $k \ge 2$ . Consider N such that $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k) \neq 0$ . Then $c_{2r+1}(m,N,k)$ has sign $(-1)^{r+1} \operatorname {\mathrm {sgn}}(\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k))$ for all but finitely many N.

Proof Since $\psi (N) \ge N$ , we can write the asymptotic formula from Theorem 1.2 as

$$ \begin{align*} c_{2r+1} &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m}\right)^r \left( \frac{k-1}{12}\psi(N)\right)^r \left[c_1 + O\left(k^{r-1} N^{r-1/2+\varepsilon}\right) \left( \frac{k-1}{12}\psi(N)\right)^{-r} \right] \\ &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m}\right)^r \left( \frac{k-1}{12}\psi(N)\right)^r \left[c_1 + O\left(k^{-1} N^{-1/2+\varepsilon}\right) \right]. \end{align*} $$

Then observe that since $\operatorname {\mathrm {Tr}} T_m \in {\mathbb {Z}}$ and $c_1 = -\operatorname {\mathrm {Tr}} T^{\prime }_m = - m^{-(k-1)/2} \operatorname {\mathrm {Tr}} T_m \neq 0$ , we must have $\left \lvert c_1 \right \lvert = \left \lvert \operatorname {\mathrm {Tr}} T^{\prime }_m \right \lvert \geq m^{-(k-1)/2}$ . And because the $O\left (k^{-1}N^{-1/2+\varepsilon }\right )$ term tends to $0$ as $N \to \infty $ , it will have magnitude less than $m^{-(k-1)/2}$ for all but finitely many N. This yields the desired result.

We note that the condition $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k) \neq 0$ here is not overly restrictive. Rouse [12, Theorem 1.6] showed that there are only finitely many k for which we could possibly have $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k) = 0$ for some N. And even for these finitely many remaining k, he showed in [12, Theorem 1.7] that $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k) \neq 0$ for $100\%$ of N. He further conjectured in [12, Conjecture 1.5] that $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k) \neq 0$ for all $N \ge 1$ and $k=12$ or $\geq 16$ .

We also note that there are various ways one could try to improve this result to where N and k both vary. The only reason we fixed k was to guarantee that $c_1 = -\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k)$ was bounded away from $0$ . If we relax the condition of k being fixed to just that $k \leq (1-\delta )\log _m(N)$ for some $\delta> 0$ , then we have the same result for all but finitely many pairs $(N,k)$ . One could also try to bound $\operatorname {\mathrm {Tr}} T^{\prime }_m(N,k)$ away from $0$ using some sort of vertical Atkin–Serre type result for the trace (e.g., along the lines of [8, Theorem 2.2]).

5 Extending to the new subspace

All of our results extend to the Hecke polynomial over the new subspace. Let $T^{\prime \,{\text {new}}}_m(N,k)$ denote the restriction of $T^{\prime }_m(N,k)$ to the new subspace $S_k^{\text {new}}(\Gamma _0(N))$ . Let $c^{\text {new}}_{r}(m,N,k)$ denote the rth coefficient of the characteristic polynomial $T^{\prime \,{\text {new}}}_{m}(N,k)(x)$ as follows:

$$ \begin{align*} T^{\prime \,{\text{new}}}_{m}(N,k)(x)=\sum_{r=0}^{d^{\text{new}}} c^{\text{new}}_r(m,N,k) x^{d^{\text{new}}-r}. \end{align*} $$

Here, $d^{\text {new}}=\dim S_k^{\text {new}}(\Gamma _0(N))$ . Just like before, we can then determine the asymptotic behavior of $c_r^{\text {new}}(m,N,k)$ as $N+k \to \infty $ . Note $c_r^{\text {new}}(m,N,k)$ is not technically defined for $N,k$ such that $\dim S_k^{\text {new}}(\Gamma _0(N)) < r$ . However, there are only finitely many such pairs $(N,k)$ [10, Theorem 1.3], so it is perfectly well-defined here to ask about $c_r^{\text {new}}(m,N,k)$ as $N+k\to \infty $ .

Let

$$ \begin{align*} \psi^{\text{new}}(N) := \prod_{p^r \parallel N} \begin{cases} p\left(1-\frac1p\right), & \text{if } r=1, \\ p^2\left(1-\frac1p - \frac{1}{p^2}\right), & \text{if } r=2, \\ p^r\left(1-\frac1p-\frac{1}{p^2}+\frac{1}{p^3}\right), & \text{if } r\geq3, \end{cases} \end{align*} $$

and note that $\psi ^{\text {new}}(N) \leq N$ and $\psi ^{\text {new}}(N) = \Omega (N^{1-\varepsilon })$ [7, Sections 18.1 and 22.13]. In [1, Lemmas 4.2 and 4.3], Cason et al. showed that for fixed m,

$$ \begin{align*} \operatorname{\mathrm{Tr}} T^{\prime \,{\text{new}}}_m(N,k) = \begin{cases} \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi^{\text{new}}(N) + O(N^{1/2}), & \text{if }m\text{ is a perfect square}, \\ O(N^\varepsilon), & \text{if }m\text{ is not a perfect square}. \end{cases} \end{align*} $$

The following two theorems then follow by an identical argument as in Theorems 1.1 and 1.2. The details are omitted.

Theorem 5.1 Fix an integer $r \ge 0$ and a perfect square $m \ge 1$ . Then for N coprime to m and $k \geq 2$ even,

$$ \begin{align*} c^{\mathrm{new}}_r(m,N,k) = \frac{(-1)^r}{r!} \left( \frac{1}{\sqrt{m}} \frac{k-1}{12} \psi^{\mathrm{new}}(N)\right)^r + O(k^{r-1}N^{r-1/2}). \end{align*} $$

Theorem 5.2 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then

$$ \begin{align*} c^{\mathrm{new}}_{2r}(m,N,k) &= \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12} \psi^{\mathrm{new}}(N)\right)^r + O(k^{r-1} N^{r-1/2}) \qquad \text{and} \\ c^{\mathrm{new}}_{2r+1}(m,N,k) &= c^{\mathrm{new}}_1(m,N,k) \cdot \frac{(-1)^r}{(2r)!!} \left(\frac{\sigma_1(m)}{m} \frac{k-1}{12}\psi^{\mathrm{new}}(N)\right)^r + O(k^{r-1} N^{r-1/2+\varepsilon}). \end{align*} $$

Then just like in Corollaries 3.2, 4.2, and 4.3, this tells us the sign of the $c^{\text {new}}_r$ in all but finitely many cases.

Corollary 5.3 Fix an integer $r \ge 0$ and a perfect square $m \ge 1$ . Then $c^{\mathrm {new}}_{r}(m,N,k)$ has sign $(-1)^r$ for all but finitely pairs $(N,k)$ .

Corollary 5.4 Fix an integer $r \ge 0$ and a non-square $m \ge 1$ . Then $c^{\mathrm {new}}_{2r}(m,N,k)$ has sign $(-1)^r$ for all but finitely pairs $(N,k)$ .

Corollary 5.5 Fix an integer $r \ge 0$ , a non-square $m \ge 1$ , and an even integer $k \ge 2$ . Consider N such that $\operatorname {\mathrm {Tr}} T^{\prime \,{\mathrm {new}}}_m(N,k) \neq 0$ . Then $c^{\mathrm {new}}_{2r+1}(m,N,k)$ has sign $(-1)^{r+1} \operatorname {\mathrm {sgn}}(\operatorname {\mathrm {Tr}} T^{\prime \,{\mathrm {new}}}_m(N,k))$ for all but finitely many N.

6 Sign patterns for more general polynomials

In response to our previous paper showing that $c_2$ tends to be negative [11], Kimball Martin suggested to us that $c_2$ might display a similar bias more generally for polynomials with totally real roots. In fact, the sign tendencies for all the coefficients given in Corollaries 4.2 and 4.3 hold more generally for a wide class of polynomials with totally real roots. Essentially the only two conditions we need to impose are that the roots are distributed over an interval $[-A,A]$ in a roughly symmetric way about the origin, and that the roots are not all clustered at the origin.

More precisely, for $A>0$ and r fixed, consider a sequence of polynomials $f_n$ with totally real roots lying in the interval $[-A,A]$ . Let $d_n$ denote the degree of $f_n$ , and let the $c_j(n)$ and $p_j(n)$ be defined as above in Section 2. We assume that $p_1(n) = o(d_n^{1/3})$ and $p_3(n) = o(d_n)$ (which will occur if the roots $x_1, \ldots , x_{d_n}$ of $f_n$ are distributed in a roughly symmetric way about the origin). Also note that $p_2(n) = \alpha _n^2 d_n$ , where $\alpha _n$ denotes the quadratic mean of the roots of $f_n$ . We assume that $\alpha _n$ is bounded away from $0$ (which will occur as long as the roots are not all clustered at the origin). Finally, observe that $\left \lvert p_j(n) \right \lvert \leq A^r d_n = O(d_n)$ for each $1 \leq j \leq r$ . These estimates

$$ \begin{align*} p_1(n) &= -c_1(n) = o(d_n^{1/3}), \\ p_2(n) &= \alpha_n^2 d_n, \\ p_{3}(n) &= o(d_n), \\ p_{j}(n) &= O(d_n) \qquad \text{for each }1 \leq j \leq r, \end{align*} $$

are essentially the content of Lemma 4.1 (except that Lemma 4.1 has much stronger error bounds, and where in that case, $\alpha _{N,k}$ tends to $\sqrt {\sigma _1(m)/m}$ as $N+k \to \infty $ [2, Theorem 1.1]).

Then using an identical argument as in Theorem 1.2, one can show that

$$ \begin{align*} c_{2r}(n) &= \frac{(-1)^r}{(2r)!!} \left(\alpha_n^2 d_n\right)^r + o(d_n^{r-1/3}) \qquad \text{and} \\ c_{2r+1}(n) &= c_1(n) \frac{(-1)^r}{(2r)!!} \left(\alpha_n^2 d_n\right)^r + o(d_n^{r}). \end{align*} $$

In particular, this means that as $d_n \to \infty $ , the coefficients of $f_n$ will tend to the sign pattern

$$ \begin{align*} &+--++--++--++\cdots \qquad \text{if }c_1\text{ is bounded below }0,\\ &++--++--++--+\cdots \qquad \text{if }c_1\text{ is bounded above }0. \end{align*} $$

For example, browsing the polynomials with totally real roots of degree 10 in LMFDB, almost all of them follow this sign pattern; see [9] (such polynomials given by LMFDB are shifted so that their roots are roughly symmetric about the origin).

If the roots of a polynomial are perfectly symmetric about the origin, then we will have $c_1 = 0$ , and the sign pattern becomes

$$ \begin{align*} &+0-0+0-0+0-0+\cdots. \end{align*} $$

For example, the roots of the Chebyshev polynomials are distributed in $[-1,1]$ in a perfectly symmetric way, and their coefficients follow precisely this pattern.

We also note what happens when the roots of a polynomial are not distributed symmetrically about the origin. If all the roots have the same sign, then the coefficients $c_r$ follow the sign pattern

$$ \begin{align*} &+-+-+-+-\ldots \qquad \text{if all the roots are positive}, \\ &+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt}+\hspace{0.4pt} \ldots \qquad \text{if all the roots are negative}. \end{align*} $$

When m is a perfect square, most of the roots of $T^{\prime }_m(N,k)(x)$ are positive, and Corollary 3.2 shows that the coefficients of $T^{\prime }_m(N,k)(x)$ tend to this first pattern.

7 A conjecture on Hecke polynomial coefficients

In [12, Conjecture 1.5], Rouse gave the generalized Lehmer conjecture: that for all $m\geq 1$ , N coprime to m, and $k = 12$ or $\geq 16$ , $\operatorname {\mathrm {Tr}} T_m(N,k) \neq 0$ . More recently, Clayton et al. [5, Conjecture 5.1] similarly conjectured that none of the Hecke polynomial coefficients vanish in the level one case. We propose the following conjecture that further extends both the generalized Lehmer conjecture, and [5, Conjecture 5.1]. The results in this article verify Conjecture 7.1 in all but finitely many cases.

Conjecture 7.1 Fix integers $m \geq 1$ and $r \geq 1$ . Then the rth coefficient of the Hecke polynomial $T_m(N,k)(x)$ is nonvanishing for all $N \ge 1$ coprime to m, and $k = 12r$ or $\geq 12 r+4$ even.

We note that these lower bounds on k are the minimum possible. For any k less than these bounds, we will have $\dim S_k(\Gamma _0(1)) < r$ , and hence $c_r(m,1,k) = 0$ , trivially. Even relaxing the lower bound on k to just requiring that $\dim S_k(\Gamma _0(N)) \geq r$ will not work; Rouse [12, Theorem 1.2] showed that for any given m and $k \in \{4,6,8,10,14\}$ , $\operatorname {\mathrm {Tr}} T_m(N,k) = 0$ for infinitely many N.

We now survey all the relevant previous results through the lens of Conjecture 7.1 (although they were not explicitly stated in these terms).

• • When $r\geq 1$ , $m=1$ , Conjecture 7.1 follows from the fact that $\dim S_k(\Gamma _0(N)) \geq r$ for $k=12r$ and $k\geq 12r+4$ .

• • In 2006, when $r=1$ and m is a non-square, Rouse [12] showed Conjecture 7.1 for all but finitely many k, and for $100\%$ of N. When $r=1, m=2$ , he also completely verified Conjecture 7.1.

• • In 2022, when $r=1$ , $m=2$ , Chiriac and Jorza [3] verified Conjecture 7.1 in the case of $N=1$ .

• • In 2023, when $r=2$ , $m\geq 2$ , Clayton et al. [5] showed Conjecture 7.1 in the case of $N=1$ for all but finitely many k. When $r=2$ , $m=2$ , they also completely verified Conjecture 7.1.

• • In 2023, when $r=1$ , $m=3$ , Chiriac et al. [4] verified Conjecture 7.1 in the case of $N=1$ .

• • In 2024, when $r=2$ , $m\ge 2$ , we [11] showed Conjecture 7.1 for all but finitely many pairs $(N,k)$ . When $r=2$ , $m=3,4$ , we also completely verified Conjecture 7.1.

• • In 2024, when $r=2$ , $m\ge 2$ , Cason et al. [1] showed a corresponding conjecture on the newspace $S_k^{\text {new}}(\Gamma _0(N))$ for all but finitely many pairs $(N,k)$ . When $r=2, m=2,4$ , they also completely verified the corresponding conjecture on the newspace.

• • In this article, when $r\geq 1$ and m is a square, Corollary 3.2 proves Conjecture 7.1 for all but finitely many pairs $(N,k)$ .

• • In this article, when r is even and m is a non-square, Corollary 4.2 proves Conjecture 7.1 for all but finitely many pairs $(N,k)$ .

• • In this article, when r is odd and m is a non-square, Corollary 4.3 shows that for k fixed, if Conjecture 7.1 holds for $r=1$ , then it also holds for each odd r for all but finitely many N. In particular, combining with Rouse’s result, this means that there exists a finite set K such that: (1) for all $k \not \in K$ , Conjecture 7.1 holds for all but finitely many N, and (2) even for $k \in K$ , Conjecture 7.1 holds for $100\%$ of N.

We observe that the last result listed here essentially reduces the problem of studying odd-indexed coefficients $c_r$ to just studying the trace, $-c_1$ .