Dynamics on the $\mathrm {SU}(2,1)$-character variety of the one-holed torus

Abstract

We study the relative $\mathrm {SU}(2,1)$-character varieties of the one-holed torus, and the action of the mapping class group on them. We use an explicit description of the character variety of the free group of rank two in $\mathrm {SU}(2,1)$ in terms of traces, which allow us to describe the topology of the character variety. We then combine this description with a generalization of the Farey graph adapted to this new combinatorial setting, using ideas introduced by Bowditch. Using these tools, we can describe an open domain of discontinuity for the action of the mapping class group which strictly contains the set of convex cocompact characters, and we give several characterizations of representations in this domain.

1. Introduction

Let $\Gamma$ be the fundamental group of the one $1$ -holed torus $S_{1,1}$ , that is, $\Gamma \cong F_2 =\langle \alpha, \beta \rangle$ , a free group of rank $2$ . The Lie group $\mathrm {SU}(2,1)$ acts on the space of group homomorphisms $\mathrm {Hom}(F_2,\mathrm {SU}(2,1))$ by conjugation. Let $\mathrm {Hom}(F_2,\mathrm {SU}(2,1))^*$ be the subspace of $\mathrm {Hom}(F_2,\mathrm {SU}(2,1))$ consisting of homomorphisms having closed conjugation orbits, and define the $\mathrm {SU}(2,1)$ -character variety of $F_2$ to be the quotient space $\mathfrak {X} = \mathrm {Hom}(F_2, \mathrm {SU}(2,1))^*/\mathrm {SU}(2,1)$ . The mapping class group $\mathcal {MCG}(S_{1,1})$ is isomorphic to the outer automorphisms of $\Gamma$ and acts on $\mathfrak {X}$ by pre-composition.

The main result of this paper is the following.

Theorem A. There exists an open domain of discontinuity $\mathfrak {X}_{BQ}$ for the action of the mapping class group $\mathcal {MCG}(S_{1,1})$ on $\mathfrak {X}$ which strictly contains the set of discrete, faithful, convex cocompact representations.

To prove this theorem we combine ideas of Bowditch [Bow98] (further generalized by Tan, Wong and Zhang [TWZ08], Maloni, Palesi and Tan [MPT15] and Maloni and Palesi [MP20]) with the explicit parametrization of the $\mathrm {SL}(3,\mathbb {C})$ -character variety of $F_2$ described by Lawton [Law06, Law07]. In order to describe the set of representations that form $\mathfrak {X}_{BQ}$ , called Bowditch representations, we need to analyze what are called ‘relative character varieties’. Given $c \in \mathbb {C}$ , we define the relative character variety $\mathfrak {X}_c$ to be the set of $\mathrm {SU}(2,1)$ -valued representations of $\Gamma$ with fixed trace of the peripheral element $[\alpha, \beta ]$ :

\begin{equation*}\mathfrak {X}_c =\{[\rho ] \in \mathfrak {X} \mid \mathrm {tr}(\rho ([\alpha, \beta ])) = c \}.\end{equation*}

As a bi-product of our work, we can show that Bowditch representations can be defined in many equivalent ways. For an isometry $\varphi$ of complex hyperbolic space $ \mathbf {H}_{\mathbb {C}}^2$ , define $l(\varphi )$ as follows:

\begin{equation*}l(\varphi ) := \inf \{ d_{ \mathbf {H}_{\mathbb {C}}^2}(x, \varphi (x)) \mid x \in \mathbf {H}_{\mathbb {C}}^2 \}.\end{equation*}

Let $\mathcal {S}$ be the set of free homotopy classes of essential unoriented simple closed curves on $S_{1,1}$ . We can define the following properties for representations in $\mathfrak {X}_c$ :

1. (BQ1) $\forall \gamma \in \mathcal {S}$ , $f(\textrm {tr }\rho (\gamma )) \gt 0$ (equivalently $\rho (\gamma )$ is loxodromic), where

   \begin{equation*}f(t)=|t|^4 - 8 \Re (t^3)+18 |t|^2 - 27;\end{equation*}

2. (BQ2) $\#\{\gamma \in \mathcal {S} \mid |\textrm {tr }\rho (\gamma )| \leqslant M(c) \} \lt \infty$ , where $M(c)\gt 0$ is defined in Section 3.4;

3. (BQ2’) $ \forall K\gt 0$ , $\; \#\{\gamma \in \mathcal {S} \mid |\textrm {tr }\rho (\gamma )| \leqslant K\} \lt \infty .$

4. (BQ3) there exists $k, m \gt 0$ such that $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ for all $\gamma \in \mathcal {S}$ ;

5. (BQ4) $\forall K\geqslant M(c)$ , the attracting subgraph $T_\rho (K)$ (defined in Section 3.5) is finite.

In Section 4.3 we prove the following result.

Theorem B. Let $[\rho ]\in \mathfrak {X}_c$ . The following are equivalent:

1. (i) the representation $\rho$ satisfies $(BQ1)$ and $(BQ2)$ ;

2. (ii) the representation $\rho$ satisfies $(BQ1)$ and $(BQ2')$ ;

3. (iii) the representation $\rho$ satisfies $(BQ3)$ ;

4. (iv) the representation $\rho$ satisfies $(BQ4)$ .

So the set of Bowditch representations $\mathfrak {X}_{BQ}$ referred to in TheoremA is the union over $c\in \mathbb {C}$ of the Bowditch representations in $\mathfrak {X}_c$ defined by any of the equivalent conditions in TheoremB. A corollary of the above theorem, given the work of [Sch22], is that our set $\mathfrak {X}_{BQ}$ coincides with the set of ‘primitive-stable representations’ defined in [KK20].

We also prove the somewhat surprising result that the topology of the character variety $\mathfrak {X}$ is non-trivial. In particular, we prove the following.

Theorem C. The $\mathrm {SU}(2,1)$ -character variety of $F_2$ $\mathfrak {X}$ strong deformation retracts onto $S^1\times S^1$ . In particular, it is not simply connected.

This is surprising because the $\mathrm {SU}(3)$ -character variety of $F_2$ satisfies the same real algebraic equations and has the same (real) dimension, but it is not homeomorphic to $\mathfrak {X}$ , since the $\mathrm {SU}(3)$ -character variety of $F_2$ is homeomorphic to an $8$ -sphere by Florentino and Lawton [FL09].

1.1 Proof strategy

We now give an overview of the proof of TheoremA and TheoremB. The proof can be organized into five main steps:

1. (0) description of the combinatorial setup: $\mathcal {C}, \mathcal {T},\mathcal {E}$ ;

2. (1) orientation on $\mathcal {E}$ , fork lemma and connectivity of the regions $\Sigma$ ;

3. (2) growth of traces for neighbors around a region and for escaping rays, and definition of an attracting subgraph of $\mathcal {E}$ ;

4. (3) Fibonacci growth and equivalent characterizations of Bowditch representations;

5. (4) proof that the set $\mathfrak {X}_{BQ}$ strictly contains convex cocompact representations.

1. Step (0): We start with a discussion of the combinatorial setup needed for our study. Given the one-hole torus $S:=S_{1,1}$ , we define three simplicial complexes associated to $S$ : the curve complex $\mathcal {C}$ , its dual $\mathcal {T}$ and the edge graph $\mathcal {E}$ . The curve complex $\mathcal {C}$ is the two-dimensional simplicial complex, where $k$ -simplices are given by $k+1$ distinct (free homotopy classes of) simple closed curves in $S$ that pairwise intersect once. Its simplicial dual $\mathcal {T}$ is a countably infinite tri-valent simplicial tree properly embedded in $\mathbf {H}^2$ defined by the property that the set $\mathcal {T}^{(k)}$ of $k$ -simplices in $\mathcal {T}$ satisfies $\mathcal {T}^{(k)} = \mathcal {C}^{(2-k)}$ for $k=0,1,2$ . Finally, the edge graph $\mathcal {E}$ has vertices corresponding to edges of $\mathcal {T}$ , and edges corresponding to adjacency, so one can see that $\mathcal {E}$ is a ‘trivalent tree of triangles’ in the sense that it is a graph made of triangles (one for each triangle $T$ in $\mathcal {C}^{(2)}$ or, equivalently, one for each vertex of $\mathcal T$ ). See Figure 1.

2. Step (1): For the first step, we show how any representation in $\mathfrak {X}_c$ induces an orientation on the edges of $\mathcal {E}$ as follows. An edge $(X, Y, Z; T, T) \in \mathcal {E}$ indicates that the edge is in the triangle $(X, Y, Z) \in \mathcal {C}^2$ and is between the triangles $(X, Y, T), (X, Z, T') \in \mathcal {C}^2$ which are adjacent to $(X, Y, Z)$ , see Figure 2. If $t = |\mathrm {tr} (\rho (T)) | \gt |\mathrm {tr} (\rho (T'))| = t'$ , then the oriented edge points towards $T'$ . In the case where there is an equality, one can choose the orientation arbitrarily (and the choice will not affect the result).

   We have a $4$ -coloring of the edges of $\mathcal {E}$ such that at each vertex there are only two triangles of two different colors meeting. We say that a vertex $v \in \mathcal {E}^{(0)}$ is a fork if there are (at least) two arrows of different colors pointing away from $v$ . The Fork Lemma (Lemma3.4) shows that if a vertex $v = (X, Y) \in \mathcal {E}^{(0)}$ is a fork, then $\min \{|x|, |y| \} \leqslant \max \{6,\sqrt {|\Re (c)|}\}$ for $x=\textrm {tr }(\rho (X))$ and $y=\textrm {tr }(\rho (Y))$ . We use this result to show, for $[\rho ] \in \mathfrak {X}_c$ , the set $\Omega _\rho (K)$ of regions with traces smaller than $K \geqslant \max \{6,\sqrt {|\Re (c)|}\}$ is connected. The proof is by contradiction and considers the distance between two connected components. The Fork Lemma is used to reach the conclusion when such a distance is greater than $2$ since in that case a fork will appear.

3. Step (2): For the second step we first study the behavior of the values of the (norm of the) traces of regions that are all neighbors of a central region $X$ . Given $[\rho ] \in \mathfrak {X}_c$ we see that if we denote by $(X_n)_{n \in \mathbb {Z}}$ the regions around $X$ , then the behavior of the sequence $(u_n)_{n\in \mathbb {N}}$ defined by $u_n = \mathrm {tr} (\rho (X_n))$ depends on the value of $f(x)$ where $x=\textrm {tr }(\rho (X))$ and $f(x) = \mathrm {Res} (\chi _x, \chi _x')$ . In fact if $f(x)=0$ , then $(u_n)_{n\in \mathbb {N}}$ grows at most quadratically, while if $f(x) \lt 0$ , then $(u_n)_{n\in \mathbb {N}}$ remains bounded. Finally, in Lemma3.8 we show that there exists a constant $D = D(c)$ , such that if the sequence $(u_n)_{n\in \mathbb {N}}$ defined above with $f(x) \gt 0$ does not grow exponentially in both directions, then there are infinitely many terms of the sequence such that $|u_n| \lt D$ . We then consider escaping rays, that is infinite geodesic rays where each edge $e_n$ is directed from $v_n$ to $v_{n+1}$ . In Lemma3.11 we show that if $\{ e_n \}_{n\in \mathbb {N}}$ is an escaping ray, then there are two cases:

   1. (1) there exists a region $\alpha$ such that the ray is eventually contained in the boundary of $\alpha$ , such that $f(x) \leqslant 0$ with $x = \mathrm {tr} (\rho (\alpha ))$ ; or

   2. (2) the ray meets infinitely many distinct regions with trace smaller than $M(c)$ , where $M(c):=\max \{6, \sqrt {|\Re (c)|}, D(c)\}$ .

   Using the description of the growth of the traces for regions around a fixed central trace, we define, for every representation $[\rho ] \in \mathfrak {X}_c$ and for every $K\gt M(c)$ , a connected attracting subgraph $T_\rho (K)$ of $\mathcal {E}$ . (Recall that $\mathcal {E}$ is a ‘trivalent tree of triangles’, so we want to define $T_\rho (K)$ so that it is a union of ‘triangles’.) The connectivity of $T_\rho (K)$ comes from the connectivity of $\Omega _\rho (K)$ , while the fact that $T_\rho (K)$ is attractive comes from the definition of $T_\rho (K)$ . We then show that for representations in $\mathfrak {X}_{BQ}$ , that is representations satisfying conditions (BQ1) and (BQ2), such a graph $T_\rho (K)$ is finite for all $K\gt M(c)$ .

4. Step (3): In the penultimate step, given a vertex $v \in \mathcal {E}^{(0)}$ , we define a Fibonacci function $F_v : \Omega \rightarrow \mathbb {N}$ . We then show that this function has the following property: if $\{\alpha, \beta \}$ is a set of free generators for $F_2$ corresponding to regions $X$ and $Y$ such that $v =(X, Y) \in \mathcal {E}^{(0)}$ , then for any element $\gamma \in \Omega$ the Fibonacci function corresponds to the word length of $\gamma$ with respect to $\{\alpha, \beta \}$ , that is $F_v(\gamma ) = W(\gamma )$ . Note that we only care about the asymptotic growth of this function, as you will see, so the choice of $v$ will not affect our results. We then define what it means for a function $g \colon \thinspace \Omega \rightarrow [0, \infty )$ to have Fibonacci growth, i.e. there exist constants $\kappa _1, \kappa _2 \gt 0$ such that

   \begin{equation*}\kappa _1 F_v (X) \leqslant g(X) \leqslant \kappa _2 F_v (X)\end{equation*}
   for all $X \in \Omega$ . If only the lower (respectively, upper) bound is satisfied, the function $g$ is said to have a lower (respectively, upper) Fibonacci growth. Given any $[\rho ] \in \mathfrak {X}$ , we denote by $\phi _\rho \colon \thinspace \Omega \rightarrow \mathbb {C}$ the function $\phi _\rho (X) := \log | \mathrm {tr} (\rho (X)) |$ . We consider the function $\phi _\rho ^+ := \max \{ \phi _\rho, 0 \}$ . We then show that this function always has an upper Fibonacci growth, but for representations in $[\rho ] \in \mathfrak {X}_{BQ}$ we show that the function has upper and lower Fibonacci growth.

   We use this result to show that we can characterize representations in $\mathfrak {X}_{BQ}$ in terms of the attracting graph $T_\rho (K)$ as follows: $[\rho ]$ is in $(\mathfrak {X}_c)_{BQ}$ if and only if the subgraph $T_\rho (K)$ is finite for all $K\gt M(c)$ , which proves TheoremB. This result is crucial to show that the set $\mathfrak {X}_{BQ}$ is open and the mapping class group acts on it properly discontinuously, and hence to prove TheoremA.

5. Step (4): In order to show that the set $\mathfrak {X}_{BQ}$ strictly contains convex cocompact representations, we use representations described by Will [Wil07, Wil12]: a family of discrete, faithful and type-preserving representations of the once-punctured torus in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ . These representations will take non-peripheral, non-trivial simple closed curves to loxodromic elements and will map the commutator to a parabolic unipotent element. In order to see that these representations are in $\mathfrak {X}_{BQ}$ , we use a characterization of both sets in term of the growth of $|l (\rho (\gamma ))|$ : the existence of $k, m \gt 0$ such that $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ for all $\gamma \in F_2$ corresponding to non-peripheral, non-trivial simple closed curves in $S_{1,1}$ . On the other hand, since the (peripheral) simple closed curve corresponding to the commutator is mapped to a unipotent element, we can see these representations are not convex cocompact. There are also other examples of representations that are in $\mathfrak {X}_{BQ} \setminus \mathfrak {X}_{CC}$ and such that the peripheral element is mapped to a parabolic non-unipotent element, see Falbel and Parker [FP03], Falbel and Koseleff [FK02b, FK02a] or Parker and Gusevskii [GP03, GP00]. See Will [Wil16] for a discussion of these references.

Figure 1. The complexes $\mathcal {C}$ (in gray) and $\mathcal {T}$ (in black), and the labeling of the connected components of $\mathbf {H}^2 \setminus \mathcal {T}$ once we fix the generators of $F_2 = \langle \alpha, \beta \rangle$ .

Figure 2. Oriented edge $(X, Y, Z; T \longrightarrow T')$ . If you forget the orientation, the same red edge can be denoted $(X; Y, Z)$ .

1.2 General setting

Let $\Gamma$ be a finitely presentable group with the discrete topology and let $G$ be a Lie group. The set $\mathrm {Hom}(\Gamma, G)$ of homomorphisms from $\Gamma$ to $G$ is a topological space by giving it the compact-open topology. The group $G$ acts on $\mathrm {Hom}(\Gamma, G)$ by conjugation, that is, $g\cdot \rho := \iota _g\circ \rho$ where $\iota _g\in \mathrm {Inn}(G)$ . Let $\mathrm {Orb}_G(\rho )$ be the $G$ -orbit of $\rho$ . Define the subspace of polystable homomorphisms

\begin{equation*}\mathrm {Hom}(\Gamma, G)^*:=\{\rho \in \mathrm {Hom}(\Gamma, G)\ |\ \mathrm {Orb}_G(\rho )=\overline {\mathrm {Orb}_G(\rho )}\},\end{equation*}

and define the polystable quotient space

\begin{equation*}\mathfrak {X}(\Gamma, G):=\mathrm {Hom}(\Gamma, G)^*/G.\end{equation*}

This quotient is called the $G$ -character variety of $\Gamma$ for historical reasons, even though in this generality it may be neither a variety, see Casimiro et al. [CFLO16], nor correspond to characters, see Lawton and Sikora [LS17].

Let $\mathcal {P}$ be a collection of properties such that if $\rho \in \mathrm {Hom}(\Gamma, G)$ satisfies $\mathcal {P}$ , then $\iota _g\circ \rho$ also satisfies $\mathcal {P}$ for all $g\in G$ . Then $G$ acts on the subspace

\begin{equation*}\mathrm {Hom}_{\mathcal {P}}(\Gamma, G):=\{\rho \in \mathrm {Hom}(\Gamma, G)\ |\ \rho \ \mathrm { satisfies }\ \mathcal {P}\}\end{equation*}

by conjugation. The polystable quotient of $\mathrm {Hom}_{\mathcal {P}}(\Gamma, G)$ by $G$ , denoted by $\mathfrak {X}_{\mathcal {P}}(\Gamma, G)$ , is called the $\mathcal {P}$ -relative $G$ -character variety of $\Gamma$ . If $\mathcal {P}$ is vacuous, one gets back $\mathfrak {X}(\Gamma, G).$

Let $\mathrm {Aut}_{\mathcal {P}}(\Gamma )$ denote the subgroup of $\mathrm {Aut}(\Gamma )$ that preserves $\mathrm {Hom}_{\mathcal {P}}(\Gamma, G)$ . Then $\mathrm {Inn}(\Gamma )\subset \mathrm {Aut}_{\mathcal {P}}(\Gamma )$ and so $\mathrm {Out}_{\mathcal {P}}(\Gamma ):=\mathrm {Aut}_{\mathcal {P}}(\Gamma )/\mathrm {Inn}(\Gamma )$ acts on $\mathfrak {X}_{\mathcal {P}}(\Gamma, G)$ by $[\alpha ]\cdot [\rho ]=[\rho \circ \alpha ^{-1}]$ . Again, if $\mathcal {P}=\emptyset$ , one obtains an action of $\mathrm {Out}(\Gamma )$ on $\mathfrak {X}(\Gamma, G)$ .

One of the main theorems in Rapinchuk [Rap15] says that, for any complex affine variety $V$ , there exists a finitely generated group $\Gamma$ and a complex reductive group $G$ , so that $\mathfrak {X}(\Gamma, G)-\{\chi _0\}$ and $V$ are isomorphic where $\chi _0$ is the point in the character variety corresponding to the trivial representation. Let $\mathcal {F}$ be the collection of finitely generated groups and $\mathcal {G}$ the collection of complex reductive affine algebraic groups. Then, since the $G$ -character variety of the trivial group $\Gamma =\{e\}$ is a point for any $G$ , the set $\{\mathfrak {X}(\Gamma, G)\ |\ \Gamma \in \mathcal {F},\ G\in \mathcal {G}\}$ generates the Grothendieck ring of affine varieties. By Bumagin and Wise [BW05] all countable groups arise as $\mathrm {Out}(\Gamma )$ as we vary finitely generated $\Gamma$ . So we have a very rich class of natural dynamical systems $(\mathfrak {X}(\Gamma, G),\mathrm {Out}(\Gamma ))$ which are interesting only when both $\mathfrak {X}(\Gamma, G)$ and $\mathrm {Out}(\Gamma )$ are non-trivial.

1.3 Some history

A classical example to consider is when $G=\mathrm {U}(1)$ and $\Gamma =\Gamma _g$ is the fundamental group of a closed orientable surface $S_g$ of genus $g$ . Then $\mathfrak {X}(\Gamma _g, G)=\mathrm {Hom}(\Gamma _g, \mathrm {U}(1))\cong \mathrm {U}(1)^{2g}$ which is identified with the Jacobian of $S_g$ , parametrizing topologically trivial holomorphic line bundles over a Riemannian surface diffeomorphic to $S_g$ . The action of $\mathrm {Out}(\Gamma _g)$ factors through $\mathrm {GL}(2g,\mathbb {Z})$ acting on a product of circles, and is classically ergodic. The Dehn–Nielsen–Baer Theorem [FM11] states that for such surfaces the (extended) mapping class group of isotopy classes of all homeomorphisms of $S_g$ is isomorphic to $\mathrm {Out}(\Gamma _g)$ .

The above example generalizes to arbitrary compact Lie groups $K$ as follows. The smooth locus of the character variety $\mathfrak {X}(\Gamma _g,K)$ is symplectic, as proven by Goldman [Gol84], and the associated invariant measure is finite, as shown by Huebschmann [Hue95]. With respect to this measure, $\mathrm {Out}(\Gamma _g)$ acts ergodically on $\mathfrak {X}(\Gamma _g,K)$ when $g\geqslant 2$ , see Goldman [Gol97] and Pickrell and Xia [PX02]. For non-orientable surfaces $N$ with $\chi (N)\leqslant -2$ and $G=\mathrm {SU}(2)$ , Palesi [Pal11] shows ergodicity with respect to a different invariant measure.

The situation for non-compact Lie groups $G$ is even more complicated, and we need two definitions to describe the situation. First, for a semisimple Lie group $G$ with no compact factors and a rank $r$ free group $F_r$ , a homomorphism $\rho : F_r \longrightarrow G$ is said to be primitive stable if any bi-infinite geodesic in the Cayley graph of $F_r$ defined by a primitive element (a member of free generating set) is mapped under the orbit map to a uniformly Morse quasigeodesic in the symmetric space $G/K$ where $K$ is a maximal compact in $G$ . Second, if $\rho$ is injective, $\rho (F_r)$ is discrete, and $\rho (F_r)$ preserves and acts cocompactly on some non-empty convex subset of $G/K$ then $\rho$ is convex cocompact.

In the case of $G=\mathrm {SL}(2,\mathbb {C})$ , Minsky [Min13] showed the set $\mathfrak {X}_{PS}(F_{n},G)$ of primitive stable representations is a domain of discontinuity for the action of $\mathrm {Out}(F_r)$ on $\mathfrak {X}(F_{r},G)$ . Minsky also proved that $\mathfrak {X}_{PS}(F_{n},G)$ strictly contains the set $\mathfrak {X}_{CC}(F_{r},G)$ of convex cocompact representations that is also a domain of discontinuity for the action of $\mathrm {Out}(F_r)$ on $\mathfrak {X}(F_{r},G)$ . When $\Gamma = F_2$ is the fundamental group of the one $1$ -holed torus $S_{1,1}$ , Bowditch [Bow98], in the case of quasi-Fuchsian representations, and more generally Tan, Wong and Zhang [TWZ08], showed that there is a domain of discontinuity $\mathfrak {X}_{BQ}(F_{2},G)$ for the action of the mapping class group of $S_{1,1,}$ on the corresponding character variety $\mathfrak {X}(F_{2},G)$ . The set $\mathfrak {X}_{BQ}(F_{2},G)$ is the set of Bowditch representations where the comparable conditions (for $G=\mathrm {SL}(2,\mathbb {C})$ ) from TheoremB hold true (although [Bow98, TWZ08] do not show $\mathfrak {X}_{BQ}(F_{2},G)$ strictly contains the convex cocompact representations). Later, Lee and Xu [LX20] and Series [Ser19, Ser20] proved $\mathfrak {X}_{PS}(F_{2},G)=\mathfrak {X}_{BQ}(F_{2},G)$ . Schlich [Sch22] generalized this result to cases where $G$ is the group of isometries of a $\delta$ -hyperbolic space. We will discuss her results below in more detail, since they are connected with the results we present.

Following the work of Bowditch, there have been other cases where interesting domains of discontinuity have been described for relative $\mathrm {SL}(2,\mathbb {C})$ -character varieties of surface groups. For example, you can see the description of a domain of discontinuity, the Bowditch set $\mathfrak {X}_{BQ}(\pi _1(S), G)$ , for the action of the mapping class group $\mathrm {Mod}(S)$ on the variety $\mathfrak {X}(\pi _1(S), G)$ , by Maloni, Palesi and Tan [MPT15] when $S$ is four $4$ -holed sphere and by Maloni and Palesi [MP20] when $S$ is the three $3$ -holed projective plane. In the case when $\Gamma$ is the fundamental group of a hyperbolic $3$ -manifold there has also been significant work finding interesting domains of discontinuity as described in this survey of Canary [Can13]. When $G$ is a semisimple group of higher rank and without compact factors, Kim and Kim [KK20] generalized Minsky’s work on primitive-stable representations and showed that they form an open domain of discontinuity for the action of $\mathrm {Out}(\Gamma )$ on $\mathfrak {X}(\Gamma, G)$ when $\Gamma$ is a free group of rank at least $2$ or the fundamental group of a closed surface of genus $g\geqslant 2$ .

Motivated by Lubotzky [Lub11], Minsky [Min13] and Gelander and Minsky [GM13] discussed primitive-stable and redundant representations in the case of free groups and $G=\mathrm {SL}(2,\mathbb {C})$ . Inspired by their work, one can imagine a decomposition of $\mathfrak {X}(\Gamma, G)$ into an open set where the action of $\mathrm {Out}(\Gamma )$ is properly discontinuous and a closed set where the action is chaotic. The only example fully understood where $G$ is not compact that exemplifies such a dynamical dichotomy is when $\Gamma$ is a free group of rank 2 and $G = \mathrm {SL}(2,\mathbb {R})$ , see Goldman [Gol03], Goldman et al. [GMST19] and Marché and Wolff [MW16, MW19].

As summarized in Goldman [Gol06], when $G$ is compact, (relative) character varieties generally exhibit non-trivial homotopy, and the action of the mapping class group exhibits non-trivial chaotic dynamics. In contrast, when $G$ is non-compact, then $\mathfrak {X}(\Gamma _g,G)$ contains open sets (like the Teichmüller space or more generally Hitchin components) which are contractible and admit properly discontinuous actions. Philosophically, the action of $\mathrm {Out}(\Gamma )$ on (relative) character varieties exhibits properties of these two extremes.

1.4 Relation with work of Schlich

Schlich [Sch22] considers representations from the fundamental group $\Gamma = \pi _1(S)$ of the one $1$ -holed torus $S=S_{1,1}$ into the isometry group $G$ of a geodesic and proper $\delta$ -hyperbolic space. Since ${\rm PSU}(2,1) =\mathrm {Isom}(\mathbf {H}^2_{\mathbb {C}})$ is the isometry group of the complex hyperbolic plane (see Section 2.3), her result applies to the setting we study. For the character variety $\mathfrak {X} = \mathfrak {X}(F_2,\mathrm {SU}(2,1))$ , Schlich defines two subsets of the character variety: the Bowditch set (using Definition (BQ3)) and the set of primitive-stable representations (generalizing Minsky’s definition to this setting). One of the main results of her thesis and paper shows that the two sets coincide. Since it is not hard to see that the set of primitive-stable representations is a domain of discontinuity for the action of $\mathrm {Out}(F_2)$ on $\mathfrak {X}$ , she also proves, as a corollary of her result, that the set of Bowditch representations also has that property. In her thesis, she also considers the case of $S = S_{0,4}$ , a four-holed sphere, and proves that in that setting the set of ‘simple-stable’ representations coincides with the set of Bowditch’s representations [Sch23].

The main difference between our work and Schlich’s work is in the way Bowditch representations are defined. This also differentiates the challenging aspects of the work. We define our Bowditch set explicitly in terms of the character variety using the symmetry and structure understood from [Law06, Law07]. Our approach is similar to that of [Bow98] and its generalizations [TWZ08, MPT15, MP20], but adapted to representations into $\mathrm {SU}(2,1)$ where the character variety is significantly more complicated. In particular, our definition is well-adapted to computer implementations and testing since it is a definition dependent on only a finite number of conditions. Schlich’s work generalizes to isometry groups of Gromov hyperbolic spaces the results obtained for $\mathrm {SL}(2,\mathbb {C})$ by Series [Ser19, Ser20]and Lee and Xu [LX20], that for the free group $F_2$ the notions of Bowditch representations and primitive stable representations are equal. In fact, her result gives a new independent proof of this result for $\mathrm {SL}(2,\mathbb {C})$ .

2. Background

In this section we fix the notation which we will use in the rest of the paper and give some important definitions.

2.1 Combinatorics of simple closed curves on the one-holed torus $S$

Let $S= S_{1,1}$ be the one-holed torus. Its fundamental group $\Gamma = \pi _1(S)$ is isomorphic to the non-abelian rank $2$ free group $F_2:=\langle \alpha, \beta \rangle$ . Abusing notation, we will let $\alpha$ and $\beta$ represent the (free homotopy class of) loops associated to the two generators so that the peripheral curve $\gamma$ going around the boundary component of $S$ is oriented to be homotopic to $[\alpha, \beta ]:=\alpha \beta \alpha ^{-1}\beta ^{-1}$ .

Let $\mathcal {S}$ be the set of free homotopy classes of essential unoriented simple closed curves on $S$ . Recall that a curve is essential if it is non-trivial and non-peripheral, or equivalently if it does not bound a disk, or a one-holed disk. We will generally omit the word essential from now on. We can identify $\mathcal {S}$ to a well-defined subset of $\Gamma /\!\sim$ , where the equivalence relation $\sim$ on $\Gamma$ is defined as follows: $g \sim h$ if and only if $g$ is conjugate to $h$ or $h^{-1}$ .

2.2 Various simplicial complexes associated to $S$

Definition 2.1. The curve complex $\mathcal {C}$ of the one-holed torus $S$ is defined by setting $k$ -simplices to be subsets of $k+1$ distinct (free homotopy classes of) essential simple closed curves in $S$ that pairwise intersect once (up to homotopy).

With that definition $\mathcal {C}$ is a two- $2$ dimensional simplicial complex isomorphic to the Farey complex [Min96].

Definition 2.2. Let $\mathcal {T}$ be the simplicial complex defined by letting the set of $k$ -simplices in $\mathcal {T}$ , denoted by $\mathcal {T}^{(k)}$ , be the set $\mathcal {T}^{(k)} = \mathcal {C}^{(2-k)}$ . The complex $\mathcal {T}$ is called the simplicial dual of $\mathcal {C}$ and is a countably infinite simplicial tree properly embedded in the hyperbolic $2$ -space $\mathbf {H}^2$ whose vertices all have degree $3$ .

Figure 1 illustrates $\mathcal {C}$ and $\mathcal {T}$ . Note that $\mathcal {T}$ can also be identified with the graph of ideal triangulations of $S$ , where edges corresponds to edge-flips. Since we will not need this identification in the rest of the paper, we will not discuss the details.

The set $\Omega :=\pi _0(\mathbf {H}^2- \mathcal {T})$ is in one-to-one correspondence with the set of vertices of $\mathcal {C}$ and hence with the set $\mathcal {S}$ ; that is,

\begin{equation*}\Omega \cong \mathcal {C}^{(0)} \cong \mathcal {S}.\end{equation*}

Given a system of generators for $F_2$ corresponding to (free homotopy classes) of simple closed curves in $S$ , we obtain a labeling of regions in $\Omega$ . For example, with $F_2 = \langle \alpha, \beta \rangle$ we obtain the labeling in Figure 1. As you can see, the labeling is defined from the starting central edge $e = \{\alpha, \beta \}$ corresponding to the intersection of the regions in $\Omega$ labeled $\alpha$ and $\beta$ , and is defined, for a certain region $X$ , as the product of the two regions adjacent to $X$ but closer to the initial edge $e$ . Note that when doing this product the region $\beta$ needs to be considered as $\beta$ on the ‘left-hand side’ of the edge $e$ and $\beta ^{-1}$ on the ‘right-hand side’ of the edge $e$ .

Definition 2.3. The edge graph $\mathcal {E}$ is the graph whose vertices correspond to edges of $\mathcal {T}$ , and whose edges correspond to adjacency. More precisely, if we denote by $\mathcal {E}^{(k)}$ the set of $k$ -simplices, we have that $\mathcal {E}^{(0)} = \mathcal {T}^{(1)} = \mathcal {C}^{(1)}$ and the edges of $\mathcal {E}$ correspond to edges that are the boundary of the same triangle in $\mathcal {C}^{(2)}$ .

Note that $\mathcal {E}$ is quasi-isometric to $\mathcal {T}$ . In fact, one can see that $\mathcal {E}$ is a ‘trivalent tree of triangles’ in the sense that it is a graph made of triangles (one for each triangle $T$ in $\mathcal {C}^{(2)}$ or, equivalently, one for each vertex of $\mathcal T$ ).

Since each edge of $\mathcal {T}$ is uniquely determined by two (free homotopy classes of) simple closed curves intersecting once, we will denote vertices of $\mathcal E$ as $v = (\alpha, \beta ) \in \mathcal {E}^{(0)}$ . It will be convenient for some proofs to denote such a vertex $v$ as $(\alpha, \beta ;\gamma, \delta )$ , where $\gamma, \delta$ are regions adjacent to $(\alpha, \beta )$ considered as an element in $\mathcal {T}^{(1)}$ . Each edge of $\mathcal E$ is uniquely determined by the choice of a triangle $T$ in $\mathcal {C}^{(2)}$ plus the choice of one of its vertices, so we will denote edges of $\mathcal E$ as $e = (\alpha ; \beta, \gamma ) \in \mathcal {E}^{(1)}$ . We will also have to denote oriented edges, so in that case we will use the notation $(\alpha, \beta, \gamma ; \delta, \delta ')$ . See Figure 2.

Additionally, we will have to color the edges of $\mathcal {E}$ with four colors so that the edges of each triangle have the same color and each triangle is adjacent to triangles with distinct coloring. This coloring can be extended to color the vertices of $\mathcal {C}$ (or equivalently to the connected components of $\mathbf {H}^2 \setminus \mathcal {T}$ ). This is possible since each such region is bounded by triangles colored with three distinct colors. Therefore, the region will be colored in the color that does not appear in the triangles going around the region. See Figure 3.

Figure 3. The complex $\mathcal {E}$ with the coloring of its edges and regions of $\mathbf {H}^2 \setminus \mathcal {T}$ .

2.3 Complex hyperbolic geometry $\mathbf {H}^2_{\mathbb {C}}$

Let $\mathbb {C}^{n,1}\cong \mathbb {C}^n\times \mathbb {C}$ be the $(n+1)$ -dimensional complex vector space with Hermitian pairing defined for vectors $(\mathbf {x},x_{n+1})\in \mathbb {C}^{n+1}$ by

\begin{equation*}\langle (\mathbf {x},x_{n+1}),(\mathbf {y},y_{n+1})\rangle :=\left (\sum _{i=1}^{n}x_i\overline {y_i}\right )-x_{n+1}\overline {y_{n+1}}.\end{equation*}

We say a vector $\mathbf {v}$ is negative if and only if $\langle \mathbf {v},\mathbf {v}\rangle \lt 0$ . Let $\mathbf {P}(\mathbb {C}^{n,1}):=(\mathbb {C}^{n,1}-\{\mathbf {0}\})/\mathbb {C}^*$ be the space of lines in $\mathbb {C}^{n,1}$ through the origin. Then the complex hyperbolic space of dimension $n$ is defined to be $\mathbf {H}_{\mathbb {C}}^n:=\{[\mathbf {v}]\in \mathbf {P}(\mathbb {C}^{n,1})\ |\ \langle \mathbf {v},\mathbf {v}\rangle \lt 0\}$ .

There is a natural copy of $\mathbb {C}^n$ in $\mathbf {P}(\mathbb {C}^{n,1})$ given by

\begin{equation*}\mathbb {A}:=\{[(\mathbf {v},v_{n+1})]\in \mathbf {P}(\mathbb {C}^{n,1})\ |\ v_{n+1}\not=0\}.\end{equation*}

The biholomorphism $\varphi :\mathbb {C}^n\longrightarrow \mathbb {A}$ is given by $\varphi (\mathbf {v})=[(\mathbf {v},1)]$ . We observe that $\mathbf {H}_{\mathbb {C}}^n\subset \mathbb {A}$ since if $v_{n+1}=0$ then $\langle \mathbf {v},\mathbf {v}\rangle \gt 0$ . Then restricting $\varphi ^{-1}$ to $\mathbf {H}_{\mathbb {C}}^n$ we see that $\mathbf {H}_{\mathbb {C}}^n$ is biholomorphic to the unit ball in $\mathbb {C}^n$ since $\langle (\mathbf {v},1),(\mathbf {v},1)\rangle \lt 0$ if and only if $\sum _{i=1}^{n}v_i\overline {v_i}\lt 1$ .

Consequently, the boundary of $\mathbf {H}_{\mathbb {C}}^n$ is homeomorphic to the unit sphere in $\mathbb {C}^n$ :

\begin{equation*}\partial \mathbf {H}_{\mathbb {C}}^n=\{[\mathbf {v}]\in \mathbf {P}(\mathbb {C}^{n,1})\ |\ \langle \mathbf {v},\mathbf {v}\rangle=0\}\cong S^{2n-1}.\end{equation*}

We let $\mathrm {U}(n,1)$ be the subgroup of $\mathrm {GL}(n+1,\mathbb {C})$ consisting of matrices $A$ such that

\begin{equation*}\langle A(\mathbf {x},x_{n+1}),A(\mathbf {y},y_{n+1})\rangle =\langle (\mathbf {x},x_{n+1}),(\mathbf {y},y_{n+1})\rangle .\end{equation*}

The image of $\mathrm {U}(n,1)$ in $\mathrm {PGL}(n+1,\mathbb {C})$ , denoted by $\mathrm {PU}(n,1)$ , is the group of biholomorphisms of $\mathbf {H}_{\mathbb {C}}^n$ (see [Gol99]).

Here $\mathrm {PU}(n,1)$ acts transitively on $\mathbf {H}_{\mathbb {C}}^n$ with stabilizer isomorphic to $\mathrm {U}(n)$ . Thus, we see that the complex hyperbolic space is a homogeneous space

\begin{equation*}\mathbf {H}_{\mathbb {C}}^n\cong \mathrm {PU}(n,1)/\mathrm {U}(n)\cong \mathrm {SU}(n,1)/\mathrm {S}(\mathrm {U}(n)\times \mathrm {U}(1)).\end{equation*}

2.4 $\mathrm {SL}(3,\mathbb {C})$ and $\mathrm {SU}(2,1)$

Let $\mathrm {SL}(3,\mathbb {C})$ be the Lie group of $3\times 3$ complex matrices of unit determinant, which is also a complex affine variety since it is the zero locus of the determinant polynomial minus $1$ .

We review a few facts about two real forms of $\mathrm {SL}(3,\mathbb {C})$ . First,

\begin{equation*}\mathrm {SU}(3)=\{A\in \mathrm {SL}(3,\mathbb {C})\ |\ A^{-1}=\overline {A}^\dagger \},\end{equation*}

where $\dagger$ means ‘transpose’. Letting

\begin{equation*}J=\left (\begin{array}{c@{\quad}c@{\quad}c}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & -1\end{array}\right ),\end{equation*}

we secondly have

\begin{equation*}\mathrm {SU}(2,1)=\{A\in \mathrm {SL}(3,\mathbb {C})\ |\ A^{-1}=J\overline {A}^\dagger J\}.\end{equation*}

Consequently, $\mathrm {tr}(A^{-1})=\overline {\mathrm {tr}(A)}$ for any $A\in \mathrm {SU}(3)\cup \mathrm {SU}(2,1)$ .

The following lemma is well-known (see, for example, [Gol99, Exercise 3.1.1]), but we include a proof since it is an essential fact in our proof of TheoremC.

Lemma 2.4. Any maximal compact subgroup of $\mathrm {SU}(2,1)$ is isomorphic to $\mathrm {U}(2)$ . In addition, the center of $\mathrm {SU}(2,1)$ is isomorphic to $\mathbb {Z}_3$ .

Proof. In general, the group $\mathrm {U}(p)\times \mathrm {U}(q)$ is a maximal compact in $\mathrm {U}(p,q)$ and so $\mathrm {S}(\mathrm {U}(p)\times \mathrm {U}(q))$ is a maximal compact in $\mathrm {SU}(p,q)$ . In the case of $\mathrm {SU}(2,1)$ we have that $\mathrm {S}(\mathrm {U}(2)\times \mathrm {U}(1))=\{(A,\lambda )\in \mathrm {U}(2)\times \mathrm {U}(1)\ |\ \mathrm {Det}(A)\lambda=1\}$ . On the other hand, this group is isomorphic to $\mathrm {U}(2)$ simply by sending $(A,\lambda )\mapsto A$ . Since all maximal compact subgroups are isomorphic, the first result follows.

Now we compute the center. Suppose $Z\in \mathrm {SU}(2,1)$ is in the center $Z(\mathrm {SU}(2,1))\leqslant \mathrm {SU}(2,1)$ . Then for all $A\in \mathrm {SU}(2,1)$ we have $AZ=ZA$ . On the other hand, this equation, which is necessarily algebraic, holds on the Zariski closure of $\mathrm {SU}(2,1)$ , which is $\mathrm {SL}(3,\mathbb {C})$ . So $Z$ is a central matrix in $\mathrm {SL}(3,\mathbb {C})$ and so a scalar multiple of the identity. The only such matrices are $\{\omega I, \omega^2 I, I\}\cong \mathbb {Z}_3$ , where $\omega$ is a primitive cube root of unity.

An automorphism of the complex hyperbolic space $\mathbf {H}^2_{\mathbb {C}}\subset \mathbb {P}(\mathbb {C}^{2,1})$ , represented by an element in $ \mathrm {PSU}(2,1):=\mathrm {SU}(2,1)/Z(\mathrm {SU}(2,1))$ , lifts to an element of $\mathrm {SU}(2,1)$ , and the fixed points in $\mathbf {H}^2_{\mathbb {C}}\cup \partial \mathbf {H}^2_{\mathbb {C}}$ correspond to eigenvectors of this lift. For each such automorphism, there are three lifts corresponding to the cubic roots of unity $\zeta _3\subset S^1\subset \mathbb {C}$ . The following definition is taken from [Gol99, Section 6.2.1].

Definition 2.5. An element in $\mathrm {SU}(2,1)$ is called:

1. (i) elliptic if it has a fixed point in $\mathbf {H}^2_{\mathbb {C}}$ ;

2. (ii) parabolic if it has a unique fixed point on $\partial \mathbf {H}^2_{\mathbb {C}}$ ;

3. (iii) unipotent if it is parabolic and all the eigenvalues are equal;

4. (iv) loxodromic if it fixes a unique pair of points on $\partial \mathbf {H}^2_{\mathbb {C}}$ .

Given a matrix $A\in \textrm {SU} (2,1)$ such that $t = \mathrm {tr}(A)$ , its characteristic polynomial, whose zeros are the eigenvalues of $A$ , is the polynomial $\chi _A(x) = x^3 - t x^2 + \overline t x -1$ . The resultant of $\chi _A$ and its derivative tells us when $\chi _A$ has multiple roots, and hence when $A$ has an eigenvalue with multiplicity greater than $1$ . Let $f\colon \thinspace \mathbb {C} \longrightarrow \mathbb {R}$ be defined by

\begin{equation*}f(t) = \mathrm {Res} (\chi _A, \chi _A') = |t|^4 - 8 \Re (t^3)+18 |t|^2 - 27.\end{equation*}

This function vanishes on a deltoid centered at $0$ of radius $3$ , see Figure 4.

Figure 4. $f(t)=0$ .

Theorem 2.6 (Theorem 6.2.4, [Gol99]). The map $\mathrm {tr}\colon \thinspace \mathrm {SU}(2,1)\longrightarrow \mathbb {C}$ is surjective. If $A,B\in \mathrm {SU}(2,1)$ are loxodromic and $\mathrm {tr}(A)=\mathrm {tr}(B)$ , then $A$ and $B$ are conjugate.

In addition, for any $A\in \mathrm {SU}(2,1)$ with $t = \mathrm {tr}(A)$ we have:

1. (i) $A$ is elliptic with distinct eigenvalues if and only if $f(t)\lt 0$ ;

2. (ii) $A$ is loxodromic if and only if $f(t)\gt 0$ ;

3. (iii) $A$ is parabolic but not unipotent if and only if $A$ is not elliptic and $t\in f^{-1}(0)-3\zeta _3$ ;

4. (iv) $A$ is a complex-reflection if and only if $A$ is elliptic and $t\in f^{-1}(0)-3\zeta _3$ ;

5. (v) $A$ is an unipotent automorphism of $\mathbf {H}^2_{\mathbb {C}}$ or the identity map if and only if $t\in 3\zeta _3$ .

The following result is a corollary to the above theorem.

Corollary 2.7 ([Gol99]). Let $A\in \mathrm {SU}(2,1)$ with eigenvalues $\lambda _1,\lambda _2,\lambda _3$ , and let $t = \mathrm {tr}(A)$ . We have the following.

1. (i) If $f(t) \gt 0$ , then $A$ is loxodromic, and its eigenvalues are all distinct: one is inside, one is outside and one is on the unit circle.

2. (ii) If $f(t)=0$ , then $A$ has an eigenvalue of multiplicity at least $2$ , and all eigenvalues are on the unit circle.

3. (iii) If $f(t) \lt 0$ , then $A$ is regular elliptic, and has three distinct eigenvalue s satisfying $|\lambda _i | = 1$ .

2.4.1 $\mathrm {SL}(3,\mathbb {C})$ -character variety

Let $S_{1,1}$ be the one-holed torus. The fundamental group of $S_{1,1}$ is isomorphic to a rank $2$ free group

\begin{equation*}F_2:=\langle \alpha, \beta \rangle =\langle \alpha, \beta, \gamma \ |\ [\alpha, \beta ]=\gamma \rangle, \end{equation*}

where $\gamma$ corresponds to a loop around the boundary circle and $\alpha, \beta$ are loops of longitude and latitude in the torus, as we discussed in Section 2.2.

The set of group homomorphisms $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is in bijective correspondence with $\mathrm {SL}(3,\mathbb {C})\times \mathrm {SL}(3,\mathbb {C})$ by sending a homomorphism $\rho$ to $(\rho (\alpha ),\rho (\beta ))$ . Since products of smooth manifolds are smooth manifolds and products of affine varieties are affine varieties, we can give the structure of a complex affine algebraic variety (and a smooth manifold) to $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ .

The group $\mathrm {SL}(3,\mathbb {C})$ acts on $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ by conjugation, which becomes simultaneous conjugation under the identification with $\mathrm {SL}(3,\mathbb {C})\times \mathrm {SL}(3,\mathbb {C})$ . Precisely, for $g \in \mathrm {SL}(3,\mathbb {C})$ and $\rho \in \mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ the action is defined by $g\cdot \rho =g\rho g^{-1}$ , and for $(A,B)\in \mathrm {SL}(3,\mathbb {C})\times \mathrm {SL}(3,\mathbb {C})$ we have $g\cdot (A,B)=(gAg^{-1}, gBg^{-1})$ .

We are interested in the moduli space of such homomorphisms up to the equivalence defined by this action. The quotient $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))/\mathrm {SL}(3,\mathbb {C})$ does not satisfy the $T_1$ -separation axiom since some points are not closed (corresponding to non-closed orbits in $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ ). There is a natural way to make it satisfy the $T_1$ -separation axiom: remove the orbits that are not closed in $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ . Let $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))^*$ be the subspace of $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ that consists of homomorphisms with closed conjugation orbits. Then the polystable quotient

\begin{equation*}\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C})):=\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))^*/\mathrm {SL}(3,\mathbb {C}) \end{equation*}

satisfies the $T_1$ -separation axiom.

However, one can show that it is homeomorphic to the Geometric Invariant Theory (GIT) quotient $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))/\!\!/\mathrm {SL}(3,\mathbb {C})$ , where the latter is given the induced metric topology from being an affine variety (called the analytic or strong topology). See Florentino and Lawton [FL14, Theorem 2.1] for a proof. Thus, $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ has the structure of a stratified manifold (and so is a metric space). Moreover, $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is homotopic to the non-Hausdorff space $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))/\mathrm {SL}(3,\mathbb {C})$ just the same (see Florentino, Lawton and Ramras [FLR17, Proposition 3.4]).

To understand the affine variety structure on $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ , which approximates the non-Hausdorff space $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))/\mathrm {SL}(3,\mathbb {C})$ , we need to understand the ring of polynomial functions on it.

Let $\mathbf {X}=(x_{ij})_{1\leqslant i,j\leqslant 3}$ and $\mathbf {Y}=(y_{ij})_{1\leqslant i,j\leqslant 3}$ be two $3\times 3$ matrices of variables (called ‘generic matrices’). Then the ring of polynomials on $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is

\begin{equation*}\mathbb {C}[\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))]\cong \mathbb {C}[x_{ij},y_{ij}|1\leqslant i,j\leqslant 3]/\langle \mathrm {Det}(\mathbf {X})-1,\mathrm {Det}(\mathbf {Y})-1\rangle .\end{equation*}

Since $\mathrm {SL}(3,\mathbb {C})$ acts algebraically by conjugation on $\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))$ , there is an action on $\mathbb {C}[\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))]$ given by $(g\cdot f)(\rho )=f(g^{-1}\rho g)$ for $g\in \mathrm {SL}(3,\mathbb {C})$ . The ring of polynomials on $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is then

\begin{align*} \mathbb {C}[\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))] & :=\mathbb {C}[\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))]^{\mathrm {SL}(3,\mathbb {C})}\\ & := \{f\in \mathbb {C}[\mathrm {Hom}(F_2,\mathrm {SL}(3,\mathbb {C}))]\ |\ g\cdot f=f\text { for all } g\in \mathrm {SL}(3,\mathbb {C})\}. \end{align*}

One of the main theorems in Lawton [Law06, Law07] states that

\begin{equation*}\mathbb {C}[\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))]\cong \mathbb {C}[x,y,z,t,A,B,C,D,s]/\langle s^2-Ps+Q\rangle, \end{equation*}

where $P,Q\in \mathbb {C}[x,y,z,t,A,B,C,D]$ and the polynomial $s^2-Ps+Q$ is irreducible.

Consequently, we have the following.

Theorem 2.8 ([Law06, Law07]). $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is a hypersurface in $\mathbb {C}^9$ that is a branched double cover of $\mathbb {C}^8$ . The branching locus is exactly the representations that are fixed by transpose.

The explicit forms of $P$ and $Q$ are

\begin{equation*}P=-3 + t D+x A - t A y - D x B+y B+x A y B - A B z - x y C+z C \end{equation*}

and $Q=9 + t^3 - 6 t D+D^3 + x^3 - 6 x A+t D x A+A^3 + t x^2 y+3 t A y - 2 D^2 A y - t x A^2 y+D x y^2 + $ $ D A^2 y^2\; +\; y^3\; -\; x A y^3\; -\; 2 t^2 x B\;+\; 3 D x B\; -\; D x^2 A B+D A^2 B - 6 y B+t D y B - x^3 y B+x A y B+$ $ x^2 A^2 y B - A^3 y B - t A y^2 B+t x^2 B^2 + t A B^2 - D x y B^2 + x A y^2 B^2 + B^3 - x A B^3\; -\; 3 t x z+D^2 x z\,+ $ $ t A^2 z+t^2 y z -3 D y z - D x A y z\; +\; x^2 y^2 z\; +\; A y^2 z\; +\; x^2 B z\; +\; 3 A B z\; - x A^2 B z - t x y B z+D B^2 z - $ $ A y B^2 z+D A z^2\; -\; 2 x y z^2\; +\; t B z^2\; +\; z^3\; +\; D x^2 C\; +\; t^2 A C - 3 D A C\; +\; 3 x y C - x^2 A y C + A^2 y C + $ $ t y^2 C \; -\; 3 t B C\; +\; D^2 B C\; -\; t x A B C\; -\; D A y B C \; -\; x y^2 B C\; +\; x B^2 C\; +\; A^2 B^2 C - 6 z C+t D z C+$ $x A z C+y B z C+t x C^2 + D y C^2 - 2 A B C^2 + C^3.$

As functions of the matrix variables $\mathbf {X}$ and $\mathbf {Y}$ (up to the ideal $\langle \mathrm {Det}(\mathbf {X})-1,\mathrm {Det}(\mathbf {Y})-1\rangle$ ), the variables $x,y,z,t, A,B,C,D,s$ are as follows:

\begin{gather*}x=\mathrm {tr}(\mathbf {X}), \;\;y=\mathrm {tr}(\mathbf {Y}), \;\; z=\mathrm {tr}(\mathbf {X}\mathbf {Y}), \;\; t=\mathrm {tr}(\mathbf {X}\mathbf {Y}^{-1}),\\A=\mathrm {tr}(\mathbf {X}^{-1}), \;B=\mathrm {tr}(\mathbf {Y}^{-1}), \;C=\mathrm {tr}(\mathbf {X}^{-1}\mathbf {Y}^{-1}), \;D=\mathrm {tr}(\mathbf {X}^{-1}\mathbf {Y}), \text { and } s=\mathrm {tr}(\mathbf {X}\mathbf {Y}\mathbf {X}^{-1}\mathbf {Y}^{-1}).\end{gather*}

In these terms, the relations are

\begin{equation*}P=\mathrm {tr}(\mathbf {X}\mathbf {Y}\mathbf {X}^{-1}\mathbf {Y}^{-1})+\mathrm {tr}(\mathbf {Y}\mathbf {X}\mathbf {Y}^{-1}\mathbf {X}^{-1}) \end{equation*}

and

\begin{equation*}Q=\mathrm {tr}(\mathbf {X}\mathbf {Y}\mathbf {X}^{-1}\mathbf {Y}^{-1})\mathrm {tr}(\mathbf {Y}\mathbf {X}\mathbf {Y}^{-1}\mathbf {X}^{-1}).\end{equation*}

Remark 2.9. In Lawton [Law06, Law07] the polynomials $P,Q$ are shown to have $\mathrm {Out}(F_2)$ symmetry and in terms of that symmetry they can be written more compactly. See also Parker [Par12] for a ‘cyclically’ symmetric coordinate change.

As shown in Florentino and Lawton [FL09], the singular variety $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ strong deformation retracts to a topological $8$ -sphere $S^8$ . In fact, this $8$ -sphere is precisely the $\mathrm {SU}(3)$ -character variety of $F_2$ : $\mathfrak {X}(F_2,\mathrm {SU}(3)):=\mathrm {Hom}(F_2,\mathrm {SU}(3))/\mathrm {SU}(3).$

2.4.2 $\mathrm {SU}(2,1)$ -character variety

We are interested in the corresponding moduli space where $\mathrm {SU}(3)$ is replaced by $\mathrm {SU}(2,1)$ .

Given a word $w\in F_2=\langle \alpha, \beta \rangle$ , define $\mathrm {tr}_w\!:\!\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))\longrightarrow \mathbb {C}$ by $\mathrm {tr}_w([\rho ]):=\mathrm {tr}(\rho (w))$ . For representations taking values in $\mathrm {SU}(3)$ or $\mathrm {SU}(2,1)$ , we have that $\mathrm {tr}_{w^{-1}}=\overline {\mathrm {tr}_w}$ for any word $w\in F_2$ . Thus, the real coordinate ring of $\mathfrak {X}(F_2,\mathrm {SU}(3))$ or the polystable quotient $\mathfrak {X}(F_2,\mathrm {SU}(2,1)):=\mathrm {Hom}(F_2,\mathrm {SU}(2,1))^*/\mathrm {SU}(2,1)$ is generated by the real and imaginary parts of

\begin{equation*}\{\mathrm {tr}_\alpha, \mathrm {tr}_\beta, \mathrm {tr}_{\alpha \beta },\mathrm {tr}_{\alpha \beta ^{-1}},\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}}\},\end{equation*}

subject to the relations

(1) \begin{equation} \Re (\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}})=P/2 \end{equation}

and

(2) \begin{equation} \mathrm {Im}(\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}})=\pm \sqrt {Q-P^2/4}. \end{equation}

Note that both $P$ and $Q$ will be real polynomials, as $P$ is the sum

\begin{equation*}P=\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}}+\mathrm {tr}_{\beta \alpha \beta ^{-1}\alpha ^{-1}}=2\Re (\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}}),\end{equation*}

while $Q$ is the product

\begin{equation*}Q=\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}}\cdot \mathrm {tr}_{\beta \alpha \beta ^{-1}\alpha ^{-1}}=|\mathrm {tr}_{\alpha \beta \alpha ^{-1}\beta ^{-1}}|^2,\end{equation*}

and hence will simplify to real polynomials in the real and imaginary parts of $\mathrm {tr}_\alpha, $ $\mathrm {tr}_\beta, $ $\mathrm {tr}_{\alpha \beta }$ and $\mathrm {tr}_{\alpha \beta ^{-1}}$ .

If we write this out in terms of the variables $x,y,z,t$ above, we obtain for $P$ and $Q$ the following expressions:

\begin{equation*}P = |x|^2 |y|^2 + |x|^2 + |y|^2 + |z|^2 + |t|^2 - 2 \Re (xy\overline z ) - 2 \Re (\overline x y t ) - 3 \end{equation*}

and $Q=9 - 6 |x|^2 - 6 | y |^2 - 6 |z|^2 - 6 | t |^2 + | t |^2 | y |^2 +| x |^2 | y |^2+|y|^2 |z|^2 + |t|^2|z|^2 +|x|^2|z|^ 2+ |t|^2|x|^2$ $ +\;|x|^2 |y|^4 \;+\; | y |^2 | x |^4+2 \Re (z^3)\;+\;2 \Re (t^3)+2 \Re(x^3)\;+\; 2 \Re (y^3) \;+\; 2\Re (t x^2 y) - 6 \Re (t x z) + 2 \Re (t^2 y z)\; +$ $ 2 \Re (x^2 y^2 z) \; -\; 4 \Re (x y z^2)\; +\; 2 \Re (x y^2 \overline {t})\; -\; 6 \Re (y z \overline {t})\; +\; 2 \Re (x z \overline {t}^2)\; +\; 6 \Re (t y \overline {x}) - 2 | x |^2 \Re ( y^3)+2 \Re (y^2 z \overline {x})$ $ -\; 2 | y |^2 \Re (\overline {x} t y) \;-\;2 | x |^2 \Re ( y z \overline {t}) \;+\; 2 \Re (z^2 \overline {t}\overline {x}) \;-\; 4 \Re ( y \overline {t}^2 \overline {x}) \;-\; 2 | x |^2 \Re (t y \overline {x}) \;+\; 2 \Re (t z \overline {x}^2)\;\;+\;\;2 \Re (y^2 \overline {t} \overline {x}^2)$ $ -\;2 | y |^2 \Re (x^3) \;+\; 2 \Re (x^2 z \overline {y}) \;-\; 2 | y |^2 \Re (t x z)\;+\;2 \Re (t z^2 \overline {y}) \;+\;6 \Re ( z \overline {x} \overline {y}) \;-\; 2 | x |^2 \Re (z \overline {x} \overline {y}) \;\;+\;\; 2 \Re (z \overline {t}\overline {y}^2)\; -$ $ 2 | y |^2 \Re ( z \overline {x} \overline {y}).$

However, just because $\mathfrak {X}(F_2,\mathrm {SU}(3))$ and $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ satisfy the same real algebraic equations and have the same (real) dimension, it does not mean they are homeomorphic. In fact, $\mathfrak {X}(F_2,\mathrm {SU}(3))\cong S^8$ , and we now show that $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ is homotopic to a torus $S^1\times S^1$ .

Theorem 2.10. The character variety $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ strong deformation retracts to $S^1\times S^1$ .

Proof. As defined above, $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ is the polystable quotient of $\mathrm {Hom}(F_2,\mathrm {SU}(2,1))$ by the conjugation action of $\mathrm {SU}(2,1)$ . As Lemma2.4 shows, $\mathrm {U}(2)$ is a maximal compact subgroup of $\mathrm {SU}(2,1)$ . Thus, by Casimiro et al. [CFLO16], $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ strong deformation retracts to $\mathfrak {X}(F_2,\mathrm {U}(2))$ .

On the other hand, $\mathrm {U}(2)\cong \mathrm {SU}(2)\times _{\mathbb {Z}_2}S^1$ , where $\mathrm {SU}(2)\times _{\mathbb {Z}_2}S^1 := \left (\mathrm {SU}(2)\times S^1\right )/ \mathbb {Z}_2,$ and where $\mathbb {Z}_2$ acts on $\mathrm {SU}(2)\times S^1$ as follows. For $\mu \in \{\pm 1\}\cong \mathbb {Z}_2$ , $A \in \mathrm {SU}(2)$ and $\zeta \in S^1$ , $\mu \cdot (A, \zeta ):= (\mu A, \mu ^{-1}\zeta )$ . From this we conclude that $\mathfrak {X}(F_2,\mathrm {U}(2))\cong \mathfrak {X}(F_2,\mathrm {SU}(2))\times _{\mathbb {Z}_2^2}(S^1\times S^1)$ by Florentino and Lawton [FL12], where

\begin{equation*}\mathfrak {X}(F_2,\mathrm {SU}(2))\times _{\mathbb {Z}_2^2}(S^1\times S^1) := \left ( \mathfrak {X}(F_2,\mathrm {SU}(2))\times (S^1\times S^1)\right )/ \mathbb {Z}_2^2, \end{equation*}

and the action of $\mathbb {Z}_2^2$ on $\mathfrak {X}(F_2,\mathrm {SU}(2))\times (S^1\times S^1)$ is the product action coming from the action of $\mathbb {Z}_2$ defined above. On the other hand, $\mathfrak {X}(F_2,\mathrm {SU}(2))$ is a closed 3-ball $B$ (see Florentino and Lawton [FL09]) and so by [FL12] we conclude that $\mathfrak {X}(F_2,\mathrm {U}(2))\cong B\times (S^1\times S^1),$ which strong deformation retracts to $S^1\times S^1$ , as required.

Given that $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ deformation retracts onto $\mathfrak {X}(F_2,\mathrm {SU}(3))$ and $\mathfrak {X}(F_2,\mathrm {SU}(3))$ is homeomorphic to an 8-sphere, we have the following corollary.

Corollary 2.11. The character variety $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ is not simply connected, and so is not homotopic to either $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ or $\mathfrak {X}(F_2,\mathrm {SU}(3))$ .

We will be studying the mapping class group dynamics on certain subvarieties of $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ , called relative character varieties. To shorten the notation, let $\mathfrak {X}=\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ in what follows. Define $\mathfrak {b}:\mathfrak {X}\longrightarrow \mathbb {C}$ by $\mathfrak {b}([\rho ])=\mathrm {tr}(\rho (\alpha \beta \alpha ^{-1}\beta ^{-1}))$ . By [Gol99, Theorem 6.2.4], $\mathfrak {b}$ is surjective.

Definition 2.12. The relative character variety with respect to $c\in \mathbb {C}$ is $\mathfrak {X}_c:=\mathfrak {b}^{-1}(c)$ .

Theorem 2.13. The number $c\in \mathbb {C}-\mathbb {R}$ if and only if $P^2-4Q\not=0$ . $\mathfrak {X}_c$ is a smooth manifold if $c\in \mathbb {C}-\mathbb {R}$ .

Proof. The variety $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ is a branched double cover of $\mathbb {C}^8$ embedded in $\mathbb {C}^9$ as described in Section 2.4.1. The branching locus $\mathcal {B}$ , where the branched double cover is exactly one-to-one, is defined by the equation $P^2-4Q=0$ . As shown in Florentino and Lawton [FL09], the locus $\mathcal {B}$ corresponds exactly to the representations that are fixed under the involution $[(A,B)]\mapsto [(A^\dagger, B^\dagger )]$ . We call these the symmetric representations (up to conjugation). Equations (1) and (2) imply that $\mathfrak {X}_c\subset \mathcal {B}$ if and only if $c\in \mathbb {R}$ . Equivalently, $\mathfrak {X}_c\cap \mathcal {B}=\emptyset$ if and only if $c\in \mathbb {C}-\mathbb {R}$ . As shown in Lawton [Law06, Law07], the reducible locus (that is, the set of reducible representations) is contained in the branching locus. Thus, if $c\in \mathbb {C}-\mathbb {R}$ , we know that $\mathfrak {X}_c$ consists of conjugation classes of irreducible representations. The irreducible locus is a smooth manifold in $\mathfrak {X}(F_2,\mathrm {SL}(3,\mathbb {C}))$ by [Law07], which implies that the irreducible locus in $\mathfrak {X}$ is a smooth manifold too. Thus, since the boundary map is a submersion on the irreducible locus (as the computation in Goldman, Lawton and Xia [GLX21, Proposition 8.1] applies to this setting too), the fibers $\mathfrak {X}_c$ are smooth submanifolds by the inverse function theorem.

Remark 2.14. The locus of reducible representations (which are generally singular) is a five-dimensional subspace of the seven-dimensional branching locus in $\mathfrak {X}$ . Thus, if $c\in \mathbb {R}$ , and so $\mathfrak {X}_c$ is in the branching locus, we expect that $\mathfrak {X}_c$ contains singular representations and so is not a manifold. Heuristically this should follow since either $\mathfrak {X}_c$ contains both an irreducible and a reducible and thus a singularity, or $\mathfrak {X}_c$ contains only reducibles and thus contains an orbifold singularity from the action of the Weyl group. However, as [FGLM22, Proposition 3.1] shows, relative character varieties containing only reducible representations can ‘accidentally’ be manifolds. We also note that irreducible representations in the branching locus may have special geometric features, as described by Paupert and Will [PW17] or Will [Wil07].

Remark 2.15. It would be interesting to explore whether $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ is a topological manifold, since $\mathfrak {X}(F_2,\mathrm {SU}(3))$ is a topological manifold and $\mathfrak {X}(F_2,\mathrm {SU}(2,1))$ is homotopic to a manifold. It would also be interesting to explore the topology $($ up to homotopy $)$ of the level sets $\mathfrak {X}_c$ for various values of $c\in \mathbb {C}$ .

In our situation, $\mathrm {Out}(F_2)\cong \mathrm {GL}(2,\mathbb {Z})$ which is isomorphic to the mapping class group of the one-holed torus. This group acts on $\mathfrak {X}$ and the subgroup of mapping classes that fix the boundary of $S_{1,1}$ acts on $\mathfrak {X}_c$ . We will now proceed to show that there exists an open domain of discontinuity $\mathfrak {X}_{BQ}$ for the action of $\mathrm {Out}(F_2)$ on $\mathfrak {X}$ which strictly contains the set of discrete, faithful, convex cocompact representations (more generally the primitive stable representations).

3. Bowditch set

Recall that the aim of this article is to define a set of representations that will provide an open domain of discontinuity for the action of $\mathrm {Out}(F_2)$ on $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ . In particular, in this section we will define and study the set of Bowditch representations, and in the next section we will prove that they define an open domain of discontinuity.

For notational convenience, given a representation $\rho \colon \thinspace F_2 \rightarrow \mathrm {SU}(2,1)$ , or its conjugacy class $[\rho ] \in \mathfrak {X}$ , and given $X, Y \in \mathcal {S}$ , we will write $x=\textrm {tr }(\rho (X))$ , $y=\textrm {tr }(\rho (Y))$ etc.

3.1 Orientation on $\mathcal {E}$

Given a representation $\rho \colon \thinspace F_2 \rightarrow \mathrm {SU}(2,1)$ , we will need to assign to each undirected edge $e \in \mathcal {E}$ a particular directed edge $\vec e_\rho$ . In Section 2.2 we identified edges $e$ of $\mathcal {E}$ with triples $e = (X; Y, Z) \in \mathcal {E}^{(1)}$ where $X,Y,Z \in \mathcal {S}$ . Since we need to discuss an orientation, it will be more useful to use the notation $e = (X, Y, Z; T, T') \in \mathcal {E}^{(1)}$ ; see Figure 2.

Definition 3.1. Let $e= (X, Y, Z; T, T') \in \mathcal {E}^{(1)}$ . Let $t = |\mathrm {tr} (\rho (T)) |$ and $|\mathrm {tr} (\rho (T'))| = t'$ . Then we define the oriented edge $\vec e_\rho$ in the following way:

• if $|t| \gt |t'|$ (respectively, $|t| \lt |t'|$ ) we orient the edge $e$ pointing towards $T'$ (respectively, towards $T$ );

• if $|t| = |t'|$ we choose the orientation of $\vec e_\rho$ arbitrarily.

That oriented edge is denoted by $\vec e_\rho = (X, Y, Z; T \rightarrow T')$ , when it points towards $T'$ .

We will see that the intersection of the Bowditch representations (up to conjugation) with the set of representations that satisfy $|t| = |t'|$ for infinitely many edges is empty. So, for a given representation, the finite number of arbitrary choices that occur when $|t| = |t'|$ will not affect the large-scale behavior of the orientation of the graph $\mathcal {E}$ , the latter being what is essential to our proof.

3.2 Connectedness and the Fork Lemma

In this subsection we will prove an important lemma, called the ‘Fork Lemma’, which will be the key to proving the connectedness of the set of regions with trace bounded by a given constant.

3.2.1 Fork Lemma

Let us first describe a preliminary result which will be useful in several proofs in this section and in Section 4.

Lemma 3.2. Let $c \in \mathbb {C}$ , $[\rho ] \in \mathfrak {X}_c$ , $C=\max \{2\sqrt [4]{3},\sqrt {|\Re (c)|}\}$ and $v=(X,Y;Z,T) \in \mathcal {E}^{(0)}$ . If the collection $x$ , $y$ , $z$ and $t$ satisfies $|x|,|y| \geqslant C$ and $|t| \geqslant |z|$ , then we have

\begin{equation*}\frac C4 \max (|x|, |y|) \lt |t| \lt 5 |x||y|.\end{equation*}

Proof. Without loss of generality, we can assume that $|x| \geqslant |y|\geqslant C$ . For the first inequality, assume by contradiction that $|t| \leqslant \frac {C}{4}|x|$ . Recall that

\begin{equation*}P=2 \Re (c) = |x|^2 |y|^2 + |x|^2 + |y|^2 + |z|^2 + |t|^2 - 2 \Re (xy\overline z ) - 2 \Re (x \overline y t ) - 3.\end{equation*}

We have

\begin{align*} P & \geqslant (|x|^2|y|^2 - 4|x||y||t| +|t|^2) + |x|^2 + |y|^2 + |z|^2 -3 \\ & \geqslant \left ((|xy| - 2 |t|)^2 - 3|t|^2\right ) + 2C^2+0 - 3 \\ & \geqslant \left (\left (|x||y| - \frac {C}{2} |x|\right )^2 - \frac {3}{16} C^2 |x|^2\right ) + 2C^2 - 3 \\ & = |x|^2 \left ( \left (|y|-\frac {C}{2}\right )^2 - \frac {3}{16}C^2\right ) + 2C^2 - 3 \\ & \geqslant C^2 \frac {C^2}{16} + 2C^2 - 3 \gt 2C^2, \end{align*}

where we used the hypothesis $|t| \leqslant \frac {C}{4}|x|$ only in the third line, and the fact that $C^4 \gt 16 \cdot 3$ in the last inequality. This gives a contradiction, as $ 2\Re (c) \leqslant 2 C^2$ , and hence proves the first inequality.

For the second inequality, assume by contradiction that $|t| \geqslant 5 |x| |y|$ (or equivalently $|x| |y| \leqslant \frac {|t|}{5}$ ). In that case, we have

\begin{align*} P & \geqslant (|x|^2|y|^2)+(|x|^2+|y|^2)+(|z|^2)+(|t|^2-2 |x||y||t| - 2 |x||y||z|) - 3 \\ & \geqslant C^4+2C^2+0 + \frac {|t|^2}{5} - 3 \\ & \gt 2C^2, \end{align*}

where the last inequality is justified since $C^4 \gt 16\cdot 3 \gt 3$ . This gives the same contradiction, as $P \leqslant 2 C^2$ .

We are now ready to state and prove the Fork Lemma. First, let us define what is a fork. For a vertex $v = (X, Y ; Z, T) \in \mathcal {E}^0$ choose representatives $\alpha, \beta \in F_2$ of $X$ and $Y$ such that $\alpha \beta$ and $\alpha \beta ^{-1}$ are representatives of $Z$ and $T$ , respectively. Recall that the traces of the four neighboring regions around $v$ are given by

\begin{equation*}\mathrm {tr} (\alpha \beta^2) = t - x \overline {y} + y z, \quad \quad \quad \mathrm {tr} \left ((\alpha^2 \beta )^{-1}\right ) = t - x \overline {y} + \overline {xz}\end{equation*}

and

\begin{equation*} \mathrm {tr} (\alpha ^{-2} \beta )=\overline {tx} - x y+z, \quad \quad \quad \mathrm {tr} (\alpha \beta ^{-2}) = -x y+t \overline {y} + z. \end{equation*}

Recall also that we have a $4$ -coloring of the edges of $\mathcal {E}$ defined in Section 2.2, and at each vertex there are only two triangles of two different colors meeting.

Definition 3.3. We say that a vertex $v \in \mathcal {E}^{(0)}$ is a fork for $\rho$ if there are (at least) two edges oriented by $\rho$ having $v$ as a source and they are of different colors (for the 4-coloring). Visually, $v$ is a fork if two arrows point away from $v$ in two different triangles. See Figure 5.

Figure 5. A vertex which is a fork (left) and one which is not (right). The two arrows pointing away from the vertex have to be in different triangles to get a fork.

Equivalently, the vertex $v = (X, Y ; Z, T) \in \mathcal {E}^{(0)}$ is a fork for $\rho$ if the following conditions hold:

\begin{equation*}\begin{cases}|\textrm {tr }\rho (XY^{-1}) | & \lt \max \{ |\textrm {tr } \rho (XY^2) |, |\textrm {tr } \rho (X^2Y) | \} \\ & \mbox { and } \\ |\textrm {tr } \rho (XY)| & \lt \max \{ |\textrm {tr } \rho (XY^{-2}) |, |\textrm {tr } \rho (X^{-2}Y) | \}.\end{cases}\end{equation*}

In terms of the four traces $(x,y,z,t)$ these conditions can be written as

\begin{equation*}\begin{cases} |t| \lt |t- x \overline y+yz| \\ or \\ |t| \lt |t - x \overline {y} + \overline {xz}|, \end{cases} \quad \mbox { and } \begin{cases} |z| \lt |z-xy+\overline {xt}| \\ or \\ |z| \lt |z - xy + \overline {y}t |. \end{cases} \end{equation*}

We will call these inequalities the fork conditions. See Figure 6.

Lemma 3.4. Let $c \in \mathbb {C}$ , $[\rho ] \in \mathfrak {X}_c$ and $v = (X,Y) \in \mathcal {E}^{(0)}$ . If $v$ is a fork for $\rho$ , then we have that $\min \{|x|, |y| \} \leqslant \max \{6,\sqrt {|\Re (c)|}\}$ .

Proof. Without loss of generality, and by using the fork conditions, we can assume that $|t| \gt | t - x\bar y+yz |$ and $|z| \gt |z -xy +\overline {xt}|$ . Let $C = \max \{6,\sqrt {|\Re (c)|}\}$ and assume by contradiction that $\min \{|x|,|y|\} \gt C \geqslant 6$ .

Figure 6. A fork with the labeling of the neighboring regions.

Note that $|t - x\bar y+yz | \geqslant |yz-x\bar y|-|t| \geqslant |y||z|-|x||y| - |t|$ . So, since we have $|t| \gt | t - x\bar y+yz|$ by the fork assumption, we deduce that $2|t| +|x||y| \gt |y||z|$ , and hence we obtain

\begin{equation*}|z| \lt \frac {2 |t|}{|y|} + |x|.\end{equation*}

Similarly, as $|z -xy +\overline {xt} | \geqslant |\overline {tx}-xy|-|z| \geqslant |t||x|-|x||y|- |z|$ and $|z| \gt |z -xy +\overline {xt}|$ , we get

\begin{equation*}|t| \lt \frac {2 |z|}{|x|} + |y|.\end{equation*}

We combine both inequalities to get

\begin{align*} |z|+|t| & \lt 2 \left ( \frac {|t|}{|y|} + \frac {|z|}{|x|} \right ) + (|x| + |y|)\\ & \lt 2 \frac {|t| + |z|}{C} + 2\max \{ |x|, |y| \}. \end{align*}

From this it follows that

\begin{equation*}|z|+|t| \lt \frac {2C}{C-2} \max \{|x|, |y|\}.\end{equation*}

Recall that we have

\begin{equation*}P=|x|^2 |y|^2 + |x|^2 + |y|^2 + |z|^2 + |t|^2 - 2 \Re (xy\overline z ) - 2 \Re (\overline {x} y t ) - 3.\end{equation*}

Since $C \geqslant 6$ , we have that $\frac {4C}{C-2} \leqslant C \lt \min \{|x|,|y|\}$ . Hence, using the inequality above, we get

\begin{eqnarray*} |2 \Re (xy\overline z ) + 2 \Re (x \overline y t )| \leqslant 2 |xy| (|z| + |t|) & \lt & \frac {4C}{C-2} |xy| \max \{|x|, |y|\}\\ & \lt & |xy| \min \{|x|, |y|\} \max \{|x|, |y|\} = |x|^2 |y|^2. \end{eqnarray*}

Moreover, using Lemma3.2, we can see that

\begin{equation*}|z|^2 + |t|^2 -3 \geqslant \max (|z|, |t|)^2 -3 \geqslant \frac {C^2}{16} \max (|x|, |y|)^2 - 3 \geqslant \frac {C^4}{16}-3 \gt 0.\end{equation*}

This implies that

\begin{equation*}P=|x|^2 |y|^2 + |x|^2 + |y|^2 + |z|^2 + |t|^2 - 2 \Re (xy\overline z ) - 2 \Re (x \overline y t ) - 3 \gt |x|^2+|y|^2 \gt 2 C^2.\end{equation*}

This gives a contradiction, as $P=2 \Re (c) \leqslant 2 C^2$ .

As we will see, the conclusion of the Fork Lemma (Lemma3.4) still holds for vertices that satisfy the fork conditions ‘up to epsilon’, and for certain arguments we will need the notion of an $\varepsilon$ -fork. More precisely, we give the following definition.

Definition 3.5. Let $\varepsilon \gt 0$ . A vertex $v \in \mathcal {E}$ is an $\varepsilon$ -fork if changing the value of one of the four regions around $v$ by less than $\varepsilon$ makes it a fork. This means that $v$ has (at least) two arrows pointing away from it or pointing towards it, but such that the difference between the values at the opposing regions of the edge, say $x, x' \in \mathbb {C}$ , are such that $\vert \vert x \vert - \vert x' \vert \vert \lt \varepsilon$ .

The Fork Lemma (Lemma3.4) above can be generalized to $\varepsilon$ -forks.

Corollary 3.6. Let $c \in \mathbb {C}$ , $[\rho ] \in \mathfrak {X}_c$ and $v = (X,Y) \in \mathcal {E}^{(0)}$ . Let $C = \max \{6,\sqrt {|\Re (c)|}\}$ and $\varepsilon _0 = \varepsilon _0(C) = C^2 \frac {C-6}{4} \gt 0$ . For all $\varepsilon \lt \varepsilon _0$ , if $v$ is an $\varepsilon$ -fork, then $\min \{ |x|, |y| \} \lt C$ .

Proof. We will follow an argument similar to the one used to prove the Fork Lemma (Lemma3.4). Assume by contradiction that $\min \{|x|,|y|\} \gt C \geqslant 6$ . Let $\varepsilon \lt \varepsilon _0$ . Assume $v$ is an $\varepsilon$ -fork. We keep the same notation for $x,y,z,t$ as before. The fork conditions give $|t| \gt |t - x \overline y+yz | - \varepsilon$ and $|z| \gt |z - xy + \overline {xt} | - \varepsilon$ . The same steps as before give

\begin{equation*}|z| \lt 2\frac {|t|}{|y|} + |x| + \frac {\varepsilon }{C} \quad \mbox { and } \quad |t| \lt 2 \frac {|z|}{|x|} + |y| + \frac {\varepsilon }{C}.\end{equation*}

We combine both inequalities and obtain

\begin{equation*}|z| + |t| \lt \frac {2C}{C-2} \max \{ |x|, |y| \} + 2 \frac {\varepsilon }{C-2}.\end{equation*}

As $\varepsilon \lt \varepsilon _0$ , we obtain directly that $|z|+|t| \lt \dfrac {|C|}{2} \max \{|x|,|y|\}$ , since $C \lt \max \{|x|,|y|\}$ .

Now, using the hypothesis $C \lt \min \{ |x|, |y| \}$ , we get

\begin{equation*}|2 \mathfrak {R} (xy\overline z)+2 \mathfrak {R} (\overline {x} y t) | \leqslant 2 |xy| (|z|+|t|) \lt |x|^2 |y|^2.\end{equation*}

Using reasoning similar to the previous proof, we can then see that $P \gt 2C^2$ , which gives a contradiction. So we have that $\min \{ |x|, |y| \} \leqslant C$ , as we wanted to prove.

3.2.2 Connectedness

Recall that we have the following identification $\Omega = \mathcal {C}^{(0)} = \mathcal {S}$ .

For any $K \geqslant 0$ and $[\rho ] \in \mathfrak {X}$ , we denote by $\Omega _\rho (K)$ the set of regions with traces smaller than  $K$ :

\begin{equation*}\Omega _\rho (K) = \{ \gamma \in \Omega \mid |\mathrm {tr} (\rho (\gamma ))| \leqslant K \}\subset \mathbf {H}^2.\end{equation*}

Since every element $\gamma \in \Omega$ corresponds to the closure of a region in the complement of the (properly embedded trivalent) tree $\mathcal {T}$ in $\mathbf {H}^2$ , as defined in Section 2.2, we consider the set $\Omega _\rho (K)$ to be a subset of $\mathbf {H}^2$ . Using the Fork Lemma, in this section we deduce that, for $K$ large enough, these sets are connected when represented as subsets of $\mathbf {H}^2$ .

Lemma 3.7. Let $c \in \mathbb {C}$ and $[\rho ] \in \mathfrak {X}_c$ . For all $C \geqslant \max \{6,\sqrt {|\Re (c)|}\}$ , the set $\Omega _\rho (C)$ is connected.

Proof. Assume by contradiction that $\Omega _\rho (C)$ is not connected. We make an argument based on the minimal distance between two connected components of $\Omega _\rho (C)$ . Recall that if $v = (X,Y ; Z, T) \in \mathcal {E}^{(0)}$ , then we have

\begin{equation*}P=|x|^2 |y|^2 + |x|^2 + |y|^2 + |z|^2 + |t|^2 - 2 \Re (xy\overline z ) - 2 \Re (\overline {x} y t ) - 3=2 \Re (c),\end{equation*}

which implies $P \leqslant 2C^2$ .

If the distance is $1$ , then we have a vertex $v = (X, Y; Z,T) \in \mathcal {E}^{(0)}$ such that $|z|, |t| \leqslant C$ and $|x|, |y| \gt C$ . In that case, we have

\begin{equation*}|2 \Re (xy\overline z)+2 \Re (\overline x y t)| \lt 4C |xy| = 5C |xy| - C|xy| \lt |x|^2 |y|^2 - 3,\end{equation*}

since $5C \lt C^2 \lt |x||y|$ and since $C |x||y| \geqslant C^3 \gt 3$ . This gives

\begin{equation*}P \gt |x|^2 + |y|^2 \geqslant 2C^2,\end{equation*}

which is a contradiction.

If the distance is $2$ , then we have an edge $(X,Y,T ; Z, Z')$ . We can assume, without loss of generality, that $|z| \leqslant C$ , $|z'| = |z+\overline {yt} - xy| \leqslant C$ and $|x|, |y|, |t| \gt C$ . In that case the inequality $|z+\overline {y}t - xy| \leqslant C$ implies that $C \lt |t| \leqslant \frac {C+|z|}{|y|} + |x| \lt |x| + 2.$ This gives

\begin{equation*}|2 \Re (xy\overline z)+2 \Re (\overline x y t)|\lt 2C |xy| + 2 |xy| (|x| + 2) \lt 2 |xy| (C+|x|+2)\lt |x|^2 |y|^2,\end{equation*}

where the last inequality follows from

\begin{equation*}2 (C+|x|+2) = |x| \left (2 + \frac {4}{|x|}+\frac {2C}{|x|}\right ) \lt |x| \left (2 + \frac {3C}{|x|}\right ) \lt |x| 5 \lt |x| C \lt |x||y|.\end{equation*}

Since $|t|^2-3\gt C^2-3 \gt 0$ and $|x|^2 + |y|^2 +|z|^2\gt 2 C^2$ , we also find that $P\gt 2C^2$ , which gives the same contradiction.

If the distance is greater than or equal to $3$ , then we can consider the shortest path between these components and orient the edges along this path according to $\rho$ . The first and last oriented edges are pointing in opposite directions (each towards one of the connected components). So there exists a vertex in the middle of the path that is a fork for $\rho$ . So there is a region with trace less than $C$ with a smaller distance to each connected component, which contradicts the minimality of the distance.

3.3 Escaping rays and neighbors

Let $P$ be a geodesic ray in the edge graph, starting at a vertex $v_0$ and consisting of edges $e_n$ joining $v_n$ to $v_{n+1}$ . Such a geodesic only passes once through each triangle of the edge graph. We say that such an infinite geodesic ray is an escaping ray if, for all $n \in \mathbb {N}$ , each edge $e_n$ is directed from $v_n$ to $v_{n+1}$ . Given a representation $[\rho ] \in \mathfrak {X}_c$ , an infinite geodesic ray is called an escaping ray for $\rho$ if it is an escaping ray with respect to the orientation induced by $\rho$ , see Section 3.1.

We will prove that the existence of escaping rays for a given representation $\rho$ will imply that either the ray will eventually be contained in the boundary of a region $X \in \mathcal {C}^{(0)}$ such that $\rho (X)$ only has eigenvalues on the unit circle (which is equivalent to $f(x) \leqslant 0$ according to Theorem2.7), or there are infinitely many regions with value bounded by a constant depending only on the boundary parameter $c \in \mathbb {C}$ .

First we consider the case of an escaping ray that is included in the boundary of a given region in $\mathcal {C}^{(0)}$ , and then we will describe the general case.

3.3.1 Neighbors around a region

In this subsection we are interested in the behavior of the values of the traces of regions that are all neighbors of a central one. Let $\alpha \in F_2$ . The neighbors of the region associated to $\alpha$ are all associated to elements of the form $\gamma _n = \alpha ^n \beta$ for all $n \in \mathbb {Z}$ .

Let $c \in \mathbb {C}$ and let $[\rho ] \in \mathfrak {X}_c$ be a representation. Let $x = \mathrm {tr} (\rho (\alpha ))$ and consider the sequence defined by $u_n = \mathrm {tr} (\rho (\gamma _n))$ for all $n \in \mathbb {Z}$ . The trace identity gives the following relation:

(3) \begin{equation} \forall n \in \mathbb {Z}, \ u_{n+3} = x u_{n+2} - \overline {x} u_{n+1} + u_n . \end{equation}

To study the solutions of this recurrence relation we define $X_n = \left(\begin{matrix} u_{n+2} \\ u_{n+1} \\ u_n \end{matrix}\right) \in \mathbb {C}^3$ and consider the matrix $M_x = \left(\begin{matrix} x & - \overline x & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{matrix}\right)$ , so that we have $X_{n+1} = M_x X_n$ . The matrix $M_x$ is conjugate to a matrix in $\textrm {SU} (2,1)$ and its characteristic polynomial is given by $\chi _x(s) = s^3 - x s^2 + \overline x s - 1$ . We can use Corollary2.7 to determine the behavior of the eigenvalues of $M_x$ , using the function $f$ defined by $f(x) = \mathrm {Res} (\chi _x, \chi _x')$ , and hence of the sequence $(u_n)_{n\in \mathbb {Z}}$ depending on the value of $f(x)$ .

If $f(x)=0$ , then the coefficients of the matrix $M_x^n$ grow at most quadratically in $n$ , and hence the growth of the sequence $(X_n)_{n\in \mathbb {Z}}$ , and subsequently of the sequence $(u_n)_{n\in \mathbb {Z}}$ , is at most quadratic, see Appendix A.1.

When $f(x) \neq 0$ , we can diagonalize the matrix $M_x$ as $M_x = P D P^{-1}$ with

\begin{equation*}P = \left(\begin{matrix} \lambda & \frac {\bar \lambda }{\lambda } & \frac {\, 1\, }{\bar \lambda } \\ 1 & 1 & 1 \\ \frac {1}{\lambda } & \frac {\, \lambda \,}{\bar \lambda } & \bar \lambda \end{matrix}\right), \quad D = \left(\begin{matrix} \lambda & 0 & 0 \\ 0 & \frac {\bar \lambda }{\lambda } & 0 \\ 0 & 0 & \frac {\, 1\, }{\bar \lambda }\end{matrix}\right) \;\; \text { and } \;\; x = \lambda + \frac {\bar \lambda }{\lambda } + \frac {\, 1\, }{\bar \lambda }.\end{equation*}

Hence we get $X_n = P D^n P^{-1} X_0,$ where $X_0 = \left(\begin{matrix} u_2 \\ u_1 \\ u_0 \end{matrix}\right)$ .

Let $(e_1, e_2, e_3)$ be the basis of $\mathbb {C}^3$ formed by the columns of the matrix $P$ , which correspond to the eigenvectors of the matrix $M_x$ . We can decompose $X_0 = m_1 e_1 + m_2 e_2 + m_3 e_3$ with $(m_1, m_2, m_3) \in \mathbb {C}^3$ . In that case, we get that

\begin{equation*}X_n = m_1 \lambda ^n e_1 + m_2 \left ( \frac {\bar \lambda }{\lambda } \right )^n e_2 + m_3 \left ( \frac {1}{\bar \lambda } \right )^n e_3.\end{equation*}

If $f(x)\lt 0$ , then $|\lambda | = 1$ and we see that the sequence $(X_n)_{n\in \mathbb {Z}}$ stays bounded in $\mathbb {C}^3$ , and subsequently the sequence $(u_n)_{n\in \mathbb {Z}}$ is bounded as the projection of $X_n$ on $\mathbb {C} \cdot e_3$ .

When $f(x) \gt 0$ , then we have $|\lambda | \gt 1$ . If $m_1 m_3 \neq 0$ we see that the sequence $X_n$ grows exponentially in both directions. However, it is possible to have $m_1 m_3=0$ and in that case the sequence will stay bounded in one direction. The purpose of the next lemma is to prove that the bound on such a sequence will only depend on the boundary parameter $c$ .

Lemma 3.8. There exists a constant $D = D(c)$ , such that if the sequence $(u_n)_{n\in \mathbb {N}}$ defined above with $f(x) \gt 0$ does not grow exponentially in both directions, then there are infinitely many terms of the sequence such that $|u_n| \lt D$ .

Proof. With the notation above, we are necessarily in the case $m_1 m_3=0$ . We can assume, without loss of generality, that $m_1=0$ , so that we have

\begin{equation*}X_0 = m_2 e_2 + m_3 e_3 \;\text { and }\;\;X_n = m_2 \left ( \frac {\bar \lambda }{\lambda } \right )^n e_2 + m_3 \left ( \frac {1}{\bar \lambda } \right )^n e_3.\end{equation*}

Since $|\lambda | \gt 1$ , we see that the sequence $X_n$ will accumulate on a circle of radius $|m_2 |$ in the complex line $e_2$ . More precisely, for all $\varepsilon \gt 0$ , there exists $n_0 \in \mathbb {N}$ , such that, for all $n \geqslant n_0$ , there exists $\theta _n \in \mathbb {R}$ such that $\Vert X_n - |m_2| e^{i\theta _n} e_2 \Vert \lt \varepsilon$ . In other words, for $n$ large enough, the vector $X_n$ gets arbitrarily close to the set $\{ |m_2| e^{i\theta } e_2, \theta \in \mathbb {R} \}$ .

Hence, to prove the lemma, it suffices to find a uniform bound on $|m_2|$ that only depends on the boundary parameter $c$ . Let $\varepsilon$ be arbitrarily small, and $n \geqslant n_0$ as above.

By definition of the sequence, we have

\begin{equation*}x = \mathrm {tr} (\rho (\alpha )), \ u_n = \mathrm {tr} (\rho (\alpha ^{n} \beta )), \ u_{n+1} = \mathrm {tr} (\rho (\alpha ^{n+1} \beta )), \text { and } u_{n+2} = \mathrm {tr} ( \rho (\alpha (\alpha ^{n+1}) \beta )). \end{equation*}

By denoting $\beta ' = \alpha ^{n+1} \beta$ we can write these values as

\begin{equation*}x = \mathrm {tr} (\rho (\alpha )), \quad u_{n+1} = \mathrm {tr} (\rho ( \beta ' )), \quad u_{n+2} = \mathrm {tr} ( \rho (\alpha \beta ' )), \quad \overline {u_n} = \mathrm {tr} (\rho (\alpha \beta '^{-1} )),\end{equation*}

so we can identify the quadruple $(x, u_{n+1}, u_{n+2}, \overline {u_n})$ as the character of a representation in $\mathfrak {X}_c$ . This means that the quadruple $(x, u_{n+1}, u_{n+2}, \overline {u_n})$ will satisfy the equations for $P$ and $Q$ with $y=u_{n+1}, z=u_{n+2}$ and $t = \overline {u_n}$ .

Using our hypothesis on $n$ , this quadruple is arbitrarily close to a quadruple of the form $\left (x, |m_2| e^{i \theta }, |m_2[ e^{i\theta } \frac {\bar \lambda }{\lambda }, |m_2| e^{-i\theta } \frac {\bar \lambda }{\lambda } \right ),$ for some $\theta \in \mathbb {R}.$

By continuity of the expression relating $P$ and $Q$ with the quadruple of trace coordinates, we know that this ‘limit’ quadruple also satisfies the equations for $P$ and $Q$ and corresponds to an equivalence class of a representation in $\mathfrak {X}_c$ .

We can first write the equation for $P$ with the quadruple $\left (x, |m_2| e^{i \theta }, |m_2[ e^{i\theta } \frac {\bar \lambda }{\lambda }, |m_2| e^{-i\theta } \frac {\bar \lambda }{\lambda } \right )$ , which gives

\begin{align*} P & = |x|^2 |m_2|^2 + |x|^2+3 |m_2|^2 - 2 \Re \left (x |m_2| e^{i\theta } \displaystyle \frac {\, \lambda \,}{\bar \lambda } |m_2|e^{-i\theta } \right ) - 2 \Re \left ( \overline {x} |m_2|e^{i\theta } \frac {\, \bar \lambda \, }{\lambda } |m_2|e^{-i\theta } \right ) -3 \\ & = |m_2|^2 \left ( |x|^2 - 4 \Re \left ( x \frac {\lambda }{\, \bar \lambda \, } \right ) +3\right ) - 3 + |x|^2. \end{align*}

So we obtain an expression for $|m_2|$ in terms of $P$ and $x$ :

(4) \begin{equation} |m_2|^2 = \dfrac {P+3 - |x|^2}{|x|^2 - 4 \Re \left ( x \frac {\, \lambda \, }{\bar \lambda } \right ) +3}. \end{equation}

The equation for $Q$ gives

\begin{align*} & Q = |m_2|^4 \left (5 + |x|^2+2 \Re \left ( 2 \left (\frac {\bar \lambda }{\lambda }\right )^3- 2 \overline x \frac {\bar \lambda }{\lambda } - x \left (\frac {\bar \lambda }{\lambda }\right )^2 \right ) \right ) \\ & + |m_2|^2 \left ( |x|^4+3|x|^2-18+2\Re \left ( 2 x^2 \frac {\bar \lambda }{\lambda }-3 x \left (\frac {\bar \lambda }{\lambda }\right )^2+6 \overline x \frac {\bar \lambda }{\lambda } - 2 |x|^2 \overline x \frac {\bar \lambda }{\lambda }+\overline x^2 \left ( \frac {\bar \lambda }{\lambda }\right )^2-x^3 \right ) \right ) \\ & + 9 - 6|x|^2+2\Re (x^3). \end{align*}

Using the expression of $|m_2|^2$ found above and replacing $x = \lambda + \frac {\bar \lambda }{\lambda } + \frac {1}{\bar \lambda }$ and writing $\lambda = r e^{is}$ with $r \gt 0$ and $s \in \mathbb {R}$ , we can obtain (for example, using Mathematica) the following formula:

(5) \begin{equation} Q = \dfrac {(Pr^4 - r^6+1)(Pr^2 + r^6 - 1)}{r^4 (r^2+1)^2}. \end{equation}

This means that $r$ is one of the finitely many solutions of the equation above. At this point, we have already proved that if the sequence $(u_n)_{n\in \mathbb {N}}$ does not grow exponentially, then $|\lambda |$ and hence $|x|$ can only take finitely many values that only depend on $P$ and $Q$ . Moreover, we cannot have $r=1$ as a solution as in that case we would have $f(x)=0$ which is excluded from our consideration.

Finally, if we look at (4) we see that its denominator can be written as

\begin{equation*}|x|^2 - 4 \Re \left ( x \frac {\lambda }{\bar \lambda } \right ) +3 = |\lambda |^2 - 2 \Re \left ( \frac {\lambda^2}{\bar \lambda }\right ) - 2 \Re \left (\frac {\lambda }{\bar {\lambda }^2}\right ) + 2 + \frac {1}{|\lambda |^2} =\dfrac {(|\lambda |^2+1)|\lambda^2 - \bar {\lambda } | ^2 }{|\lambda |^4}, \end{equation*}

which makes it clear that the denominator of the expression can only be equal to $0$ when $\lambda \in \zeta _3$ and hence when $|\lambda | = 1$ . This means that if $r = |\lambda |$ is one of the finitely many solutions of (5), then the expression of $|m_2|^2$ is bounded by a value that depends only on $P$ and $Q$ , and hence that depends only on $c$ .

We can conclude that there exists $D = D(c)$ , only depending on $c$ , such that $|m_2| \lt D$ . Thus, if the sequence $(u_n)_{n\in \mathbb {Z}}$ does not grow exponentially for $n \longrightarrow + \infty$ , then $|u_n|$ will converge to a value smaller than $D$ .

Remark 3.9. We can use the previous proof to produce an algorithm to find the constant $D$ . Precisely, first solve for $r^2$ using (5) and then plug it into (4) to obtain a formula for $|m_2|$ in terms of $P,Q$ and $s$ . A value for $D$ can be taken to be $|m_2|+\epsilon$ . So we need to find the maximum of $|m_2|$ on the circle (parametrized by $s$ ), which can always be done numerically.

For the rest of the paper, for all $c \in \mathbb {C}$ , we set

(6) \begin{equation} M(c):=\max \{6, \sqrt {|\Re (c)|}, D(c)\}. \end{equation}

Lemma3.8 implies the following result.

Lemma 3.10. Let $c \in \mathbb {C}$ . There exists a function $H_c \colon \thinspace \mathbb {C} \rightarrow \mathbb {R} \cup \{ \infty \}$ with the following properties, for all $x \in \mathbb {C}$ :

1. (a) $H_c(x) = H_c(\overline x)$ ;

2. (b) $H_c(x) \geqslant M(c)$ ;

3. (c) $H_c(x) = \infty$ if and only if $f(x)\leqslant 0$ ;

4. (d) if $f(x) \gt 0$ , then for any sequence $u_n$ as above satisfying (3) and coming from a representation $[\rho ] \in \mathfrak {X}_c$ , there exists $n_1, n_2 \in \mathbb {Z} \cup \{ \pm \infty \}$ such that:

   1. (i) $|u_n| \leqslant H_c(x)$ if and only if $n_1 \leqslant n \leqslant n_2$ ;

   2. (ii) $|u_n|$ is monotonically decreasing for $n \lt n_1$ and increasing for $n \gt n_2$ .

There are similar statements in previous works ([Bow98 3.14], [TWZ08, 3.20], [MPT15, 4.15] and [MP20, 3.4.1]) that all rely on the same basic principle. We refer the interested reader to the proofs given in these papers that are in simpler cases.

Proof. We will omit certain computational details for clarity.

First, we simply set $H_c (x) = \infty$ when $f(x) \leqslant 0$ .

Now, let $x \in \mathbb {C}$ , such that $f(x) \gt 0$ . Assume that we have $[\rho ] \in \mathfrak {X}_c$ , a region $X \in \Omega$ such that $x = \mathrm {tr} (\rho (X))$ and the sequence $(u_n)$ defined as before. So $(u_n)$ satisfies (3) and is such that $(x, u_1, u_2, \overline {u_0} )$ is the character of the representation $\rho$ . As discussed above, there exists $(m_1, m_2, m_3) \in \mathbb {C}^3$ such that

\begin{equation*}u_{n+1} = m_1 \lambda ^n + m_2 \left ( \frac {\bar {\lambda }}{\lambda } \right )^n + m_3 \left ( \frac {1}{\bar \lambda } \right )^n.\end{equation*}

Assume first that $m_1 m_3 \neq 0$ . Up to reindexing the sequence we can assume that $|m_3| \leqslant |m_1| \leqslant |\lambda |^2 |m_3|$ . The quadruple $(x,u_1, u_2, \overline {u_0} )$ satisfies the equations for $P$ and $Q$ , so using an argument similar to the one for finding the bound on $|m_2|$ in the proof of Lemma3.8, we can show that there exists a constant $K_c (x)$ such that $\frac {|m_2|}{\sqrt {|m_1m_3|}}$ and $|m_1 m_3|$ are both less than $K_c (x)$ . One can see that

\begin{align*} \frac {|u_{n+2}| - |u_{n+1}|}{\sqrt {|m_1 m_3|}} & \geqslant |\lambda |^n \sqrt {\frac {|m_1|}{|m_3|} } ( |\lambda | - 1) - 2 \frac {m_2}{\sqrt {|m_1 m_3|}} - \sqrt {\frac {|m_3|}{|m_1|}} \frac {1}{|\lambda |^{n+1}} (|\lambda |+1) \\[5pt] & \geqslant |\lambda |^n (|\lambda |-1) - 2 K_c(x) - (|\lambda |+1). \end{align*}

Hence, if $|\lambda |^n$ is larger than some constant $L_c (x):= \frac {2K_c (x) + (|\lambda | + 1)}{|\lambda |-1}$ , then we have that $|u_{n+2}| \geqslant |u_{n+1}|$ .

On the other hand, as $|u_{n+1}| \leqslant |m_1| |\lambda |^n + |m_2| + |m_3| \frac {1}{|\lambda |^n}$ , we infer that there exists $H_c(x)$ such that if $|u_{n+1}| \geqslant H_c (x)$ , then $|\lambda |^n \geqslant L_c (x)$ . Hence, for indices such that $|u_n| \geqslant H_c (x)$ we have that the sequence is monotonically increasing, which is the desired condition.

We can choose $H_c (x)$ to be always bigger than $M(c)$ so thatm in the case $m_1 m_3=0$ , Lemma3.8 gives the desired result. The sequence $u_{-n}$ behaves in a similar way, simply replacing $x$ with $\bar x$ , and hence this shows that $H_c( x) = H_c (\bar x)$ .

3.3.2 Escaping rays

Using the constant $M(c)$ defined in (6) we now prove the following.

Lemma 3.11. Let $c \in \mathbb {C}$ and $[\rho ] \in \mathfrak {X}_c$ . Set $M(c):=\max \{6, \sqrt {|\Re (c)|}, D(c)\}$ . Suppose that $\{ e_n \}_{n\in \mathbb {N}}$ is an escaping ray. Then there are two cases:

1. (i) there exists region $\alpha$ such that the ray is eventually contained in the boundary of $\alpha$ , such that $f (x ) \leqslant 0$ with $x = \mathrm {tr} (\rho (\alpha ))$ ; or

2. (ii) the ray meets infinitely many distinct regions with trace smaller than $M(c)$ .

Proof. Let $(e_i)_{i\in \mathbb {N}}$ be an escaping ray. First, assume that the ray is eventually contained in the neighborhood of some region $\alpha$ , and note $x = \mathrm {tr} (\rho (\alpha ))$ . If $f(x) \leqslant 0$ , then we are in the first case. Otherwise, we have $f(x) \gt 0$ . Note that the sequence $(|u_n|)_{n\in \mathbb {N}}$ of traces of regions around $x$ cannot be exponentially growing with respect to $n$ , or else the ray would not be escaping. From the previous lemma, it means that there are infinitely many distinct regions with trace smaller than  $D (c)$ .

Now assume that the escaping ray $(e_i)_{i \in \mathbb {N}}$ does not eventually end in the neighborhood of a given region. We consider a $4$ -coloring of the complex $\mathcal {E}$ as described in Section 2.2. Let $(X_i)_{i\in \mathbb {N}}, (Y_i)_{i\in \mathbb {N}}, (Z_i)_{i\in \mathbb {N}}, (T_i)_{i\in \mathbb {N}}$ be the four sequences of regions corresponding to the four regions around the head of edge $e_i$ , with the letters $X,Y,Z,T$ each corresponding to a given color. Let $x_i = \mathrm {tr} (\rho (X_i))$ ; and similarly for $y_i, z_i, t_i$ .

As the escaping ray does not end in the neighbors of a given region, we see that the four sequences $|x_i|, |y_i|, |z_i|, |t_i|$ are infinite, decreasing and bounded below. So the modulus of each sequence is converging towards some limit.

For all $\epsilon \gt 0$ , there exists $n_0$ such that, for all $n \geqslant n_0$ , the vertex $v_n$ between the two consecutive directed edges $e_n$ and $e_{n+1}$ is an $\epsilon$ -fork. For $\epsilon$ small enough, we can then apply the $\epsilon$ -fork lemma to conclude that for all $n \geqslant n_0$ there is at least one region around the vertex $v_{n}$ that has trace smaller than $C:= \max \{6, \sqrt {|\Re (c)|} \}$ . As the escaping ray is not contained in the neighborhood of a given region, we know that there will be infinitely many distinct such regions.

3.4 Bowditch representations

We can now define Bowditch representations.

Definition 3.12. Let $c \in \mathbb {C}$ , and let $\rho \colon \thinspace F_2 \rightarrow \mathrm {SU}(2,1)$ be a representation such that $[\rho ]\in \mathfrak {X}_c$ . We say that $\rho$ is a Bowditch representation if:

1. (i) $\forall \gamma \in \mathcal {S}$ , we have that $\rho (\gamma )$ is loxodromic and

2. (ii) the set $\Omega (M) = \{ \gamma \in \mathcal {S} \ | \ \mathrm {tr} (\rho (\gamma )) \lt M \}$ is finite,

where $M = M(c)$ as defined in (6).

We denote by $\mathfrak {X}_{BQ,c} = \mathfrak {X}_{BQ, c}(F_2, \mathrm {SU}(2, 1))$ the set of conjugacy classes of representation in $\mathfrak {X}_c$ that are Bowditch and, more generally, we denote

\begin{equation*}\mathfrak {X}_{BQ} = \bigcup _{c \in \mathbb {C}} \mathfrak {X}_{BQ, c}(F_2, \mathrm {SU}(2,1)).\end{equation*}

The following result is an immediate consequence of Lemmas3.8 and 3.11.

Lemma 3.13. If $[\rho ] \in \mathfrak {X}_{BQ}$ , then there will be no escaping rays for $\rho$ . Moreover, for any region $X \in \Omega$ and neighboring regions $\{Y_n\}$ around $X$ , the absolute value of traces $|\mathrm {tr}\left (\rho (Y_n)\right )|$ will have exponential growth in both directions.

3.5 Attracting subgraph

3.5.1 Attracting arc

Recall that we have the following identifications: $\Omega = \mathcal {C}^{(0)} = \mathcal {S}$ . We will make use of Lemma3.10 to define an attracting subgraph in the neighborhood of certain regions.

Let $c \in \mathbb {C}$ , $[\rho ] \in \mathfrak {X}_c$ and $K \geqslant M(c)$ . For each region $X \in \Omega (K)$ , we define

\begin{equation*}J_\rho (K, X) := \bigcup \left \{ e \in \mathcal {E}^{(1)} | \ e = (X, Y_n, Y_{n+1}) \mbox { and } |y_n|, |y_{n+1}| \leqslant \max \{H_c(x), K\} \right \}.\end{equation*}

This is a connected non-empty subarc in $\partial X$ with the following properties:

1. (i) every directed edge $\vec e_\rho$ such that $e \in \partial X$ that is not in $J_\rho (K, X)$ points towards $J_\rho (K, X)$ ;

2. (ii) if $Y \in \Omega (K)$ and $e \in \partial X$ has one vertex $v = (X, Y) \in \mathcal {E}^{(0)}$ , then $e \in J_\rho (K, X)$ .

This definition is similar to the definitions used in Tan, Wong and Zhang [TWZ08] and Maloni, Palesi and Tan [MPT15]. However, in our setting, we need to define a subgraph that is slightly larger than this subarc, but with similar properties, because of the way we define a subtree to be ‘attracting’. Recall that the graph $\mathcal {E}$ is a ‘trivalent tree of triangles’ in the sense that it is a graph made of triangles, one for each triangle $T$ in $\mathcal {C}^{(2)}$ . We want to define a bigger set $\widetilde {J}_\rho (K, X) \subset \mathcal {E}^{(1)}$ with the property that, if one edge $e$ is in $J_\rho (K, X)$ , we want to add, in $\widetilde {J}_\rho (K, X)$ , the whole triangle that $e$ is part of. It is clear that $J_\rho (K, X) \subset \widetilde {J}_\rho (K, X)$ . We say that a triangle $\Delta \subset \mathcal {E}^{(1)}$ is in the boundary $\partial X$ of the region $X$ if one of its edges has this property. So we define the set $\widetilde {J}_\rho (K, X)$ as a connected union of triangles in $\partial X$ satisfying the following properties.

1. (i) For each triangle $\Delta$ in $\partial X$ such that $\Delta$ is not in $\widetilde {J}_\rho (K,X)$ , then, for each vertex $v$ of $\Delta$ , the directed edge from $v$ forming a geodesic towards $\widetilde {J}_\rho (K, X)$ is pointing towards $\widetilde {J}_\rho (K, X)$ . ¹

2. (ii) If $Y \in \Omega (K)$ and $\Delta$ is a triangle adjacent to $X$ and $Y$ , then $\Delta$ is in $\widetilde {J}_\rho (K, X)$ .

We can now explain better why we need to define this extension. Let $e$ be an edge which is part of a ‘triangle’ $\Delta$ . It could happen that $e\in J_\rho (K, X)$ but no other edge of $\Delta$ is. In this situation, the geodesics from the vertex of $\Delta \setminus e$ to $J_\rho (K, X)$ are not unique and it could happen that it is not true any longer that both those geodesic paths point towards $J_\rho (K, X)$ , as we will require in the definition of $\rho$ -attracting below.

3.5.2 Attracting subgraph

Definition 3.14. A non-empty subgraph $T \subset \mathcal {E}$ is said to be $\rho$ -attracting if, for every vertex $v\in \mathcal {E}^{(0)}$ such that $v \notin T$ and for every geodesic in $\mathcal {E}$ from $v$ to $T$ , each $\rho$ -oriented edge along the geodesic points towards $T$ .

For all $K \geqslant M(c)$ we constructed a subarc $\widetilde {J}_\rho (K, X)$ for all regions $X \in \Omega = \mathcal {S}$ . We can now define the graph

\begin{equation*}T_\rho (K) := \bigcup _{X \in \Omega (K)} \widetilde {J}_\rho (K, X).\end{equation*}

This set is a subgraph of $\mathcal {E}$ and we can prove the following result.

Proposition 3.15. If $K \geqslant M(c)$ , then the subgraph $T_\rho (K)$ is connected and $\rho$ -attracting.

Proof. First, let us show that $\mathcal {T} = T_\rho (K)$ is connected. Let $e, e'$ be two edges of $\mathcal {T}$ . Then $e$ and $e'$ belong to a triangle adjacent to regions $X, X' \in \Omega (K)$ . As $\Omega (K)$ is connected, we can find a sequence of regions $X = X_0, X_1, \dots, X_n = X'$ such that $X_k$ is adjacent to $X_{k+1}$ for all $k$ . Hence there are triangles $\Delta _0, \ldots, \Delta _n$ , such that each $\Delta _k$ is adjacent to both $X_k$ and $X_{k+1}$ , and by construction this triangle is in $T_{\rho }(K)$ . Since the union of triangles in $\widetilde {J}_{\rho } (K, X_k)$ is connected, the triangles $\Delta _k$ and $\Delta _{k+1}$ are connected in $\widetilde {J}_{\rho } (K, X)$ , and hence the two edges $e$ and $e'$ are connected by a path in $\mathcal {T}$ .

Second, let us show that $\mathcal {T} = T_\rho (K)$ is $\rho$ -attracting. Let $e = (X,Y,Z ; T, T')$ be an edge that intersects $\mathcal {T}$ at a vertex $v = (X,Y) \in \mathcal {E}^{(0)}$ , but that is not in $\mathcal {T}$ (so $e$ is in the circular neighborhood of $\mathcal {T}$ ). This means that the triangle $\Delta$ adjacent to $X,Y,T$ is in $\mathcal {T}$ , but the triangle $\Delta '$ adjacent to $X,Y,Z$ is not.

By construction, one of the regions $X,Y,T$ is in $\Omega _\rho (K)$ . If $X$ or $Y$ is in $\Omega _\rho (K)$ then we can assume, without loss of generality, that $\Delta$ is in $\widetilde {J}_\rho (K, X)$ and $\Delta '$ is adjacent to $X$ . In that case, the construction of $\widetilde {J}_\rho (K, X)$ ensures that the directed edges of $\Delta '$ are pointing towards $\Delta$ . If $X$ and $Y$ are not in $\Omega _\rho (K)$ , then we necessarily have $T \in \Omega _\rho (K)$ . In that case, we cannot have $|t'| \lt |t|$ , as we would have $T' \in \Omega _\rho (K)$ , and this would contradict the connectedness of $\Omega _\rho (K)$ . So all the edges in a circular neighborhood of $T$ are pointing towards $\mathcal {T}$ .

Now assume that $e$ is an edge belonging to a geodesic from a vertex to $T$ that does not point towards $\mathcal {T}$ . Then, as the last edge of that geodesic is pointing towards $\mathcal {T}$ , there is necessarily a fork along that geodesic, which means that one of the regions adjacent to $e$ is in $\Omega _\rho (K)$ , and this again contradicts the connectedness of $T_\rho (K)$ .

We can now infer directly the following property for Bowditch representations.

Proposition 3.16. Let $c \in \mathbb {C}$ , $[\rho ]$ in $\mathfrak {X}_{BQ}(F_2, \mathrm {SU}(2,1))$ and let $M = M(c)$ be the constant defined in ( 6 ). For any $K\geqslant M$ , the subgraph $T_\rho (K)$ is finite $($ in addition to being connected and $\rho$ -attracting $).$

Proof. Given such a representation $\rho$ , we have that the set $\Omega (K)$ is finite, and each $X \in \Omega (K)$ is such that $f(\mathrm {tr} (\rho (X))\gt 0$ and the neighbors around the region $X$ have exponential growth. Hence, the subgraph $\widetilde {J}_\rho (K, X)$ is finite. This proves that the subgraph $T_\rho (K)$ is a finite union of finite subgraphs and so is finite.

4. Fibonacci growth

4.1 Fibonacci functions

In this section, we give the definition of the Fibonacci function $F_v : \Omega \rightarrow \mathbb {N}$ that is associated to a vertex $v = (\alpha, \beta ) \in \mathcal {E}$ , and say what it means for a function $g \colon \thinspace \Omega \rightarrow [0, \infty )$ to have Fibonacci growth.

The map $F_v$ is constructed by induction on the distance $d_v (X)$ from a region $X \in \Omega$ to $v$ , which is the minimal graph distance in $\mathcal {E}$ from $v$ to a vertex $v' = (X,Y)$ . We have naturally $d_v (\alpha ) = 0$ and $d_v (\beta ) = 0$ . Given any region $Z \neq \alpha, \beta$ , there are precisely two regions $X,Y$ such that $X,Y,Z$ are the three regions around a triangle in $\mathcal {E}$ and $d_v (X)\lt d_v (Z)$ and $d_v (Y) \lt d_v (Z)$ . So we define

\begin{equation*}F_v (Z) = \begin{cases} 1 & \mbox {if } d_v (Z)=0, \\ F_v (X) + F_v (Y) & \mbox {if } d_v (Z) \neq 0 \text { and } X, Y \text { are as defined above.} \end{cases}\end{equation*}

Definition 4.1. Suppose $g \colon \thinspace \Omega \rightarrow [0, \infty )$ and $\Omega ' \subset \Omega$ . Let $v \in \mathcal {E}$ . We say that:

• $g$ has an upper Fibonacci bound on $\Omega '$ if there is some constant $\kappa \gt 0$ such that $g(X) \leqslant \kappa F_v (X)$ for all $X \in \Omega '$ ;

• $g$ has a lower Fibonacci bound on $\Omega '$ if there is some constant $\kappa \gt 0$ such that $g(X) \geqslant \kappa F_v (X)$ for all $X \in \Omega '$ ;

• $g$ has Fibonacci growth on $\Omega '$ if it has both upper and lower Fibonacci bound on $\Omega '$ .

The following result will be our basic tool to prove the Fibonacci growth for the logarithm of the norm of trace functions associated to representations in the Bowditch set. This result also shows that the definition of Fibonacci growth is independent of the vertex $v$ , since the definition only depends on the asymptotic growth of the function and not on the local behavior. In fact, if in the definition above we would use a different function $F_w$ defined with respect to a different vertex $w$ , then the only difference is that we might have to use different constants.

Lemma 4.2. (Corollary 2.1.2 in Bowditch [Bow98].) Suppose $g \colon \thinspace \Omega \rightarrow [0, \infty )$ satisfies an inequality of the form $g(Z) \leqslant g(X)+g(Y)+c$ $($ respectively, $g(Z) \geqslant g(X)+g(Y)+c)$ for some fixed constant $c$ , whenever the regions $X, Y, Z \in \Omega$ meet at a vertex of the trivalent tree $\mathcal {T}$ . Then $g$ has lower $($ respectively, upper $)$ Fibonacci growth.

Another important property of Fibonacci functions is explained by the following result, which relates $F_v$ to the word length of the elements of $\mathcal {S}\subset \Gamma /\!\sim$ ; see Section 2.2 for the definition of $\Gamma /\!\sim$ . In fact, we can write out explicit representatives in $\Gamma$ of elements of $\Gamma /\!\sim$ corresponding to given elements of $\mathcal {S}$ , as we described there. Let $v = (X, Y; T, T') \in \mathcal {E}^{(0)}$ . The regions $X, Y \in \mathcal {S}$ are represented, respectively, by a pair of free generators $\alpha$ and $\beta$ for $\Gamma$ . As explained in Section 2.2, the regions $T$ and $T'$ are represented by $\alpha \beta$ and $\alpha \beta ^{-1}$ , respectively, and similarly all other regions in $\Omega$ can be written as words in $\alpha ^{\pm 1}$ and $\beta ^{\pm 1}$ . Note that all the words arising in this way are cyclically reduced. Let $W(\omega )$ denote the minimal cyclically reduced word length of an element $\omega$ with respect to the generating set $\{\alpha, \beta \}$ . From this we deduce the following.

Proposition 4.3. Suppose $\{\alpha, \beta \}$ is a set of free generators for $\Gamma$ corresponding to regions $X$ and $Y$ . Let $v$ be the vertex $v =(X, Y) \in \mathcal {E}^{(0)}$ . If $\omega _Z \in \widehat \Omega$ corresponds to a region $Z\in \Omega$ , then $W(\omega _Z) = F_v(Z)$ .

4.2 Upper and lower Fibonacci growth

For any $[\rho ] \in \mathfrak {X}$ , we denote by $\phi _\rho \colon \thinspace \Omega \rightarrow \mathbb {C}$ the function $\phi _\rho (X) := \log | \mathrm {tr} (\rho (X)) |$ . We consider the function $\phi _\rho ^+ := \max \{ \phi _\rho, 0 \}$ . The purpose of this section is to prove that, if the representation $[\rho ]$ is in the Bowditch set $\mathfrak {X}_{BQ}$ , then the map $\phi _\rho ^+$ has Fibonacci growth.

First, we show that any representation in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ has upper Fibonacci bound. We will use Lemma3.2.

Proposition 4.4. For any $[\rho ]$ in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ , the function $\phi _\rho ^+$ has an upper Fibonacci bound on $\Omega$ .

Proof. Let $v$ be an arbitrary vertex in $\mathcal {E}^{(0)}$ . For all regions $Z \in \Omega$ not adjacent to $v$ , let $X, Y$ be the regions adjacent to $Z$ such that $d_u (X) \lt d_u (Z)$ and $d_u (Y) \lt d_u (Z)$ . Applying Lemma3.2 to the vertex $u = (X,Y; Z, T) \in \mathcal {E}^{(0)}$ , we obtain that $|z| \leqslant 5 |x| |y|$ and hence $\phi _\rho ^+ (Z) \leqslant \phi _\rho ^+ (X) + \phi _\rho ^+ (Y) + \log (5)$ . Using Lemma4.2, we can see that $\phi _\rho ^+$ has an upper Fibonacci bound.

The lower Fibonacci bound is not satisfied for a general representation in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ , but it will be true for representations in the Bowditch set. Recall that a vertex $v \in \mathcal {E}^{(0)}$ corresponds to an edge of the trivalent tree $\mathcal {T}$ . Hence if we remove such a vertex from $\mathcal {E}$ we split the graph into two connected components. We will refer to properties satisfied on one such component as occurring ‘on one side’. To prove this, we need the following lemma which shows that if a vertex $v$ is ‘attracting on one side’ (meaning that all the geodesics from vertices on one component of $\mathcal {E} \setminus \{v\}$ are pointing towards $v$ ), then the Fibonacci bound is satisfied for the regions adjacent to that component.

Lemma 4.5. Let $[\rho ] \in \mathfrak {X}(F_2, \mathrm {SU}(2,1))$ . Assume that $v$ is $\rho$ -attracting on one side, and all the regions adjacent to that component of $\mathcal {E} \setminus \{v\}$ have values greater than $C$ . Then there exists $s\gt 0$ such that for all vertices $p \neq v$ in that component, if $X,T,Z'$ are the three regions around $p$ such that $d(Z', v) \gt \max \{d(v,T), d(v,X)\}$ , then we have

\begin{equation*}\log |\phi _\rho ^+(Z')| \gt \log |\phi _\rho ^+(X)| + \log |\phi _\rho ^+(T)| - s.\end{equation*}

Proof. The notation is chosen so that the arrow points away from the region $Z'$ and towards the region $Z$ . We have the relation $z' = z+ \overline {xt} - x y$ . The orientation of the arrow gives that $|z'| \gt |z|$ . So we have

\begin{equation*}|z'| = |z + \overline {xt} - xy| \geqslant |x| (|t| - |y|) - |z|.\end{equation*}

Now assume that $|t| \gt |z|$ . This allows us to directly use Lemma3.2, so that

\begin{equation*}2 |z'| \gt |z'| + |z| \gt |x| \left ( \frac {C}{4} \max \{|x|,|y|\} - |y| \right ) \geqslant |x| |t| \left ( \frac {C-4}{2C} \right ).\end{equation*}

This gives $\log |z'| \gt \log |x| + \log |t| - \log \left ( \frac {C-4}{2C} \right )$ .

Note that here we have $|z'|\gt |y|$ , and hence, by induction, if for a given vertex we have $|t|\gt |z|$ , then for all subsequent vertices we also have that inequality.

Finally, we prove that all vertices, except possibly $v$ , satisfy that inequality. If $v$ is such that $|t|\lt |z|$ , then the previous lemma states that $|z| \gt \frac {C}{4} \max \{|x|, |y| \}$ . The two vertices at distance $1$ from $v$ , which corresponds to $(x,t,z')$ and $(y,t,z''),$ satisfy $|z'|\gt |y|$ and $|z''| \gt |x|$ , which is the required inequality to apply the previous reasoning to these two vertices.

We are now going to use Lemma4.5 to prove the lower Fibonacci bound of representations in $\mathfrak {X}_{BQ}$ .

Proposition 4.6. Let $[\rho ] \in \mathfrak {X}_{BQ}$ . Then the function $\phi _\rho ^+$ has lower Fibonacci bound on $\Omega$ .a

Proof. Let $[\rho ] \in \mathfrak {X}_{BQ, c}$ . We denote $M = M(c)$ and we consider the subgraph $T_\rho (M)$ , which is finite, connected and $\rho$ -attracting by Propositions3.15 and 3.16. By construction, all the vertices on the boundary of $T_\rho (M)$ satisfy the hypothesis of Lemma4.5 and hence the map $\phi _\rho ^+$ has lower Fibonacci growth on each connected component of $\mathcal {E} \setminus T_\rho (M)$ . As there are only finitely many such connected components (one for each vertex on the boundary of $T_\rho (M)$ ), we can then obtain the lower Fibonacci bound on all of $\Omega$ .

4.3 Characterizations of Bowditch set

We can now state different characterizations for the representations in the Bowditch set. Our first characterization is a slight reformulation of the definition.

Proposition 4.7. Let $[\rho ] \in \mathfrak {X}_c(F_2, \mathrm {SU}(2,1))$ . Then $[\rho ] \in \mathfrak {X}_{BQ}$ if and only if the following two conditions are satisfied:

1. (i) for all $\gamma \in \mathcal {S}$ , we have $f(\mathrm {tr} (\gamma )) \gt 0$ ;

2. (ii) for all $K \geqslant 0$ , the set $\Omega _\rho (K)$ is finite.

Proof. For the first condition, Corollary2.7 implies that an element $A \in \mathrm {SU} (2, 1)$ is loxodromic if and only if $f(\mathrm {tr}(A)) \gt 0$ .

For the second condition, we can see that if $[\rho ] \in \mathcal {X}_{BQ, c}$ , then $\Omega _\rho (M(c))$ is finite and hence $\phi _\rho ^+$ has lower Fibonacci bound. From this it follows that for all $K \geqslant M(c)$ we have that the set $\Omega _\rho (K)$ is finite, as we wanted. In addition when $0 \leqslant K \lt M(c)$ , then we have the inclusion $\Omega _\rho (K) \subset \Omega _\rho (M(c))$ .

For the next characterization, we use the fact that $\mathrm {SU}(2, 1)$ can be identified with the group of orientation-preserving isometries of the complex hyperbolic plane, and relate that to the following notion of well-displacing representations. This will be the link with the notion of Bowditch representation developed by Schlich [Sch22].

Recall that for every element $\varphi \in \mathrm {Isom}(\mathbf {H}_{\mathbb {C}}^2)$ one can consider the displacement $l(\varphi )$ , which is defined by

\begin{equation*}l(\varphi ) := \inf \{ d_{ \mathbf {H}_{\mathbb {C}}^2}(x, \phi (x)) \mid x \in \mathbf {H}_{\mathbb {C}}^2 \}.\end{equation*}

Recall also that, for any element $\gamma \in F_2$ , we denote by $W(\gamma )$ the minimal cyclically reduced word length with respect to some generating set. In these terms, we now prove the following alternative characterization.

Proposition 4.8. A representation $[\rho ]$ belongs to $\mathfrak {X}_{BQ}$ if and only if there exists $k, m \gt 0$ such that $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ for all $\gamma \in \mathcal {S}$ .

Proof. For the forward direction, we use the fact that if $[\rho ] \in \mathfrak {X}_{BQ}$ , then Proposition4.6 implies that $\phi _\rho ^+$ has lower Fibonacci bound on all of $\Omega$ , and hence that there exists $\kappa \gt 0$ such that $\phi _\rho ^+(X) \geqslant \kappa F_v (X)$ for all $X \in \Omega$ . Proposition4.3 tells us that for the appropriate $v \in \mathcal {E}^{(0)}$ we have that $F_v (\gamma ) = W(\gamma )$ for all $\gamma \in \mathcal {S}$ . We can use the fact that $\mathrm {log}^+(|\mathrm {tr}(A)|) \leqslant |l(A)|$ for all $A \in \mathrm {SU}(2, 1)$ to see that for all $\gamma \in \mathcal {S}$ we have

\begin{equation*}|l(\rho (\gamma ))| \geqslant \mathrm {log}^+(|\mathrm {tr}(\rho (\gamma ))|) \geqslant \kappa W(\gamma ).\end{equation*}

Conversely, if we have a representation $\rho$ which satisfies $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ for all $\gamma \in \mathcal {S}$ , then we will show that $\rho$ satisfies conditions $(1)$ and $(2)$ from Definition3.12. First, if we have an element $\gamma \in \mathcal {S}$ such that $f(\mathrm {tr}(\rho (\gamma ))) \leqslant 0$ , then we can find sequences of regions adjacent to $\gamma$ such that the word length increases, but the length remains bounded. Second, assume that $[\rho ] \in \mathfrak {X}_c$ . If the set $\Omega (M(c))$ is infinite, then we can see that the inequality $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ cannot be satisfied for all $\gamma \in \mathcal {S}$ , because we only have a finite amount of elements with bounded word length.

Remark 4.9. Note that the proof above shows that if $[\rho ] \in \mathfrak {X}_{BQ}$ , then there exists $k\gt 0$ such that $|l (\rho (\gamma ))| \geqslant k W(\gamma )$ for all $\gamma \in \mathcal {S}$ . In particular, we do not need the additive constant in the statement of the theorem. On the other hand, the result above is stronger for the converse implication when there is the additive constant, and this is why we are leaving the statement as it is written.

Using this, we can show that all primitive-stable representations are in the Bowditch set. This characterization corresponds to the definition given by Schlich of Bowditch representations, where she proves that Bowditch representations from the free group $F_2$ into isometry groups of Gromov hyperbolic spaces are primitive stable. So a corollary of the characterization above and of Schlich’s result is that $\mathfrak {X}_{BQ}(F_2, \mathrm {SU}(2,1)) = \mathfrak {X}_{PS}(F_2, \mathrm {SU}(2,1))$ .

Proposition 4.10. Let $c \in \mathbb {C}$ and $K \geqslant M(c)$ , and let $[\rho ] \in \mathfrak {X}_c$ . We have $[\rho ]$ is in $\mathfrak {X}_{BQ}$ if and only if the subgraph $T_\rho (K)$ is finite.

Proof. The forward implication follows from Proposition3.16, so we only need to prove that if $T_\rho (K)$ is finite for $K \geqslant M(c)$ , then $[\rho ] \in \mathfrak {X}_{BQ}$ . We prove the contrapositive. Suppose $[\rho ] \notin \mathfrak {X}_{BQ}$ , so then we have two cases.

The first case is that there exists $X \in \Omega$ such that $\rho (\gamma )$ is not loxodromic (or equivalently that $f(\mathrm {tr}(\rho (\gamma ))) \leqslant 0$ ). In this case the arc $\widetilde {J}(K, X)$ (and hence the subgraph $T_\rho (K)$ ) is infinite.

The second case is that the set $\Omega (K)$ is infinite, so we can see again that the subgraph $T_\rho (K)$ is infinite because for each $X \in \Omega (K)$ the arc $\widetilde {J}(K, X)$ is non-empty.

Having addressed both cases, we have proved the proposition.

In the final part of this section we will show why the Bowditch set $\mathfrak {X}_{BQ}$ strictly contains the set $\mathfrak {X}_{CC}$ of convex cocompact representations in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ . Recall that convex cocompact representations can be defined in many equivalent ways, especially since $\mathrm {SU}(2,1)$ is a rank $1$ Lie group. For example we say that a representation $[\rho ]\in \mathfrak {X}(F_2, \mathrm {SU}(2,1))$ is convex cocompact if there exists $k, m \gt 0$ such that $|l (\rho (\gamma ))| \geqslant k W(\gamma ) - m$ for all $\gamma \in F_2$ (or, equivalently, if the (any) orbit map $\tau _\rho$ is a quasi-isometric embedding). We can see from the definition that $\mathfrak {X}_{CC} \subset \mathfrak {X}_{BQ}.$ We now want to show that this is a proper inclusion. We will use a result of Will [Wil07, Wil12] that describes a three-dimensional family of discrete, faithful and type-preserving representations of the once-punctured torus in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ . These representations will take simple closed curves to loxodromic elements and will map the commutator to a parabolic unipotent element. Let $F_2:=\langle \alpha, \beta \rangle$ . Since simple closed curves in $S_{1,1}$ correspond to primitive elements of $F_2$ , the representations described by Will are in the Bowditch set. One can see this using one of our characterizations of $\mathfrak {X}_{BQ}$ , or the equivalence of $\mathfrak {X}_{BQ}$ with the set of primitive stable representations in $\mathfrak {X}(F_2, \mathrm {SU}(2,1))$ implied by Schlich [Sch22]. Remember that the peripheral element corresponds to the commutator, which is not a primitive element. The fact that the commutator $[\alpha, \beta ]$ is mapped to a unipotent element by Will’s representations prevents them from being convex cocompact representations. If one would like to see other examples of representations that are in $\mathfrak {X}_{BQ} \setminus \mathfrak {X}_{CC}$ and such that the peripheral element is mapped to a parabolic non-unipotent element, one can use the work of Falbel and Parker [FP03], Falbel and Koseleff [FK02b, FK02a] or Parker and Gusevskii [GP03, GP00]. See also Will [Wil16] for a discussion of these references. These examples imply the following result.

Proposition 4.11. The set $\mathfrak {X}_{BQ}$ strictly contains the set of convex cocompact representations $\mathfrak {X}_{CC}$ .

4.4 Openness and proper discontinuity

In this section we prove that the set $\mathfrak {X}_{BQ}$ is open and the group $\mathrm {Out}(F_2)$ acts properly discontinuously on it. First we prove the following lemma which will imply the openness of $\mathfrak {X}_{BQ, c}$ .

Lemma 4.12. Let $c \in C$ and let $K \geqslant M(c)$ . For each $[\rho ] \in \mathfrak {X}_{BQ, c}$ , if $T_\rho (K)$ is non-empty, there exists a neighborhood $U_\rho$ of $[\rho ]$ in $\mathfrak {X}_c$ such that, for all $[\sigma ] \in U_\rho$ , the subgraph $T_\sigma (K)$ is contained in $T_\rho (K)$ .

Proof. Let $[\rho ] \in \mathfrak {X}_{BQ, c}$ and let $K \gt M(c)$ such that $T_\rho (K)$ is non-empty. First, it is easy to see that, for $\sigma$ close enough to $\rho$ , the subgraphs $T_\sigma (K)$ and $T_\rho (K)$ have non-empty intersection.

Now consider the set $V \subset \mathcal {E}^{(0)}$ of vertices of $\mathcal {E}$ which are on the boundary of $T_\rho (K)$ . Let $v \in V$ . We can show that if $[\sigma ] \in \mathfrak {X}_c$ is close enough to $[\rho ]$ , then $v$ is also not in the interior of $T_\sigma (K)$ . From the definition of $T_\rho (K)$ we can see that if $v = (X, X')$ , then we have $| \mathrm {tr}(\rho (X))| \gt K$ or $| \mathrm {tr} (\rho (X'))| \gt K$ . If we choose $\sigma$ close enough to $\rho$ , then we can show that the same inequality will be true for $\sigma$ as well. Since the subgraph $T_\rho (K)$ is finite, there is only a finite number of vertices in $V$ , so we can choose $\sigma$ close enough to $\rho$ such that any vertex $v \in V$ is not on the interior of $T_\sigma (K)$ .

Now, since $T_\sigma (K)$ is connected and $T_\sigma (K) \cap T_\rho (K)$ is non-empty, we have that the subgraph $T_\sigma (K)$ is contained in $T_\rho (K)$ .

The last result of this paper proves proper discontinuity for the action of $\mathrm {Out}(F_2)$ on $\mathfrak {X}_{BQ, c}$ .

Theorem 4.13. The set $\mathfrak {X}_{BQ, c}$ is open in $\mathfrak {X}_c$ and the action of $\mathrm {Out}(F_2)$ is properly discontinuous on $\mathfrak {X}_{BQ, c}$ .

Proof. The previous lemma directly implies openness, using the characterization of representations in $\mathfrak {X}_{BQ, c}$ given by Proposition4.10.

In order to prove that the action of $\mathrm {Out}(F_2)$ is properly discontinuous, let $\mathsf {C}$ be a compact subset of $\mathfrak {X}_{BQ, c}$ . We want to prove that the set

\begin{equation*}\Gamma _{\mathsf {C}} = \{ g \in \mathrm {Out}(F_2) \ | \ g \mathsf {C} \cap \mathsf {C} \neq \emptyset \} \end{equation*}

is finite.

Let $K \geqslant M(c)$ be such that, for any $[\rho ] \in \mathsf {C}$ , the tree $T_\rho (K)$ is non-empty. Now around each element of $\mathsf {C}$ there exists a neighborhood $U_\rho$ given by the previous lemma. So we have that the set $(U_{\sigma })_{[\sigma ] \in \mathsf {C}}$ is an open covering of $\mathsf {C}$ . Using the compactness of $\mathsf {C}$ , we take a finite subcover $(U_{\rho _i})_{i\in I}$ , where $I$ is a finite set.

Now, for each element $\rho _i$ , we take the tree $T_{\rho _i} (K)$ and consider the union

\begin{equation*}\mathcal {T}(K) = \bigcup _{i \in I} T_{\rho _i} (K).\end{equation*}

By construction, for each element $[\sigma ] \in \mathsf {C}$ , the subgraph $T_\sigma (K)$ is contained in $\mathcal {T}(K)$ . Since the subgraph $\mathcal {T}(K)$ is a finite union of finite subgraphs, it is itself finite. It follows that the set

\begin{equation*}\Gamma _1 = \{ g \in \Gamma \ | \ g \mathcal {T}(K) \cap \mathcal {T}(K) \neq \emptyset \} \end{equation*}

is finite. Finally, as $T_{g\sigma } (K) = g T_\sigma (K)$ , we have that $\Gamma _{\mathsf {C}} \subset \Gamma _1$ , and hence $\Gamma _{\mathsf {C}}$ is finite, which proves that $\mathrm {Out} (F_2)$ acts on $\mathfrak {X}_{BQ, c}$ properly discontinuously.