Cenozoic climate change and the evolution of North American mammalian predator ecomorphology

Abstract

The trend of global cooling across the Cenozoic transformed the North American landscape from closed forest to more open grasslands, resulting in dietary adaptations in herbivores in response to shifting resources. In contrast, the material properties of the predator food source (muscle, skin, and bone) have remained constant over this transition, suggesting a corresponding lack of change in predator dietary adaptations. We investigated the North American mammalian predator fossil record using a tooth-shape metric and body mass, predicting that the former would exhibit stability. Instead, we found that mean molar morphology became more blade-like, with our tooth-shape metric sharply increasing in the late Eocene and remaining high from the Oligocene onward. Subsequent tests in extant carnivorans reveal taxa with more bladelike teeth are prevalent in more open environments. Our results reveal an unexpected functional shift among North American predators in response to large-scale environmental changes across the Cenozoic.

Non-technical Summary

Understanding how predators respond to climate change is crucial given their disproportionate impact on ecosystems from their place atop the food chain. Past studies, which relied on simply keeping track of the number of species though time, concluded that predators were largely unaffected by climate change. However, our research challenges this notion by examining how tooth shape changed in North American mammalian predators over the Cenozoic (65 million years ago to the present). In our study, we collected and analyzed body-mass and tooth-shape data for both living and fossil mammalian predators. From our analysis on living mammalian predators, we found those with bladelike molars were more commonly found in open habitats. When we applied this result to our fossil mammalian predators, we found evidence for a similar relationship occurring in the past. Throughout the Cenozoic, the North America landscape transitioned from tropical forests to a patchwork of both larger open areas and smaller forests as the climate began to cool. Our study found that in response to this transition, fossil predator molars became more blade-like, demonstrating a response to a changing habitat. This discovery challenges the traditional belief that predators remained unaffected by climate change in the past and highlights the importance of accurately understanding historical responses to habitat change in predators.

Introduction

The North American mammalian fossil record has provided insights into community-level responses to climate and habitat changes over the past 65 Myr (Janis et al. 2000; Figueirido et al. 2012, 2019; Jardine et al. 2012; Samuels and Hopkins 2017). Sharp punctuations of global warming and cooling events characterized the early and mid-Cenozoic (Zachos et al. 2001, 2008; Retallack 2007; Westerhold et al. 2020), while climatic changes in the mid- and late Cenozoic transformed the North American landscape from forested Eocene habitats to open Miocene grasslands (Graham 1999; Edwards et al. 2010; Strömberg 2011). Over this environmental transition, ungulates evolved higher crowned teeth and increased mean body mass as adaptive responses to these new niches (Alroy 1998; Mihlbachler et al. 2011; Jardine et al. 2012). Similar parallel evolutionary trajectories are apparent in dental crown height among rodents and lagomorphs (Jardine et al. 2012; Tapaltsyan et al. 2015; Samuels and Hopkins 2017). The common trajectories of large and small herbivores demonstrate the connection between macroevolutionary change among herbivores and vegetation shifts across long timescales (Blois and Hadly 2009); however, similar explorations of dietary (dental) evolution within predator communities are lacking.

The strength of climate and habitat change as drivers of both adaptation and diversification appears to be less pronounced in carnivores than herbivores as evidenced from the fossil record (Werdelin and Lewis 2005; Palombo et al. 2009; Liow and Finarelli 2014; Finarelli and Liow 2016; Balisi and Van Valkenburgh 2020). A potential explanation for this decoupling is the static nature of the predator food resource throughout the Cenozoic (Van Valkenburgh 1999). The physical properties of the diet of meat eaters (skin, flesh, and bone) have not changed significantly over time (Van Valkenburgh 1995). In contrast, as forests gave way to more open habitats (including grasslands), tougher vegetation (e.g., grasses) became more widespread, exogenous grit became more prevalent, and grazing herbivores diversified as browsers declined (Janis et al. 2000).

The fossil mammalian predator record provides an avenue to test whether evolutionary responses to climate change can be generalized across trophic levels. Previous studies on North American predator evolution have focused on relatively narrow time windows (Van Valkenburgh 1988; Friscia and Van Valkenburgh 2010; Christison et al. 2022) or restricted taxonomic groups (Van Valkenburgh et al. 2004; Figueirido et al. 2015), have not incorporated ecomorphological analysis (Van Valkenburgh and Janis 1993; Hunt 2004), and/or have relied on summary measures of morphological disparity (Wesley-Hunt 2005). These studies do not directly address climate or environmental adaptations similar to increasing crown height in ungulates and glires (Mihlbachler et al. 2011; Samuels and Hopkins 2017). To assess whether predators display dental adaptations to habitat change in a manner similar to herbivores, additional data beyond species richness and disparity metrics are needed: namely, functional traits (Eronen et al. 2010).

In this study, we investigate whether broad patterns of community structure in North American mammalian predators (“Creodonta” and Carnivora) shift in response to climatic change through the lens of two functional traits: body mass and the lower first molar (m1) relative blade length (RBL). Due to the potential resource stability of the physical properties of skin, muscle, and bone through geologic time, we hypothesize that the trajectory of a dietary trait (e.g., dental morphology) would remain constant and unaffected by Cenozoic climatic change. In contrast, traits under multiple selective pressures, one of which is diet (e.g., body size), might respond to habitat opening over the Cenozoic, mirroring similar patterns in herbivore body size evolution. Associations between our functional traits (body mass and RBL) and diet and habitat type, respectively, are assessed using extant taxa to confirm the functional significance of our chosen traits. We document long-term trends in predator RBL and body mass over the Cenozoic and specifically test for evidence of predator immediate response to six major Cenozoic climatic events: (1) the Paleocene–Eocene thermal maximum (PETM); (2) the early Eocene climatic optimum (EECO); (3) the middle Eocene climatic optimum (MECO); (4) the Eocene/Oligocene transition (EOT); (5) the middle Miocene climatic optimum (MMCO); and (6) the Northern Hemisphere glaciation (NHG).

Materials and Methods

Fossil Occurrences and Ages

Occurrences for “Creodonta” and Carnivora found in the contiguous United States were downloaded from the New and Old Worlds (NOW) and MIOMAP fossil mammal databases (Carrasco et al. 2007; NOW Community 2020). We use the informal terms “Creodonta” and “creodont” to refer to those non-carnivoran carnivores in the groups Hyaenodonta and Oxyaenida. We used the taxonomic names matched with each occurrence from NOW and MIOMAP and standardized them through the Paleobiology Database (Peters and McClennen 2016) and the literature to generate a species-level taxon list (Supplementary File 1). In the NOW fossil database, localities are assigned to individual biochrons, whereas MIOMAP localities are assigned maximum and minimum absolute ages estimated through various biostratigraphic or radiometric dating methods. In order to utilize both databases, we assigned each fossil locality to one of the North American Land Mammal Ages (NALMA) subdivisions (Gunnell et al. 2008). All taxa were assigned as present or absent within each NALMA subdivision, and taxa are assumed to have ranged throughout an entire subdivision. By binning fossil occurrences using the NALMA subdivision, we can account for repeated entries from both NOW and MIOMAP.

To assess potential sampling biases, we performed two analyses. First, we performed a linear regression for the number of predator species versus the number of localities sampled at each NALMA subdivision from the NOW database. We then regressed the residuals of the linear regression with time. These residuals, if uncorrelated with time, have been previously found to be a good proxy for sampling-corrected relative richness (Samuels and Hopkins 2017). Second, to assess whether each NALMA subdivision displays similar sampling behavior, we repeated the first two analyses with terrestrial North American ungulates as a comparable mammalian group with a similar body-mass range and therefore similar preservation potential. We then performed a linear regression on the residuals from the predator analysis versus the ungulate analysis to see whether sampling behavior is constant for each NALMA subdivision for all taxonomic groups.

Morphological Data

We selected two functional traits for our study: body mass and RBL. Body mass is commonly chosen as a functional trait due to its importance in multiple aspects of mammalian ecology, including but not solely limited to diet (Smith and Lyons 2011). In mammalian predators, body mass correlates with prey size, home-range size, and life-history traits (Gittleman and Harvey 1982; Gittleman 1986; Carbone et al. 1999). We used previously published body-mass estimates of extant carnivorans (Slater and Friscia 2019), and we estimated fossil predator body mass using the regression equations for the first lower molar for Carnivora and the largest lower molar for the “Creodonta” (Van Valkenburgh 1990). We also supplemented our dataset with previously calculated body-mass estimates where they applied (Supplementary File 1). For all subsequent analyses, body mass was log transformed using base 10.

RBL was estimated as the ratio of the mesiodistal length of the trigonid to the maximum mesiodistal length of the lower first molar for the order Carnivora (Supplementary Fig. 1) and the largest lower molar for the “Creodonta,” respectively (Van Valkenburgh 1989; Friscia and Van Valkenburgh 2010). This ratio captures the relative proportion of the molar dedicated to slicing rather than grinding and has been found to correlate well with dietary mode in extant carnivorans (Van Valkenburgh and Koepfli 1993; Sacco and Valkenburgh 2004; Friscia et al. 2007; Meachen-Samuels and Van Valkenburgh 2009). Taxa with higher RBL values have diets with a greater proportion of meat content, and those with lower RBL values tend to have a more omnivorous diet (Van Valkenburgh 1991). In total, we collected RBL and body-mass estimates for 394 terrestrial fossil North American predators and 182 extant carnivorans, with 21 overlapping species (see Supplementary File 1).

Functional Significance of Body Mass and RBL

To test for association between our functional traits and habitat and diet, we assigned extant carnivoran species to one of three dietary modes (hypercarnivore, mesocarnivore, or hypocarnivore) and one of three habitat types (closed, mixed, or open) based on the literature (species list and designations found in Supplementary File 1). Analyses of variance (ANOVAs) and phylogenetic ANOVAs were run on body mass for both dietary mode and habitat type to explore overall differences between categories using the R package geiger (Pennell et al. 2014). As RBL is a ratio, we used the nonparametric Kruskal-Wallis and Games-Howell tests to compare RBL values between both groupings. The RBL for all extant members of the family Felidae is 1.0 due to the lack of a talonid. Therefore, we repeated our comparisons of RBL with felids excluded to remove any bias due to their inclusion.

Analysis of Taxonomic and Morphological Shifts

To investigate whether shifts in predator body mass and RBL distributions occurred at major climatic events, we used the Handley likelihood approach (Handley et al. 2009) to identify breakpoints between NALMA subdivisions that reflect significant taxonomic and morphological shifts. Because the Handley method requires discrete distributions, we assigned species to body size and RBL categories. Body mass was divided into five categories (0–1 kg, >1–7 kg, >7–21 kg, >21–100 kg, >100 kg) that reflect differences in predator energetic costs and behaviors documented in extant carnivorans (Carbone et al. 1999, 2007). For RBL, we placed each species into one of six bins each spanning a length of 0.1 from an RBL of 0.4 (the minimum RBL observed) to 1 (the maximum possible).

However, shifts in functional traits distribution over time could simply be due to taxonomic replacement and not a functional response to an environmental change. If phylogenetic constraint is strong, a turnover event could result in a concurrent shift in functional trait distributions. Extant mammalian predators exhibit relatively high phylogenetic constraint, in which various functional traits display an early burst of adaptive evolution (Slater and Friscia 2019). Nevertheless, fossil predators display multiple examples of iterative evolution of specific dietary ecomorphs (such as hypercarnivores and bone-crackers) within various families, indicative of potential evolutionary flexibility in mammalian predators (Van Valkenburgh 2007). To account for this confounding variable, we also tested for shifts in taxonomic composition. Here, each family represents a category, with the category counts being the number of species found within each family. By testing for shifts in both functional trait distribution and taxonomic composition, we can determine whether an observed shift is likely to represent a functional response to potential environmental change within distinct clades or is driven by turnover.

Following the Handley likelihood approach (Handley et al. 2009), we assumed that these category counts from each NALMA subdivision are drawn from a multinomial distribution, where each category count has a relative abundance value in each of the distributions. We then tested whether two NALMA subdivisions are drawn from the same underlying distribution or were drawn from two distinct distributions, with both outcomes having an associated likelihood. For example, if two NALMA subdivisions contained a different number of species but had the same frequency of category counts once the species were assigned to their given categories, then the higher-likelihood outcome is one where they share the same multinomial distribution. Because the samples are ordered in time, only samples from adjacent NALMA subdivisions can share a common distribution. Each grouping of adjacent NALMA subdivisions found to share a common global distribution is referred to as a “regime,” and the points between regimes are referred to as “regime shift.” We constructed candidate models wherein each model varies in the number and placement of regime shifts. We then ranked each model using its Akaike information criterion (AIC). Using this model-ranking approach allowed us to find the optimal regime configuration across all NALMA subdivisions for the given category counts. The Handley likelihood approach found the best-fit model with corresponding regime configurations for our three community indices: taxonomic composition, body mass, and RBL.

Finally, we assessed whether any of the regime shifts from the Handley likelihood approach correspond to major Cenozoic climatic events to determine whether predators demonstrate rapid responses to dramatic habitat change. We considered six major climatic events (Raymo 1994; Zachos et al. 2008; Westerhold et al. 2020): the PETM at 56 Ma, the EECO from 53 to 51 Ma, the MECO from 40.5 to 40.1 Ma, the EOT at 33.7 Ma, the MMCO from 17 to 14 Ma, and the onset of the NHG from 2.7 to 2.5 Ma. The PETM, EECO, MECO, and MMCO all represent global warming events, while the EOT and the NHG represents global cooling events. All analyses were performed in R (R Core Team 2013).

Results

Evolutionary Trends in Predator Species Richness, Body Mass, and RBL

We found significant correlations between richness and the number of localities sampled for both predators (r = 0.705, F = 32.6, p < 0.0001) and ungulates (r = 0.709, F = 31.32, p < 0.0001) (Supplementary Fig. 2). Residuals from those regressions did not significantly correlate with time for both predators (r = 0.0837, F = 0.233, p = 0.632) and ungulates (r = 0.145, F = 0.663, p = 0.422), indicating that they are good proxies for relative predator or ungulate richness. A further linear regression demonstrates predator residuals are significantly correlated with predator richness (r = 0.709, F = 33.4, p < 0.0001), indicating that sampling-corrected relative predator richness mirrors predator richness. The significant correlation holds after accounting for temporal autocorrelation by regressing first differences of both predator richness and predator residuals (r = 0.680, F = 27.64, p < 0.0001). There was also no significant correlation between predator and ungulate regression residuals (r = 0.00745, F = 0.00172, p = 0.967). The pattern of predator richness over time more likely indicates true changes in predator diversity across the Cenozoic rather than variation in sampling intensity in each NALMA subdivision. Therefore, the close match of sampling-corrected predator richness and raw predator richness allows for direct analysis of raw predator richness.

Predator richness at the onset of the Cenozoic in North America began low, with six species all in the family Viverravidae, and monotonically increased in the Paleocene (Fig. 1A). In the early Eocene, richness rose to 37 species (four families) in the early Wasatchian, and then subsequently sharply decreased across the middle and late Eocene to a minimum of five species (three families) in the early Uintan. Predator richness then rose steadily again in the Oligocene, with a peak richness of 63 species (six families) in the late early Arikareean. After this peak, predator richness dropped across the early Miocene to 31 species (five families) in the late Hemingfordian before increasing again in the middle and late Miocene. A third noticeable drop in richness began in the late Blancan and persists to the present, with predators reaching a minimum species richness of 21 (seven families) in the early Irvingtonian. There is no consistent relationship between predator richness and major Cenozoic climatic events. Across the four global warming events (Zachos et al. 2001), the PETM and MMCO show an increase in richness, while predator richness decreases across the EECO and MECO.

Figure 1. Plot of predator richness (A), body mass (B), and relative blade length (RBL) (C) across the Cenozoic. Gray lines indicate species durations. The black line represents species richness, and the brown line represents family richness within each North American Land Mammal Ages (NALMA) subdivision for A. For B and C, the solid line represents mean log-transformed body mass and RBL, respectively, within each NALMA subdivision, and the dashed line represents median log-transformed body mass and RBL, respectively, within each NALMA subdivision. Global warming events are indicated in red, and global cooling events in blue: PETM, Paleocene–Eocene thermal maximum; EECO, early Eocene climatic optimum; MECO, middle Eocene climatic optimum; EOT, Eocene/Oligocene transition; MMCO, middle Miocene climatic optimum; and NHG, Northern Hemisphere glaciation. Median RBL is 1.00 in both the Duchesnean and early Chadronian due to the small number of species present (n = 6), four of which are hyaenodontids with RBL values of 1.00.

Mean body size increased steadily from the Paleocene to the late Eocene, with the maximum mean body mass occurring in the early Chadronian (Fig. 1B). After the Chadronian, mean body mass began to plateau at the Eocene/Oligocene boundary and remained stable to the present. Despite the stability of mean body mass in the Miocene, maximum predator body mass continued to rise, contributing to a sustained increase in both body-mass range and variance (Supplementary Fig. 3). The maximum range of body mass occurred in the early Barstovian, when both the diminutive procyonid Bassariscus minimus and the massive amphicyonid Amphicyon ingens were present. Similarly, variance in body mass expanded after the Eocene/Oligocene boundary and reached its maximum at the present (Supplementary Fig. 3). There are no obvious declines or increases in mean or maximum body mass associated with the six climatic events.

Mean RBL increased quickly across the Eocene, reaching a peak in the early Chadronian with the appearance of four hyaenodontids with RBL values of 1.00 (Fig. 1C). After the peak in RBL in the early Chadronian, mean RBL never returned to the lower values (<0.646) measured in the Paleocene and early Eocene and instead appears to hover around 0.700. The one exception is a slight drop in mean RBL values in the latest Oligocene through the early Miocene. Although mean RBL after the Eocene/Oligocene boundary remained steady, the variance in RBL increased over the next 34 Myr (Supplementary Fig. 4). Unlike body mass variance, variance in RBL contracted twice across the Cenozoic. The first drop occurred at the early Chadronian as a result of the loss of relatively low RBL species, such as basal amphicyonid Angelarctocyon australis (Fig. 1C). The second drop in RBL variance occurred in the late Arikareean with the extirpation of the high-RBL members of the feliform family Nimravidae (Supplementary Fig. 3). Apart from an apparent upward shift in RBL coincident with the MECO, there is no generalizable association between declines or increases in mean or maximum RBL and climatic events. Unfortunately, there are relatively few predator species known from the time before and after the MECO (late Uintan–Duchesnean), so the marked upward shift here in RBL may reflect preservation bias.

Functional Significance of Body Mass and RBL Based on Extant Carnivorans

Log-transformed body mass in extant carnivorans (Fig. 2) did not differ significantly among dietary categories (ANOVA; F(2, 181) = 0.664, p = 0.516) or among habitat types (ANOVA; F(2, 181) = 2.965, p = 0.0533). After phylogenetic ANOVAs are computed, the results hold for both dietary categories (F(2, 181) = 1.257, p = 0.287) and habitat type (F(2, 181) = 1.917, p = 0.15).

Figure 2. Box plots of functional traits body mass and relative blade length (RBL) compared against diet (A, C) and habitat type (B, D), with felids removed. Post hoc Games-Howell test performed to provide pairwise combinations for RBL for diet and habitat after significance was found in both groups mean differences.

A Kolmogorov-Smirnov test reveals that RBL significantly differs from normality (K-S test; p = 0.00011, contrasted with log-transformed mean body mass (K-S test; p = 0.05). Mean RBL differed significantly among both dietary categories (Kruskal-Wallis test; χ²(2, 181) = 84.667, p < 2.2*10⁻¹⁶) and habitat type (Kruskal-Wallis test; χ²(2, 181) = 14.352, p = 0.000764). Hypercarnivores tended to have larger RBL relative to either mesocarnivores or hypocarnivores, and mesocarnivores had a significantly higher mean RBL value compared with hypocarnivores, in agreement with prior work on the correlation of blade length to diet (Van Valkenburgh and Koepfli 1993; Sacco and Valkenburgh 2004; Friscia et al. 2007; Meachen-Samuels and Van Valkenburgh 2009). Mean RBL was significantly higher in open-habitat species than mixed- or closed-habitat species, and this remained true after removing the Felidae, all of which have an RBL of 1.0. As a nonparametric phylogenetic ANOVA is not available, we removed the Felidae and found the resulting distribution does not significantly differ from normality (K-S test, p = 0.714). Computing phylogenetic ANOVAs for this new RBL distribution finds that the mean RBL differs significantly between both dietary categories (F(2, 148), p = 0.0099) and habitat type (F(2, 148), p = 0.0198).

Subsequent nonparametric Games-Howell post hoc tests for RBL confirm pairwise group means differences, with significant differences (p < 0.01) between all pairwise dietary categories, even after removing the Felidae (Fig. 2C). The Games-Howell test finds a significant difference in RBL only between open and closed habitats (p = 0.01). Removing the Felidae results in significantly higher mean RBL when comparing open with mixed habitats as well as open with closed habitats (p < 0.001) (Fig. 2D).

Predator Regime Shifts

A focus on mean body mass and RBL can overlook potential shifts in the shape of the underlying distribution over time. To assess whether predator functional trait distributions responded to rapid habitat change, we applied the Handley likelihood approach to determine whether any shifts in predator regimes coincided with major climatic events across the Cenozoic (Handley et al. 2009). This likelihood approach also allows us to account for taxonomic turnover as a potential driver of predator trait response. Our best-performing models identified seven shifts (“regime shifts”) in both taxonomic composition and RBL distribution, respectively, and six regime shifts in body-mass distribution for predator communities (Fig. 3). Two of the seven regime shifts in taxonomic composition corresponded with climatic events, the PETM and MECO (Fig. 3A). Both body-mass and RBL regimes have a single regime shift coinciding with a climatic event, the MMCO and EECO, respectively (Fig. 3B,C). All corresponding climatic events are global warming events, with no regime shifts occurring at the two global cooling events, the EOT and NHG. More detailed information on the nature of the regime shifts (changes in distribution of taxa, body mass, and RBL values) can be found in Supplementary Table 1.

Figure 3. Plot of taxonomic composition and functional trait distribution of predators across the Cenozoic. A–C, Predator abundance in each taxonomic or functional trait category. Dashed lines indicate regime shifts determined by the best-fit regime change model using Akaike information criterion (AIC).

Shared Regime Shifts

If concurrent shifts occur in both taxonomic composition and a functional trait (body mass or RBL), then the regime shift for the trait regime may reflect taxonomic replacement and not a functional response to an environmental change. As phylogenetic constraint is strong within living carnivorans (Slater and Friscia 2019), a taxonomic turnover event could result in a concurrent shift in trait regime. Based on the Handley likelihood approach, there are only four regime shifts shared across two or three of our variables (taxonomic composition, body mass, and RBL). Additionally, only two of these shared regime shifts involved taxonomic composition.

The earliest shared regime shift is between body-mass and RBL distributions and occurred at 45.38 Ma within the Uintan (Fig. 3B,C). It reflects an increase in the proportion of species with larger body mass and higher RBL values. The second shared regime shift at 32.10 Ma (Orellan/Whitneyan transition) is associated with substantial diversification of the Canidae alongside a decline in species with RBL values >0.90. Taxonomic composition, RBL, and body-mass distributions all share a regime shift at 22.20 Ma within the Arikareean that is driven by the extinction of the catlike forms (Nimravidae) and thus species with RBL values >0.80, alongside the removal of predators larger than 100 kg. This regime shift coincides with the beginning of the North American “cat gap,” a period within the Miocene characterized by the absence of catlike forms (Van Valkenburgh 1999). The final shared regime shift marks the end of the cat gap at 17.34 Ma in the Hemingfordian, when felids first appear in North America and RBL values increase.

Discussion

We hypothesized that unlike herbivores, predators would not evolve novel dental adaptations in response to Cenozoic global cooling or warming, largely because the material properties of their primary foods (skin, flesh, and bones) would be unaffected by climate change (Van Valkenburgh 1995). We tested this hypothesis on North American fossil predators (Carnivora and “Creodonta”) using a single dental measurement, m1 RBL, and estimated body mass, as the latter might vary with climate and perhaps diet. To our surprise, we uncovered a substantial upward shift in mean RBL in the late Eocene, during a time of transition from relatively closed, continuous tropical forest habitats to more complex habitats consisting of smaller forests separated by larger open areas (Tomiya et al. 2021). After its sharp rise in the late Eocene, RBL then declines to a relatively steady mean value that remains above that observed in the Paleocene and early Eocene predators (Fig. 1C).

To explore possible causes of the upward shift in RBL values in the late Eocene as well as the sustained increased mean RBL from the Oligocene onward, we examined the association between RBL values and habitat in a sample of extant carnivorans and found species assigned to open habitats tended to have higher RBL values than those assigned to mixed or closed habitats, barring felids (Fig. 2). The reasons for this association are not entirely clear, but it might reflect a difference in food resources in open versus closed habitats. Relative to drier, more open habitats, forests exhibit greater vegetation structure, a known determinant of species diversity (Tews et al. 2004). The greater diversity of available resources (e.g., insects and fruits) would favor the retention of a talonid basin on the lower carnassial for crushing (Friscia et al. 2007). Conversely, the talonid reduces as dietary breadth narrows, resulting in more efficient processing of energetically favorable meat (Van Valkenburgh and Koepfli 1993; Carbone et al. 1999; Friscia et al. 2007). Although more data on the difference in availability of food types in open versus closed habitats are needed, it is intriguing that the marked rise in mean RBL might be related to a shift in habitat openness.

Recovering exact associations between habitat type and functional traits is challenging in predators (Polly 2010). Therefore, our finding that predator RBL reflects habitat type indicates another potential candidate trait for habitat reconstruction. Previous candidates include cranial shape (Tseng and Flynn 2018), calcaneal gear ratio (Polly 2010, 2020), and hunting behavior (Van Valkenburgh 1985), all which require complete skulls or postcranial elements. Reliable and easily measured proxies for ecology are vital for understanding the potential niches of fossil taxa, especially in vertebrate groups, which often suffer from poor and incomplete preservation (Eronen et al. 2010; Fraser et al. 2021). Compared with these skeletal traits, RBL is easier to measure, requiring only a lower first molar with a preserved trigonid, and provides similar information regarding environmental partitioning within mammalian predators.

Habitat reconstruction within the late Eocene is challenging due to the sparseness of floral assemblages, especially in comparison to the early and middle Eocene. In addition, late Eocene floras often exhibit non-analogous morphologies, making comparisons to modern counterparts difficult (DeVore and Pigg 2010). Alternative evidence such as the decline in arboreal mammal richness in general and primate richness in particular supports a loss of closed-canopy forest (Gingerich 1979; Townsend et al. 2010), while paleosol evidence suggests a more mosaic pattern in the late Eocene (Retallack 2007). We suggest that the rapid rise in predator mean RBL represents a predator functional response to habitat restructuring. The family driving this RBL increase is the Hyaenodontidae, in which all members have an RBL >0.70 and that makes up more than half the predators present in the Uintan (Fig. 3) The increase in mean RBL is not simply due to family turnover, as our best-fit models find shifts in taxonomy regimes at different times within the Uintan compared with the regime shifts for RBL and body-mass regimes. However, potential collection biases may obscure the true pattern within the middle Eocene; for instance, numerous fossils collected from the Washakie Formation (spanning the Bridgerian and Uintan) are still undescribed (Tomiya et al. 2021). In the Oligocene and Miocene, where the phytolith record improves, phytolith data indicate the spread of open-habitat grasses during the late Oligocene and Miocene (Strömberg 2004), an observation consistent with the both the paleosol record (Retallack 2007) and the eventual rise of hypsodonty in ungulates (Strömberg 2002). The consistently elevated mean RBL for North American predators from the late Eocene onward provides evidence for a sustained functional response in predators that tracked the shift from a forested greenhouse world to one characterized by the predominance of more open habitats.

Similar to RBL, mean predator body mass peaks in the late Eocene (Fig. 1B). However, maximum predator body masses arise in the Miocene, long after the Eocene appearance of maximum RBL values (i.e., 1.0) (Fig 1C). Within the late Eocene, we also find the disappearance of predators <1 kg, with a continued low abundance of small predators in the Oligocene (Fig. 3C). It is not clear why small predators are so rare during this interval. After the Oligocene, body-mass variance and range increases (Supplementary Fig. 3), indicating a potential expansion into various niches in the Miocene, when smaller-bodied forms return. A general increase in both range and variance in the Miocene also occurs in RBL, except during the Miocene cat gap (Van Valkenburgh 1999), in which range and variance for both RBL and body mass decrease (Supplementary Figs. 3, 4).

Our models found a general lack of coincidence in the timing of functional trait regimes and major climatic events. As mean RBL appears to be associated with habitat openness, the documented shifts in RBL regimes further suggest that dietary morphology in mammalian predators was impacted by the changes in vegetation and habitat structure. However, none of the predator functional regimes were bounded by major climatic events, suggesting that predators respond to climatic shifts over longer timescales, in contrast with mammalian herbivores, which tend to respond more rapidly, as exemplified by the immediate decline in burrowing and arboreal rodents following the EOT (Samuels and Hopkins 2017). Previous work on fossil canids corroborates our finding, where shifts in locomotor behaviors tracked the gradual spread of open habitats rather than abrupt climatic events (Figueirido et al. 2015). Alternatively, it is possible that local adaptations to regional and temporal environmental fluctuations are obscured when viewing taxonomic and functional patterns of predator evolution on a continental scale. Regional surveys of paleovegetation reveal a diversity of local habitats and vegetational histories despite the occurrence of global climatic events and environmental perturbations. For example, the paleobotanical record of the Great Plains differs widely compared with the Pacific Coast (Wing 1998). Therefore, it is possible that a regional analysis would reveal a more immediate response in predators toward higher RBL in concordance with the evolution of open habitats.

Not surprisingly, the early Miocene cat gap was identified by distinct taxonomic and RBL regimes (Fig. 3). An interval devoid of catlike forms in North America (Van Valkenburgh 1999), the cat gap begins with the extinction of the Nimravidae at the end of the late early Arikareean at 22.2 Ma and the arrival of the amphicyonids Cynelos and Amphicyon and the ursid Phoberocyon from Eurasia (Zhanxiang 2003; Woodburne 2004; Fig. 3). The cat gap lasts until the end of the early Hemingfordian at 17.34 Ma, when a second dispersal event from Eurasia of true felids and multiple mustelid genera, including Dinogale and Mionictis, occurs (Rothwell 2003; Woodburne 2004). The removal of the Nimravidae likely provided opportunities for alternative taxa to fill the vacated functional space of very high RBL (>0.80) catlike forms, but that specific region remained unoccupied for the duration of the cat gap (Wesley-Hunt 2005). This is somewhat surprising, given that the lower carnassials with RBL values >0.80 have evolved repeatedly in both non-catlike carnivorans (e.g., hyaenids and Cryptoprocta ferox) and “creodonts,” and may reflect a phylogenetic constraint within the remaining predators (Slater and Friscia 2019).

Although the cat gap is characterized by an absence of species with the extreme carnassial morphology of nimravids, it does not follow that their functional role as large hypercarnivores was not filled. In fact, there are amphicyonids, canids, and mustelid species that appear during the cat gap with specializations for hypercarnivory. Among the amphicyonids, the genus Borocyon and subfamily Temnocyoninae exhibit dental specialization for meat slicing and bone crushing along with postcranial features that suggest an ability for sustained pursuit (Hunt 2009, 2011). Additional large hypercarnivores appear within the cat gap, such as the canids Euoplocyon and Enhydrocyon, as well as the jaguar-sized mustelid Megalictis ferox, all of which exhibit well-developed trenchant heels on their lower carnassials indicative of a hypercarnivorous diet (Van Valkenburgh 1991). The locomotion and hunting behavior of these predators would have differed from those of catlike forms. Large felids utilize their forelimbs both in pursuit and holding prey; however, in caniform carnivorans, forelimb adaptations for a fore–aft gait limit rotation of the antebrachium for prey manipulation (Van Valkenburgh 1985). Despite morphological and attendant behavioral differences from large felids, the ecological role of consuming large prey as hypercarnivores was likely occupied by these amphicyonids, canids, and large mustelids (Van Valkenburgh 1991; Carbone et al. 1999). In fact, hunting behavior differences may not have had a strong niche-partitioning effect within and immediately after the cat gap. When catlike forms reappeared in North American in the Hemingfordian, they exhibited both high and low ankle gear ratios, displaying a wide range of locomotory behaviors and the return of the catlike hunting behavior (Polly 2020).

Conclusion

Mammalian predator taxonomic composition and both body-mass and RBL distributions underwent multiple shifts across the last 65 Myr in North America. After the late Eocene peak in mean RBL, predator assemblages exhibited elevated mean RBL that continued after the Oligocene, indicating a sustained change in predator trait distribution in response to the spread of open habitats (Strömberg 2004). Aside from the late Eocene shift in RBL, shifts in taxonomic and functional regimes appeared to be decoupled from major climate change events in the Cenozoic. This decoupling likely reflects a less immediate response in mammalian predators to environmental change compared with mammalian herbivores (Werdelin and Lewis 2005; Samuels and Hopkins 2017). Only 3 of the 11 functional regime shifts coincided with taxonomic change, with 2 of them defining the early Miocene cat gap. After inspection of taxa before and after the functional regime shifts, replacement at the genus level appears to be driving the changes in functional regimes. The low proportion of overlap demonstrates that shifts in ecomorphology are not simply due to family-level turnover events and instead points to a functional response within predator families to the changing Cenozoic climate, albeit over longer timescales.

Finally, using multiple traits allows for greater insights into the evolution of predator dietary and habitat niches. Predator body mass and RBL both increase in variance through time, but larger taxa are not always those with the highest RBL values. In future studies, going beyond examining the mean and variance of traits and quantifying the order of functional niche occupation over time can more directly address patterns of expanding niche breath of mammalian predators across the Cenozoic.