Aim: Species distribution models can guide invasive species prevention and management by characterizing invasion risk across space. However, extrapolation and transferability issues pose challenges for developing useful models for invasive species. Previous work has emphasized the importance of including all available occurrences in model estimation, but managers attuned to local processes may be skeptical of models based on a broad spatial extent if they suspect the captured responses reflect those of other regions where data are more numerous. We asked whether species distribution models for invasive plants performed better when developed at national versus regional extents.

Location: Continental United States.

Methods: We developed ensembles of species distribution models trained nationally, on sagebrush habitat, or on sagebrush habitat within three ecoregions (Great Basin, eastern sagebrush, and Great Plains) for nine invasive plants of interest for early detection and rapid response at local or regional scales. We compared the performance of national versus regional models using spatially independent withheld test data from each of the three ecoregions.

Results: We found that models trained using a national spatial extent tended to perform better than regionally trained models. Regional models did not outperform national ones even when considerable occurrence data were available for model estimation within the focal region. Information was often unavailable to fit informative regional models precisely in those areas of greatest interest for early detection and rapid response.

Main conclusions: Habitat suitability models for invasive plant species trained at a continental extent can reduce extrapolation while maximizing information on species’ responses to environmental variation. Standard modeling methods can capture spatially varying limiting factors, while regional or hierarchical models may only be advantageous when populations differ in their responses to environmental conditions, a condition expected to be relatively rare at the expanding boundaries of invasive species’ distributions.

Early detection and rapid response, invasion risk, model transferability, species distribution models

Organisms’ responses to environmental variation underlie patterns of distribution and abundance and are the basis for correlative statistical tools such as species distribution models (SDMs; Franklin 2010). Among the challenges of such models are that 1) the relationships between environmental conditions and organismal response can vary over space and time, and 2) outcomes under new conditions are difficult to predict (Sequeira et al. 2018; Yates et al. 2018). These twin challenges, transferability and extrapolation, can point to opposing solutions when interest is in predicting habitat suitability in a region beyond the core of a species’ range (Werkowska et al. 2017; Sequeira et al. 2018). Transferability challenges could favor limiting both estimation and prediction to within the region of interest when such data are available and where responses to environmental conditions are thought to be distinctive (Barbet-Massin et al. 2018). However, while a regional approach may capture key limiting factors, it excludes the full range of environmental conditions under which data are available and hence can lead to unnecessary extrapolation and errors in estimated suitability (e.g., Fitzpatrick et al. 2007; Broennimann and Guisan 2008).

Predicting suitability for invasive species exemplifies challenges with both transferability and extrapolation (Elith et al. 2010). Wherever invasive species are still spreading, correlative models can conflate this lag in time (i.e., lack of equilibrium) with a lack of suitability. A common recommendation is to develop the most inclusive view of invasion risk by estimating models based on both the native and invaded ranges to capture the species’ complete environmental associations and minimize extrapolation (Fitzpatrick et al. 2007; Broennimann and Guisan 2008). However, where populations are differentiated in their responses to environmental conditions, surveillance and management may be more effectively guided by locally or regionally tuned approaches because of poor model transferability (e.g., Connor et al. 2019; Collart et al. 2021). Studies focused on native species have found modeling intraspecific subsets of the data based on genetic or regional groupings improved distribution model predictions (Chardon et al. 2020). Further, regional models performed better at predicting distributions within the margins of species ranges, where different environmental predictors were most important (Vale et al. 2014; Connor et al. 2019). Marginal or poorly sampled populations may also contribute little to model estimation if training data are heavily dominated by a better sampled portion of a species’ range (Pearman et al. 2010; Hällfors et al. 2016); conversely, limiting model estimation to sparse data in a subset of the range may lead to low model quality. Given the potential for population differentiation within species’ invaded ranges (e.g., Colautti and Barrett 2013), models of invasive species’ distributions may face important trade-offs between inclusivity versus regional applicability, as well as practical data limitations in newly invaded areas. Meta-analyses of studies that trained models using native range only, invaded range only or global range did not find that global models perform better than models generated in the range of interest, and indicate that superiority of global model performance could be a statistical artefact because test data are not independent (Liu et al. 2020b).

Early detection and rapid response (EDRR) activities aim to prevent establishment, spread, and impact through surveillance and rapid management action, and can minimize invasions in new regions (Reaser et al. 2020). Sagebrush (Artemesia sp.) habitats in the western United States (U.S.) provide habitat for many wildlife species and face multiple stressors including invasive species, altered fire regimes, climate change, and energy development (Davies et al. 2011; Coates et al. 2016). Crist et al. (2019) have developed a list of invasive plants that have no, patchy, or limited presence on a state-by-state basis within sagebrush habitat. Their approach emphasized the potential for ongoing spread and geographic differences in invasion status, as species that are well established within one state may still warrant EDRR elsewhere. For these regional ‘EDRR species’, species distribution models can guide surveillance by identifying areas with high invasion risk (Brooks and Klinger 2009). However, one concern we have heard from within the management community is that models trained with a broad geographic extent could miss regionally and locally relevant limiting factors if important signals were swamped by other portions of the range.

For a set of nine species recognized as EDRR targets within sagebrush habitats (Crist et al. 2019), we characterized each species’ relationship to sagebrush communities to understand habitat associations and degree of sage specialization. We then compared the performance of national (here used to refer to the conterminous U.S.) versus regional species distribution models. We compared regional models to national, instead of global, models because of the availability of a wider breadth of predictors within the conterminous U.S., including higher quality data than are available globally (e.g., for soils), and finer spatial resolution of predictors focused on the U.S. compared to global versions. Appropriate methods to account for sampling biases are also likely to differ between a native range, where a species is likely closer to equilibrium, and a novel range, which complicates background selection when pooling records from native and invaded ranges (Elith et al. 2010; Jarnevich et al. 2017). In addition, all species in question have been in the U.S. since at least 1957 (based on earliest occurrence records; GBIF.org 2022), giving them time to potentially develop local adaptations and providing numerous occurrence points for model estimation (Liu et al. 2020a; b). Thus, we fit species distribution models for each species across the U.S., from all sagebrush within the U.S., and separately within sagebrush habitats in each of three ecoregions (Great Basin, eastern sage, Great Plains). Models trained on sagebrush only were fit to allow for any response curves specific to sagebrush habitats (within which models were also tested). We evaluated model performance using withheld spatially independent validation data within each region. We hypothesized that given sufficient data and variation in environmental responses, a regional model evaluated with test data from within the region could outperform a national model. Our results evaluate whether national models can sufficiently capture invasion risk across ecoregions, or whether estimation of models for each region improves the credibility of the outputs for on-the-ground management.

The data underpinning the analysis reported in this paper are available by a U.S. Geological Survey data release through the Science Base Repository at https://doi.org/10.5066/P90AL0PN.

Most of our focal invasive plants had higher proportions of occurrences in sagebrush habitats compared to occurrences of all invasive plants of the same life form, pointing towards preference for sagebrush habitats after accounting for potential variation in sampling intensity with habitat type. Ventenata dubia occurred in sagebrush habitats in a greater proportion relative to occurrence points of other graminoid invasive species, as did T. caput-medusae to a lesser extent (Suppl. material 1: Fig. S2). Of the forb/herb species, C. virgata, C. juncea, and S. aethiopis also had positive ratios for sagebrush, indicating that these five species are disproportionately problematic within sagebrush habitats, even after considering sampling biases in occurrence locations. The three Centaurea species, C. juncea, and H. glomeratus all had positive ratios for the eastern sage region compared to other invasive forb species. All species had a positive association with shrubland, which includes sagebrush dominated locations, except C. juncea which had a positive association with the herbaceous land cover classes (Suppl. material 1: Fig. S3). It did, however, still have a positive ratio of occurrences in sagebrush everywhere but the Great Plains region (Suppl. material 1: Fig. S2).

Only two species, C. diffusa and R. repens, had enough locations in all three regions to fit models to all model estimation extents (Suppl. material 1: Table S3, Fig. S1). Patterns in predictions between the different training extents varied by species. R. repens mapped predictions varied with the training extent (Fig. 2), but the total area within each region predicted to be suitable by each model ensemble varied less for R. repens than for C. diffusa (R. repens points were closer to 1:1 line in Fig. 3b; Suppl. material 1: Fig. S4a, g). Centauria diffusa model ensembles that were trained on occurrences in sage showed poor extrapolation to other habitats in that they were less restricted to sage compared to the national model ensemble (i.e., sage only models, represented by red triangles in Fig. 3b, fell above the 1:1 line in Fig. 3b); interestingly, several of the national model ensembles for this species predicted less suitable area than their regional counterparts. Variable patterns could be seen for other species, with no clear visual differences in the geographic extent of predicted suitability among models trained on different regions (Suppl. material 1: Fig. S4). Some regional models predicted a lot of suitable habitat outside their training region, potentially extrapolating incorrectly (Suppl. material 1: Fig. S4); models extrapolated to other regions could show higher or lower suitable area than continental models, with extrapolation leading to more variability than interpolation (i.e., the points farthest from the 1:1 line in Fig. 3b are small, indicating they arose via extrapolation). Important predictors between training extents were relatively similar (Suppl. material 1: Fig. S5).

Figure 2.

Predicted suitability for Rhaponticum repens within the eastern sage region (green region in Fig. 1) from models trained using data from the A Eastern sage B Great Basin C Great Plains D All sage, and E the continental U.S. (CONUS) (training regions shown in Fig. 1). Maps (A–E) show ensembled model predictions, defined as the number of models predicting suitable habitat; F shows training and test data for Rhaponticum repens within Eastern sage; test data were withheld from estimation of all models and used to create consistent performance assessment sets for each species and region.

Models tested on the region where they were trained were not better than continental U.S. models (paired t-test p-value = 0.07, mean difference = -0.14, i.e., continental models marginally better). Continental U.S. models outperformed models trained on the test region in seven of ten cases (Fig. 3). We had 11 regional test datasets across the nine species which met our criterion of 30 test points within the withheld spatial strip (Suppl. material 1: Table S3). Of these, the continental models or all sagebrush models were better than regional models (including those trained in other regions) in most cases (Fig. 3, Suppl. material 1: Fig. S6). A regional model performed better than any other in five cases, of which three were actually models trained in other regions and extrapolated to the test region. The two models trained on the same region as the test strip had a lower Boyce index than models trained on either all sage or other regions. All models for S. aethiopis were poor, with all Boyce index values well below zero, despite decent performance according to typical assessment metrics based on the training data (cross validation AUC > 0.75, with an average per region > 0.89).

Figure 3.

A Regional models did not outperform continental-scale models, even when many points were available within the training region. Boyce index values were calculated for the training region’s test strip for both the matching region model ensemble (x-axis) and continental United States model ensemble (y-axis) for each species (color). Species without sufficient occurrence points within the test strip were excluded. Values above the 1-1 line indicate continental U.S. model had better performance; for most species and regions, models with a continental extent performed better even when the number of regional training points was high (i.e., points are above the 1:1 line, even for big points). B suitable area predicted by national models (either entire continental U.S. or sagebrush habitat within the U.S.) compared to regional models, where larger size indicated if the focal region considered for area calculation was the same (interpolation) or different (extrapolation) from the regional modeling training region. Values above 1-1 line indicate the national model predicted more suitable habitat.

While V. dubia had enough locations to meet our criteria to develop models for the Great Plains region (n = 4,246), the occurrences were all within a relatively small geographic extent, and there were not enough locations for validation (Suppl. material 1: Table S3, Fig. S1). This small geographic extent was problematic in fitting models, where we were unable to obtain enough target background locations within the area around the general extent of occurrences within the region. Three of the five KDE models had poor fit (e.g., training AUC = 0.67 (GLM), 0.695 (MARS), 0.64 (RF)) that we were unable to improve; the other two KDE models had moderate performance (AUC < 0.8).

Regionally trained models for invasive plants of management concern did not perform better than national models when evaluated with independent data from within the training region. Continental-scale models tended to outperform regional ones even when the number of regional training points was high (Fig. 3), supporting the general recommendation to use a broad spatial extent for training models of invasive species (Fitzpatrick et al. 2007; Broennimann and Guisan 2008). Mapped predictions from models trained on a focal region were more similar to continental scale predictions within that region, compared to extrapolated results from models trained in other regions (Fig. 2; Suppl. material 1: Fig. S4). When comparing area predicted as suitable by models trained on different geographic extents for the same target region, there was not a consistent pattern, but extrapolation led to more variable results (Fig. 3b). When interpolating, including training points beyond the focal region did affect predictions within that region, as we found differences in both the spatial pattern and the overall level of predicted suitability between continental and regional model outputs. The tendency for higher performance of continental models points to these modifications being generally positive for within-region model performance and indicates that models with a broader extent are less prone to swamp regional patterns than to usefully reduce model extrapolation.

For most species, we had insufficient data to estimate and evaluate a model in one or more of our focal sagebrush regions. For example, V. dubia lacked estimation data in the eastern sage region, and is established within only a small area of the Great Plains, where active EDRR efforts have yielded a large number of data points (Hart and Mealor 2021). However, because our validation design utilized spatial strips to provide a more independent, and therefore more realistic, estimate of performance, we had insufficient regional validation points to assess model performance within the Great Plains. In addition, strong spatial clustering of points early in an invasion, such as with V. dubia in the Great Plains, can reflect propagule pressure and the idiosyncrasies of dispersal, with many unoccupied locations due to dispersal limitation (Elith et al. 2010; Václavík and Meentemeyer 2012). Species distribution models trained on only a portion of a species’ range are therefore likely to be less accurate in early invasion stages.

While this study focused on the geographic extent of estimation data, comparisons with previous work highlight how other modeling decisions shape predicted invasion risk. Here, we thresholded individual models in our ensembles based on a rule that categorized 10% of training presences as occurring in unsuitable habitat. This threshold rule is appropriate for EDRR activities where search is the end use of models and a targeted approach can focus search efforts towards areas with a relatively higher degree of suitability (Sofaer et al. 2019b). In contrast, Jarnevich et al. (2021) quantified invasion risk across management units, and therefore used a more precautionary approach, the 1^st percentile threshold, to avoid minimizing invasion risk via errors of omission. In contrast to the 10^th percentile threshold, the first percentile classifies 1% of training presences as being in unsuitable habitat and thus classifies a larger portion of a study area as suitable. Both thresholds are based only on presence information, as true absences are unavailable. The more targeted threshold used here resulted in a smaller extent of predicted suitability for the same species, and illustrates how different thresholds may be implemented depending on intended use (Sofaer et al. 2019b).

Our study varied the geographic extent of estimation data to compare continental and regional models. Our findings align with results for native species, where in the absence of a priori evidence for niche divergence, researchers recommended creating models across a species’ range (Collart et al. 2021; Connor et al. 2019). However, we held predictor variables constant between geographies, and the inclusion of geospatial variables believed important for controlling a species distribution may produce a better model than one for a larger extent lacking that information. Indeed, our continental models do not include species’ global ranges because we highly value predictor variables that are available for the U.S. but are not available, inconsistent, or of lower quality globally (e.g., information on soils). For these species we lacked information that would indicate we needed to vary predictors geographically.

Alternatives to regional models include allowing for non-stationarity in environmental responses via hierarchical modeling, geographically weighted regression (Osborne et al. 2007) or spatially-varying coefficient models (Gelfand et al. 2003). Hierarchical models can estimate both overall environmental responses and variation in those relationships among groups (e.g., via random slopes in a mixed modeling framework). Both regional and hierarchical modeling approaches typically require defining intraspecific groups, but little emphasis has been placed on the approaches used to define subpopulations, which should be well justified (Chardon et al. 2020). Here, we considered intraspecific divisions based on ecoregions; among native species, studies have diverged in whether splitting by ecoregion (Smith et al. 2019a) or by genetic similarity (Chardon et al. 2020) yields the best performance. Partial pooling, a hierarchical approach that incorporates multiple intraspecific groups within a single mixed model, provides a method intermediate between splitting and lumping (Smith et al. 2019b). The research to define subpopulations takes time and resources which may not be available for many invasive species, particularly when time to action is critical in limiting invasion costs (Pergl et al. 2020). These alternatives add complexity and potentially require more resources to first define groupings appropriately and then create multiple or hierarchical models for a single taxon compared to a continental approach. There is a continuum of automation versus human time and insight in developing species distribution models (Young et al. 2020), from large extent models for suites of species using the same predictors for all models (e.g., Allen and Bradley 2016) to very detailed models for a single species (e.g., Smith et al. 2019b; Chardon et al. 2020). The best path forward depends on the objectives, data availability, a priori information about populations and species, and the available resources and timeline.

In selecting a modeling approach, it is important to distinguish between populations that have different limiting factors and populations that have different responses to environmental conditions. Across a species’ range, it is typical that different limiting factors are suspected to constrain population growth; for example, an early macroecological hypothesis posited that biotic interactions more often defined southern range limits while abiotic conditions more often defined northern range limits (reviewed by Schemske et al. 2009). Cases where, for example, one area may be too dry while another is too cold can be handled by standard range-wide modeling approaches, as demonstrated by our study. It is only where the definition of ‘too cold’ varies among populations that more tailored or complex models may be warranted as highlighted by other studies of native species. Ideally, common garden experiments and similar tools would be used to test for differentiation but these types of studies for every invasive species would be time and cost prohibitive.

The degree of variation in responses to environmental conditions and the amount of data available underlie the selection of appropriate strategies for species distribution modeling (Fig. 4). Consistent responses to ecological conditions (e.g., Connor et al. 2019; Collart et al. 2021) or capturing a broader range of environmental conditions occupied by a species (e.g., Broennimann and Guisan 2008) support range-wide modeling (bottom right), while evidence for regional differentiation lends support to regional or hierarchical modeling methods where data are available (e.g., Chardon et al. 2020 upper right). However, there is a key tension between data availability and relevance for EDRR. Model outputs inform EDRR when they can be used to guide surveillance efforts and assess spatial patterns of invasion risk during a rapid response. Yet at these early stages of an invasion, there is necessarily little to no data on species’ occurrences within the focal area or the data occur within such a small extent that model fitting is difficult (e.g., V. dubia in the Great Plains region; left side of Fig. 4). Regional models will typically be most relevant at later stages of an invasion, where there has been more opportunity for population divergence, range filling, and data collection (moving from left to right within Fig. 4). Clear justification and communication of model assumptions between model producers, local knowledge holders, and decision-makers can help clarify what kinds of differences warrant regional or hierarchical models. Delayed actions may increase costs associated with invasions and decrease the ability to meet management goals for newly introduced species to a region (Ahmed et al. 2022). Regional models did not perform better than national models, and thus national models may have use to inform management decisions for early detection of invasive species.

Figure 4.

Conceptual depiction of the utility of different modeling methods and of the trade-offs between data availability within a focal region and relevance of model outputs for Early Detection and Rapid Response (EDRR) within that region. Range-wide modeling is appropriate where there is little variation in the relationship between a species’ occurrence and environmental conditions. Where local populations are differentiated in their responses to the environment, hierarchical or regionalized models are expected to produce the most relevant predictions for within the region, and the selection among model types may depend on data availability, institutional capacity, and time horizon for delivering results. The relevance of model outputs for EDRR is high only very early in an invasion, when few data are available; therefore, range-wide modeling is expected to remain the primary tool used to anticipate habitat suitability for non-native species.

This research was funded by the U.S. Fish and Wildlife Service, the U.S. Geological Survey/ U.S. Fish and Wildlife Service Science Support Partnership Program, and the U.S. Geological Survey Invasive Species Program. We thank Janet Prevey and two anonymous reviewers for comments on earlier versions of this manuscript. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government.