Environmental impacts resulting from biological invasions have been conceptualised as measurable (Ricciardi et al. 2013) or significant (Simberloff et al. 2013) changes to an ecosystem property such as species composition or ecosystem functioning. Such impacts can be multidirectional (Jeschke et al. 2014) as alien species can increase or decrease an ecological feature. Furthermore, not every negative impact constitutes serious damage because societies usually accept minor negative impacts caused by alien species (Bartz et al. 2010). A certain threshold must thus be exceeded before a negative impact, such as a decrease in a native species population size, becomes significant and can thus be addressed as damage (Bartz et al. 2010; Figure 1). The German Nature Conservation Act, for example, calls for action only against alien species that endanger ecosystems, habitats or other species. Likewise, the list of IAS of Union concern according to EU Regulation 1143/2014 focusses on alien species with significant negative impacts on biodiversity. Assessing impacts of alien species that justify any type of action is thus a key issue in risk assessment (Powell 2004, Hulme 2011, Genovesi et al. 2015, Tanner et al. 2017).

Figure 1.

From environmental changes to environmental damages by invasive alien plants. In an assessment approach, invasion-mediated changes become environmental damages or benefits when human values are incorporated. Human values matter in selecting relevant assessment endpoints and categories of impact, in distinguishing mere changes in ecological properties from negative or positive impacts, and in setting thresholds that separate impacts from significant impacts. Only significant negative impacts represent damage or harm (after Bartz et al. 2010).

Despite remarkable progress in classifying and understanding the environmental impacts of alien species (Pyšek et al. 2012, Ricciardi et al. 2013, Simberloff et al. 2013, Foxcroft et al. 2017), the great complexity of the issue is still challenging (Hulme 2011, Jeschke et al. 2014, Courchamp et al. 2017), raising the question of how assessment approaches address invasion-mediated impacts. In detail, we analysed how existing approaches considered the following issues: (i) Covered biodiversity levels: are impacts at all levels of biodiversity considered, as targeted by the Convention on Biological Diversity (CBD), i.e. genes, species and ecosystems, and how are these impacts incorporated into the assessment? (ii) Impact magnitude: are parameters such as magnitude of overall impact, effect size or irreversibility of impacts – thereby distinguishing impacts from significant impacts – incorporated as indicated by relevant legislation (e.g. EU Regulation 1143/2014)?

We differentiated three dimensions of context dependence (Figure 2). The first is the context of the alien species itself: what potential due to its characteristics (e.g. seed production, competitiveness) does a specific species have in order to induce environmental changes? That different species differ in characteristics and performance and thus need to be assessed individually is widely accepted (Simberloff et al. 2011). Furthermore, intraspecific differentiation should also be considered in assessment approaches because it can lead to different environmental impacts. Infertile varieties of an invasive alien species might be ‘safe’–but not necessarily. Dispersal of vegetative propagules, for example, is a powerful pathway to invasions in Fallopia taxa (Pyšek et al. 2003). Moreover, introduced subspecies of a native species may produce significant negative environmental impacts as reported for European subspecies of Phragmites australis in North America (Pyšek et al. 2018).

Figure 2.

Context dependence of environmental impacts of invasive alien plants. Invasion impacts differ with different context dimensions: a the context of the alien species itself b the environmental context within the actual or potential range of the alien species, and c the context of the values that are incorporated in impact assessments and that may be different among and within societies. All contexts may change with time.

Some IAS ‘blacklists’ cover national scales, translating impact assessments from at least one well-documented case of impact at the local scale to the country scale (e.g. Essl et al. 2011, Nehring et al. 2013). In this way, evidence of negative impacts at the local or regional scale is generalised to larger spatial scales. This generalisation may be justified by the precautionary approach (Essl et al. 2011), but ignores the fact that invasive species may perform quite differently in other parts of their range (Hulme et al. 2013). Accordingly, invasion impacts may be over- or underestimated when ignoring environmental variation of invaded habitats (Thiele et al. 2011). We thus considered as a second dimension of context dependence the environmental context. Prominent examples include two tree species: Cinchona pubescens shows differences across vegetation zones: it is threatening endemics in Galàpagos (Jäger et al. 2009), while facilitating endemics in managed Hawaiian forests (Fischer et al. 2009). Robinia pseudoacacia performs differently under diverse climatic conditions in Europe, with both positive and negative effects (Cierjacks et al. 2013, Vitková et al. 2017). Sádlo et al. (2017) correspondingly argued for considering the local context when deciding whether to manage R. pseudoacacia.

Third, we considered the context of societal values, which is of fundamental importance within any impact assessment as values differ among societies and over time (Estévez et al. 2015). Assessment approaches link environmental impacts with values of affected ecosystem properties (‘resources’). Beyond this, linking impacts with values requires decisions that are themselves value-laden. Examples include (i) the selection of assessment endpoints and impact categories; (ii) the differentiation between changes to ecological properties and negative or positive impacts; and (iii) the setting of thresholds of significant impacts (Figure 2; Opdam et al. 2009; Bartz et al. 2010). In the latter case, it is not only the magnitude or severity of impacts that is important, but also the value of the affected resources (Robu et al. 2007, Lawler 2009, Bartz et al. 2010). For example, the risk of Red-listed species being displaced by invasive species might justify greater management efforts than would be appropriate if only ubiquitous species were affected.

Considering context dependence within impact assessments is challenging as many interfering factors vary, e.g. the local biotic and abiotic parameters or the time since introduction or appearance of an alien species at a site (Hulme et al. 2013, Kumschick et al. 2015, Pyšek 2016). The assessment of impacts is also complicated by the fact that ecological and social contexts may change with time. For example, alien species that do not currently cause negative impacts may become problematic with ongoing climate change and vice versa (Bellard et al. 2013). The changing valuation of the Chinese tree Ailanthus altissima in the USA in the wake of the Opium Wars in the 19^th century (Shah 1997) demonstrates how changing societal values may modulate assessments of alien species. To date, knowledge about the context dependence of invasion impacts is rather fragmentary, and a better prediction of impacts requires considerable further research (Kumschick et al. 2015). Moreover, uncertainty about the alien status of a species may exist and may result in unknown invasion impacts of cryptic alien species (Essl et al. 2018, Jarić et al. 2019).

To better understand context dependencies in the assessment approaches, we asked the following questions: (i) Species context: does the approach consider a species’ potential to cause environmental impacts, and how is this potential addressed in the assessment approach? (ii) Environmental context: does the approach account for the potential or actual distribution of the alien species and the identity of habitats, species or other resources that may be affected? (iii) Context of societal values: does the approach differentiate between positive and negative effects and account for the value of (potentially) affected resources?

Managing IAS can involve high costs (Woldendorp and Bomford 2004, Panetta 2009) that may account for a considerable part of the overall costs associated with IAS (Hoffmann and Broadhurst 2016). Yet, management actions are not necessarily successful (van Wilgen et al. 2012, McConnachie et al. 2012, Kerr et al. 2016, Kraaij et al. 2017). Thus it is reasonable to consider the prospects for successful management within the scope of risk assessments. This is particularly true for assessment approaches whose main objective is to guide management decisions. Taking account of management prospects is relevant for pre-introduction assessments as well, as the absence of suitable management methods might justify a denial of introduction (Heikkilä 2011). Accordingly, Tanner et al. (2017) recommended considering issues of risk management (e.g. the availability and cost-effectiveness of preventative measures) when prioritising species in the context of IAS EU legislation 1143/2014.

Many factors may impede successful management of IAS, including the availability of effective methods and sufficient funding to conduct all necessary measures within the required time frame (Panetta and Timmins 2004, Cacho et al. 2006, Gardener et al. 2010). We thus analysed (i) whether management prospects are considered in assessment approaches and (ii) which factors relevant to successful management are addressed.

The transparency of assessment approaches is essential for application by different users. Powell (2004, p. 1306) highlights the problem of subjectivity by emphasising that ‘different assessors may mean very different things by “low” environmental impact, for example’. In particular, qualitative approaches risk a high subjectivity that reduces the comparability of assessment scores. Transparency further supports the communicability and acceptance of assessment results. For transparency, a clear terminology is required, especially for value-laden terms such as ‘impact’ (Jeschke et al. 2014), ‘damage’ (Sagoff 2005, Bartz et al. 2010) and ‘invasive’. Using such ambiguous terms without exact definition may lead to confusion in policy debates or even undermine management efforts (Ricciardi and Cohen 2007, Hulme 2011). More generally, normative assumptions that underlie impact or risk assessments should be disclosed (EPA 2000, Jardine et al. 2003) as relevant values differ among and within societies (Schüttler et al. 2011, Kumschick et al. 2012, Estévez et al. 2015).

We thus analysed if (i) the assessment methods of reviewed assessment approaches and the incorporation of applied criteria are transparent, (ii) relevant terms are clearly defined, and (iii) underlying values are disclosed.

We conducted a query in the Web of Science (WoS, accessed 11 July 2018, search in all databases) for literature containing the search terms woody OR weed* OR non-native OR invasive OR exotic OR alien OR nonindigenous AND assess* OR evaluat* OR analy* OR predict* OR prioritiz* OR scor* OR classif* OR rank* OR screen* AND risk* OR impact* OR effect* OR hazard* OR consequence* OR invasion* OR invad* OR introduction* OR entry OR threat OR potential* OR tool in its title (the asterisk ensures that all relevant endings of a root term are considered). Though we concentrated on impact assessment we included the term ‘risk’ in our search. Because risk is a function of both consequence and likelihood (Hulme 2011), the assessment of consequences, i.e. effects or impacts, should also be addressed in risk assessment approaches.

This search yielded about 3,450 papers. From this result we excluded articles from research areas such as “acoustics”, “system cardiology” or “transplantation”. By reading the title and abstract of the remaining 680 papers, we narrowed our focus to 158 articles dealing with the assessment of impacts or risks resulting from the introduction or spread of alien species. For our analysis we chose from this subset all approaches that were developed to assess impacts or risks of alien plants or alien species in general. We did not consider approaches explicitly developed for other taxa such as mammals, birds or fishes. We further ruled out papers that focussed on testing the validity of already existing approaches. All in all, our search led to 19 papers (Tucker and Richardson 1995, Reichard and Hamilton 1997, Pheloung et al. 1999, Kil et al. 2004, Olenin et al. 2007, Parker et al. 2007, Molnar et al. 2008, Ou et al. 2008, Randall et al. 2008, Stone et al. 2008, Feng and Zhu 2010, Magee et al. 2010, Miller et al. 2010, Skurka Darin et al. 2011, Koop et al. 2012, Sandvik et al. 2013, Blackburn et al. 2014, Nentwig et al. 2016, Davidson et al. 2017). We included eight further papers that we found through cross-references and that met the selection criteria (Panetta 1993, Kowarik et al. 2003, Weber and Gut 2004, Virtue et al. 2008, Kumschick et al. 2012, EPPO 2012, Nehring et al. 2013, Branquart et al. 2016). Ultimately, we reviewed 27 papers that provide approaches to assess risks and impacts of alien plants or alien species in general.

We analysed our set of 27 assessment approaches according to the key issues, criteria and parameters shown in Table 1. In the supplemental material, we document detailed results of our analysis (see Suppl. material 1: assessment results) and offer examples of how we applied the criteria (see Suppl. material 2: assessment criteria).

As our study is mainly based on a literature search in the WoS, relevant scientific work might not be captured when published in reports, working papers or other publications that are not listed in the WoS or that are written in other languages than English. Beyond this, papers addressing the topic but not using the defined search terms in their title might have been missed. We did include relevant papers in our analysis that were found through cross-referencing but not listed in WoS. Thus, we believe that the chosen subset of articles reflects a broad scope of existing approaches.

The assessment approaches can be grouped into three main categories according to their main objectives. The first group comprises predictive systems that aim to support decisions about the introduction of an alien species to an area. Such decisions are relevant for the initial introduction of a species at the national scale (Pheloung et al. 1999) and for subsequent secondary releases, e.g. in different regions of a country (Kowarik et al. 2003). The second group provides prioritisation tools to support decisions about the management of alien species that are already present in a given region (Skurka Darin et al. 2011). Some authors described their approach as meeting both prediction and prioritisation objectives (Ou et al. 2008, Feng and Zhu 2010). As a third category, we grouped approaches that function as information tools that present the impacts, invasiveness etc. of alien species without explicitly guiding decisions on introduction or management (Parker et al. 2007).

The approaches fundamentally differed in their methods for merging criteria and deriving final assessment results. They can be assigned to three major categories (Figure 3): decision trees, scoring systems and matrix tools. Decision trees are hierarchical systems based on yes/no questions (e.g. Tucker and Richardson 1995). Scoring systems derive assessment results by adding or multiplying scores for different parameters (e.g. Feng and Zhu 2010). Finally, some approaches use a two-dimensional matrix in which the main criteria are combined to generate assessment results (e.g. Sandvik et al. 2013). More than half (n = 15) of all analysed approaches were scoring systems. While only three approaches were designed as a matrix tool, others did combine several methods. For example, Virtue et al. (2008) used a scoring system to assess ‘weed risk’ but used a matrix to combine ‘weed risk’ and ‘stage of introduction’ to derive recommendations on management actions. Four approaches included a small decision tree as a pre-evaluation step, e.g. to determine which species should be further assessed, while the core assessment relied on a scoring system (e.g. Weber and Gut 2004, Randall et al. 2008).

Figure 3.

Methods used by assessment approaches of invasive alien plants. To determine final assessment results, all assessment approaches were based on one or a combination of the following methods: (a) decision tree, (b) scoring system and (c) matrix tool..

As each assessment method has strengths and weaknesses (Fox and Gordon 2009, Hulme 2011, Kumschick and Richardson 2013, Buerger et al. 2016), there is no preferable method per se. The performance of a method may be influenced by the availability of relevant information or other factors. In scoring systems, for example, the final score assigned to an alien species usually depends on the number of questions answered. Thus, in the case of poorly studied species, risks may be underestimated (Dawson et al. 2009).

In this section, we describe how the 27 assessment approaches incorporate environmental impacts in relation to biodiversity levels and magnitude of impacts.

Biodiversity levels

According to the CBD, biodiversity comprises genetic diversity, species diversity and ecosystem diversity, and the interdependencies within and between these levels of biodiversity. Alien plants may, for example, interact with other species at different trophic levels or change ecosystem processes (Vilà et al. 2011, Pyšek et al. 2012, Schirmel et al. 2016). Of the approaches, only 12 considered the impacts of alien species at all levels of biodiversity (e.g. Randall et al. 2008, Blackburn et al. 2014). More than three-quarters of all approaches considered the species or ecosystem level (Figure 4a), while the genetic level was covered by only half of the papers. This is a clear shortcoming as hybridisation is broadly acknowledged to be a relevant impact mechanism (Huxel 1999, Meyerson et al. 2012). Our analysis suggests that some invasion risks may be underestimated as many approaches did not cover the main biodiversity levels equally, although this has changed over time. The more recent approaches (since Kumschick et al. 2012, see Suppl. material 1) regularly considered all biodiversity levels.

The way in which approaches incorporated environmental impacts clearly differed. Many approaches accounted for the displacement of other species through a discrete criterion (e.g. ‘interaction with native species’; Sandvik et al. 2013) and thus referring to concrete effects. Others relied on species characteristics that may indicate (potential) impacts. One example is the ‘ability [of a species] to form large, dense, persistent populations’ (EPPO 2012). Deriving potential impacts from species characteristics may be appropriate when data about effects in the reference area are missing, but such indications can mislead decision makers. Whether a given biological feature, e.g. the potential of an alien species to form dense populations, translates to a relevant effect starkly differs among species (Hejda et al. 2009).

Relating impact assessments to observed effects instead of potential effects is thus preferable but depends on the objective of the assessment and the availability of data. Some approaches that are intended to support decisions on whether an alien species can be introduced refer to anticipated impacts of this species (e.g. Stone et al. 2008, Koop et al. 2012). Such pre-introduction assessments usually rely on transferring experiences from other regions (Kulhanek et al. 2011, Kumschick and Richardson 2013) and are burdened with uncertainties due to the context dependence of invasion impacts. At the very least, it is important that the regions be comparable in terms of climate, soil, habitats etc. Accordingly, Sandvik et al. (2013) prioritised data as follows: a) the area of interest, b) regions with comparable eco-climatic conditions, c) other regions with different eco-climatic conditions, and d) other (preferably closely related) species with comparable ecological and demographic characteristics. The decrease in data reliability along this spectrum is a strong argument for filling the gaps in databases.

The number of criteria considered under environmental impacts also differed among the assessment approaches. While Blackburn et al. (2014) covered many mechanisms that may lead to impacts at the species level (e.g. competition, predation, hybridisation, transmission of diseases, interaction with other alien species), other approaches emphasised ‘competition’ as the main impact mechanism at the species level (e.g. Virtue et al. 2008, Magee et al. 2010). Some approaches were much less detailed, when covering environmental impacts, for example, by ‘ecological disturbance on ecosystems’ (e.g. Kil et al. 2004). There was thus a gap between the inclusion of a relevant issue, e.g. different biodiversity levels, and the scope and detail of criteria that were used to detect or quantify the impact. Only a few studies (e.g. Randall et al. 2008, Kumschick et al. 2012, Blackburn et al. 2014) combined a full coverage of biodiversity levels with a broad range of criteria for quantifying invasion impacts.

Impact magnitude

Quite often legislation on biological invasions (e.g. EU regulation 1143/2014) requires the significance of impacts to be considered as a prerequisite for any decision or action against specific IAS. In addition to the value of the affected resources, the overall magnitude or severity of the impacts is important for assessing the significance of an impact (Robu et al. 2007, Lawler 2009, Bartz et al. 2010). Some approaches summed scores for different impact types (e.g. competition, hybridisation) to calculate final overall impact classes. Blackburn et al. (2014), for example, differentiated minimal, minor, moderate, major and massive impacts. Such classes help distinguish between negative and significant negative impacts. This also holds for systems which assign impact scores (e.g. Randall et al. 2008, Kumschick et al. 2012).

Most approaches, however, did not provide explicit information on the magnitude of impacts (Figure 4b). Alternatively, information on certain parameters may enable conclusions on the magnitude of impacts. Among these are effect size, irreversibility, and cumulativeness of impacts, the latter caused by different alien species in the area of interest; and the abundance and distribution of the alien species as drivers of impact (Parker et al. 1999, Hulme 2011). While more than half of all approaches incorporated effect size and spatial extent, other impact parameters were underrepresented. Only three approaches (e.g. Magee et al. 2010) considered the cumulative effect of several alien species in the reference area; no approach considered the interplay of alien species with other pressures such as land use or pollution. Again three approaches explicitly stipulated the abundance of an alien species in the region as an issue, usually in the form of a request for data on distribution (e.g. Olenin et al. 2007). Blackburn et al. (2014) included the irreversibility of impacts most extensively by using it as a characteristic feature to discriminate between massive and major impacts for each criterion.

Figure 4.

Assessment of environmental impacts of invasive alien plants. Incorporation of impacts in analysed assessment approaches (n = 27), related to a covered biodiversity levels and b parameters relevant to magnitude of impact.

Environmental impacts: synthesis

Environmental impacts were considered in different ways within the assessment approaches. Quite often impacts were addressed in terms of species characteristics related to potential effects rather than a direct assessment of impacts. The former is reasonable when data about concrete effects in the reference area are missing, for instance in pre-introduction assessments, but it also might be error-prone as species impacts are context dependent. Given that IAS can considerably threaten all levels of biodiversity it is striking that impacts on genetic diversity were neglected by many approaches. Although it may be more difficult to account for impact mechanisms such as hybridisation than, for example, a decline in native species populations, covering all relevant impact mechanisms and assessment endpoints (i.e. affected resources of concern) is of vital importance to generate resilient assessment outcomes. Although there are different options for assessing the significance of impacts, the overall magnitude of impacts should be considered. However, our analysis shows that this measure was not regularly included. Likewise, important impact parameters such as cumulativeness or irreversibility were underrepresented.

It is common knowledge in invasion science that invasion impacts are context-dependent as they depend on (i) the characteristics of the invading species (Simberloff et al. 2011), (ii) the environments in which the invasion occurs (Pyšek and Richardson 2010, Thiele et al. 2010, Hulme et al. 2013, Kumschick et al. 2015), and (iii) the societal values that may be affected by the invasion (Estévez et al. 2015). We analyse here whether existing assessment approaches considered these three dimensions of context dependence (Figure 2).

Context of species

All assessment approaches (except Reichard and Hamilton 1997) considered the species-related context dependence of invasion impacts (Figure 5). Some approaches took into account effect-related species characteristics, e.g. a species’ ability to form large and dense monocultures (e.g. Weber and Gut 2004); other approaches assessed concrete effects, e.g. decrease in abundance of affected species (e.g. Kowarik et al. 2003, Blackburn et al. 2014). Most assessment approaches accounted for effect-related species characteristics as well as concrete effects and additionally included information about the establishment or distribution of a species in the reference area.

Context of environments

To incorporate the environment-related context of invasion impacts, assessments should consider information on the (potential) distribution (1) of alien species and (2) of (potentially) affected environmental resources. Only half of all approaches (Figure 5) considered the former, mainly by accounting for the potential (e.g. Panetta 1993) or actual distribution of an alien species (Parker et al. 2007), or both (Weber and Gut 2004). The spatial scale for considering the distribution of species, however, varies from the local to the global context, depending on the purpose of the assessment approach. According to the German Nature Conservation Act, alien species that have not been classified as invasive (i.e. that do not threaten biodiversity) may be planted in the wild if risks can be excluded. To assess risks in such cases, the assessment approach by Kowarik et al. (2003) focused on a local context: the place of release and the area of subsequent potential distribution. In contrast, for marine ecosystems, where dispersal limitation is less relevant for invasion processes, Molnar et al. (2008) considered the actual distribution of species in marine ecosystems in a global context.

Figure 5.

Assessment of context dependence of invasive alien plants. Incorporation of context dependence of environmental impacts in the analysed assessment approaches (n = 27) in relation to different dimensions of context dependence: a species b environments and c societal values. (IAS = invasive alien species).

Only one-quarter of the assessment approaches further addressed the environmental context of impacts by referring to (potentially) affected resources, such as species or habitats of conservation concern. Approaches by Ou et al. (2008) and Randall et al. (2008) included questions about the proportion of the species’ current range where negative impacts have been measured. A few other approaches (e.g. Kowarik et al. 2003, Miller et al. 2010, Skurka Darin et al. 2011) went further by overlaying the occurrence of alien species with the occurrence of (potentially) affected resources. Yet, even this does not allow for a proper context-related assessment, as many factors influence context dependence (e.g. time since introduction, propagule pressure; Pyšek 2016). Moreover, the co-occurrence of alien species and resources of conservation concern at a site does not necessarily lead to negative effects on resources (Ramírez-Cruz et al. 2019). In the end, linking both, i.e. impacts and potentially affected environments, would allow for a better consideration of environmental contexts than would assessments that are merely based on (effect-related) species characteristics or impacts observed in other regions.

Distinguishing impacts across environmental conditions would also allow for multiple responses to IAS. This can be appropriate when the effects of a given species may be positive, negative or neutral depending on the environmental context. Incorporating the environmental context in assessments would allow positive impacts to occur and help allocate management efforts to counteract negative impacts. This is most feasible at the local scale. Assessment approaches designed to support management decisions consider species that are already present, which should enable a more concrete differentiation of the environmental context – either for a specific regional context or at the typological level by considering different biotope types [as proposed by Sádlo et al. (2017) for managing invasions by Robinia pseudoacacia]. In contrast, approaches aimed at a pre-introduction assessment usually refer to the national scale, thus largely requiring that different environmental settings be considered at a rather rough level.

Our analysis shows that only a few post-introduction assessment approaches allowed for such a concrete differentiation of the environmental context. Kil et al. (2004) for instance simply asked whether a species is ‘widely distributed’ within the country. On the one hand, this approach stands out for its simplicity as it is applicable with little information. On the other hand, it does not enable a differentiated assessment that accounts for different environmental conditions.

Context of societal values

Environmental impacts on species assemblages or ecosystem properties can be positive or negative (Ricciardi et al. 2013, Simberloff et al. 2013). Several studies focussed on these issues with a main emphasis on economic aspects; only six explicitly differentiated between negative and positive impacts (Figure 5; e.g. Kil et al. 2004, Davidson et al. 2017). Kumschick et al. (2012) went furthest by scaling every criterion with regard to positive and negative effects and applying a weighting factor according to stakeholder preferences.

Nearly half of all approaches considered the value of (potentially) affected resources (Figure 5). Six of these provided a rather basic approach: for example, Molnar et al. (2008) and Davidson et al. (2017) considered the conservation value of affected resources to distinguish the highest from the second highest impact level. Another six approaches stipulated a more systematic assessment of values. Skurka Darin et al. (2011) determined the distance between populations of alien species and valuable resources such as ‘concentrations of threatened and endangered species and rare plant communities’ and used this measure for setting management priorities. Stone et al. (2008) posed the question ‘Could the species reduce the biodiversity value of a natural ecosystem, either by reducing the amount of biodiversity present (diversity and abundance of native species), or degrading the visual appearance?’ Randall et al. (2008) included a separate criterion ‘conservation significance of the communities and native species threatened’ and ascribed to this criterion the second highest weighting within the subcategory ‘impacts’. Sandvik et al. (2013) went a step further, basing their final impact categories on the value of the affected resources, i.e. ‘the ecological effect is classified as milder if none of the species affected by the alien species is threatened or a keystone species’. Finally, Kowarik et al. (2003) derived the significance of impacts from a matrix combining the magnitude of impacts with the conservational value of the affected resources. This approach used several criteria for assessing the conservation value of affected resources, whereas other approaches relied only on the status of a species or habitat as threatened or rare.

Context dependence: synthesis

The performance of invasive species may vary depending on environmental conditions. Moreover, societal values, which vary from society to society – and within societies – affect the perception of invasive species. Thus, the operationalisation of context dependence remains an important challenge for assessment approaches. Our analysis shows that nearly all approaches incorporated species-related context dependence by considering species identity, species traits, or the ability of a species to cause environmental impacts. A step forward would be to incorporate the actual (or potential) exposure of relevant resources to alien species in assessment approaches. Although many approaches requested at least basic information about the (potential) distribution of the given alien species only three approaches explicitly included the exposure of (potentially) affected resources. At the management level, exact information about the occurrence of alien species as well as (potentially) affected resources should be available.

Given that about half of all approaches mentioned the support of management decisions as an important objective, it is surprising that environment-related context dependence was not more strongly represented. Finally, all approaches inherently incorporated values, ranging from the choice of relevant assessment endpoints to the classification of impacts based on thresholds. The latter mainly depends on the magnitude of impacts but also the value of the resources affected. Yet, only a few approaches comprehensively incorporated the value of such resources. Species or habitats can be valuable without being threatened or rare, e.g. due to a global responsibility for their conservation or because they are protected for cultural reasons. Thus, the exclusive focus of the analysed approaches on criteria such as endangerment or rareness may be seen as a further deficit, in particular with respect to prioritisation of management actions.

Successful management of biological invasions is a basic supposition for preventing, mitigating or removing negative impacts of IAS. Moreover, feasibility of management may be a prerequisite for listing an invasive species, e.g. according to EU Regulation 1143/2014. As management success depends on many factors (Table 1), the availability of effective and practicable eradication or control methods, and of sufficient personnel and financial resources are essential requirements for successful management (Panetta and Timmins 2004). Against this background we analysed whether and how management prospects were incorporated in the studied set of assessment approaches (Figure 6).

About half of all approaches directly considered prospects for successful management. Among these, most focussed on several relevant parameters, but there were substantial differences in how clearly the parameters were operationalised. For example, the approaches provided by Ou et al. (2008) or Skurka Darin et al. (2011) included different criteria, i.e. availability of effective methods, costs of control or eradication. In contrast, Parker et al. (2007) simply asked whether an alien species is easy or difficult to control without explaining which parameters should be considered to answer this question. All approaches developed to support management decisions should directly address the existence and feasibility of adequate measures, yet our analysis indicates that some do not (e.g. Kil et al. 2004, Weber and Gut 2004).

The approaches also differed significantly in how they considered parameters that influence the feasibility and success of management (Figure 6). As expected, some of these parameters (e.g. number, accessibility and detectability of infestations) played no role in assessment approaches exclusively supporting decisions on the introduction or entry of alien species (n = 7; e.g. Panetta 1993). It is nonetheless striking that, besides ‘species traits’ and, perhaps, the ‘size of (potentially) infested area’, approaches rarely considered parameters influencing management. The key question on the availability of effective control or eradication methods for setting priorities in alien species management was only explicitly included in seven approaches (e.g. Feng and Zhu 2010, Nehring et al. 2010). In the same vein, the availability of sufficient funding within the required time frame (including follow-up measures) is a broadly acknowledged premise for management success (Child et al. 2001, Gardener et al. 2010). But only four approaches (e.g. Ou et al. 2008, Skurka Darin et al. 2011) requested information on cost and/or time commitment for managing a given alien species and incorporated a basic estimate of the available financial and personnel resources in the assessment approach.

Figure 6.

Assessment of prospects for successful management of invasive alien plants. Incorporation of factors relevant to successful management of invasive alien species within analysed assessment approaches (n = 27).

Management measures can bring about unwanted side effects on biodiversity, e.g. by enhancing the spread of other invasive species (Zavaleta et al. 2001, Jäger and Kowarik 2010, Pearson et al. 2016). Thus, the management of a particular IAS should always take into account the co-occurrence of other alien species (Ballari et al. 2016). Three approaches incorporated this issue by including criteria such as ‘impacts of management on native species’ (e.g. Randall et al. 2008). Finally, it is important to consider the restorability of native habitats and species communities after management (Zavaleta et al. 2001, Carroll 2011, Panetta et al. 2019). Only Ou et al. (2008) incorporated this concept with their criterion ‘cost and time commitment of restoration’. Other studies incorporated information about the irreversibility of impacts (e.g. Olenin et al. 2007, Davidson et al. 2017), and thus, to a certain extent, may endorse conclusions concerning restorability of affected resources.

Management: synthesis

Only half of all studies directly considered prospects for successful management or the efforts to be taken. Additionally, important parameters such as unwanted side effects of management or the restorability of species communities and habitats after successful management were widely ignored. Thus, the majority of the studies lacked essential information to truly support management decisions. Strikingly, this also held for many approaches aimed at prioritisation of management.

Transparency of assessment approaches not only fosters acceptance of assessment results but also improves communication among stakeholders involved in alien species assessment. Here we analysed if (i) the way in which criteria were incorporated into assessment approaches is replicable, (ii) relevant terms were clearly defined, and (iii) underlying values were disclosed.

Transparency of criteria and assessment methods

The transparency of how assessment criteria were incorporated differed among and within the reviewed approaches (Suppl. materials 1, 2). For instance, the approach of Ou et al. (2008) contained quantified criteria (e.g. a percent scale of the ‘proportion of current range where the species caused negative impact’) as well as qualitative or semi-quantitative, and thus also ambiguous, criteria. One example of the latter is the criterion ‘impact on economy and other aspects’ which was scaled as follows: one scoring point (SP): ‘little or without impact on local economy and other aspects’, four SP: ‘weak impact on one aspect’, six SP: ‘significant impact on one aspect’, eight SP: ‘significant impact on two aspects’ and ten SP: ‘significant impact on more than two aspects’. But when is an impact weak or significant? Without explicit explanation, any assessment based on this criterion remains nebulous.

When a quantification of criteria is not possible, questions should have clear and unambiguous explanations and guidelines as to how they should be answered (Weber et al. 2009). In light of this challenge, supplemental guidance addressing questions was published (Gordon et al. 2010) for the well-established Australian Weed Risk Assessment (Pheloung et al. 1999). For the majority of analysed approaches, however, such guiding material is not available.

Transparency is not only crucial in the operationalisation of individual criteria, but also in the way in which the final assessment results are derived. In contrast to the results for individual criteria, nearly all approaches met this requirement.

Definition of terms

All analysed papers used the term ‘invasive’, but only ten approaches provided a definition (e.g. Reichard and Hamilton 1997, Magee et al. 2010; see Suppl. material 1). This is a shortcoming as fundamentally different definitions of ‘invasive’ exist that are either impact-related (Mack et al. 2000) or refer to the spread and population establishment of alien species (Richardson et al. 2000). All assessment approaches applied terms such as ‘impact’, ‘damage’ or ‘harm’ to address effects on relevant resources induced by alien species, yet only six papers explicitly defined those terms (e.g. Kowarik et al. 2003, Nehring et al. 2010, Sandvik et al. 2013). This illustrates the necessity for clarifying ambiguous terms to enhance communication among scientists and other stakeholders (Jeschke et al. 2014).

Disclosure of values

Assessments of impacts are strongly value-laden. Disclosing these values and explaining the reasoning behind them, specifically as they relate to key terms, is crucial for transparency and acceptance (Bartz et al. 2010, Estévez et al. 2015). This especially holds true for the identification of decision-relevant impacts where stakeholders may hold different underlying values. Moreover, the disclosure of underlying values is of vital importance when scaling and calibrating criteria or deriving final assessment results. We found that applied values were disclosed in only a few approaches, resulting in a deficit of transparency. Only four approaches (Kowarik et al. 2003, Nehring et al. 2010, Sandvik et al. 2013, Branquart et al. 2016) explicitly referred to relevant legislation when deriving relevant assessment endpoints or setting thresholds. To some extent, underlying values were revealed by considering the conservation value of affected resources, and a few approaches substantiated their choice of criteria in this way (e.g. Kowarik et al. 2003; Randall et al. 2008). Only one approach went further and directly incorporated views of stakeholders into the assessment procedure (e.g. Kumschick et al. 2012).

Transparency of assessment approaches: synthesis

Noticeably, no approach consistently defined all criteria used. Every approach included at least some criteria with a wide scope for interpretation. In part this was certainly due to the fact that not all relevant information can be quantified adequately. Thus, quantified criteria may require data that, in a concrete case, might not be available or may be difficult to collect. This highlights the need for explicit guidelines for the application of criteria. Further, in many approaches, ambiguous or value-laden terms, such as ‘invasive’, ‘impact’ or ‘damage’, were not defined, nor were underlying values revealed.