Understanding uncertainty in the Impact Classification for Alien Taxa ( ICAT ) assessments

The Environmental Impact Classification for Alien Taxa (EICAT) and the Socio-Economic Impact Classification of Alien Taxa (SEICAT) have been proposed to provide unified methods for classifying alien species according to their magnitude of impacts. EICAT and SEICAT (herein “ICAT” when refered together) were designed to facilitate the comparison between taxa and invasion contexts by using a standardised, semi-quantitative scoring scheme. The ICAT scores are assigned after conducting a literature review to evaluate all impact observations against the protocols’ criteria. EICAT classifies impacts on the native biota of the recipient environments, whereas SEICAT classifies impacts on human activities. A key component of the process is to assign a level of confidence (high, medium or low) to account for uncertainty. Assessors assign confidence scores to each impact record depending on how confident they are that the assigned impact magnitude reflects the true situation. All possible sources of epistemic uncertainty are expected to be captured by one overall confidence score, neglecting linguistic uncertainties that assessors should be aware of. The current way of handling uncertainty is prone to subjectivity and therefore might lead to inconsistencies amongst assessors. This paper identifies the major sources of uncertainty for impacts classified under the ICAT frameworks, where they emerge in the assessment process and how they are likely to be contributing to biases and inconsistency in assessments. In addition, as the current procedures only capture uncertainty at the individual impact report, interspecific comparisons may be limited by various factors, including data availability. Therefore, ranking species, based on impact magnitude under the present systems, does not account for such uncertainty. We identify three types of biases occurring beyond the individual impact report level (and not captured by the confidence score): biases in the existing data, data collection and data assessment. These biases should be recognised when comparing alien species based on their impacts. Clarifying uncertainty concepts relevant to the ICAT frameworks will lead to more consistent impact assessments and more robust intra- and inter-specific comparisons of impact magnitudes.


Introduction
Understanding the impacts of alien species in their recipient environments is a key research theme in invasion science (Strayer et al. 2006;Pejchar and Mooney 2009;Vilà et al. 2011;Kumschick et al. 2015). However, making comparisons between taxa is difficult as invasions are context-dependent and measurements of impact are not collected using a consistent method (Courchamp et al. 2017). As such, different frameworks have been developed to guide invasion biologists towards more standardised approaches which facilitate comparisons amongst invasion scenarios (Nentwig et al. , 2016Blackburn et al. 2014). In 2014, Blackburn and colleagues proposed a systematic method for classifying impacts across alien taxa, based on the effects of alien species on native biota. The resulting Environmental Impact Classification System for Alien Taxa (EICAT) (Blackburn et al. 2014;Hawkins et al. 2015) is conceptually based on the International Union for the Conservation of Nature's (IUCN) Red List of Threatened Species, which uses a ranked classification scheme to determine the global conservation status for individual species (IUCN 2012). Since its publication, the EICAT protocol has been formalised (IUCN 2020a, b;Hawkins et al. 2015) and applied to various groups including birds (Evans et al. 2016(Evans et al. , 2018a, amphibians (Kumschick et al. 2017), gastropods (Kesner and Kumschick 2018), some mammals (Hagen and Kumschick 2018), marine fishes (Galanidi et al. 2018) and bamboos ). More recently, Bacher et al. (2018) proposed an adapted version of the EICAT framework to address socio-economic impacts (SEICAT) caused by alien species. The currency used to measure impact for this scheme is observed changes to human activities and/or well-being and, to date, SEICAT has been applied to amphibians, birds, marine fishes, some mammals and gastropods, in conjunction with the EICAT assessments Galanidi et al. 2018;Hagen and Kumschick 2018;Kesner and Kumschick 2018).
In the ICAT classification schemes, assessors first conduct a comprehensive literature search to collate all impact records for a given alien species. They then classify each of these impact records into one of the five ICAT semi-quantitative scenarios, according to the magnitude of the impact. For instance, under EICAT, impact magnitudes are hierarchically structured, based on the level of organisation of the native population(s) (i.e. individuals or populations) in which they cause an effect: MC (Minimal Concern; negligible level of impact, but no impact on the performance of native individuals is detected), MN (Minor; the performance (e.g. growth, reproduction) of native individuals is decreased by the alien, but no impact at the native population level is detected), MO (Moderate; the alien causes a decline in at least one native population), MR (Major; the alien causes a local extinction of at least one native population, but this local extinction is reversible, which means that the native species could recolonise the area if the alien population were removed), MV (Massive; the alien causes an irreversible local extinction of at least one native population). If there is no relevant information to derive an impact score, then a species is classified as Data Deficient.
A key aspect of each assessment involves assigning a confidence score for each recorded impact to provide an estimate of uncertainty. Both frameworks adopt a similar approach as the Intergovernmental Panel on Climate Change (IPCC) and the European and Mediterranean Plant Protection Organization (EPPO) to deal with uncertainty (Mastrandrea et al. 2010;Holt et al. 2012;Kenis et al. 2012). The assessor must assign a confidence score of either high, medium or low, based on guiding probabilities (Table 1), to each impact report, depending on how confident they are that the assigned impact magnitude is true i.e. could the actual impact be lower or higher than what is classified. Although several key sources of uncertainty are identified in the guidelines (IUCN 2020a;Hawkins et al. 2015;Bacher et al. 2018), whether the current consideration of uncertainty is sufficient has not been critically evaluated.
Inadequately accounting for uncertainty when assigning impact magnitudes could lead to incorrect judgement calls and potentially to non-relevant prioritisation and mismanagement of species. Todd and Burgman (1998) demonstrated how incorporating uncertainty into the conservation status of species can cause differences in the assessment outcome, potentially altering conservation priorities. McGeoch et al. (2012) described the uncertainties associated with alien species listing and demonstrated how they produce inconsistencies at the taxonomic and geographic scale. Insufficient handling of uncertainty may not only be detrimental for the native taxa (EICAT) and human societies (SEICAT) that are affected by alien species; it can lead to public distrust in invasion science and reduce the success of future management and restoration programmes (Liu et al. 2011). Failure to effectively capture and communicate uncertainty may lead to ill-informed decisions, causing people to potentially undermine management objectives (Ascher 2004), which is of particular concern to invasive species management where public support is critical for achieving management outcomes (Bremner and Park 2007;Kraus and Duffy 2010;Novoa et al. 2017;Russell and Stanley 2018).
To address potential sources of uncertainty relevant to the ICAT assessments, we evaluate the current consideration when assigning confidence scores, identifying where uncertainties may arise during the assessment process. In the first part of this manuscript, we explain the key concepts and definitions of uncertainty relevant to the ICAT frameworks and map these along the assessment process. We then proceed to identify new sources of uncertainty currently not considered under the framework guidelines and discuss how these may play a role in both the evaluation of information and the final ICAT scores. In doing so, Table 1. The three current confidence levels (high, medium, low) assigned to individual impact reports using the ICAT frameworks. Guiding probabilities are given in the guidelines to aid the assessor in interpreting their level of confidence into one of the three qualitative categories.

Confidence level
Approximate probability of the impact being correct High~90% Medium~65-75% Low~35% we develop a more comprehensive understanding of uncertainty relevant to ICAT assessments, which may be of conceptual relevance to other aspects of risk assessment, particularly when extracting and evaluating impact information from various sources.

General types of uncertainty and how they can be expressed
Uncertainties arise because our knowledge of systems is incomplete and we often deal with imperfect information; thus, uncertainty is inherent to all scientific research (van der Bles et al. 2019). In some cases, uncertainty can be minimised through the collection of additional information, yet it is impossible to eliminate uncertainty altogether (Regan et al. 2002). In cases where uncertainty cannot be reduced, best practice involves quantification of-and when this is not possible, sufficient acknowledgement of-where uncertainties remain and how they may alter the interpretation of evidence (Fischhoff and Davis 2014). Common expressions of uncertainty in science are usually communicated through quantitative terms such as confidence intervals, standard deviations and probability distributions, but generally, they capture only parts of the overall uncertainty (e.g. measurement error).
A taxonomy of uncertainty applicable to ecological research was described by Regan et al. (2002), who distinguish between two key types of uncertainty: epistemic and linguistic (Table 2). Given their broad applicability to ecological concepts, these expressions of uncertainty are relevant to ICAT assessments and have recently been considered in developing a framework for uncertainty in invasion science (Latombe et al. 2019). Epistemic uncertainties arise because of our limited knowledge of the system of interest. They can generally be reduced with increasing information; however, obtaining a complete understanding of such systems is almost always impractical, hence the necessity to use simplified models to characterise the true state (Regan et al. 2002). Different types of epistemic uncertainty are relevant to the understanding of alien species impacts in general. These include natural variation, measurement error, systematic error, model uncertainty and subjective judgement (Table 2; Regan et al. 2002). Linguistic uncertainties arise because language is imprecise and changes over time cause terminology to be both used inconsistently and open to interpretation (Regan et al. 2002). The different types of linguistic uncertainty include vagueness, context-dependency, ambiguity, indeterminacy of theoretical terms and underspecificity (Table 2). It is clear that linguistic uncertainty has pervaded invasion science, given the numerous attempts to standardise concepts and definitions to improve consistency across the discipline Colautti and MacIsaac 2004;Richardson et al. 2010;Blackburn et al. 2011).

Considering uncertainty for ICAT assessments
Uncertainty directly relevant to the ICAT assessments can be considered at two levels: 1) the impact report level and, 2) the species level. The impact report level is the individual record of impact (of an alien species at a specific location and point in time) that is documented in some form-such as a journal article of grey literature-and assigned an impact score. In contrast, the species level summarises all the individual records of impact for a particular alien taxon (IUCN 2020a).

Uncertainties relevant at the impact report level
The different types of epistemic and linguistic uncertainty emerge across various stages relevant to an ICAT assessment; first, uncertainties will arise when the impact observation is initially observed and/or measured; second, when the impact is communicated in some form of report and third, when the ICAT assessment is conducted ( Figure 1). Any uncertainty that arises at any one stage will continue to be present at all subsequent stages, with uncertainty propagating throughout the process, from the initial impact observation to the final ICAT assessment. Thus, all uncertainties that arise prior to the impact assessment are encapsulated in the subsequent stages ( Figure 1). All uncertainties relevant here are included in the impact report box of Figure 1.
Uncertainty initially emerges in the form of natural variation, which corresponds to spatial and temporal changes occurring within the study system. An appropriate study design will identify a suitable temporal and spatial scale under which impacts of the alien species can be characterised (Christie et al. 2019).
The next step at which uncertainties emerge is when the impact is observed and measured. Here, four new sources of epistemic uncertainties are identified: measurement error, systematic error, model uncertainty and subjective judgement ( Figure 1). Each of these uncertainties may not necessarily be relevant for every impact report as the Table 2. Different types of epistemic and linguistic uncertainties and their definitions which are relevant to the ICAT assessment process (Regan et al. 2002).

Epistemic
Linguistic Natural variation Vagueness Variations in the variables measured in the study system (e.g. temporally, spatially).
Arises since language allows borderline cases. Particularly relevant to ordinal categories (e.g. high, medium, low) where arbitrary and/or poorly defined cut-offs exist.

Measurement error
Ambiguity Imperfections in the measurement equipment or observational techniques which generates random deviation in the measurement data from the true value. Includes operator error and instrument error.
When words have more than one meaning and it is unclear which meaning is intended.

Systematic error
Context dependence Bias in the measuring equipment or sampling procedure that generates non-random deviations from the true value (e.g. via poorly-calibrated equipment). This also includes error resulting from the deliberate judgement of a person to exclude (or include) data.
Lack of specificity related to the context in which something is to be understood. For example, understanding the meaning of something being "small" requires knowledge as to whether the description refers to an insect or a plant.

Model uncertainty
Underspecificity Arises due to the necessary simplifications (models) used to represent physical and biological systems.
Occurs when there is unwanted generality i.e. there is a lack of specificity to ensure complete understanding.

Subjective judgement
Indeterminacy of theoretical terms Occurs as a result of the interpretation of data, often when data are scarce and/or error prone. Particularly relevant to expert judgement.
Arises as the meaning of terms can change over time. For instance, this source of uncertainty is particularly relevant to taxonomic terms, which may be subject to revision, leading to changes in the names of species or higher-level groups.
ICAT assessments allow the use of different information sources (see Table 3 for the key differences in impact records between EICAT and SEICAT that should be considered). For instance, media reports of a change in local human activities-in response to an alien species-deriving from interviews with residents will not be subject to model uncertainty.
Although currently not directly addressed in the framework guidelines (IUCN 2020a; Hawkins et al. 2015;Bacher et al. 2018), linguistic uncertainties are important for assessors to consider when informing the confidence score. Linguistic uncertainties are of direct relevance for ICAT assessments: they occur when the impact observations, or measurements, are described in a report with imprecise and inconsistent language. Often linguistic uncertainty will be difficult to reduce retrospectively. In some cases, linguistic uncertainty (such as a vaguely described methodology of the impact study) may mask the ability to identify epistemic uncertainties.

The assessment process
Under the published guidelines, assessors are instructed to capture the key sources of epistemic uncertainty for each impact report and ascribe these to one overall level of confidence (IUCN 2020a; Hawkins et al. 2015;Bacher et al. 2018). Following the succession of guidelines, the consideration of uncertainty has been somewhat revised. The most recently-revised EICAT guidelines (IUCN 2020a) identify five major sources of uncertainty Figure 1. Uncertainties propagate across the process of an impact assessment. The first source of uncertainty emerges due to natural variation associated with the occurrence of an alien species' impact on native biota. Uncertainties arise at three key stages when information on the impact of an alien species is captured 1) the impact observation stage; i.e. when the impact is measured 2) the impact report stage; i.e. when the impact is communicated in some form of report and finally, 3) at the ICAT assessment stage; i.e. when the assessment is conducted. Any uncertainty that arises will be carried through to the subsequent stages, as illustrated through the encapsulation of uncertainties across the process.  (scien�st or other) Assessor that the assessor must consider when assigning a confidence score: i) data quality and type ii) spatial and temporal scale and iii) confounding effects iv) study design and v) overall coherence of evidence. These sources of uncertainty are also relevant for SEICAT; however, given that the currency used to measure impact differs between the two frameworks (native species' populations vs. human activities), interpretation and importance of different uncertainties may vary to fit the criteria and concepts for each framework (Table 3).

Impact
When evaluating the magnitude of an impact, the assessor interprets the information contained in the impact report and, when possible, translates this information into one of the five ICAT magnitudes. As impact reports were not aimed at testing the assessment criteria (e.g. which level of organisation of the native population is affected by the alien), the assessor has to interpret the information at hand, a process which inevitably introduces a new source of uncertainty. It may be difficult for ICAT assessors to identify limitations generated by the way the impact was measured and reported. Ideally, authors of an impact study will address limitations with their research; however, ICAT assessors must critically assess all available information (e.g. study design, statistical analyses) to identify potential weakness in the inference of the data. It is at this stage-where the impact measurement is reported-that linguistic uncertainties become relevant and should ideally be recognised by assessors, who should be aware of how language may influence their interpretation of the information.
Assessments will be further compounded by systematic error (i.e. when the assessor systematically decides to include or exclude information that they should otherwise exclude or include) and subjective judgement (Regan et al. 2002). These sources of uncertainty initially become relevant when the assessor conducts a literature review to extract the records of impact for an alien taxon, then decides which fit the framework criteria. For instance, there may be some confusion as to what sources of impact should be included in assessments. Under the EICAT guidelines, impacts are defined as changes to the environment that reduce native biodiversity or alter ecosystem functioning to the detriment of a native species (Hawkins et al. 2015). Therefore, the inclusion of laboratory and mesocosm experiments presents a grey area when considering impact reports. In many cases, such experiments can be informative towards identifying the mechanism(s) through which an alien species impacts on native biodiversity and if native individuals are (potentially) suffering in their performance. However, laboratory and mesocosm studies will always be limited to revealing impacts of MC or MN, given that EICAT measures impacts based on native communities. Therefore, a decline of a natural population or its local extinction cannot be inferred from artificial settings, but such experiments may be useful to provide information about the mechanisms of impact. If assessors include laboratory-or mesocosm-derived sources of information in EICAT assessments, they should be clearly specified as such. Subjective judgement arises due to the interpretation of information; it emerges at the initial impact observation and continues to appear throughout the assessment procedure as each person involved in the process introduces their own form of subjective judgement (Figure 1). An ICAT assessor's subjective judgement is the primary form of uncertainty that we can minimise by clarifying concepts appropriate to assigning confidence scores and improving the consistency amongst assessors when using the two assessment schemes. Subjective judgement is also Table 3. Major sources of uncertainty are identified in the IUCN (2020a) EICAT guidelines. Each source of uncertainty is relevant to both the EICAT and the SEICAT schemes; assessors must consider each source when assigning confidence scores. The two frameworks differ in their currencies used to measure impact (native populations [EICAT] versus human activities [SEICAT]). Therefore, contextual understanding of how these uncertainties may influence confidence scores is required. We highlight some aspects of how considering uncertainty may differ between EICAT and SEICAT below. . Further, they must consider whether there is any evidence for additional driver(s) of change causing the observed effects. A major challenge in understanding the impacts of alien species is to disentangle the driving causes of biodiversity declines. Studies/reports range from being simple negative correlations between alien and native populations to beforeafter-control-impact studies, which may influence the data quality and interpretation Christie et al. 2019). Often, observed correlations between alien species and native biodiversity loss are reported, but the cause of change is not the alien. For instance, the driving cause of change may be habitat modification that facilitates the alien, which works simultaneously to cause a negative impact on native species. An example would be an alien species that establishes and thrives in an urban area may not be the driver of native bird declines; rather, it could be that loss of resources due to urbanisation are causing the native birds to decline.
Alien species altering human activities should be considered in the same way as for EICAT, i.e. the assessor must ask the question "is the alien driving the recorded changes?" However, with SEICAT, given that people can directly communicate the reason for reducing or discontinuing an activity, it may be possible to get a better understanding of the causality behind the recorded impact magnitude with much higher confidence.

Study design
The ICAT frameworks evaluate the different levels of impact, whereby each step change in a category reflects an increase in the order of magnitude of the particular impact so that a new level of organisation is involved (individuals, population, community). A study/report may describe an impact affecting one organisation level (e.g. performance of the individual), but gives no information of relevance to a higher level (e.g. if the impact reduces the population size). This aspect of uncertainty can be captured by considering the directionality of uncertainty for each impact report.
A study that is designed to assess the impacts of an alien on the individual performance, but does not capture any information about impacts to the population cannot be assigned higher than an MN. This does not mean that the true impact is not higher and thus, the impact report cannot be assigned a high confidence. High confidence scores can be assigned when the criterion of the magnitude higher than the one assigned has been investigated and found to be not true.
Reports relevant for SEICAT may not capture the true level at which the alien is causing an impact. Often, individual people are interviewed to obtain information on the alien's impacts and their experience may not represent the true state of the entire community. this presents a form of linguistic uncertainty present in the guidelines). Rather, we assert that assessors should focus only on the quality of the report, given the invasion scenario.

Source of uncertainty
Data used to derive EICAT scores are most frequently sourced from primary (i.e. not secondary referencing) and grey literature.
A decrease in the size of human activity may not be quantified but inferred from the evidence. For example, studies of diseases and parasites transmitted by aliens affecting humans will rarely report quantitatively on how they affect activities, although the authors may infer such effects. Data used to derive SEICAT scores are more likely to be anecdotal forms of evidence; personal communications and media reports often contain information of relevance to SEICAT. Although anecdotal evidence may be thought of as lower quality information , given people can directly communicate behavioural changes in response to alien species, evidence deriving from such information may reveal the true state of impact. However, as SEICAT uses the change in activity size as the measure of impact, information on how many people participate in the activity and on the local population size, is required for high confidence reports.

Spatial scale
Understanding if the impact has been recorded at a relevant spatial scale to capture the assigned impact magnitude accurately.
Assessors should ask if the study was conducted on a scale over which native species in the region of interest can be characterised. This requires a basic understanding of what constitutes a local population for a given species. A population can be difficult to delimit given suitable habitat for a species is usually fragmented across a landscape and further, populations are often managed within geopolitical jurisdictions. It may be particularly difficult to discern if an alien taxon causes a decline in population from available data with high confidence. Surveys may make it appear as if the population has declined, when in reality, species that are mobile may avoid areas when an alien species occurs. The 'focal region' for SEICAT can be highly variable given densities of human communities. Impacts may be assessed on scales ranging from small villages to large metropolitan areas. Therefore, data about the number of people affected (i.e. those that reduce their activity) and the population size across the geographic scale should be included in the assessment when the information is available.

Source of uncertainty
For EICAT

Temporal scale
In earlier guidelines, the temporal scale at which the impact was previously recorded was not considered important since the ICAT frameworks assign magnitude, based on the highest impact (Hawkins et al. 2015;Bacher et al. 2018). However, it is important to assess how the "true" impact varies with the temporal scale. Uncertainty at the temporal scale is important in two aspects, whereby the study may not capture a relevant time period to detect maximum impact or the study provides an inaccurate snapshot that is not reflective of impact; for instance, the study focused on one season or was just too short to be able to capture any change.
Changes in native population size may be limited to only a short period (e.g. seasonally), which generally has little effect on reducing the overall population size. Assessors should consider that the impact report may provide only a snapshot in time and determine how relevant the impact is at a suitable temporal scale.
The same issues relating to temporal scale for EICAT are relevant for SEICAT.

Coherence of evidence
At the individual impact report level, assessors must determine whether all the evidence points towards the same direction or whether evidence may be contradictory or ambiguous. A study relevant for EICAT may present conflicting evidence based on different variables measured to determine impact. For instance, a study measuring more than one physiological variable of a native species in response to an alien may indicate both negative and positive effects (e.g. a reduction in height of plant growth but increase in leaf area size).
There may be conflicting reports from individuals as to whether an alien species is causing reductions in activity size.
relevant to uncertainties when summarising impacts at the species level (see below). Additionally, it must be considered how the written synthesis of ICAT assessments and the justifications of classifications may propagate linguistic uncertainty further.

Directionality of uncertainty
Uncertainty in impact assessments means that the true impact can be higher or lower than the one assigned. However, assessors may be confident that an impact magnitude is not lower than the one assigned, but could be higher (or vice versa). Thus, uncertainty can be asymmetrically distributed around the assessment value; it may be larger in one direction than in the other. This directionality aspect of uncertainty is currently not captured using the confidence scores, yet may provide important insight to impacts. Using EICAT as an example, it may be that the assessor assigns a minor impact score (MN) to an impact record that robustly demonstrates that an alien taxon affects the performance of individuals of a native species and, thus, is not negligible (i.e. not MC). However, given the study did not address (i.e. measure) whether the impact is causing a decline in the local population, it is not possible to know whether the 'true' impact caused by the alien taxon is higher (MO, MR or MV). For instance, studies that assess physiological responses of native species to invasive species do not necessarily relate such effects beyond the individual (i.e. effects on fitness resulting in declining populations) (Graham et al. 2012). Such cases are quite distinct to impact records that sought to quantify population responses to an alien species, yet found no evidence in support of population decline. Since documenting directionality in uncertainty related to each impact record may improve our overall understanding of potential impacts, this information may be particularly useful once several records of impact are obtained for a single species. Directionality in uncertainty, therefore, presents an important facet of uncertainty to recognise when using the ICAT schemes.

Uncertainties relevant at the species level
Presently, there is no consideration of uncertainty beyond the confidence score assigned to each impact report (IUCN 2020a;Hawkins et al. 2015;Bacher et al. 2018). The ICAT assessment schemes adopt the precautionary principle, whereby the overall classification of an alien taxon is based on the highest magnitude the taxon has reached. Therefore, there is no distinction between species with the same highest impact magnitude, regardless of whether there are few or many accounts of impact. It is also important to acknowledge additional sources of uncertainty which influence the ability to conduct assessments for alien taxa. As these uncertainties occur beyond the individual impact report level, they are not captured by the confidence score as currently described. Uncertainties due to the biases in the collected and the existing (or produced) impact reports contribute to the quality of final assessments, making them of direct relevance when comparing taxa based on ICAT scores. If alien taxa are com-pared, based on the highest magnitude they have been observed to cause (Hawkins et al. 2015;Bacher et al. 2018), it is pertinent that their highest impact magnitude caused in nature is documented and that these data have been adequately collected and assessed using the ICAT frameworks. It is likely that the more impact reports for an alien species that are produced, collected and assessed, the higher the chance that the maximum impact of the alien taxon will be detected and correctly classified. We recognise three important aspects to evaluate when looking at species-level comparisons: biases in existing data, data collection and data assessment.

Biases in the existing data
The availability of impact records will vary widely within (Evans et al. 2018b) and between taxa (Vilà et al. 2010) and will not necessarily be reflective of impact severity (Evans and Blackburn 2019). Indeed, of the larger taxonomic groups that have been assessed (amphibians, bamboos, birds), the majority of species are classified as data deficient (Evans et al. 2016;Kumschick et al. 2017;Canavan et al. 2019). As biases in biological records (Isaac and Pocock 2015) and within invasion biology are evident (Pyšek et al. 2008), some taxa will be disproportionately represented when conducting literature searches necessary for ICAT assessments. Gaps may be driven by funding availability with regions associated with higher economic status investing more in invasive species research (Pyšek et al. 2008;Bellard and Jeschke 2016). Further, it is usual for a lag time between an alien species becoming established and research effort on the species in the new environment to be observed Lyons et al. 2019). Due to this and other reasons, such as the nature and duration of the peer-review process, the dissemination of impacts reports is often delayed . Even well-studied species may not have impacts measured that can be easily transferred to ICAT scores, potentially rendering it data deficient or with few reports from which to derive an impact magnitude. For instance, alien species may be well documented to impact via various mechanisms (e.g. predation, competition) under laboratory settings, but poorly represented under natural conditions. Often, biological aspects, related to mechanisms of impact, are well-researched (e.g. dietary overlap, aggressive behaviour) for alien species, but the effects on native biodiversity are not measured, rendering such studies irrelevant to EICAT assessments. Our main suggestion regarding the bias in-or lack of-existing and relevant impact data, is to adapt future impact reports to EICAT criteria: studies should focus more on the changes in the impacted native populations (in natural conditions) and less on the alien populations.

Biases in the data collection
Inconsistencies amongst assessors may be driven from the initial stage of data collection (the literature review), with variation attributed to different search strategies employed by individual assessors (Kumschick et al. 2017). Reproducibility in science is a major topic of discussion (Baker 2016;Fanelli 2018) and how systematic literature searches are conducted is often poorly detailed leading to non-reproducible results (Cooper et al. 2018;Faggion and Diaz 2019). Assessors should be specific on how they conduct their literature searches to promote transparency, which in turn, will facilitate more robust inter-specific comparisons if data requires additional reviewing. Furthermore, documentation of the sources used to score species and the final data for assessments should be published with studies using the assessment schemes (see also . Another major difficulty in data accessibility may arise from language barriers that affect the assessor's ability to collate impact reports. This is likely to be particularly applicable for SEICAT assessments, where it is expected that relevant reports of impacts on human well-being will, more often, be published in local languages. Discussions with people in local languages to identify socio-economic issues arising from the presence of alien species may facilitate assessments of species that are otherwise data deficient and help better understand additional human dimensions of biological invasions. Much regional evidence on the impacts of alien species will be confined to sources of information, such as local government reports and student theses.

Biases in the data assessment
Additional inconsistencies amongst assessors may occur because the criteria of the ICAT frameworks are interpreted and applied differently; individual assessors will inevitably introduce their own level of bias to the process of both assigning impact categories and confidence scores. A recent study by González-Moreno et al. (2019) found variation in scoring species' impacts amongst assessors for different assessment schemes, including EICAT. Although a level of subjectivity is inevitable, some of this uncertainty may be reduced through improvement in the protocol, such as the refinement of guidelines, which is already reflected in the succession of EICAT guidelines (Blackburn et al. 2014;Hawkins et al. 2015;IUCN 2020a). However, clarification about the changes and ensuring these are effectively communicated will be important to maximise consistency (see Volery et al. 2020, as the application of different versions of the guidelines may further lead to inconsistencies across different assessments. Conducting workshops, training sessions and developing online tools that help guide assessors through the process-giving examples where uncertainty is most likely to arise-might help reduce these uncertainties. Refinements can be made as feedback from assessors identifies more issues that require additional explanation or adaptation. It is worth noting that, given the variation observed amongst assessors when applying scoring schemes (Matthews et al. 2017;González-Moreno et al. 2019), confidence scores are likely to be subject to a similar level of inconsistency. The accompanying probabilities ( Table 1) to each of the three qualitative confidence scores are intended to reduce variation in the interpretation of terms. Indeed, differences in the interpretation of the descriptions of uncertainty are known to occur amongst individuals (Budescu and Wallsten 1985). Presenting linguistic descriptions and corresponding likelihoods can, therefore, reduce the misinterpretation of confidence scoring (Budescu et al. 2014). The degree of consistency amongst assessors when assigning confidence scores should be examined to determine whether refining the expressions of confidence is necessary to reduce potential misinterpretation.

Conclusions
To produce robust impact assessments and facilitate the comparison of impacts between taxa, procedures must adequately account for uncertainties (McGeoch et al. 2012). We have highlighted key sources of uncertainty to consider when conducting the ICAT assessments and emphasised the importance of acknowledging all forms of uncertainty even when not directly relevant to informing confidence scores. As uncertainties propagate throughout the various stages of any ICAT assessment (deriving from both the impact measurer/reporter and the ICAT assessor), it is important that they are clearly defined and acknowledged to improve the overall impact assessment procedure. However, it should be noted that it will be impossible to address all types of uncertainty in any framework due to unforeseeable changes in the system under investigation or other unknown unknowns.
As the ICAT frameworks become more readily applied across different taxonomic groups, uncertainties must be appropriately considered to improve the overall ability to correctly classify impacts. By improving the consideration of uncertainty under the ICAT guidelines, we may increase the functionality of the tool for researchers and practitioners. All other things being equal (i.e. control effort, cultural values, positive impacts etc.), species that will be the best candidates for prioritisation will be those that have the highest impact with high corresponding confidence.