Corresponding author: Pablo González-Moreno (
Academic editor: P. Hulme
Standardized tools are needed to identify and prioritize the most harmful non-native species (
González-Moreno P, Lazzaro L, Vilà M, Preda C, Adriaens T, Bacher S, Brundu G, Copp GH, Essl F, García-Berthou E, Katsanevakis S, Moen TL, Lucy FE, Nentwig W, Roy HE, Srėbalienė G, Talgø V, Vanderhoeven S, Andjelković A, Arbačiauskas K, Auger-Rozenberg M-A, Bae M-J, Bariche M, Boets P, Boieiro M, Borges PA, Canning-Clode J, Cardigos F, Chartosia N, Cottier-Cook EJ, Crocetta F, D’hondt B, Foggi B, Follak S, Gallardo B, Gammelmo Ø, Giakoumi S, Giuliani C, Guillaume F, Jelaska LS, Jeschke JM, Jover M, Juárez-Escario A, Kalogirou S, Kočić A, Kytinou E, Laverty C, Lozano V, Maceda-Veiga A, Marchante E, Marchante H, Martinou AF, Meyer S, Michin D, Montero-Castaño A, Morais MC, Morales-Rodriguez C, Muhthassim N, Nagy ZA, Ogris N, Onen H, Pergl J, Puntila R, Rabitsch W, Ramburn TT, Rego C, Reichenbach F, Romeralo C, Saul W-C, Schrader G, Sheehan R, Simonović P, Skolka M, Soares AO, Sundheim L, Tarkan AS, Tomov R, Tricarico E, Tsiamis K, Uludağ A, van Valkenburg J, Verreycken H, Vettraino AM, Vilar L, Wiig Ø, Witzell J, Zanetta A, Kenis M (2019) Consistency of impact assessment protocols for non-native species. NeoBiota 44: 1–25.
Coupled with the increasing evidence of adverse impacts exerted by some non-native species (
Robust
Several international and national organizations and research groups have developed
Characteristics of impact assessment protocols used in the study. Each protocol is characterized in terms of the a) taxonomic group the protocol could be used for, b) the impact categories included (environmental alone or environmental and socio-economic), c) the final scoring scale (i.e. three levels, five levels, and more than 5 levels), d) whether the final score is based on the maximum score of impacts, e) whether the protocol included questions on species spread as part of a risk assessment (yes/no), f) the number of questions contributing to the final score, and g) the mean assessor expertise on species required to fill the questionnaire (1–5 scale based on 63 online anonymous questionnaire responses).
Protocol | Full name | Taxonomic groups | Impact categories | Final scoring scale | Final scoring based on maximum score | Spread questions included | Number of questions | Expertise on species required | Reference |
---|---|---|---|---|---|---|---|---|---|
BINPAS | Biological Invasion Impact/Biopollution Assessment | Aquatic animals | Environmental | 5 | yes | yes | 5 | 3.50 | ( |
EICAT | Environmental Impact Classification for Alien Taxa | All | Environmental | 5 | yes | no | 9 | 3.37 | ( |
EPPO-EIA | European Plant Protection Organisation-Environmental Impact Assessment for plants (EPPO-EIA-PL) and terrestrial invertebrates (EPPO-EIA-IN) | Terrestrial plants and invertebrates | Environmental | 5 | yes | no | 8 (Plants); 9 (invert.) | 3.16 | ( |
EPPO-PRI | EPPO-Prioritization scheme | Plants | Environmental and socio-economic | 3 | yes | yes | 11 | 3.00 | ( |
FISK (and related) | Fish Invasiveness Screening Kit (FISK); Freshwater Invertebrate Invasiveness Screening Kit (FI-ISK); Marine Fish Invasiveness Screening Kit (MFISK); Marine Invertebrate Invasiveness Screening Kit (MI-ISK) | Aquatic animals | Environmental and socio-economic | 3 | no | yes | 49 | 4.12 | ( |
GABLIS | German-Austrian Black List Information System | All | Environmental | 3 | yes | yes | 12 | 3.22 | ( |
GB-NNRA | Great Britain Non-native Species Risk Assessment | All | Environmental and socio-economic | 5 | no | yes | 33 | 3.90 | ( |
GISS | Generic Impact Scoring System | All | Environmental and socio-economic | >5 (discrete with max 60) | no | no | 12 | 3.46 | ( |
Harmonia+ | Belgian risk screening tools for potentially invasive plants and animals | All | Environmental and socio-economic | >5 (continuous | yes | yes | 20 | 3.46 | ( |
ISEIA | Belgian Invasive Species Environmental Impact Assessment | All (not marine for this study) | Environmental | 3 | no | yes | 4 | 2.81 | ( |
NGEIAAS | Norway Generic Ecological Impact Assessment of Alien Species | All | Environmental | 5 | yes | yes | 11 | 4.34 | ( |
A few comparative analyses have addressed differences in the structure of impact assessment protocols (
Eleven commonly used scientifically based protocols developed or applied in Europe for the evaluation of
Each protocol was characterized according to several variables (Table
A total of 57 species from different taxonomic groups not native to terrestrial, freshwater, and marine environments in Europe were selected (Suppl. material
There is a large variation in methods to implement the different protocols; some are available as downloadable freeware (-ISK toolkits, the ‘NAPRA’ version of the GB-NNRA), as online applications (e.g. Harmonia+, BINPAS), whereas some have to be constructed following the text guidelines (e.g. GISS, EICAT), and others can be obtained as spreadsheets (e.g. GB-NNRA) or databases (e.g. NGEIAAS). To harmonize use of the protocols and facilitate data retrieval, a comprehensive Excel® spreadsheet template was developed to include all the protocols (see Suppl. material
Using the protocols selected in the spreadsheet template, 89 assessors independently assessed between three to 11 species (mean = 3.9) of the taxonomic group in their area of expertise (i.e. terrestrial plants, aquatic plants, terrestrial vertebrates, terrestrial insects, other terrestrial invertebrates, freshwater invertebrates, freshwater fish, marine species and pathogens) (Suppl. material
Before retrieving the data, each assessment was checked for completeness. Once all
For each
Differences in the mean
Similarities in the scoring of
The mean coefficient of variation (
Coefficient of variation (
According to Tukey
Mean regression coefficient and confidence interval (95%) of taxonomic groups (random effects) in the best linear mixed model explaining the coefficient of variation of scores of 57 invasive non-native species for 11 different protocols including all significant species, assessor and protocol characteristics (see Table
Average coefficient and Akaike weights for each species, assessor and protocol variable within the best linear mixed models (AIC
Variable | Coefficient | Adjusted SE |
|
|
Weight |
---|---|---|---|---|---|
Intercept | 0.36 | 0.06 | 5.76 | <0.001 | |
Number of assessments | 0 | ||||
|
|||||
Web of Science records (available knowledge) | -0.06 | 0.05 | 1.18 | 0.24 | 0.06 |
|
|||||
Mean assessor expertise | -0.04 | 0.02 | 2.21 | 0.03 | 0.14 |
0 | |||||
|
|||||
Scoring scale | See results section | 1 | |||
Expertise required | -0.14 | 0.02 | 7.76 | <0.001 | 1 |
Using maximum impact score (yes-no) | -0.12 | 0.02 | 4.93 | <0.001 | 1 |
Spread (yes-no) | 0.12 | 0.05 | 3.57 | <0.001 | 0.95 |
Impact type | 0 | ||||
Number of questions | 0 |
The pair-wise correlations in
Spearman correlation matrix and hierarchical cluster of species scorings for the protocols common for all species. The color scale indicates the correlation between the species scorings obtained for each protocol pair. In brackets, the mean of all pair-wise correlations.
Spearman correlation matrix and hierarchical cluster of the species scorings for the protocols common per species group. The color scale indicates the correlation between the species scorings obtained for each protocol pair. In brackets, the mean of all pairwise correlations per group.
The comparison of impact assessment protocols for
Scoring consistency across assessors and for some taxonomic groups was surprisingly low. It is not clear why these large discrepancies occurred even when the assessors were experts in invasion biology within their taxonomic domain. Many factors can influence the interpretations of context dependence found in the scientific literature, which can lead to subjective and inconsistent answers even amongst expert assessors (
Part of the variability in consistency was explained by protocol characteristics and the approaches implemented. Protocols with three score levels were more likely to show consistency among assessors than those with five or more levels. However, a three-category scoring system might not be sufficient to discriminate between
Protocols containing questions that required greater expertise on the species yielded higher scoring consistency than simpler protocols. Protocols requiring greater expertise demanded very detailed information about the species (e.g. expected population lifetime in NGEIAAS) that, when available, is very likely to be available only in few studies. Owing to the restricted number of sources of information, the variability in the final score might be low. Complex protocols might be less user-friendly and more time-consuming, but this in itself could increase focus and decrease subjectivity. Exceptions exist, e.g. the -ISK screening (
Regarding assessor and
The high inconsistency found among assessor’s scores raises high concerns and suggests that assessments conducted by single assessors should be interpreted with caution (
Variations among protocols in species scoring are mainly due to the inclusion, or not, of socio-economic impacts. Although socio-economic and environmental impacts are generally correlated (
Among all protocols, Harmonia+, FISK and GABLIS led to very different scores in comparison to the other protocols. This difference was partly related to the different impact categories considered but also to the inclusion of questions beyond impact (e.g. management in GABLIS and FISK). Finally, the GB-NNRA protocol showed a variable relation with other protocols across taxa: low correlation with protocols only considering environmental impacts for plants and terrestrial invertebrates but high for vertebrates. The final score in the GB-NNRA was not automatically calculated as in the other protocols. Instead, assessors were asked to provide overall summary scores and confidence rankings for the
Several key factors should be taken into account when selecting or designing a
Part of the inconsistency might also come from the way the protocol is used in practice (e.g. standardized forms, clear guidelines, selection of assessors, individual vs. group assessments). We propose three main ways to reduce this type of inconsistency. First, irrespectively of the protocol, selecting a group of assessors with high expertise will yield more consistent results. Second, inconsistencies due to linguistic uncertainties (e.g. definitions, formulations, rating) can be reduced by improving the guidelines and with adequate training of the assessors (
This article is based upon work from the COST Action TD1209: Alien Challenge. COST (European Cooperation in Science and Technology) is a pan-European intergovernmental framework. The mission of COST is to enable scientific and technological developments leading to new concepts and products and thereby contribute to strengthening Europe’s research and innovation capacities. PGM was supported by the CABI Development Fund (with contributions from ACIAR (Australia) and Dfid (UK) and by Darwin plus, DPLUS074 ‘Improving biosecurity in the SAUKOTs through Pest Risk Assessments’. MV by Belmont Forum-Biodiversa project InvasiBES (PCI2018-092939). CP by Sciex-NMSch 12.108. JMJ and WCS by BiodivERsA (FFII project; DFG grant JE 288/7-1). JMJ by DFG project JE 288/9-1,9-2. CR and MB by Fundação para a Ciência e a Tecnologia grants SFRH/BPD/91357/2012 and SFRH/BPD/86215/2012, respectively. PS by MESTD of Serbia, grant #173025. JP by RVO 67985939 and 17-19025S. JCC was supported by a starting grant in the framework of the 2014 FCT Investigator Programme (IF/01606/2014/CP1230/CT0001).
Supplementary materials
statistical data
Figure S1: hierarchical cluster of the species scores for the six protocols common to all taxonomic groups. Figure S2: hierarchical cluster of the species scorings for plants and aquatic animals without correcting for sample size bias. Table S1: list of non-native species. Table S2: correlation analyses.
Supplementary materials
Spreadsheet template
Spreadsheet template to fill the 11 impact assessment protocols for non-native species considered in the study.