The relevance of using various scoring schemes revealed by an impact assessment of feral mammals

Impact scoring schemes are useful for identifying to what extent alien species cause damage. Quantifying the similarity and differences between impact scoring schemes can help determine how to optimally use these tools for policy decisions. Using feral mammals (including rats and mice) as a case study, environmental and socio-economic impacts were assessed using three schemes, namely the Generic Impact Scoring System (GISS), Environmental Impact Classification for Alien Taxa (EICAT) and Socio-Economic Impact Classification for Alien Taxa (SEICAT). The results show that socio-economic impacts scores differ between the respective schemes (GISS and SEICAT) possibly because they assess different aspects of social life and economy. This suggests that both scoring schemes should ideally be applied in concert to get a complete picture of socio-economic impacts. In contrast, environmental impact scores are correlated between GISS and EICAT assessments and this similarity is consistent over most mechanisms except for predation and ecosystems, suggesting that one scoring scheme is sufficient to capture all the environmental impacts. Furthermore, we present evidence for the island susceptibility hypothesis as impacts of feral mammals were found to be higher on islands compared to mainlands.


Introduction
Alien species cause various and sometimes devastating changes to the environment where they are introduced and influence social and economic aspects of human life (e.g. Pyšek and Richardson 2010, Vilà et al. 2010. To minimise the negative effect of alien species, the Convention on Biological Diversity in its Aichi target 9 has proposed steps to mitigate their impacts, including identifying harmful alien species and prioritising their management (www.cbd.int/sp/ targets/). To reach these goals, standardised measures for impact assessment and species prioritisation are needed.
This need has recently been met by the development of impact scoring schemes (e.g. Hawkins et al. 2015, Nentwig et al. 2016; see Leung et al. 2012 for a review on risk assessments more broadly). Such schemes are typically based on published evidence of impacts or expert opinion and are meant to be transparent, robust and easy to use (Vanderhoeven et al. 2017). However, they often differ in the parameters used for the assessment and the way published evidence is translated into impact magnitude (e.g. Kumschick et al. 2017, Turbé et al. 2017. Differences in the outcomes using these schemes can potentially influence policy decisions and, for this reason, there is a need to quantify whether impact assessment schemes are comparable. Three scoring schemes are considered in this study. The Generic Impact Scoring System (GISS) was first developed to assess the environmental and economic impact of alien mammals in Europe  and is one of the most widely used scoring schemes to date (Nentwig et al. 2016). It has been applied to various taxa including vertebrates, plants and invertebrates (e.g. Kumschick and Nentwig 2010, Vaes-Petignat and Nentwig 2014, Measey et al. 2016, Rumlerová et al. 2016. By comparison, the Environmental Impact Classification for Alien Taxa (EICAT), developed by Blackburn et al. (2014), focuses only on the environmental impact of species. It was adopted by the IUCN (https://portals.iucn.org/congress/motion/014) to enable a classification of all alien species worldwide (Hawkins et al. 2015, Evans, Kumschick and. The third scoring scheme is the Socio-Economic Impact Classification of Alien Taxa (SEICAT), which exclusively assesses the socio-economic impact of alien species .
As a case study to compare the impact scoring schemes, we use feral mammals (including mice and rats) alien to South Africa. Alien mammals are known to cause damage to many ecosystems worldwide (Howald et al. 2007, Witmer et al. 2007, Skead et al. 2011, Capellini et al. 2015 and they have been shown to have the highest impacts across various taxonomic groups in a European study . For example, feral dogs (Canis familiaris) have contributed to the decline of turtle and tortoise species in India, Costa Rica as well as the Galapagos Archipelago (e.g. MacFarland et al. 1974, Fowler 1979. In South Africa, they are known to transmit diseases to jackals and bat-eared foxes (Sabeta, Bingham and Nel 2003). Furthermore, the impacts of alien mammals have been reported as particularly severe on islands , Hays and Conant 2007, Reaser et al. 2007). Impacts of alien species in general are thought to be more detrimental on island environments by causing higher numbers of extinctions due to higher endemism, simpler food webs and slow diversification rates of species compared to mainlands . This is known as the island susceptibility hypothesis (Elton 1958, Jeschke et al. 2012. The aims of our study were threefold. Firstly, we compared the outcomes of the three scoring schemes by a) comparing environmental and socio-economic impacts of feral mammals (including mice and rats) between the respective schemes and b) disentangling differences in impact scores between impact mechanisms (such as competition and predation) for environmental impacts, expecting to find similar levels of impacts between the schemes. Secondly, a test of the island susceptibility hypothesis was conducted by looking at the differences between socio-economic and environmental impacts caused on islands and mainlands, hypothesising that impacts are higher on islands. Lastly, following the finding that some taxa receive more research attention than others (Pyšek et al. 2008), it was determined whether there is a publication bias in our study. However we do not expect a bias since mammals are generally well studied.

Species selection and literature search
Using data from various sources, including Spear and Chown (2009), Picker and Griffiths (2011), Spear et al. (2011), Skead et al. (2011, Department of Environmental Affairs (2016) and Invasive Species South Africa (www.invasives.org.za), a list of domestic mammals alien to and feral in South Africa was compiled. This includes eight species which were initially introduced for their use as pets and/or are of agricultural significance. Additionally, we included rats (Rattus rattus and R. norvegicus) and mice (Mus musculus) due to their global significance as invasive pests ( Figure 1). These species all have established alien populations in South Africa, but also represent some of the most prevalent domestic mammals with feral populations globally and some of the most damaging alien mammals ) and are referred to as "feral mammals" in this manuscript for simplicity. Even though the selection of species was based on aliens in South Africa, the literature search was based on these species' global alien range and the classification therefore represents the entire alien range.
In order to assemble information on the global impacts of these species, a review of published literature was undertaken. A search string, developed by Evans et al. (2016), was adopted (see also Appendix 1 for further detail) and papers were selected based on manual filtering of titles and abstracts. Databases searched included Google Scholar, Scopus and Web of Science. Publications on the impact of domestic mammals or pets in captivity were excluded and only impacts of stray or feral populations were considered. The reference list of the papers selected was further analysed to search for additional records of impacts. The search was terminated when the same sources were repeatedly found. All impact references were assigned a score by the main assessor and Figure 1. Total GISS environmental and socio-economic impacts of invasive mammal species alien to South Africa. Total scores represent summed scores of maximum impacts given in each subcategory for each species separately. Abbreviations represent impact scores using EICAT (black) and SEICAT (light grey) as minor (MN), moderate (MO), major (MR) and massive (MV). checked by a second assessor. Discussions on scores only occurred when there were disagreements or uncertainties around the score. For each impact found, we noted if it occurred on an island or mainland.

Impact schemes, categories and levels
GISS includes both environmental and socio-economic impacts, with EICAT and SEI-CAT focusing on one impact type each (see Suppl. material 1: Table S1 for the differences in the descriptions of the impact categories of the three schemes). GISS measures impacts by assessing the damage each species causes using six environmental (impacts on plants or vegetation; predation; competition; transmission of diseases; hybridisation; impacts on ecosystems; labelled E_GISS hereafter) and six socio-economic (impacts on agricultural production; animal production; forestry production; human infrastructure and administration; human health; human social life; SE_GISS) impact categories with six subcategories each (based on impact mechanisms or socio-economic sectors) Nentwig 2010, Nentwig et al. 2016). Impact magnitudes range from 0 to 5, zero meaning that no known or detectable impacts were recorded whereas scores of five were equal to the most severe impacts. For GISS, scores were aggregated in two ways: a) Maximum scores refer to highest scores a species achieved in any subcategory and b) sums of the maximum scores received in all subcategories of the environmental and socio-economic categories, respectively and these give an overall potential impact score, termed summed score and ranging from 0 to 30.
In addition, total scores per species referred to summed scores of the socio-economic and environmental scores combined, with a potential range from 0 to 60 .
EICAT focuses on environmental impacts consisting of 12 mechanisms and five impact magnitudes, namely minimal concern (MC), minor (MN), moderate (MO), major (MR) and massive (MV), where MC is equivalent to no detectable impact on native individuals and MV equates to most severe impacts equalling a community compositional change (Blackburn et al. 2014, Hawkins et al. 2015. According to the guidelines by Hawkins et al. (2015), only the maximum impacts across all mechanisms per species were considered for this scheme. Lastly, SEICAT  investigates the socioeconomic effects of species and is similar to EICAT in terms of impact levels. SEICAT is based on alien species' influence on all constituents of human well-being by using changes in peoples' activities to evaluate the impacts. As for EICAT, only maximum impacts were analysed. Impact scores for EICAT and SEICAT were transferred into numerical scores from 1 to 5 (where MC was translated to 1 and MV translated to 5) for statistical analyses.

Statistical analyses
Maximum and summed environmental scores per species were used to compare E_ GISS to EICAT. The same process was followed for the socio-economic comparison of SE_GISS and SEICAT. Paired Wilcoxon's signed rank tests were used to test the similarity of the maximum and summed scores obtained in GISS to maximum scores in EICAT and SEICAT respectively.
To examine the differences in magnitude between environmental and socio-economic impacts, EICAT scores were compared to SEICAT scores and E_GISS scores to SE_GISS scores using a non-paired Wilcoxon's signed rank test. For the GISS comparisons, only maximum scores were used for this test.
In order to assess what drives the potential similarity and differences between E_ GISS and EICAT scores, scores pertaining to specific mechanisms were contrasted. This was done by unifying similar mechanisms across the schemes (Table 1). A single mechanism in GISS, for example, is sometimes represented by more than one mechanism in EICAT. As we are interested in whether there are differences in how the two schemes treat each record of impact, each record was treated as one impact entry (as opposed to a maximum per species and mechanism). The scores relating to each of these were compared using paired Wilcoxon's signed rank tests.
A non-paired Wilcoxon's signed rank test was conducted to test the difference between impacts caused on islands and mainlands. Due to the small sample size when analysing impacts per species, each publication containing information on impact was used as a separate record instead of using maximum impacts per species.
A Kendall's tau was used to examine the relationship between the aggregated scores per species and the number of publications. This was done separately for each scoring scheme to test for publication bias . All analyses were conducted in R studio (version 0.99.903) and R (version 3.3.1) (R Core Team 2016). Table 1. Concatenation of the environmental impact mechanisms in GISS and EICAT that are similar following Nentwig et al. (2016) and Hawkins et al. (2015), as used to compare detailed outcomes of the two scoring schemes for each source or information. "Interaction with other alien species" in EICAT could not be attributed to specific mechanisms in GISS and was therefore not included here.

Results
The total impact of the species using GISS ranged from 15 to 37 with the highest impact being from Sus scrofa and the lowest from Felis catus ( Figure 1). No difference between the scoring schemes could be found when comparing EICAT scores to maximum E_GISS scores (paired Wilcoxon's signed rank test; V = 7.5, p = 0.424). Sixty four percent of the species had equivalent scores and, where differences occurred, it was only by a single magnitude. In contrast, when comparing EICAT scores to summed E_GISS scores, we found a significant difference (V = 66, p = 0.038). E_GISS summed scores ranged from 9 to 23 and all but three species (Bos taurus, C. familiaris and Equus asinus scored MR) had an impact magnitude of MV under EICAT.
However, SEICAT and maximum scores of SE_GISS were significantly different (paired Wilcoxon's signed rank test; V = 50.5, p = 0.015). Only 9% of the species' scores were equivalent, whereas more than 81% of the species scored higher in GISS than in SEICAT. This higher score was mostly by a single magnitude (e.g. 4 in GISS and MO in SEICAT), except for S. scrofa where the schemes differed by two magnitudes (5 versus MO). A difference remained when comparing SEICAT with summed SE_GISS scores (V = 54, p = 0.007). While summed SE_GISS scores in this case never exceeded 15, SEICAT scores ranged from MN to MR.
When testing how EICAT and GISS treat scores for various mechanisms mentioned in Table 1  When comparing environmental and socio-economic impacts, EICAT scores were significantly higher than SEICAT scores (Wilcoxon's signed rank test, W = 1.5, p < 0.0001) and the same was true using GISS (W = 105.5, p = 0.001).
A total of 318 papers were used for impact scoring (see Appendix 2). An average of 32 publications per species was found for environmental impacts in comparison to 7.5 publications per species for socio-economic impacts. None of the environmental impact

Discussion
Firstly, following the publication of EICAT (Hawkins et al. 2015) and SEICAT , this is the first application of these schemes for mammals. Until now, EICAT has only been used to assess the impacts of amphibians and birds introduced globally (Evans et al. 2016) and gastropods alien to South Africa (Kesner and Kumschick in revision) and SEICAT exclusively for the latter two Kesner and Kumschick in revision). Our study provides a starting point to adding another taxonomic group to the list of alien species with evidence based impact classifications and shows that EICAT and SEICAT are applicable to mammals. Furthermore, this study provided support for the already commonly used scoring scheme GISS (Nentwig et al. 2016) and adds assessments of many mammal species which were excluded in previous studies (Nentwig et al. 2010 excluded domesticated mammals).
Besides proving the applicability of the schemes to further taxa, our analysis reveals which impact measures might be necessary and most useful for management decisions. The comparison of environmental and socio-economic impact magnitudes, for example, shows that it is not sufficient to study one aspect to get a full picture of impacts (see also Vilà et al. 2010. Previous studies, such as those by  and Kumschick et al. (2015), found that, within schemes, environmental and socio-economic impacts were comparable, with species with high environmental impacts generally showing high economic impacts as well. This study found that feral mammals generally have larger environmental than socio-economic impacts. Yet, the difference in environmental and socio-economic impacts does not mean that the socio-economic impacts are low  with some species, such as C. familiaris, still scoring MR. Furthermore, different societal sectors (which includes conservation, health, agriculture and social) also have different priorities and for that reason, will be interested in different aspects of impacts covered by different scoring schemes . The scoring schemes used here (Kumschick et al. 2012), as well as others previously developed (e.g. D'hondt et al. 2015) therefore allow for the explicit weighting of categories. However we do not consider this to be the task of scientists, but rather the decision-makers and therefore consider all sectors to be of equal weight for this study.
The difference in magnitude recorded for socio-economic impacts between the scoring schemes has various possible causes. While both schemes are based on the same literature and are able to score all socio-economic impacts found for the selected species, GISS and SEICAT are based on different endpoints and use different "currencies" to compare impacts, with GISS addressing the actual economic damage of species and SEICAT transcribing these changes into effects on human well-being and activities being affected by the damage (Nentwig et al. 2016). SEICAT has thus moved away from a mainly economic and value-driven (monetary) approach and assesses how people react to the damage caused by the invasive species rather than the actual damage itself. As an example, the damage that feral donkeys (E. asinus) cause to agricultural production might seem high at first, as seen in the GISS impact scores of 4, but it has not stopped farmers from continuing to produce agricultural products in any way (leading to low SEICAT scores of MN) (Tisdell and Takahashi 1988). Hence, SEICAT assumes that if peoples' behaviour does not change as a result of the impact caused, the impact is not bad enough; or conversely, an impact does not have to cause huge monetary costs to be perceived as bad by certain vulnerable communities which have limited possibilities to cope with the problem . As both scoring schemes cover important aspects of socio-economic impacts, but in fundamentally different ways with GISS focusing on actual damage and monetary losses and SEICAT focusing on resulting changes to activities more generally and peoples' well-being, we suggest that using a combination of GISS and SEICAT assessments could prove useful to obtain a more complete and distinct picture of socio-economic impacts of alien species. Although this might not be the most practical solution, both schemes rely on the same evidence base. Alternatively, one scheme could be chosen based on the specific needs and scope of the assessment, with the respective endpoints assessed by each in mind.
In contrast, environmental impact scores were comparable between EICAT and GISS in our study, especially when the maximum impact was considered, suggesting only one scheme is needed. This supports the decision by the IUCN to adopt one scheme, namely EICAT, for a global classification of all alien taxa according to their environmental impacts. However, a previous study comparing the two schemes using amphibians as a case study highlights important differences for certain mechanisms between the schemes which should be considered in future applications ). These differences were mainly attributed to uncertainties in the scoring of disease impacts in general (for transmission of diseases) and the differences between the two schemes in assigning the highest impact levels for hybridisation, with GISS depending on the size of the hybrid population and EICAT only referring to the fertility of F1 offspring. The main difference, which was found between certain mechanisms in this study, could be attributed to the split in some mechanisms in GISS (namely "Impacts on ecosystems" and "Impacts on animals through predation, parasitism or intoxication") into several mechanisms in EICAT and which allows for more detailed assessments. Furthermore, EICAT consistently focuses on the recipient native community, while GISS assesses changes in nutrient fluxes and other abiotic changes as well, without a necessary link to the native biota. This can, in certain cases (mainly for plant impacts), be an advantage as studies sometimes lack a link from changes in nutrient availability to effects on the native community (e.g. Castro-Diez et al. 2009 for two exotic trees). On another note, EICAT also includes distinct categories of impact through bio-fouling and interactions with other alien species, which are not separated out in GISS (Blackburn et al. 2014;Nentwig et al. 2016). This allows for a more detailed assessment of impacts of a larger variety of taxonomic groups which is another advantage of EICAT.
In terms of the various ways to aggregate scores, EICAT and SEICAT suggest using the maximum across all categories as summary classification (Blackburn et al. 2014, Hawkins et al. 2015 while GISS originally promoted the use of summed scores , but see Kumschick et al. 2016, Nentwig et al. 2016). Both have their strengths and shortcomings and can introduce different biases (as outlined in Nentwig et al. 2017 andTurbé et al. 2017). Consequently, for some taxa, we found marked differences when scores are aggregated in different ways. The cat (F. catus) for example had the lowest recorded sum score in GISS of all taxa here considered, even though it is widely recognised as one of the most damaging alien species globally (e.g. Lowe et al. 2004) and has contributed to many extinctions of birds especially on islands (e.g. Dickman 1996). This seeming discrepancy is due to the limited range of mechanisms through which feral cats cause harm to native communities, namely mainly through predation, which leads to relatively low summed scores in GISS. Therefore we would like to highlight the importance of documenting all the sources used for each assessment and the details on all scores obtained to make a more informed policy decision, regardless of which tool is used.
Even though impacts of alien mammals are generally well studied compared to other taxa (e.g. Kumschick et al. 2015), there is a potential publication bias which can influence the magnitude of impacts recorded -the less is known about a species the lower the likelihood a severe impact is found or vice versa. We expect this to be more of a problem for less conspicuous and generally understudied taxa like invertebrates. It needs to be further evaluated how such (potential) publication biases could be addressed and avoided (see e.g. Evans et al. 2018).
Given that the selected species all have alien populations in South Africa, the results shown here could be useful to provide information for local policy-making and prioritisation. Little evidence exists on impacts of these species in the country, but data from elsewhere show that all these mammals have caused severe impacts leading to the disappearance of at least one species locally and some even contributed to global extinctions (Figure 1). Even though impacts on island have generally been more severe, they are not restricted to these regions and we expect many of the changes caused elsewhere could also happen in South Africa or have already occurred but not been documented. As an example, knowing that feral dogs hybridise with wolves and coyotes in Europe and America , Freeman and Shaw 1979, Randi and Lucchini 2002, it is possible that domestic dogs could hybridise with other native species such as the African wild dog and jackals. Evidence from other African countries in fact shows that hybridisation and disease transmission is occurring between these species , Randall et al. 2006. As another example, feral pigs (S. scrofa) have the highest summed impact (Figure 1) showing that the range of mechanisms through which they have impacts is quite widespread. For example, impacts on ecosystems have shown to be massive due to uprooting damage leading to the elimination of a rare plant, Navarretia plieantha in the United States of America . Other impacts include impacts through predation, herbivory and competition. These are very generalist impacts which are not dependent on specific conditions in the recipient environment. In South Africa, however, environmental impacts of feral pigs have only been recorded to be minor as the damage reported does not affect species composition yet (Spear and Chown 2009). This might be a function of the species' limited distribution or a bias due to lacking research. It therefore seems timely to consider this vast amount of evidence and evaluate management options for these species in sensitive areas.

Island susceptibility hypothesis
Only few studies have tested the island susceptibility hypothesis explicitly (Jeschke et al. 2012 found only 9 studies, most on birds), even though islands are generally thought to be more susceptible to invasions. Furthermore, previous studies testing this hypothesis have looked at "invasion success" or establishment rather than impact, finding limited support (Sol 2000, Jeschke 2008. A recent study on birds also used EICAT to classify species according to impacts and, as in our study, it confirmed impacts to be more severe on islands compared to mainlands (Evans et al. 2016). This might suggest that establishment and invasion success (cf. Blackburn et al. 2011) are not increased on islands, but alien species are causing more harm to the native biota. For example, ground-nesting birds and giant tortoises are particularly vulnerable to predation and trampling by invasive rodents and other mammals (MacFarland et al. 1974, Angelici et al. 2012. However, further studies would need to be undertaken to confirm this pattern more broadly.

Conclusion
This study highlights the similarity and differences amongst three impact scoring schemes when using feral mammals as a case study and which can be used to make recommendations as to how prioritisation for actions can be improved. While using more than one scoring scheme to assess the same impacts seems cumbersome and unnecessary, it can help us to get an improved understanding of the various dimensions of such impacts, especially on socio-economic systems. Although this can be time-consuming, the most labour-intensive part of the impact scoring process is collating the relevant literature. All the schemes used here are based on the same data to assess and score impacts ) and, once data is accumulated for the GISS assessment, the same references can be used to complete the other assessments.

Appendix 1
The search string used to assemble information on the global impacts of the mammals assessed in this study. Adopted from Evans et al. (2016).