The cost and complexity of assessing impact

The environmental and socio-economic impacts of invasive species have long been recognised to be unequal, with some species being benign while others are disastrous. Until recently there was no recognised standard impact scoring framework with which to compare impacts of species from very different taxa. The advent of the Environmental Impact Classification for Alien Taxa (EICAT) and Socio‐Economic Impact Classification of Alien Taxa (SEICAT) schemes allows for the possibility of assessing impact through a standard approach. However, both these schemes are still in their infancy and the associated costs of the research that informs them is unknown. We aimed to determine the study costs and complexity associated with assessing invasive species’ socio-economic and environmental impacts. We used amphibians as a model group to investigate papers from which EICAT and SEICAT scores could be drawn up to 2019. Our analysis shows that studies that resulted in higher impact scores were more costly. Furthermore, the costs of studies were best predicted by their complexity and the time taken to complete them. If impact scores from EICAT and SEICAT are allowed to inform policy, then we need to carefully consider whether species with low scores represent true impact, or require more research investment and time. Policy makers needing accurate assessments will need to finance larger, more complex, and rigorous studies. Assessing impacts in low and middle income countries may need investment using international research collaborations and capacity building with scientists from high income areas.


Introduction
Invasive species have long been recognised to produce a wide range of environmental and socio-economic impacts (Elton 1958). Early attempts to provide lists of '100 worst invasives ' (Lowe et al. 2000) were extremely popular, but subjective in terms of which species were included -and why. Instead, comparing impacts between invasive species requires a framework that provides equivalence at the environmental or socioeconomic level for impacts of organisms across Kingdoms: from Caulerpa racemosa var. cylindracea (Kingdom: Bacteria) to Felis catus (Kingdom: Animalia). Evidence of impacts can range from anecdotal observations, to laboratory and field experiments, which quantify environmental degradation or socioeconomic impacts (Hawkins et al. 2015;Bacher et al. 2018). This wide range of study types show extreme variation in research costs and are not equivalent in terms of the rigour of their findings . Assessments made without robustly designed ecological and socio-economic inquiries should be treated with caution (see Christie et al. 2019).
More complex ecological and socio-economic research designs, including those of alien species impacts, are likely to require considerable investment. Full scale field trials with complex designs (e.g., Before-After Control-Impact or Randomised Control Trials) are desirable, as these more robust designs lead to greater power to detect true effects' direction and magnitude (Christie et al. 2019). However, studies on invasions are more likely to be 'natural experiments' with simple designs, such as pre-and post-invasion (e.g., Before-After, Control-Impact or After). Even these simple field experiments are considerably more costly to implement than laboratory experiments or simulations of field conditions, such as mesocosms (Christie et al. 2019). Similarly, designing and implementing social science surveys to elicit the types and severity of socio-economic impacts associated with invasive species requires considerable time and expertise. The costs of designing and administering surveys depend on the study design (e.g., the number and size of human populations to be surveyed, efforts required to obtain a representative sample from the population, means by which the surveys are implemented). Therefore, estimating the impacts of invasions at the landscape scale is likely to cost far more than a simulation or experiment that has lower power to detect impacts of larger magnitude. It seems likely then, that more complex experiments at greater scales will follow smaller investigations that show indications of impacts, but that these initial cheaper inquiries will lack the power or confidence of the more expensive endeavours.
New attempts to produce indices of invasion impacts have consolidated around separate environmental and socio-economic impact classifications of alien taxa (Blackburn et al. 2014;Hawkins et al. 2015;Bacher et al. 2018), and one (i.e., Environmental Impact Classification for Alien Taxa; EICAT) has subsequently been adopted by the International Union for Conservation of Nature (IUCN). Unlike the IUCN Red List, both EICAT and SEICAT (Socio-Economic Impact Classification of Alien Taxa) predominantly use peer reviewed literature to make their assessments. Studies need to be carried out at scales that are appropriate to the invasive population, and the invaded environment, and those studies that cannot meet these requirements are scored with a lower confidence level (Hawkins et al. 2015). When the cost of conducting a study on invasive population impacts is variable, it seems likely that cheaper research will result in impacts of lower magnitude, and/or confidence: i.e. scores using these studies will not reflect true impact. Such a result would likely produce unequal diagnosis of the relative impacts of invasive species in a world that already radically differs in investment in conservation (McCarthy et al. 2012). This is particularly worrisome considering the role that EICAT in particular can play in terms of informing policy and legislation and conservation planning  There is already evidence of global inequality in assessments of alien impacts. Early attempts to classify impacts of taxonomic groups at a global level have emphasised the paucity of coverage for birds (30%; Evans et al. 2016) and amphibians (38%; Kumschick et al. 2017). Regional assessments indicate high levels of coverage in economically wealthy areas (e.g. environmental impacts 79% and socio-economic impacts 75% for European invasive plants; Rumlerová et al. 2016).  highlighted this geographic disparity in the availability of data on alien birds, demonstrating that data availability was related to alien bird residence time, richness, and economic status of the country. Notably, all but two alien birds have been found to have minor impacts on human well-being (Evan et al. 2020). These studies imply that there is likely disparity between the true impact of any alien species, and the impact realised by implementing scoring frameworks, such as the ICAT schemes, because they rely on the true impact to be captured by published research. This difference may relate to the country where impacts are realised, if insufficient research funding is available, or the complexity of the study (if studies are poorly designed or insufficiently long). Therefore, the research and policy community implementing or using these frameworks face a problem moving forward: how can we achieve a representative impact level for all alien species, without sufficient or adequate peer reviewed literature? Or on an economic level: will the impact score of species be independent of research costs?
In this paper, we attempt to respond to these questions by examining the EICAT and SEICAT status of alien amphibians globally, and the costs associated with contributing underlying studies. Previous scoring of amphibians used literature up until May 2015 (see Measey et al. 2016), but literature on alien amphibians as a group have a near exponential growth (van Wilgen et al. 2018), and are therefore useful as a model for trends in invasion literature more widely. Here we aim to determine: (1) whether studies that result in higher impact scores cost more to conduct, (2) whether impact is related to study complexity or time taken to conduct the study, (3) whether EICAT and SEICAT scores demonstrate changes through time (up to 2019), and (4) whether different EICAT and SEICAT scores occur with equivalent confidence. Lastly, we use four case studies to illustrate how EICAT and SEICAT scores have been assessed for particular species based on research publications.

Compilation of Literature
We followed the methods of Measey et al. (2016) to build an updated species list for established alien amphibians (105 spp), including consulting new compendia on the topic (+ 19 spp;Capinha et al. 2017), giving us a total of 124 species. We searched Web of Science in September 2019 for publications since 2015 (inclusive) using the most recent taxonomic name (or combinations of older names if any taxonomic movement had occurred since 2015; cf Frost 2019) and a composite term for invasive species [e.g. TS = ("Ambystoma tigrinum") AND TS = (alien OR invasive OR non-native OR exotic OR non-indigenous OR feral)]. We read titles and abstracts of resulting literature from the searches and obtained electronic copies of any studies that appeared to indicate a study on invasive amphibians (see van Wilgen et al. 2018 for details).

Scoring EICAT and SEICAT
After compiling all literature, we followed Hawkins et al. (2015) to assign an EICAT level in the corresponding impact category to fit any content relating to impact: Minimal Concern, Minor, Moderate, Major and Massive (hereafter MC, MN, MO, MR and MV, respectively). If no EICAT impact level could be ascribed, the paper was retained with a score of Data Deficient (DD). In the case that a score was assigned, we also scored a confidence level (low, medium or high), based on the criteria provided by Hawkins et al. (2015). In brief, the confidence score considers the type of data, the spatial scale of the study, and unanimity of evidence. This process was carried out for every paper published since May 2015 on any alien amphibian, and added to existing data on impact scored for studies prior to this (Kumschick et al. 2017;Bacher et al. 2018). Note that one paper could be scored for more than one alien amphibian species, or more than one category, as appropriate to the data it contained. Therefore, the total number of papers is less than total impact scores.
The same procedure was conducted for SEICAT, but following the guidelines set down by Bacher et al. (2018). We remained cognisant that papers which had been scored for EICAT could also carry SEICAT scores, and vice versa.

EICAT and SEICAT scoring analysis
We reasoned that over any given period of time, if there was no change in EICAT or SEICAT scoring, studies conducted in any period were likely to have equal chances of receiving the same proportions of EICAT or SEICAT scores as those received overall. Alternatively, an effect could be supported if the proportion of higher impacts (MR and MV) increased relative to the proportion of lower impacts (MC, MN and MO). Because the number of studies (see van Wilgen et al. 2018), and those to which EICAT or SEICAT scores could be attributed, have been increasing over time, we found it necessary to use only studies after 2000 when there were consistently more than three scores per year. Thereafter, we conducted a linear regression on the proportion of high to low impacts ((lower-higher)/total) per year in R (v 3.6.3; R Core Team 2020). Similarly, we tested whether the proportion of higher confidence scores (medium and high) have changed through time compared to lower confidence scores.

Costs of Conducting Studies
In each case where EICAT and SEICAT scores were determined for studies published in the last four years, we contacted the corresponding author of the study, to obtain research costs. We did not consider asking authors of earlier studies as we were concerned that accurate records of funding would not be available. We asked for the costs of the entire project in the form of a questionnaire (see Suppl. material 1), approved by Stellenbosch University's Social, Behavioural and Education Research (SBER) committee (project: 2019-13163). Specifically, we requested information on the amount of time spent by different members of the research team in conducting the study and producing peer-reviewed papers and the costs of their time (i.e., wage rates). We further requested information on costs of travel, accommodation, equipment used in the field or the lab, publication, and any other relevant costs associated with the study. If the equipment used in the study could be used for other research (e.g. computers, cameras, centrifuges) we asked the authors to estimate the percentage of total use of the equipment that was attributable to the study. Finally, we asked the monetary units used to pay these costs and the years in which the costs were incurred.
All study costs were converted to United States Dollars (USD) using the purchasing power parity (PPP) rates of currency conversion for the year(s) in which study costs were incurred, and the consumer price index (CPI) was used to convert these costs to 2018 USD by accounting for inflation since the time of the study. We obtained CPI measures from the United States Bureau of Labor Statistics. PPP measures were obtained from Organisation for Economic Co-operation and Development (OECD) data. The OECD calculates PPP measures based on the relative prices of consumer goods and services, equipment goods, occupations, and construction projects in different countries. PPP measures deviate from standard currency exchange rates because PPP takes relative price levels and costs of living in different countries into account. That is, by converting wages using the PPP we determined what equivalent USD income would have allowed the individual to maintain the same standard of living in terms of goods and services they could purchase if they were living in the USA rather than the country where the study was conducted (i.e. Australia, India, South Africa, etc.). Similarly, by converting non-wage research expenses to USD using the PPP we accounted for different price levels in different countries. This allowed us to compare study costs without the confounding effect of different salaries and price levels for goods and services across different countries of the world, and means that the costs we present can be directly compared.
For studies for which we obtained costs, we scored the complexity of the study design using the categories provided by Christie et al. (2019). The categories included (from simple to complex) were: After, Before-After, Control-Impact, Before-After-Control-Impact, and Randomised Control Trial. Along with study design complexity, time taken to conduct the study was also scored. We did not include time taken to publish the study or any delays between the finalisation of data collection and writing up of the study.

Costs and Impact analysis
In order to determine whether costs (USD PPP) are aligned with EICAT impact score or confidence, we conducted generalized linear mixed effects models (GLMM) using the lme4 package (Bates et al. 2015) in R (v 3.6.3; R Core Team 2020). Prior to analyses, model assumptions (e.g. normality, homogeneity and independence) were assessed for all variables. The independence of impact score and confidence was assessed with a Spearman's Rank Correlation test. The response variable, Cost (USD PPP), was transformed using natural logs to meet the underlying statistical assumptions of normality. Restricted Maximum Likelihood (REML) methods were used to compare models with different fixed effect structures. A full model included two fixed factors, EICAT impact score and confidence as explanatory variables. We did not anticipate the need to assess models with interaction effects between our predictor variables. We used the continent on which the studies had been conducted as a random effect to account for potential autocorrelation within continents (following Evans and Blackburn 2020). Variance of random effects were minimal (s. d. < 2.00). Relative importance of competing models was evaluated using Akaike information criterion (AIC) and log likelihoods. To evaluate the variance of data explained by each model, R 2 m (marginal: variance explained by the fixed effects) and R 2 c (conditional: variance explained by the fixed and random effects) was calculated using the package MuMin (Barton and Barton 2019).
The effect of both study design complexity and the time taken to conduct the study (as predictor variables) on EICAT impact scores (response variable) were analysed using GLMMs, as above. Model assumptions (normality, homogeneity and variable independence) were assessed. The residuals from this analysis were not normally distributed and we were unable to normalize them by transformation. A full model included fixed effects, design complexity and time. All models included the random effect continent. Variance of the random effect, continent, was minimal (s. d. < 2.00).

Scoring
We found 334 publications published since May 2015 that were on alien amphibians included in our updated list. This included two species (Desmognathus monticola; Bufo japonicus) that had not been previously on the list of Measey et al. (2016) or Cap-inha et al. (2017). Of the publications found, 109 papers could be scored (with 112 scores on 49 species) for impacts using the EICAT framework (see Suppl. material 2). Of all publications found, only eight new papers contained information that could be used to score socio-economic impacts according to SEICAT.

Cost of studies
We found that cheaper studies equated with lower EICAT impact scores (p = 0.0060; model 1 Table 1; Figure 1). The model that included confidence (model 3) was not appreciably worse (within 2 delta AICs) but the R 2 estimates indicate that confidence did not explain more of the data (Table 1). Confidence and impact score were not correlated (S = 8789.3, p = 0.4457). No relationship between the confidence score and the study cost was found (see Table 1).
Our data showed that study design (complexity) and time taken to conduct the study were significantly related to EICAT score (p = 0.0007). The highest EICAT scores (MR and MV) were obtained from Before-After studies which took longer to conduct, and lower scores typically came from shorter studies 'After' invasions had taken place (Table 2). Although the model with study design alone (model 1) was within 2 delta AICs, it was not favoured as it explained less observed variance in the data (Table 2).

Impact score changes across time
We noticed three general trends in EICAT scores over time. Firstly, that the amount of literature in the last four years that generated impact scores (not including DD) was equivalent to nearly a third of total scores (112 out of 362). Second, we found a sig- Table 1. Results of general linear mixed models for costs (USD PPP) of studies contributing to EICAT impacts of amphibians. Impact and Confidence are both calculated when scoring papers using EICAT criteria (see Hawkins et al. 2015). Delta AIC is the difference in Akaike information criterion values (AIC) between models, and Wi (Akaike weight) is the relative support a model has from the data compared to the other models in the table. Marginal ( R2m ) and conditional ( R2c ) variance of the fixed effects are reported for each model.  nificant negative trend (F 1,17 = 7.451, p = 0.014) with increasing proportions of lower impact studies over time: i.e. studies that scored MR or MV are reduced compared to the total number of scores since 2000. Lastly, we found that the proportion of studies scored with low confidence has a significant negative trend since 2000 (F 1,17 = 11.48, p = 0.003). Our overview of all data (Figure 2) shows that impact scores are higher since the 1990s than in the decades preceding this.

Model number Variables (fixed effects) Log likelihoods Number of parameters
When compared with previous scores (Kumschick et al. 2017), we found that 25% (11 of 45 species) of all amphibians had increased their EICAT impact or confidence in the four years since the original score was determined (Table 3a). This included a growth in the assessed list by 12% (five species), which were previously not assessed (NA) due to a lack of literature. Notably, scores changed for the cane toad, Rhinella marina, from Major (MR) with high confidence to Massive (MV) with medium confidence (see below). Additionally, the confidence score on Major impacts of the African clawed frog, Xenopus laevis, changed from medium to high (see below).

SEICAT Scoring
Of the eight new papers for SEICAT scoring, one species (out of the previously scored 6 species) increased its SEICAT impact score, and two studies provided data on species for which there was no previous score (Table 3b). In the four years since the original study, there has been a 25% (three species) increase in species scored for SEICAT since the search made by Measey et al. (2016), and subsequently scored for SEICAT by Bacher et al. (2018).

Confidence scores
For EICAT, we found that the number of papers decreased for each increasing impact score, except for the category MC, which is similar in size to MO (Figure 3). However, as the impact score increased, the proportion of studies with low confidence decreased and the proportion of high confidence studies increased. Both the data collected from costs of studies (n = 35), and the number of changes in scores (Table 2) were of limited inferential value on their own. We therefore provide four case studies that illustrate the diversity of studies that we found.

Case Studies
Indian bullfrogs -The Indian bullfrog, Hoplobatrachus tigerinus, native to the Indian subcontinent, has invasive populations on the Andaman archipelago and Madagascar (Mohanty et al. in press). The use of this large-bodied species for human consumption has been the primary pathway for its introduction and secondary transfers. Despite its high invasion potential, H. tigerinus invasion dynamics have only been studied in detail on the Andaman archipelago (Mohanty et al. in press  assigned H. tigerinus an EICAT score of Minor with low confidence; the species was considered 'Data Deficient' under SEICAT (Bacher et al. 2018). The EICAT score was based on observations of extremely high densities of H. tigerinus at a few sites in Madagascar (Griffin 2010), leading to inference of competition. Mohanty et al. (in press) revised both EICAT and SEICAT assessments of the species on the basis of three recent publications, Measey 2018, 2019a, b). The revised 'maximum recorded impact' was Moderate with medium confidence for EICAT (drastic reduction in survival of endemic frogs Microhyla chakrapanii and Kaloula ghoshi), costing 2018 USD 11 736. This revised assessment resulted from a mesocosm experiment, carried out over three months, that aimed to assess impact (Mohanty and Measey 2019b), equivalent to a Control Impact study (moderate complexity). A new SEICAT score of Moderate with medium confidence (ceasing of poultry-keeping) was assigned at a cost of 2018 USD 22 626, based on a year-long questionnaire-based survey of 91 villages on the Andaman archipelago. However, this study's bigger aim was to reconstruct the invasion history of the species (Mohanty and Measey 2019a). The invasive population (on the Andaman archipelago) is unlikely to attain a higher EICAT or SEICAT score at this time, given its relatively recent invasion (Mohanty and Measey 2019a). However, the associated confidence scores could be improved by conducting largescale, before-after-control-impact experiments and focussed socio-economic surveys, for EICAT and SEICAT scores respectively. African clawed frogs -The African clawed frog, Xenopus laevis, is native to southern Africa, but was exported globally from South Africa from the 1930s (van Sittert and Measey 2016) first for pregnancy testing, then as a model amphibian in research laboratories (Gurdon and Hopwood 2000), and lastly in the pet trade (Measey 2017). The first studies on invasive populations were conducted in California and published in the 1980s (McCoid and Fritts 1980), but the discovery of increasing invasive populations around the world (van Sittert and Measey 2016) led to this becoming one of the most well studied alien anurans (van Wilgen et al. 2018). The European Union invested ~$1.1 million USD in a 5 year project (INVAXEN) to understand European invasions of X. laevis. As part of this larger effort, researchers designed a study to measure the impacts of the French population of X. laevis on native amphibians at 76 sites in a transect from the introduction site (~1980) and through the invaded area and into uninvaded areas of the same region (Courant et al. 2018), equivalent to a Control-Impact study (moderate complexity). Researchers found a significant negative relationship between amphibian diversity and increasing X. laevis density, and that amphibian diversity increased as distance from the introduction site increased. Two amphibians (Bufo spinosus and Triturus cristatus) were not detected when X. laevis was present. The extirpation of these native species from within the invaded range in France increased the EICAT impact score to Major, with medium confidence. The study was conducted within a single year and is estimated to have cost 2018 USD 25 777. No literature allows for SEICAT impact scores to be assessed for invasive X. laevis, although fishermen in the region have complained that since the invasion, the quality of leisure fishing has reduced due to fewer fish being caught and interference by unintended capture of X. laevis (J. Courant pers. com.). Given the current scale and locations of invasions, the EICAT score is not expected to increase with further study, but more studies from other invasions may increase the confidence. A targeted study is still required to produce a SEICAT score.
Cane toads -Cane toads, Rhinella marina, are native to the Americas, ranging from the southern United States to central Brazil (Shine 2010). During the early 20 th century, these toads were extensively moved around the world as a biocontrol agent for invertebrate pests. In 1935, 101 R. marina were imported into Australia from Hawaii (Easteal 1981;Shanmuganathan et al. 2010) and by 1947 over 60 000 Australian-born juvenile R. marina had been produced and were released (Easteal 1981;Shanmuganathan et al. 2010). The potential threat these toads posed to Australian ecosystems was identified from the outset of their arrival, with Froggatt (1936) stating that "this great toad, immune from enemies, omnivorous in its habits, and breeding all the year round, may become as great a pest as the rabbit or cactus"; however it took several decades before notable impacts on wildlife became apparent (Breeden 1963;Pockely 1965;Rayward 1975). Owing to these early concerns, the toad's 80 year long range expansion across northern Australia has been both well-documented and modelled (e.g. Sabath et al. 1981;Freeland and Martin 1985;Urban et al. 2008;Doody et al. 2019), with substantial resources being invested into toad research. For example, between 1986 and 2009 the Australian Government invested over ~$13.8 million USD into R. marina research, management, community groups, and education (R. Shine pers. comm.). The ecological impacts have been far-reaching (Shine 2010), including competition with native species (Greenlees et al. 2006;Bleach et al. 2014;Taylor et al. 2017), multiple native species population declines (Phillips et al. 2009;Fukuda et al. 2016;Taylor et al. 2017), localised extinctions (Oakwood and Foster 2008;Doody et al. 2017), trophic disruption (Greenless et al. 2006Doody et al. 2015), and community-level reorganisation (Brown et al. 2013;Feit et al. 2018;Brown and Shine 2019).
The previous EICAT assessment identified R. marina as one of the top 10 amphibian species with the highest Maximum Recorded Impacts, receiving the listing of Major impacts in both the Predation and Poisoning/toxicity categories (Kumschick et al. 2017). Our current reassessment has up-listed the R. marina Maximum Recorded Impacts from Major with high confidence to Massive with medium confidence within the category of Poisoning/toxicity. This shift was a result of a study based in the Northern Territory and aimed at quantifying community-level impacts and examining ecological responses post-invasion (Doody et al. 2017); equivalent to a long-term Before-After study (moderate complexity). The impacts of toads in the region resulted in extirpations (e.g., complete removal of two species of predator and the dramatic reduction of a third), community-level changes (e.g., the mesopredator-release of one species), and no indication of ecological recovery 12 years post-invasion (Doody et al. 2017). The study also notes that R. marina are too prolific and widespread to eradicate (Shine and Doody 2011;Doody et al. 2017), resulting in their impacts being functionally irreversible by the Hawkins et al. (2015) definition. The study was conducted intermittently over 12 years and is estimated to have cost 2018 USD 408 737. We note that this cost estimate is conservative because we only included costs that could definitely be attrib-uted to the published research findings. Further research looking for similar findings across a wider geographic area or an example of a complete species extinction would likely serve to increase the EICAT confidence rating for the Massive impact score (currently rated at medium confidence). Additionally, the previous SEICAT assessments of R. marina had listed the maximum impact score as Major with medium confidence (Bacher et al. 2018) and it is likely that further research into the socio-economic impacts of R. marina are required and may yield higher SEICAT impact and confidence scores if the appropriate attention is given to this topic.
Guttural toads -The guttural toad, Sclerophrys gutturalis, is a common African bufonid naturally distributed across central and southern Africa (Telford et al. 2019). Guttural toads were first introduced to Mauritius and Reunion in 1922 and 1927, respectively, as an attempt at biocontrol for invertebrate pests (Cheke and Hume 2008;Kraus 2009;Dervin et al. 2014;Telford et al. 2019). These toads are now widespread across both islands and a recently introduced population in Cape Town, South Africa, became established in 2000(de Villiers 2006Vimercati et al. 2018;Telford et al. 2019). Due to their extensive predation on Mauritian land-snails (Griffiths 1996), Kumschick et al. (2017) assigned S. gutturalis an EICAT score of Moderate with low confidence, based on a study that equates to After design (low complexity). Although there are growing concerns regarding the potential impact of S. gutturalis on native species in the Mascarene Islands and Cape Town, South Africa (Griffith 1996;de Villiers 2006), hardly any research has been conducted regarding its impact. A recent publication noted that property owners are more willing to support the removal of toads in Constantia due to their loud reproductive call that can be heard over hundreds of metres . The objectives of this study were to provide a historical overview of invasive amphibian dispersal pathways in southern Africa, give an overview of legislation in South Africa regarding amphibian invasions and assess the status of three invasive amphibian populations in South Africa, namely Hyperolius marmoratus, Xenopus laevis and Sclerophrys gutturalis. Although Measey et al. (2017) did not aim to describe the socio-economic impact of S. gutturalis, the anecdotal evidence presented in this study allowed us to assign a SEICAT score for S. gutturalis. The change from Data Deficient to Minor for Health/Social, spiritual and cultural relations with a low confidence level came at an estimated cost of 2018 USD 58 110. Given sufficient directed work, particularly on Reunion and Mauritius where local endemism is high, we consider that the EICAT and SEICAT scores and confidence in them will increase.

Discussion
We found that for EICAT impact scores, but not confidence in those scores, the costs of studies that produce higher impacts are more expensive. Moreover, higher EICAT impact scores were obtained from studies with more complex designs that took longer to conduct. For our amphibian dataset, we found that highest impact scores have only been elucidated in the last 20 years of research, and that studies with higher confidence are also more recent. The implications of our findings are that it will not be possible for the IUCN to categorise the true impact of alien species using EICAT (and, should they choose to adopt it, SEICAT) without increased investment in research. This is especially true in areas of the world that do not currently have the resources to invest in more costly research -a pattern already seen for data availability in birds . Moreover, for many invasive species we cannot expect that studies conducted rapidly will convey the true EICAT score at that point in time. Our data indicates that preliminary studies should act to recommend whether or not more, longer term studies are warranted. For example, in the past four years, new literature has changed the status of even the most well studied invasive amphibian, R. marina (see van Wilgen et al. 2018) from MR to MV (i.e. from a reversible impact to an irreversible impact), in a study that took more than 12 years to conduct (Doody et al. 2017) at a cost of USD 408 737. If the same pattern continued across all taxa as we have shown for amphibians, the consequence will be that species with potentially high EICAT or SEICAT impact scores will go unrecognised because of insufficient research investment. It seems likely that this will impact the assessment using the ICAT schemes of all invasions in regions from lower middle-income economies, unless research funding is specifically targeted towards invasion science in these regions, or funds obtained in wealthier regions are leveraged to develop international research collaborations.
There may be reasons why amphibians are not representative of all invasive groups. Although the rise in data available is currently exponential (van Wilgen et al. 2018), sparse older data (pre 2000) does not allow for a quantitative assessment of impact during this time. For example, all but one amphibian scored as MV (except R. marina) include the impact mechanism of hybridisation (see Kumschick et al. 2017). As this requires a genetic diagnostic not widely available before the late 1990s, studies had not previously proved this mechanism, and so MV was nearly absent. Other taxa that have more impact mechanisms contributing to higher impact scores could be investigated to determine changes in impact over time. While we show that higher impacts were associated with more expensive studies, there are other possible explanations that may explain this pattern. For example, the perception of high impacts (independent of scoring schemes) may stimulate greater investment in research. The information provided here about R. marina and X. laevis supports this supposition, but other species (e.g. Asian spiny toad, Duttaphrynus melanostictus) achieve high impacts with few studies (see Measey et al. 2016). It is important to note that both R. marina and X. laevis have received this investment where they have invaded high income countries, while the impact from their introduction to less wealthy regions remains comparatively understudied.
In this study, we show that a relatively short period of time (less than four years) is enough to make considerable changes to the global list of EICAT and SEICAT amphibian scores. Papers from which EICAT and/or SEICAT can be scored have grown by around 25%, with increases in the number of species assessed as well as increasing impact scores for species already assessed. Unsurprisingly, a small number of species were assessed for the first time, including two species that were not featured in previous lists of established alien amphibians (Kraus 2009;Measey et al. 2016;Capinha et al. 2017). Currently, the IUCN aims to update Red List entries every five years. Our study suggests that this periodicity is appropriate to the ICAT schemes, but that ad hoc updates may be appropriate when project findings suggest that this may result in a status change. It can be hoped that when EICAT is more widely acknowledged in the scientific community (through use by the IUCN), that more studies will be designed to incorporate the specific impact mechanisms and suggest appropriate scoring, much as they are currently for the IUCN Red List.
Overall, we found that higher EICAT scores displayed a higher proportion of studies with high confidence. This is evidence of our supposition that only studies with greater investment and more complex designs (cf Christie et al. 2019) are able to demonstrate Major or Massive impacts. We acknowledge that it is possible for studies demonstrating Massive impacts to be very cheap, such as the extinction of a well-known endemic on an island as the result of an invasion, and that these are not infrequent (cf Pyšek et al. 2017). However, although such impacts are possible for amphibians, none has been recorded as their largely invertebrate prey items are not comparatively well studied, thereby introducing another source of impact bias. It is noteworthy that the lowest EICAT score, Minimal Concern (MC), has fewer contributing papers than Minor (MN), with the majority of them having low or medium confidence. In our dataset, most studies resulting in low impact scores (MC and MN) were not aiming to determine impacts of invasive species. It is possible that because our dataset involved only established species, it is more difficult to demonstrate the absence of impact (MC) than Minor impact (MN), explaining the skew in the numbers of papers in these categories. This is in contrast to studies resulting in MO, MR or MV that are primarily focussed on impact. The mechanisms of most research funding mean that simple preliminary studies from small funds showing some impact (perhaps with low confidence) are required before researchers will motivate for funds to conduct more complex experimental research. Policy frameworks are extremely important to motivate for such funding (e.g. CBD, IPBES 2013).
We continued to find very few studies that provided impact scores for SEICAT (Bacher et al. 2018), and this is similar to other studies for different taxa that have attempted to implement this framework (see . This is perhaps unsurprising as our methods have an inherent bias to sampling from the scientific over the socio-economic literature. But there is a growing interest in socio-economic studies in invasion science, and we expect to see these assessments growing in confidence over time. Currently, it is not possible to say whether SEICAT will suffer from the same issues (e.g., costs linked to impact score, and impact score linked to study complexity) that we observed in EICAT, but it is important that this be assessed in the future.

Summary
Here, we provide the first set of estimates of the cost of published studies used when implementing the ICAT schemes. We found evidence that more expensive and complex studies are needed to score the highest levels of impacts for amphibians. This is something that should concern those who wish to implement the use of these scores as we show that the highest scores may require high levels of funding, investment of time and expertise to achieve. If scores inform policy, then this may result in a circularity where poor investment in impact forming research results in true impacts not being revealed.