The risk analysis framework presented here was specifically designed for the purpose of listing alien species under the regulatory framework of the South African National Environmental Management: Biodiversity Act (NEMBA, Act 10 of 2004) Alien and Invasive Species Regulations (hereafter called the NEM:BA A&IS Regulations; Department of Environmental Affairs 2014; for details of how the framework aligns with the regulations, see Suppl. material 1). The development of the RAAT framework was initiated in 2015, in response to the promulgation of the NEM:BA A&IS Regulations. The regulatory lists of 2014 were informed by expert opinion, but the decisions taken and recommendations made were not clearly documented (see Kumschick et al. 2020a for a discussion). As the regulatory lists specify taxa which need to be controlled and for which other restrictions are in place, it has social and economic implications and has been contested in a number of cases [van Wilgen and Wilson (2018); see also Novoa et al. (2015) for a discussion on listing alien Cactaceae]. A framework was therefore required to (retrospectively) provide evidence for listing in a consistent transparent manner (e.g. Woodford et al. 2017).

During the development of this framework, regular meetings with decision-makers [mainly representatives from the Biosecurity Division of what was, at the start of the process, the South African Department of Environmental Affairs (DEA), but became the Department of Forestry, Fisheries and the Environment (DFFtE) in 2020] were held to ensure their needs were taken into account and the framework was relevant for the intended purpose. The first version of the framework was used by graduate students at the Centre for Invasion Biology at Stellenbosch University (CIB) to assess taxa from a wide range of taxonomic groups and feedback from this exercise was used to refine it, providing additional clarification and guidance. The second version was reviewed by the Alien Species Risk Analysis Review Panel (ASRARP), a panel of South African experts set up by the South African National Biodiversity Institute (SANBI) to review risk analyses for alien taxa [both those performed in relation to the import of species not yet present in the country and those performed in relation to the regulation and listing of alien taxa under the NEM:BA A&IS Regulations (Kumschick et al. 2020a)]. The panel includes independent experts on biological invasions and risk analyses, with representatives from private and public entities and experts on a wide range of taxonomic groups. The issues raised by the ASRARP on the framework were mainly related to details in the wording which could lead to misunderstandings. These were subsequently addressed, and a new draft was reviewed by representatives from different organisations, including the DEA, members of the ASRARP, the SANBI, and the CIB. Finally, RAAT was signed off by the ASRARP before submission as a report to the DEA in March 2017. A revised version was subsequently uploaded to a pre-print server to make it widely accessible (Kumschick et al. 2018).

Initially, several risk analyses were piloted by ASRARP members, but after the first three risk analyses were approved, subsequent risk analyses were submitted by SANBI staff, students, and post-docs not affiliated with ASRARP to ensure a separation between the review panel and the assessors. The risk assessors (who had various backgrounds and levels of education, including alien species managers, taxonomists, post-graduate students, and researchers), were trained to use the framework during five courses that were run over 2018–2019 (Table 1, several additional courses were held in 2020 based on an accepted draft of this paper). The courses provided valuable feedback in terms of how the framework should be worded to avoid inconsistencies and to clarify the calculations of likelihood and risk specifically. Moreover, as the risk analyses were submitted for review at the meetings of the ASRARP and reviewed by independent experts, the framework has been further refined by adding sections on management that could help clarify specific issues on sub-specific entities. The framework presented here has thus been tested and refined in practice over two years (Suppl. material 2).

RAAT is yet to be either formally adopted in South African legislation or included as an official guiding document, but it is being used by officials to justify applications to revise the listing of taxa under their mandate. Even though RAAT was initially designed for the purpose of listing alien species under the NEM:BA A&IS Regulations, the intention was always to create a system that can be used more generally to aid decisions regarding management prioritisation and the listing of taxa under policy frameworks. Therefore, throughout the framework, the questions posed and options for answers were designed to be generic and applicable across regions. However, in the Suppl. material 2, these are worded specifically with the South African context in mind for local decision-makers and managers to determine the appropriate categories as referenced in the NEM:BA A&IS Regulations.

RAAT consists of a series of questions which need to be answered by the person assessing an alien taxon of interest. The accuracy of an analysis relies, amongst other factors, on ensuring that a thorough literature review on the taxon under assessment is conducted. Some information can be extracted from national and international databases on native and alien species, such as the Global Invasive Species Database (http://www.iucngisd.org/gisd/), CABI’s Invasive Species Compendium (https://www.cabi.org/isc/), Global Biodiversity Information Facility (https://www.gbif.org/), and the Red List of Threatened Species (http://www.iucnredlist.org/). However, primary literature should preferably be consulted and included. Information from the native range can be useful, including indigenous knowledge.

If insufficient information is published on the taxon, closely related taxa should be considered, for example, congeners (e.g. Bomford 2008). However, it needs to be clearly stated when such information is used, and which species was selected as a surrogate and why. Species with similar life history traits and behaviour are preferred. All information must be documented and referenced to be able to review how recommendations were developed and when assumptions were made and to facilitate updating the analysis as suitable information becomes available.

Taxonomists and other experts should be consulted for the risk analysis process to fill gaps in literature, especially for sections initially scored data-deficient for a given taxon. Expert opinion is beset with biases that are well understood and described (Burgman 2016). To minimise such biases, all information sources need to be documented, including listing which experts were consulted and their expertise in the respective topic. It is also possible, and preferable in many cases, that taxa are assessed in working groups rather than by a single assessor to minimise bias (Burgman et al. 2011). In the South African case, and based on international best practice (e.g. Defra 2015), review of analyses through the ASRARP provides another mechanism to avoid bias (Kumschick et al. 2020a).

Assessors can also, of course, be biased and there is often considerable uncertainty when interpreting data (McGeoch et al. 2012; Vanderhoeven et al. 2017) and which is difficult to avoid. Clear guidance on how to respond to each question in the RAAT and formalised descriptions of each response option is provided in the form of scenarios to minimise assessor bias. It is important to indicate how confident the assessor is in the response provided (Carrington and Bolger 1998). The confidence score should give an indication on how confident the assessor is that the answer provided is correct. This generally depends on the amount and quality of data available on the taxon. We followed the guidelines as described in the European Plant Protection Organisation (EPPO) pest risk assessment decision support scheme and as published in Hawkins et al. (2015) for confidence ratings (see also Suppl. material 5).

The RAAT is divided into five sections and includes all aspects of risk analysis, namely risk assessment (sections 2 and 3), risk management (section 4), and risk communication (sections 1 and 5) (Fig. 1). The sections are abbreviated with three-letter acronyms: 1) Background (BAC) provides information on the assessor, the taxon under consideration, and information needed to perform the analysis; 2) Likelihood (LIK) assesses biological, ecological, and behavioural traits of the taxon that could lead to its arrival, establishment, and spread; 3) Consequences (CON) include the recorded and potential impacts of the taxon; 4) Risk management (MAN) includes questions related to the ability to control a taxon, whether the taxon is beneficial in some situations, and provides recommendations for management and/or listing of taxa; 5) Reporting provides guidance on how to communicate the outcomes of the analysis. This last section does not consist of questions, but is a compilation of the results of the previous four sections and provides an easily digestible summary for the communication of recommendations to stakeholders. Each section is discussed below.

Figure 1.

A schematic of the Risk Analysis for Alien Taxa framework described here. For each section a number of parameters need to be assessed (more detail in Table 2).

It is important to clearly outline the scope of the analysis to clarify what is assessed, for which region, and by whom. This section therefore includes the region of interest, the taxon for which the analysis is performed, and information on the taxon, as this forms the basis for data collection (Table 2).

The region for which the risk analysis is performed is referred to as the Area (developed from the concept by D’Hondt et al. 2015). In most cases, analyses will be undertaken at a national level (e.g. South Africa), but the structure of the framework allows the analyses to be undertaken for different spatial units (e.g. for a national park or for the southern African region). However, the Area must be clearly specified and all questions referring to the Area specifically consider information with respect to the region chosen.

The taxon under assessment is referred to as the Taxon. The Taxon can be a species, sub-species, infra-specific entity, genus or any other taxonomic level. Risk analyses are mostly carried out on individual species as a standard taxonomic entity as, mostly, this is the level at which information is available, but this is not always appropriate, feasible or desirable. For example, different taxonomic levels are preferable: if the taxonomy of a group is not well resolved (e.g. some genera within the family Cactaceae, Novoa et al. 2015); if species are difficult to distinguish but the whole group (i.e. genus or family) poses a significant threat (e.g. certain taxa of mites or plant pathogenic rust fungi); and if there are important differences between sub-species or infra-specific entities (e.g. varieties and cultivars; see Datta et al. 2020 and Gordon et al. 2016). Ideally the analysis should consider whatever taxonomic grouping for which the risk is the same (e.g. Wilson et al. 2011), though in practice, this is very hard to achieve and species level assessments are therefore most common.

The section on likelihood assesses the probability of the Taxon to arrive, establish, and spread in the Area, with two questions for each process (arrival, establishment, and spread), resulting in six questions in total (LIK1–LIK6 in Suppl. material 2). These include questions on habitat and climate suitability and likelihood of entry and spread via aided and unaided pathways. Each answer is expressed as a probability value p, with all the levels and scenarios described in the narrative section and each level representing an order of magnitude difference. If the answer is not known after consulting literature and experts, following a precautionary principle, the answer is treated as p = 1 for the rest of the assessment, though noting that no answer was supplied and so highlighting an obvious area where more research is needed (Hulme 2012). For each probability level, we give general examples to provide guidance. These are structured as follows:

• Extremely unlikely (p = 0.000001): as likely as winning the lottery, if you play it once.

• Very unlikely (p = 0.0027): as likely as a new person you meet having their birthday on the same day as yours.

• Unlikely (p = 0.027): as likely as rolling two sixes when playing dice.

• Fairly probable (p = 0.5): as likely as getting heads when flipping a coin, i.e. fifty-fifty.

• Probable (p = 1 for calculation purposes): more likely to happen than not.

The probability levels of all the questions in this section are combined to calculate the likelihood of an invasion occurring. The final likelihood is calculated as the product of the maximum scores for each stage, i.e. p(arrival) [= max(LIK1, LIK2)] × p(establishment) [= max(LIK3, LIK4)] × p(spread) [= max(LIK5, LIK6)] (Suppl. material 2)].

RAAT thus incorporates some basic considerations of probabilities by multiplying the likelihoods of a taxon to cross the barriers in the invasion process, i.e., if the taxon cannot cross a certain barrier, the likelihood of establishment is decreased (Suppl. material 2: Fig. S2).

As it is important to get a comprehensive understanding of the potential harm caused by an alien taxon, it has been suggested that both environmental and socio-economic impacts should be included in risk assessments (e.g., Kumschick and Richardson 2013; Roy et al. 2018). The assessment of current and potential impacts, or consequences, is based on recent developments of impact scoring schemes (Blackburn et al. 2014; Nentwig et al. 2016; Bacher et al. 2018). EICAT is used for the assessment of environmental impacts (Blackburn et al. 2014; Hawkins et al. 2015). It was adopted by the International Union for the Conservation of Nature (IUCN) as a standard for the classification of alien taxa (IUCN 2020a, b), to be used alongside the Red List for the conservation of biodiversity. For socio-economic impacts, we initially used parts of the Generic Impact Scoring System (GISS) (Nentwig et al. 2016; see Kumschick et al. 2018). Since then, a new scoring scheme, more similar to EICAT and more consistent in the way impact levels are assigned, was published, namely the SocioEconomic Impact Classification of Alien Taxa (SEICAT) (Bacher et al. 2018). The version of the framework presented here therefore uses SEICAT instead of the GISS (Suppl. material 2), although all approved risk analyses reported in Table 1 are based on the GISS.

These impact scoring schemes have been shown to be intuitive to use, robust (Kumschick et al. 2017a, b), and transparent, and have proven to be applicable for a wide range of taxa (e.g., Kumschick et al. 2015; Evans et al. 2016; Kumschick et al. 2017a; Rumlerova et al. 2017; Hagen and Kumschick 2018; Kesner and Kumschick 2018; Nkuna et al. 2018). This makes them suitable for use as a component in a risk analysis framework. Another common feature of these impact assessment schemes is that all available evidence of impacts in the global alien range (including the Area) of the Taxon is collated and used for scoring (Hawkins et al. 2015; Nentwig et al. 2016; Bacher et al. 2018; see also Table 3 for an overview of the different impact levels). The guidelines cover each mechanism and sector through which alien taxa can affect the recipient regions, including competition, herbivory, and hybridisation for environmental impacts; and safety, material assets, and health for socio-economic impacts.

These impact classification schemes, however, only consider impacts for which evidence is available (see also Kumschick et al. 2020b). Due to the lack of comprehensive impact studies for most species in most regions (e.g. Pyšek et al. 2008; Evans et al. 2016; Bacher et al. 2018; Kumschick et al. 2017a), the impact of alien species is likely under-reported. We therefore included the possibility to use data from congeners or other closely related species with similar life history traits to the RAAT framework (similar to Bomford 2008). Furthermore, to estimate potential and currently unrecorded impacts of the Taxon in the Area, we include considerations on the Taxon’s traits, behaviour, ecology, and impacts recorded in the native range (Table 3). This results in three to five questions related to impact – depending on data availability for the Taxon itself (Table 2). As we are interested in what the worst that could happen is, the maximum of the different impact scores is used as the consequence score.

The consequence score, together with the final probability from the Likelihood section, calculated as described above, are used to assess the level of risk (low, medium, high; as shown in Table 4). If the risk is low, no prioritised management or regulations are recommended and there is no requirement to complete the risk management section of the framework. If the risk is medium or high, however, the risk management section must be completed.

Generally, the distinction between whether or not (as opposed to how) to regulate a Taxon relies on the risks it poses to the recipient environment and economy. For taxa that are not yet present in an area and for which decisions on importation are required, this can be a relatively straightforward process: if the Taxon poses a high risk, it should not be allowed for import, but if it is low risk, it can be considered safe for import (e.g. Keller and Kumschick 2017). However, decisions regarding taxa that are already present and potentially well established in an area and are in use for various purposes, also depend on how easily they can be managed. Since management does not happen in isolation from the rest of society, social perceptions and benefits provided by the Taxon need to be assessed and accounted for in these cases (e.g., Zengeya et al. 2017). Unlike in the risk assessment section of the framework, where clear answers and probabilities are provided to determine the level of risk, the inclusion of benefits is dependent on the agenda of various stakeholders, priorities of decision-makers and the influence of key stakeholders (e.g. Kumschick et al. 2012; Woodford et al. 2017). To keep the process transparent, we make provision for these aspects to outline how the inclusion of benefits influences management decisions and which benefits were included (Suppl. material 2).

Furthermore, once a taxon has been identified as posing a medium or high risk, one needs to consider what can be done to manage the risk. For taxa already present in the Area (i.e., for which prevention is no longer an option), this will often require a detailed evaluation of management options, the development of management plans, an assessment of financial resources, and a process of prioritisation of potential interventions (Wilson et al. 2017). Such detailed assessments are beyond the intended scope of the RAAT framework, as they also depend on political decisions and the allocation of resources. However, the RAAT framework provides for some basic management considerations which allow for a broad classification of how to treat certain risks. Therefore, the aim of this section is to provide some guidance as to which broad management goals should be investigated and what information is required in order to prioritise management actions.

The assessment of risk management is more open-ended, but needs to be documented in detail to assure transparency of decisions. In the RAAT framework, this includes socio-economic and environmental benefits, the feasibility to stop future immigration of the Taxon, and basic considerations regarding management feasibility (Suppl. material 2). The latter are based on Wilson et al. (2017) and Panetta and Timmins (2004) and include: a) accessibility of populations, b) whether detectability is time-dependent, c) time to reproduction, and d) propagule persistence of the Taxon. A scoring approach leads to a basic assessment of the ease of management.

Further to the assessment of these traits, it is important to note that for an assessment of eradication feasibility, a detailed study including, for example, the delimitation of all alien populations of the Taxon, population estimates, management trials, and some estimate of the return on investment of different competing strategies, should be conducted (Wilson et al. 2017). Eradication should not be set as a target if not evaluated in detail, as this could lead to a waste of limited resources (e.g., Cacho et al. 2007). To aid this process, there is a question in the framework asking if an eradication feasibility study has been performed for the Area (MAN4 in Suppl. material 2) and a further question on control options available (MAN5 in Suppl. material 2).

The answers provided in the risk management section feed into Fig. 2, which leads to broad recommendations on how to manage a Taxon. These differ, based on whether the Taxon is already present in the Area, whether prevention or eradication are feasible goals, and whether the Taxon has benefits to the Area, such that it might be a conflict species that could be allowed with a permit under certain conditions (Fig. 2).

Figure 2.

A decision tree for determining the appropriate regulatory response for species which are considered to be of medium or high risk during the risk assessment process. The information in brackets refers to question numbers in the RAAT framework (Table 1 and Suppl. material 2).

Once the level of risk has been determined and options for management and benefits evaluated, it is crucial to clearly communicate the outcomes of the analysis to stakeholders, including the general public, policy-makers, traders, and users of the Taxon. We identify two important components of risk communication. First, stakeholders need to be engaged during the risk analysis process for assessors to obtain information on the Taxon and to gain the support of stakeholders in the process (e.g., Novoa et al. 2018). There are often formal regulatory processes of stakeholder engagement and, in contentious cases, an independent scientific assessment might be needed (Scholes et al. 2017), but if conflicts are to be avoided, engagement should happen close to the outset of the process. Second, risk communication is important to provide stakeholders with sufficient information to understand the recommendations and be in a position to know under which circumstances decisions would change, for example, how new information will influence risk. Therefore, communication needs to be simple enough to reach understanding, but needs to provide enough information to underpin the decision.

In the RAAT framework, we incorporated several communication strategies to reach these goals. We provide a decision tree which uses information from the analysis to make recommendations on the management strategy for the Taxon. Fig. 2 describes how to arrive at recommendations for the management and regulation from the answers provided in the risk analysis. This depicts a simplified decision-making process which can be easily understood by policy-makers and stakeholders, while the details to feed into the flow diagram are documented and provided in detail in the full analysis. Furthermore, in addition to providing all details of the risk analysis with information on each parameter, we provide a template for an easy-to-digest summary and reporting sheet, including the conclusions from each section, with short descriptions on the Taxon itself, impacts, risks, ease of management, and benefits. An example of a summary sheet is given in the Suppl. material 3.

Beside the need to set appropriate management goals after risk analysis, there are some other considerations to be made specifically in the South African context. The NEMBA A&IS Regulations set out four potential listing statuses, all linked to specific conditions (Department of Environmental Affairs 2014; Kumschick et al. 2020b): Category 1a: eradication targets; Category 1b: control targets (potentially with exemptions); Category 2: control targets for which certain activities are allowed under permits with conditions; Category 3: control targets with exemptions. During the development and testing of the RAAT framework, it became clear that, with a desktop study (such as the RAAT framework) alone, these categories cannot always be conclusively determined. We therefore recommend that many of the management specific recommendations should be developed on a case by case basis for the species regulated. This includes, for example, suitable permit conditions for category 2 species, management goals for category 1b species (e.g., containment or asset protection, and the need for area-specific management), and the situations under which species can be exempt from conditions (this included category 3 species which are effectively listed the same as category 1b species with some specified exemptions according to the NEM:BA and its A&IS Regulations). Such exemptions could include trees declared as national monuments and protected as “heritage” (e.g., Dickie et al. 2014) should they prove not to contribute to the invasion. A related issue is that of subspecific entities – certain cultivars or varieties could be considered safe for cultivation even if the “parental stock” is invasive (e.g. Datta et al. 2020; Gordon et al. 2016). There is provision within the RAAT framework to assess sub-specific entities separately, but often data on underlying traits are missing (e.g., proof of sterility).

We believe that the RAAT framework is not the place to develop the details of such risk management issues in depth. This should rather be an integral part of the development of national management programmes for particular taxa that can elucidate where and when control should be targeted and when, perhaps, control will be ineffective (for South African examples of such plans, see, for example, van Wilgen et al. 2011; Le Maitre et al. 2015; Terblanche et al. 2016; and the discussion in van Wilgen and Wilson 2018).