Research Article
Print
Research Article
A review of invasive species reporting apps for citizen science and opportunities for innovation
expand article infoLeif Howard, Charles B. van Rees§, Zoe Dahlquist, Gordon Luikart, Brian K. Hand
‡ University of Montana, Missoula, United States of America
§ University of Georgia, Athens, United States of America
Open Access

Abstract

Smartphone apps have enhanced the potential for monitoring of invasive alien species (IAS) through citizen science. They now have the capacity to massively increase the volume and spatiotemporal coverage of IAS occurrence data accrued in centralised databases. While more reporting apps are developed each year, innovation across diverse functionalities and data management in this field are occurring separately and simultaneously amongst numerous research groups with little attention to trends, priorities and opportunities for improvement. This creates the risk of duplication of effort and missed opportunities for implementing new and existing functionalities that would directly benefit IAS research and management. Using a literature search of Early Detection and Rapid Response implementation, smartphone app development and invasive species reporting apps, we developed a rubric for quantitatively assessing the functionality of IAS reporting apps and applied this rubric to 41 free, English-language IAS reporting apps, available via major mobile app stores in North America. The five highest performing apps achieved scores of 61.90% to 66.35% relative to a hypothetical maximum score, indicating that many app features and functionalities, acknowledged to be useful for IAS reporting in literature, are not present in sampled apps. This suggests that current IAS reporting apps do not make use of all available and known functionalities that could maximise their efficacy. Major implementation gaps, highlighted by this rubric analysis, included limited implementation in user engagement (particularly gamification elements and social media compatibility), ancillary information on search effort, detection method, the ability to report absences and local habitat characteristics. The greatest advancement in IAS early detection would likely result from app gamification. This would make IAS reporting more engaging for a growing community of non-professional contributors and encourage frequent and prolonged participation. We discuss these implementation gaps in relation to the increasingly urgent need for Early Detection and Rapid Response frameworks. We also recommend future innovations in IAS reporting app development to help slow the spread of IAS and curb the global economic and biodiversity extinction crises. We also suggest that further funding and investment in this and other implementation gaps could greatly increase the efficacy of current IAS reporting apps and increase their contributions to addressing the contemporary biological invasion threat.

Keywords

biosurveillance, citizen science, early detection and rapid response, invasive species, mobile device, species occurrence, wildlife technology

Introduction

Invasive alien species (IAS) are a leading contributor to biodiversity loss (Bellard et al. 2013; Simberloff et al. 2013; IPBES 2019) and cause annual economic damage in the order of hundreds of billions of US dollars in each of many countries around the world (Pimentel et al. 2005; Bradshaw et al. 2016; Sepulveda et al. 2020). The rate of new introductions shows no sign of saturation across time (Seebens et al. 2017) and the impacts and spread of IAS are expected to increase under climate change (Rahel and Olden 2008; Jourdan et al. 2018). The prevailing paradigm for IAS research, monitoring and management is Early Detection and Rapid Response (EDRR; Crall et al. 2012; Reaser et al. 2020a), which calls for coordinated, standardised and verifiable occurrence data across large spatial scales to support monitoring, biosurveillance and risk assessment (Martinez et al. 2020; Reaser et al. 2020a; Wallace et al. 2020).

Reports from volunteers (commonly called community or citizen scientists) make growing contributions to meeting these monitoring data needs, from providing first detections of new invasions (Vendetti et al. 2018; Eritja et al. 2019) to providing additional data that improves species distribution models (e.g. Roy-Dufresne et al. 2019; Zhang et al. 2020). The advent and rapid growth of mobile technology and smartphone software applications (hereafter apps) have greatly enhanced the potential for IAS reporting and the collection of crowdsourced (i.e. derived from many contributions) IAS occurrence data at unprecedented scales (Silvertown 2009; Teacher et al. 2013; Adriaens et al. 2015; Marchante et al. 2017). The integration of mobile apps with centralised databases is a major technological innovation contributing to the potential increase in available community science data for meeting the data demands of EDRR (Andrachuk et al. 2019; Wallace et al. 2020).

However, there are concerns that the current use of IAS mobile reporting apps is not maximising the potential of this powerful new technology for upscaling EDRR implementation needed to combat the worsening invasive species crisis (Martinez et al. 2020). The rapid growth, development and increasing proliferation of IAS apps has occurred quickly and with little coordination and communication amongst developers. This poses a major risk of development in apps that duplicate effort, result in errors (bugs) and is done in an in isolated environment where developers are unaware of the learning experiences and best practices proposed by others (Luna et al. 2018; Johnson et al. 2020). Martinez et al. (2020) suggested that the current technological toolbox to deal with IAS is still incomplete and inadequate and mobile apps constitute a major opportunity to address the needs of the field through technology.

There are a growing number of published articles describing IAS reporting apps (e.g. LaForest et al. 2011; Goëau et al. 2013; Scanlon et al. 2014; Wallace et al. 2016; Barre et al. 2017; Schade et al. 2019; Mäder et al. 2021), necessitating a solid conceptual framework for assessing how effectively existing and future apps can contribute to the broader vision of EDRR and global-scale invasion research and monitoring. Adriaens et al. (2015) provided a valuable review of IAS reporting apps in Europe, but many of these no longer exist (Schade et al. 2019) and mobile technology has made substantial strides in the last six to seven years, with the advent of 5G networks and a rapidly growing user-base now in excess 2.8 billion people (Alavi and Buttlar 2019).

We synthesised existing literature across the disciplines of invasion biology, citizen science and mobile app development to design a comprehensive rubric for assessing IAS app functionalities that could greatly improve the contribution of reporting apps to ongoing EDRR efforts (Martinez et al. 2020; Reaser et al. 2020a). Rubrics have been used to evaluate apps from other disciplines, especially education and healthcare (Lee and Cherner 2015; Stoyanov et al. 2015; Robinson et al. 2020) and can serve as a tool to assess the functionality of individual apps and the existing app corpus with respect to disciplinary and user needs. We applied this rubric to all free English-language apps available through the Apple App Store and Google Play in North America. We highlight trends and implementation gaps amongst reviewed apps and suggest key pathways for future innovation using existing technology. This review and the resulting rubric are intended to guide future IAS reporting app development and help address the demand for high-quality mobile platforms for collecting IAS occurrence data and while making the best use of the technological resources available to developers.

Methods

We modelled our rubric format after Lee and Cherner (2015), who divided rubrics into thematic units called domains, which contain any number of dimensions corresponding to particular elements, features or functionalities of mobile apps. We compiled information on app features and functionalities from our literature search (see Fig. 1) into four domains: Data Collection, Identification, Reporting, and User Engagement (Fig. 1). These domains were established a priori, based on recent EDRR literature referencing the proposed framework and the integration of mobile technology for reporting IAS (e.g. Martinez et al. 2020; Meyers et al. 2020; Morisette et al. 2020; Reaser et al. 2020; Wallace et al. 2020, Fig. 1).

Figure 1.

App review workflow. Each box header displays the workflow stage, examples of search terms used, and the number of papers used for that stage. The top-left box shows the search terms used for identifying rubric domains. These papers were reviewed to identify broad categories into which smartphone features could be organized for the rubric. The bottom left panel depicts the search string used to identify app dimensions within these domains (N = 498 papers, see also Table 1 for a detailed list of dimensions). In the central bottom panel, these dimensions are grouped by domain.

The Data Collection domain includes app functionalities pertaining to the type, method, geographic scale and taxonomic scope of data that an app can collect, while the Reporting domain focuses on how user-submitted data are input, collected and managed. The Identification domain pertains to features that aid in taxonomic identification, with features like built-in field guides or machine learning for image recognition. Finally, the User Engagement domain entails all participant-focused features, including options for guidance, help and feedback, ease of use and features to promote participation and sustained use, such as games and social networking elements.

We then conducted targeted searches on the Web of Science (WOS) and Google Scholar to identify the dimensions for our rubric (Fig. 1). We compiled a list of 498 papers which were distributed between two of the authors to determine relevance and extract app features described as potential dimensions for the rubric. To determine relevance, the abstracts and introductions of each paper were first visually scanned for references to smartphone or mobile apps, invasive species research, citizen science or other similar terms (Suppl. material 1: Table S1). The 91 relevant publications (Suppl. material 1: Table S1) were then examined more closely to identify pertinent dimensions which were added to a running list (Table 1). Due to the use of multiple terms within different sources for similar dimensions, we consolidated similar functionalities into single rubric dimensions. For example, games, contests and rewards were grouped together as gamification.

Table 1.

App dimensions organized by the four rubric domains with source information (relevant literature) and rubric scoring scale used to rank smartphone mobile apps. Domains are indicated by superscript prefix as follows: DC = Data Collection; ID = Identification; Rep = Reporting; Eng = User Engagement. Letters within the parentheses following each dimension name correspond to that dimension in Figure 3.

Dimension Definition Rubric Scoring Scale Relevant Literature
DCAbsence Data (A) Users can submit negative reports or the absence of a specific IAS. 0 = not present; 3 = present. Wallace et al. 2016.
DCAbundance/ Area (B) Users can enter the number of individuals, abundance, or area covered by the observed IAS. 0 = not present; 3 = present. Schade et al. 2019; Wallace et al. 2016.
DCCatch Per Unit Feature/ Time Spent for Observation (C) User can include information on time spent looking for IAS. This can be used to calculate catch per unit effort and potentially estimate abundance. 0 = not present; 3 = present. Bannerot, S. P., and Austin, C. B. 1983
DCClimate/ Habitat Data (D) Reporting interface includes climate and habitat/site context-related metadata fields (i.e., temperature, water flow rate, substrate, etc.) 0 = not present; 3 = present. Adriaens et al. 2015; Andrachuk et al. 2019; Reaney et al. 2019.
DCExternal Sensors (E) Users can sync external devices that collect data or detect IAS and/or the app allows upload of additional data types (sound recordings, rapid genetic identification results from biofouling or propagule analysis, eDNA/PCR/ddPCR results). 0 = not present; 3 = present. Adriaens et al. 2015; Andrachuk et al. 2019; Brick et al. 2020; Joseph et al. 2020; Kamolov and Park 2019; Liew et al. 2020; Martinez et al. 2020; Pastick et al. 2020; Rowley et al. 2019; Shao et al. 2020.
DCInternal Sensors (F) App has access to utilize smartphone’s thermometer, gyroscope, air humidity sensor, internal clock, barometer, and GPS to gather background data for sighting. 0 = not present; 1 = one internal sensor used; 2 = two internal sensors used; 3 = three or more internal sensors used Andrachuk et al. 2019; Adriaens et al. 2015; Bergquist et al. 2020; Hu et al. 2019; Kvapilova et al. 2019; Reaney et al. 2019; Schade et al. 2019; Schneider 2014; Wallace et al. 2016; Wu et al. 2019.
DCLarge Geographical Range (G) Data collection is not limited by the spatial focus of the app. 0 = local; 1 = state/ province wide; 2 = regional; 3 = national or international Adriaens et al. 2015; Schade et al. 2019.
DCManual Notes/ Data Entry (H) Allows users to input manual notes to capture observation/situational data that is not part of the formatted reporting form. 0 = not present; 3 = present. Scott et al. 2020.
DCSighting Type/ Status Documentation (Alive/Dead and/or LifeStage) (I) User can document the life stage, infestation stage or condition of the species observed (alive vs dead). 0 = not present; 3 = present. Pochon et al., 2017
DCSampling Method Documentation (J) User can indicate type of sampling method (i.e., visual observation, hook and line, snorkeling, trail camera, etc.) 0 = not present; 3 = present. Shuster et al., 2005
DCTaxonomic Range (K) Data collection is not limited by the taxonomic focus of the app. Data can be recorded for all types of IAS. 0 = single species; 1 = single taxonomic group (e..g, genus, family) ; 2= multiple, non-nested taxonomic groups; 3 = any species or taxon Adriaens et al. 2015; Wallace et al. 2016.
DCTracks Target Over Time (L) Allows monitoring specific target or location over time to track spread and changes to abundance or area covered by an IAS. Prompts follow up searches or reporting over time. App allows the user to report follow up visits or allows the second user/visit to validate sightings through comments on existing record 0 = not present; 3 = present. Adriaens et al. 2015; Liew et al. 2020; Lin et al. 2020; Wallace et al. 2016.
IDAI/Photo ID (M) App identifies taxa or returns results based on photo and machine learning or uses machine learning to train algorithms to gather data. 0 = not present; 3 = present. Hosseinpour et al. 2019; Veenhof et al. 2019.
IDIAS List/ Field Guide (N) App includes a list of known and common invasives with pictures and information or the app includes an interactive key that allows users to choose from IAS morphological attributes and the app makes suggestions to assist with identification. 0 = not present; 3 = present. Adriaens et al. 2015; Schade et al. 2019; Wallace et al. 2016.
IDMap w/ Observations (O) App has a map screen with points for verified IAS sightings. Ideally, this map is interactive allowing the user to access observational data by tapping the point. 0 = not present; 3 = present. Adriaens et al. 2015; Reaney et al. 2019; Schade et al. 2019; Wallace et al. 2016.
IDPhoto Upload (P) App has access to the onboard camera, and the user can take the picture and upload an image of the encountered IAS with timestamp and GPS data. 0 = not present; 3 = present. Adriaens et al. 2015; Andrachuk et al. 2019; Schade et al. 2019; Schneider 2014; Wallace et al. 2016.
IDReport Verification (Q) Reports submitted via app are verified by trained authority before being added to the database or posted on the user interface. 0 = none or relies on user selection of species from list; 1 = expert only or AI only verification; 2 = multiple levels of verification; 3 = multiple levels of verification that are indicated on observation/record within app. Adriaens et al. 2015; Schade et al. 2019; Wallace et al. 2016.
IDSearch/List Filter (R) User interface allows searching for specific IAS taxa, IAS type or by geographic region. 0 = not present; 3 = present. Zamberg et al. 2020.
IDUnknown Reporting (S) Previously undocumented or unidentified IAS can be reported. Allows reports of unknown species that are not listed in the app. 0 = not present; 3 = present. Rowley et al. 2019.
RepAutomated Outlier Rejection (T) App uses algorithms combined with internal or external sensors to exclude non-targeted data/ reports (i.e., uses GPS to exclude reports of desert species in tidal marsh). 0 = not present; 3 = present. Kvapilova et al. 2019; Pastick et al. 2020; Wu et al. 2019.
RepIntegrates Previous Reports (U) Data from established IAS reports/sightings and historical presence/absence data, which can be visualized by users 0 = not present; 1 = data available as a static distribution map; 2 = data integrate user observations that were previously submitted; 3 = data integrate user observations that were previously submitted plus data from other sources (e.g., government surveys) Wallace et al. 2016.
RepOffline Reporting (V) App stores data from reports when offline to be uploaded when the user returns to service. 0 = not present; 3 = present. Adriaens et al. 2015; Wallace et al. 2016; Schade et al. 2019.
RepReporting Form (W) App has a formatted reporting structure that includes all data required for EDRR/ report has required fields to standardize the data reported. 0 = not present; 3 = present. Wallace et al. 2016.
RepReports to Central Database (X) Reports are submitted to a national IAS database for verification and use by appropriate IAS decision- making entities. 0 = no database; 1 = Stores data that could be accessed and filtered for IAS data; 2 = App/project has dedicated IAS database; 3 = App sends data directly to central/ national or management/ agency IAS-centric database. Adriaens et al. 2015; Schade et al. 2019; Wallace et al. 2016; Wallace et al. 2020; Zamberg et al. 2020.
RepWebsite/ Dashboard (Y) Website reporting component and online frontend user dashboard to access IAS information and support the IAS app. 0 = not present; 1 = link to parent site with program or developer info only; 2 = link to parent program site w/ reporting ability; 3 = link to program site with reporting and user interface Adriaens et al. 2015; Rowley et al. 2019; Schneider 2014; Wallace et al.,2016.
EngDevice Compatibility (Z) Available on both IOS platforms (Android/ iPhone). Users are not limited by the type of smartphone owned. 0 = not available; 1 = One IOS platform only; 3 = both major platforms Adriaens et al. 2015; Wallace et al. 2016; Zamberg et al. 2020.
EngFeedback Feature (AA) Users can contact admin or developer with comments or suggestions and this feature is readily accessible within the user interface. 0 = not present; 2 = buried in secondary screens; 3 = accessible from main menu
EngGamification (AB) App includes features to promote user engagement through competition (i.e., Leader Boards, Rankings, Quizzes, or Contests to promote use. Badges, Trophies, Unlockable Content, Tracking Progress). 0 = not present; 3 = present. Aebli 2019; Adriaens et al. 2015; Andrachuk et al. 2019; Bayuk and Altobello 2019; Cellina et al. 2019; Szinay et al. 2020; Wallace et al. 2016.
EngHelp Content (AC) App includes guidance materials on how to use the app or link to Frequently Asked Questions / troubleshooting solutions for common questions and user-related concerns. 0 = not present; 2 = a help link is available to separate support page; 3 = in-app help functionalities and information are available Adriaens et al. 2015.
EngIAS Related Title (AD) App title implies purpose is IAS reporting. 0 = title has no mention or indication of relation to IAS; 1 = mentions an invasive species or taxon; 2 = uses the acronym IAS in the title; 3 = includes the term “invasive” or “invasion” Wallace et al. 2016.
EngNews Feed/ Notifications (AE) In-app feature to build a sense of community. Interface where recent sightings are highlighted, and app or IAS related news can be viewed by the end user/ In app notifications from admin to user or via social media notifications. 0 = not present; 3 = present. Joseph et al. 2020; Szinay et al. 2020.
EngSocial Media Outlet (AF) Users can upload/post verified reports to social media platforms directly from the IAS app. Allows users to share status, trophies, number of verified reports. App allows login using social media platform login info to connect directly to users’ social media outlet of choice. 0 = none; 1 = Function to share observations or keep private within the app; 2 = has a share icon that allows user to send messages or share via individual’s personal social media accounts; 3 = App has dedicated social media platform accounts for posting shared observations Adriaens et al. 2015; Andrachuk et al. 2019; Cellina et al. 2019; Joeckel and Dogruel 2020; Martinez et al. 2020; Szinay et al. 2020.
EngUpdated Regularly (AG) Developers and Admin regularly update the app to fix bugs and add new dimensions as they become available and relevant. 0 = Last updated four or more years ago; 1 = Last updated three years ago, 2 = Last updated two years ago; 3 = Updated in the last year Castaneda et al. 2019.
EngUser Account/ Login (AH) Users can create a private unique user ID and password to protect information stored on the app. Can be set to stay logged in or prefill login info to increase ease of reporting via preferences. 0 = no user account system; 1 = users log in for every use; 2 = user ID’s saved for automatic login; 3 = User ID’s saved and linked to e-mail address or other contact information Andrachuk et al., 2019; Joeckel and Dogruel, 2020; Schade et al. 2019; Wallace et al. 2016.
EngUser-Centered Design (AI) User-friendly interface. Easily navigable design. Users can easily send reports without going through a bunch of screens or submitting unnecessary information. 0 = text only; 1 = simple user interface with report form; 2 = basic and intuitive user interface; 3 = multiple-page user interface with buttons, images, visual guides, and dropdowns Adriaens et al. 2015; Bergquist et al. 2020; Birnie et al. 2019; Scott et al. 2020; Wallace et al. 2016.

Our final rubric consisted of 35 dimensions which are listed by domain along with definitions and source information in Table 1. Most dimensions were scored by the presence (3 points) or absence (0 points) of key functionalities, although some used a scale including 1, 2 and 3 points for dimensions with multiple levels (e.g. different geographical scales, wherein local scales received a score of 0, state or province scales, a score of 1, regional scales, a score of 2 and national or international, a scale of 3; Table 1).

Next, we compiled a list of all free, English-language IAS reporting apps on the Google Play and Apple iTunes online app stores using a methodised search (Fig. 1). We defined IAS reporting apps as those which were specifically focused on detecting and monitoring IAS and offered user functionality to report an IAS occurrence. The final eligibility of each app was determined by the ability to report observations directly from the app, to eliminate apps that were not used for IAS reporting (e.g. apps only for identification and outreach and no reporting functionality were excluded). We also specifically included iNaturalist, Flora Incognita and Plantnet, which are recommended and commonly used for reporting invasives by some agencies, though they were not specifically designated for IAS reporting. This yielded a final sample of 41 IAS apps (Fig. 1).

We then collected additional information from online store descriptions and metadata for all apps to gain insight into regional trends, the types of agencies using app data, app publishers, download trends and temporal trends in app release and availability. Download statistics were based on Google downloads and were not available for four apps. Download statistics are reported by Google in numerical bins (i.e. ≥ 5, ≥ 100, ≥ 1,000), so we calculated summary statistics as approximations.

Apps were then downloaded and three reviewers independently applied our rubric to each app. Scores for each rubric dimension were determined, based upon the presence and functionality of each feature within the app and feature descriptions from mobile app stores. Each reviewer received training in how to interpret dimension scores and categories to increase consistency. Reviewers then completed the rubric for all apps independently. We assessed the concordance amongst reviewer scores to check for any major inconsistencies using Spearman’s non-parametric correlation with the package Hmisc (Harrell Jr 2021) implemented in R (version 4.0.3; R Core Team 2021). We assessed reviewer concordance for all total, subtotal and dimension-specific scores. We calculated mean scores for all total, subtotal and dimension-specific scores and used these as the primary method of comparison and ranking among apps.

We then examined the distribution of rubric scores for individual apps, as well as within domains and individual dimensions. For domain- and dimension-specific scores, we report scores for the top apps after reporting scores for the entire sample. This allows for comparison of overall app corpus performance and top apps with respect to the idealised suite of mobile app functionalities specified in our rubric and with respect to the top-performing apps being used. Here, we present total rubric scores and domain subtotal scores as percentages and provide raw scores in parentheses.

Results

We found strong concordance between app total scores amongst all three reviewers, with pairwise Spearman’s correlation values ranging from 0.72 to 0.82 (p values, all < 0.0001; Suppl. material 2: Table S2). This concordance held for individual dimension scores, with rank correlations ranging from 0.78 to 0.93 (p values, all < 0.0001; Suppl. material 2: Table S2). Total rubric scores for the 41 IAS reporting apps in our sample ranged from 27.93% to 66.35% of the maximum score (29.33 – 69.67 points; Fig. 2), with a mean of 46.64% ± 10.88% (48.98 ± 11.42 points; Fig. 2). Total rubric scores amongst all apps followed a near-normal distribution (Fig. 2). The top five apps were: GLEDN, EDDMapS, IveGot1, MAEDN, Outsmart Invasive and Species. Total scores for these top-performing apps ranged from 61.90% to 66.35% of maximum (65.00 – 69.67 points) with a mean of 63.56% ± 1.83% (66.73 ± 1.92; Fig. 2). Raw data for all reviewed apps are available in supplemental materials (Suppl. material 3, 4: Table S3, S4).

Figure 2.

Distribution of total rubric scores across all apps. Rubric scores were near-normally distributed around the mean (dashed vertical line).

The Data Collection Domain had 36 available points from 12 dimensions (Table 1). Scores in this domain across all apps ranged from 18.53% to 68.61% of maximum (6.67 – 24.70 points) with a mean of 39.97% ± 13.22% (14.39 ± 4.76; Fig. 3a), while scores for the top-performing apps ranged from 50.00% to 68.53% (18.00 – 24.67 points) with a mean of 57.78% ± 9.04% (20.80 ± 3.25; Fig. 3a). No apps were readily compatible with external sensors and only seven apps included documentation of the sampling method by which species were detected. Other app dimensions with relatively low implementation (< 40% of sampled apps) included documentation of Catch per Unit Effort (13 apps) and Climate and Habitat data and Absence reporting (implemented by 15 and 14 apps respectively; Fig. 3a; see Table 1 for definitions).

Figure 3.

Distributions for domain subtotal scores across all reviewed apps w/ distribution of scores for each dimension within the domain a Data collection b User engagement c Reporting d Identification. Mean is indicated by the dashed vertical line for domain subtotals and by the points for dimensions.

Apps could score a maximum of 30 points from 10 dimensions in the User Engagement domain. Observed scores ranged from 16.67% to 65.57% of available points (5.00 – 19.67 points) with a mean of 44.20% ± 12.67% (13.26 ± 3.80; Fig. 3d) and top scores from 47.77% to 65.57% (14.33 – 19.67) with a mean of 57.33% ± 6.74% (17.20 ± 2.02; Fig. 3b). Only two apps (iNaturalist and Squishr; 5% of sampled apps) included gamification functionalities and only eight (< 20%; Invasive Plants of Arizona, ERWP Invasives Reporter, PlantNet, iNaturalist, Squishr, CSMON-LIFE Observation, FeralScan Pest Mapping and NJ Invasives) included compatibility with social media (Fig. 3d).

Eighteen possible points from six dimensions were available within the Reporting domain. Observed scores ranged from 22.22% to 77.78% (4.00 – 14.00 points) with a mean of 54.65% ± 16.13% (9.84 ± 2.90; Fig. 3c) and top-performing apps ranging from 70.39% to 77.78% (12.67 – 14.00) with a mean of 74.44% ± 3.31% (13.40 ± 0.60; Fig. 3c). The lowest scoring dimension within this domain (with mean ~ 1 or below) across all apps was Automated outlier rejection (only iNaturalist and Report Invasives BC or ~ 5% of our sample included this feature; Fig. 3c).

The Identification domain had a maximum of 21 points from seven dimensions and observed scores ranged from 9.52% to 88.90% of maximum (2.00 – 18.67 points) with a mean of 54.70% ± 18.23% (11.49 ± 3.83 points; Fig. 3b) of available domain points, while top-performing apps ranged from 68.24% to 77.76% (14.33 – 16.33 points) with a mean of 73.02% ± 4.05% (15.33 ± 0.85; Fig. 3d). Only seven apps (Aqua Invaders; AquaHunter; Asian Hornet Watch; EDDMapS; Flora Incognita; iNaturalist; PlantNet or ~ 17% of sampled apps) implemented an artificial intelligence or machine learning approach to photo identification, which was the lowest scoring dimension in this domain of functionality.

We found that 28 of 41 (68.29%) sampled apps were from North America, followed by five apps from the European Union, two apps from the United Kingdom, three from Australia and one app focused on Eastern and Southern Africa (Suppl. material 3: Table S3). Data collected via 21 of 41 (51.22%) apps are sent to government agencies while nine apps were associated with NGOs, four with university programmes, two with the European Union International Invasives Programme and three apps with private entities. All five of the top-performing (i.e. highest scoring) apps were released by Bugwood LLC (University of Georgia Center for Invasive Species and Ecosystem Health, Tifton, Georgia, USA). We found that 28 apps (68.39%) allowed reporting of any taxon and did not specify a habitat type (i.e. focused on all taxa and biomes); three apps (7.32%) focused on all taxon types, but within the aquatic biome only; five apps (12.20%) focused only on plants; three apps (7.32%) focused on invertebrates only; one app focused on a single animal species (Suppl. material 3: Table S3).

The number of downloads for each app was highly right-skewed and ranged from 5 to 1,000,000 (mean = 27,600 ± 162,000). Only two apps (iNaturalist and Asian Hornet Watch) exceeded 100,000 downloads and two other apps had more than 10,000 downloads. Twenty-seven of the reviewed apps had 1,000 or fewer downloads (Suppl. material 3: Table S3). Four apps did not have download information available. The earliest two released apps in our review were released in 2010 and 2011, respectively, both released by Bugwood LLC and most apps released before 2015 were published by this developer.

Discussion

We synthesised existing literature in invasion biology, citizen science and mobile app development to generate a rubric describing the functionality of an idealised IAS reporting app and applied this rubric to the available, free, English language IAS reporting apps on two major app-indexing software platforms (Google Play and Apple App Store). We measured the breadth of implementation of various technologies and functionalities amongst the current corpus of IAS reporting apps to identify opportunities for improvement and innovation in mobile apps for IAS detection and monitoring.

Our review highlights the major implementation gaps and provides a formalised rubric for holistically and quantitatively assessing app design, relative to best practices and recommendations from literature and the scientific community. The repeatability and transparency of this rubric for future assessments is especially helpful given the proliferation of IAS reporting apps and their variable use lifetimes. Five of the 24 European IAS apps, reviewed by Adriaens et al. (2015), no longer existed a year later (Schade and Tsinaraki 2016). Furthermore, a careful assessment of existing app functions and re-use of knowledge is important to prevent “re-inventing the wheel” as app development continues in a piecemeal and fragmented fashion (Johnson et al. 2020). Our review also further indicates that even top IAS reporting mobile apps make use of only about 70% of the useful features and functionalities mentioned and recommended in the literature, suggesting that there is substantial room to improve the performance of IAS mobile apps, even with existing technology and knowledge.

A worthwhile caveat is that, although our rubric summarises current suggested features and best practices for IAS mobile reporting apps, an app need not receive a perfect score to be functional and effective. A hypothetical app achieving a perfect score in our rubric would be easy to use, include value-added and gaming functionalities to encourage user uptake and sustained participation, enlist multiple onboard smartphone sensors to collect ancillary information, use machine-learning functionalities for automated taxonomic identification, provide visualisations of past reports and sightings for target taxa, facilitate researcher-user interaction to reduce data collection bias and would collect data in standard formats that enabled data sharing and interoperability with other monitoring systems. This is no doubt much to ask of any developer or project, but patterns and trends in our study nonetheless point in the direction of helpful innovations for invasive species apps going forward.

Many important functionalities found in only a few sampled apps, reinforcing the notion that better use of available technology could make major contributions to IAS research and management, particularly the implementation of EDRR approaches (Lahoz-Monfort et al. 2019; Martinez et al. 2020). Notably, artificial intelligence or machine learning for photo identification was a poorly implemented functionality present in a small proportion of surveyed apps, despite its great success in driving user uptake and participation in apps like iNaturalist. This represents a major implementation gap for invasive species apps, both in that it would greatly enhance species identification and, thus, the reliability of species reports and that it might provide a functionality that increases public participation and utility to users.

The substantial variation observed amongst rubric scores for sampled apps further suggests that there is little consistency in app functionality and design between developers, a finding that echoes the observations of other researchers that IAS mobile app development is not well coordinated amongst projects (Adriaens et al. 2015; Johnson et al. 2020). Better coordination and consistency amongst IAS reporting app developers would prevent duplication of effort and accelerate innovation and implementation of useful functionalities. The availability of open-source code or templates for local agencies to develop apps, based on frameworks developed by larger and better-funded organisations, might address this need while also reducing implementation gaps.

The five top-scoring apps were set apart by including functionality for reporting absences or non-detections, unknown or unidentified taxa and detection metadata (i.e. survey method, time and effort). Rubric dimensions and corresponding mobile app functionalities that were absent from this higher-scoring subgroup are indicative of major gaps in IAS reporting app implementation and development. These also included automated quality control features like outlier flagging (to highlight potential first detections of an unreported species in an area for expert review) or rejection (for species that cannot occur in the indicated area; for example, a marine species on top of a mountain), the use of integrated mobile device sensors (e.g. thermometer, altimeter and barometer) and user-focused elements, such as social media compatibility and game features.

We observed the lowest proportional rubric performance in the Data Collection domain, which includes app features pertaining to how and what data are included in a user report. These low scores were driven by only a small number of apps allowing absence (non-detection) or abundance reporting or collecting ancillary information on habitat variables and little use of onboard sensor technology (even amongst top apps, as noted above). Absence (or non-detection) data are important in their own right for biosurveillance (i.e. confirming that a species has not reached or established in an area); such periodic verification of IAS absence or non-detection constitutes the biosurveillance needed for EDRR implementation.

Beyond monitoring (biosurveillance), absence data are also valuable as a complement to presence data, enabling much more robust statistical modelling of species distributions (Elith et al. 2017). Such models lie at the core of a proactive approach to IAS research and management because they enable spatially-explicit risk assessment and forecasting (Latombe et al. 2017; Battini et al. 2019; van Rees et al., in press). Many existing databases and reporting apps collect and accommodate presence-only data (Adriaens et al. 2015; Wallace et al. 2016). Although distribution models are constructed using presence-only data from community science data and mobile reporting apps (Kress et al. 2018; Malek et al. 2018), limitations exist compared to presence-absence distribution models. Presence-only modelling involves more statistical assumptions, with a higher likelihood of inaccurate model outputs and over-inflated model evaluation statistics due to the necessity of defining background or pseudo-absence points (Elith et al. 2017). Finally, a more systematic sampling and reporting of non-detections could greatly improve the modelling potential of IAS mobile app-generated data (Wallace et al. 2016).

Abundance and other quantitative data can, in turn, enable more explicit modelling of population behaviour, facilitating a mechanistic understanding of invasion dynamics (Latombe et al. 2017; McGeoch and Jetz 2019). Ancillary information on weather or other physical habitat characteristics can provide in-situ environmental covariates to enhance these types of modelling or even downscale spatial covariates derived from remote-sensing data (Atkinson 2013).

The onboard sensors and instrumentation available in contemporary mobile devices are increasing in diversity and quality and now include a barometer, gyroscope, accelerometer, microphone and ambient light sensor, along with gigabytes of data storage capacity (Lane et al. 2010). Bioacoustic analysis can detect and identify species in targeted and passive recordings (e.g. Platenberg et al. 2020), a process that can be increasingly automated using machine-learning approaches (Martinez et al. 2020). Camera traps, infrared cameras and other external sensors can now readily be linked to smartphones and could enhance IAS detection by allowing for the capture of images remotely and allow for the detection of cryptic species, based on thermal signatures, respectively (Martinez et al. 2020).

Reviewed apps also had generally low scores in the User Engagement domain, indicating that there is substantial room for innovation and growth in the degree and manner in which the volunteer community is engaged in IAS data collection. At the time of review, Invasive Plants of Arizona, ERWP Invasives Reporter, PlantNet, iNaturalist, Squishr, CSMON-LIFE Observation, FeralScan Pest Mapping and NJ Invasives allowed users to share observations via social media feeds. Other apps have begun to include this feature in more recent updates (e.g. Flora Incognita). Only iNaturalist and Squishr integrate leaderboards which introduce a competitive element to promote user engagement and retention. iNaturalist allows users to access and comment on reports/confirm or dispute taxonomic identification (Pimm et al. 2015).

The success and efficacy of highly popular reporting apps like eBird (Sullivan et al. 2014) and iNaturalist show the volumes of data that can be generated where user participation is high (e.g. > 1 million records on iNaturalist within ~ 7 years; Pimm et al. 2015); although these apps record more than just invasive species, their success is testament to what can be done with biodiversity apps when useful functionalities are provided to users. Limited implementation of such user engagement and user experience features is no doubt a major obstacle to similarly mainstreaming IAS monitoring amongst the nature-interested public. User motivation is a primary factor influencing the uptake and sustained use of mobile apps (Luna et al. 2018) and gamification (adding competitive or progress-orientated elements to the user experience) and social media connections (allowing socialisation and sharing of the activity) are effective motivators (Adriaens et al. 2015). The Budburst app (Han et al. 2011) offers badges and ranks to app users, based on their level of activity and performance in locating plant species and allows users to share their findings on the social media site Flickr. The social and aspirational motivations provided by game elements and sharing were the highest-ranked sources of motivations amongst surveyed contributors (Han et al. 2011); the potential for competitive ‘listing’, which has long been popular amongst birdwatchers, was captured in the eBird app, which is no doubt part of its enthusiastic and sustained uptake amongst users (Sullivan et al. 2014).

In addition to increasing user engagement and increasing data submissions, gamification elements could also allow better coordination between researchers and community scientists, increasing the value of individual reports for management and policy objectives (Groom et al. 2019). For example, gamification features could increase rewards for community science surveys and reporting in areas where data are more helpful for modelling or biosurveillance. These could include places with scarce data, lower visitation rates or for which repeated visits are needed for time-series analysis (Callaghan et al. 2019). Such mechanisms could be integrated with value-of-information analyses to provide spatiotemporal prioritisations and corresponding rewards to data collection that maximises value to decision-making or related statistical modelling (Dietze et al. 2018). Game elements and rewards could also encourage absence reporting to combat biases against reporting negative results or promote the validation and verification of flagged reports through follow-up visits. The latter feature was poorly represented amongst our surveyed mobile apps. Rewards, such as badges, contest rankings, personal lists or social media recognition, align well with researcher needs to increase sustained use and activity within apps, while increasing the benefit to volunteer participants. It is also worth acknowledging that there is a potential trade-off between user motivation, app usability and data quality, wherein highly effective gamification methods may provide perverse incentives to generate data that maximise rewards, even if the data themselves are not authentic (Adriaens et al. 2015).

Certain key functionalities for reporting data and automating quality control were also largely absent from our sampled apps: few apps allowed users to submit reports offline or save them for later submission and only two included automated quality control mechanisms, such as outlier rejection or flagging. Inclusion of these features could increase the quantity and quality of data from IAS reporting apps. For example, the use of machine learning to flag or eliminate false reports, could reduce the time spent on verification of submitted reports, especially where data volume exceeds the capabilities of experts or trained volunteers who are typically responsible for verification (e.g. Kress et al. 2018; Malek et al. 2018). Offline reporting capabilities are necessary to avoid spatial biases in reporting, wherein remote areas outside of typical mobile phone service are under-reported (Graham et al. 2011). Detecting novel invaders is an additional priority for EDRR risk analysis and horizon scanning (Roy et al. 2020). The ability to report an unknown species using an app provides real-time accurate data to support these processes. For example, the non-native Central American milk snake, recently discovered by community scientists in the Everglades (Brasilero 2021) would not have been reportable via many current IAS apps because it was not on their list of potential invasive taxa.

Taxonomic identification is a priority for EDRR risk assessment and eliciting the proper level of response to a detection. Photo ID and machine-learning algorithms could streamline the reporting process by cutting out the need for users to identify an IAS prior to being able to submit a report and improving report accuracy (Terry et al. 2020). For example, iNaturalist users can take a picture with their phone and are then offered possible taxa that match the uploaded image (Pimm et al. 2015). This type of functionality can increase reporting rates by reducing the burden of effort on users and provides an incentive for app use by providing reference photos and information on encountered organisms. Choe et al. (2020) developed a mobile app for identifying endangered parrots at customs checkpoints using a cognitive neural network algorithm and similar image recognition technology could help users learn to identify and report species of concern as they are encountered.

Additional data collected from app descriptions indicated that non-standardised data from many mobile apps are being sent to a plethora of non-interconnected regional or local databases (Luna et al. 2018; Johnson et al. 2020). This corroborates findings from other reviews of community-sourced IAS data (Adriaens et al. 2015; Luna et al. 2018; Johnson et al. 2020) that, although large volumes of data are being collected, their storage and management is highly fragmented and inconsistent. With few exceptions, we also found little information on the metadata and data management practices used by each app developer, echoing findings by Schade et al. (2017) in Europe that most apps are opaque with respect to data use and not amenable to data reuse. IAS occurrence databases amongst different apps and organisations are designed to meet different goals, objectives and standards (Ricciardi et al. 2000; Latombe et al. 2017), but data must ultimately be standardised and centralised to make them useful for EDRR applications at larger scales (Fuller and Nielson 2015; Reaser 2020; Wallace et al. 2020). The interoperability of community science IAS data from mobile apps and transparency by app developers as to how and where data are stored are high priorities for the field (Groom et al. 2017; Johnson et al. 2020). Apps built on the EDDMapS platform (Laforest et al. 2011), which send data to a national database, are a notable exception and a positive example for future reporting apps.

This review was limited to English language IAS reporting apps available in North America through the Apple App Store and Google Play, introducing a geographical and linguistic bias to our study sample. Further work should expand this review to apps in other languages and available in other parts of the world, although the number of existing IAS mobile apps and their users are also strongly biased towards Western Europe and North America (Johnson et al. 2020). Our data were also somewhat biased by the uneven distribution of apps amongst developers, including one developer (Bugwood LLC, n = 15 apps) which accounted for the majority of top-scoring apps.

Another caveat is the need for more publicly available information (e.g. use metrics), which could greatly facilitate further analysis of app performance and sustained use. In the absence of such data on actual use for each sampled app, this analysis was limited to their range of functionalities and basic information on number of downloads. Download statistics are, however, a flawed metric of the success or performance of an app, as effective data collection could take place on a small-scale, regional basis with relatively few downloads with an enthusiastic user base. Our inability to access user statistics or submitted data for the surveyed apps made such metrics unfeasible, but finding ways to share such information in ways that protect the privacy of users would help scientists investigate correlates of success across biodiversity apps. Despite these limitations, our results provide a useful framework for investigating the functionality of existing IAS apps and the degree to which they manifest best practices from EDRR and app development literature.

Future efforts in IAS reporting app development should emphasise better use of existing technologies, data sharing and management and interoperability and game features that can both increase user participation and coordination between researchers and app users. The development and implementation of gamification functionalities could greatly increase app uptake and sustained use and is compatible with potential mechanisms to improve the quality of data recorded by non-professionals through spatial prioritisation and reward systems. Further research on the prevalence of different motivating factors in IAS reporting app participation would support efforts to increase uptake and provide valuable guidance for marketing and gamification. Given the bellicose terminology and adversarial popular thinking around invasive species (Janovsky and Larson 2019), the optimal strategies for effective and ethical management and community science research of IAS could differ substantially from work in other systems of ecological community science for ethical reasons (e.g. Han et al. 2011). In other words, very different lessons might be learned about user motivations and how they can best be managed for citizen science applications, given that efforts are focused on detection and hopeful eradication, rather than preservation. Increasing the implementation of machine learning for image and sound recognition and, thus, the automation of detection from community science observations is also a major priority (Schade et al. 2019).

The cost of designing apps, especially ones providing the multitude of functionalities described above, is a potential obstacle to further innovation. App design and creation often cost in the order of tens to hundreds of thousands of US dollars (Odenwald 2019). The development of a generalised, customisable app template with multiple options for functionalities (including gamification and user rewards), but with consistent metadata, back-end data management and storage infrastructure could simultaneously reduce the data fragmentation amongst IAS mobile apps (Johnson et al. 2020), while also encouraging their use and uptake by regional organisations and the larger citizen science community. Such a centralised app template could be financially supported by governments, philanthropists and a group or consortium of organisations, thus reducing the financial burden on any one group and allowing a pooling of institutional and monetary resources. Importantly, the broader economic benefits of any successful IAS reporting app with large and consistent community participation would far outweigh initial investment costs, when considering avoided ecological, agricultural and other costs.

Although our framework gave greater credit to apps with larger taxonomic ranges, regionally-focused apps may have an advantage in connecting and identifying with the interests and attitudes of local users, increasing the volume and quality of participation. For example, Aquahunter, an app produced by a county-level invasive species department in Minnesota (USA), integrates features of larger focal scale apps, such as a photo recognition tool, the ability to share an observation on Twitter/Facebook and an interactive map with observations. Such implementations of social media may be more effective at smaller scales, where users are more likely to be socially connected prior to using the app. A template model allowing customisation for regional applications would maintain these advantages, while overcoming ongoing problems of data fragmentation and lack of interoperability amongst existing apps.

Conclusions

Smartphone apps, if widely used, are amongst the most promising approaches to monitor, predict and reduce the spread of invasive alien species. Wide-spread use of mobile apps could massively increase the spatiotemporal coverage of IAS data collection, yielding new modelling insights into invasion dynamics. Future apps would attract a greater and more consistent user base with the addition of gaming functions (e.g. leaderboards, reward systems), social media connections (e.g. sharing functionalities), the ability to report absences and valuable ancillary data on surrounding habitats, survey methods and survey effort. With broader participation, more informative reporting forms and more consistent and structured data management, IAS reporting apps could make much larger contributions to Early Detection and Rapid Response efforts worldwide. This, in turn, could save local, regional and national economies millions to billions of dollars annually, while protecting valuable ecological and agricultural systems for future generations.

Acknowledgements

This research was funded in part by NASA award #80NSSC19K0185 to GL, J. Kimball and BKH and a USGS Northwest Climate Adaptation Science Center award G17AC000218 to CBvR. We thank Rebekah Wallace (UGA Center for Invasive Species & Ecosystem Health), Helen Roy (UK Center for Ecology and Hydrology), Tim Adriaens (Instituut Natuur-En Bosonderzoek/INBO), Steve Amish (University of Montana), Katie Graybeal (U. Montana) and Phil Matson (Flathead Lake Biological Station) for helpful conversations and suggestions that enhanced the manuscript. We would also like to extend a special thanks to Anna Sansalone (U. of Wake Forest) for assisting in calibrating and testing the app rubric.

References

  • Adriaens T, Sutton-Croft M, Owen K, Brosens D, Kilbey D, Groom Q, Ehmig C, Thürkow F, Schneider K (2015) Trying to engage the crowd in recording invasive alien species in Europe: experiences from two smartphone applications in northwest Europe. Management of Biological Invasions 6(2): 215–225. https://doi.org/10.3391/mbi.2015.6.2.12
  • Aitkenhead M, Donnelly D, Coull M, Hastings E (2014) Innovations in Environmental Monitoring Using Mobile Phone Technology – A Review. International Journal of Interactive Mobile Technologies 8(2): 42–50. https://doi.org/10.3991/ijim.v8i2.3645
  • Alavi AH, Buttlar WG (2019) An overview of smartphone technology for citizen-centered, real-time and scalable civil infrastructure monitoring. Future Generation Computer Systems 93: 651–672. https://doi.org/10.1016/j.future.2018.10.059
  • Andrachuk M, Marschke M, Hings C, Armitage D (2019) Smartphone technologies supporting community-based environmental monitoring and implementation: a systematic scoping review. Biological Conservation 237: 430–442. https://doi.org/10.1016/j.biocon.2019.07.026
  • August T, Harvey M, Lightfoot P, Kilbey D, Papadopoulos T, Jepson P (2015) Emerging technologies for biological recording. Biological Journal of the Linnean Society 115(3): 731–749. https://doi.org/10.1111/bij.12534
  • Baruch A, May A, Yu D (2016) The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform. Computers in Human Behavior 64: 923–931. https://doi.org/10.1016/j.chb.2016.07.039
  • Battini N, Farías N, CB G, Schwindt E, Bortolus A (2019) Staying ahead of invaders: using species distribution modeling to predict alien species’ potential niche shifts. Marine Ecology Progress Series 612: 127–140. https://doi.org/10.3354/meps12878
  • Bilodeau P, Roe AD, Bilodeau G, Blackburn GS, Cui M, Cusson M, Doucet D, Griess VC, Lafond VMA, Nilausen C, Paradis G, Porth I, Prunier J, Srivastava V, Stewart D, Torson AS, Tremblay E, Uzunovic A, Yemshanov D, Hamelin RC (2019) Biosurveillance of forest insects: part II—adoption of genomic tools by end user communities and barriers to integration. Journal of Pest Science 92: 71–82. https://doi.org/10.1007/s10340-018-1001-1
  • Bradshaw CJA, Leroy B, Bellard C, Roiz D, Albert C, Fournier A, Barbet-Massin M, Salles J-M, Simard F, Courchamp F (2016) Massive yet grossly underestimated global costs of invasive insects. Nature Communications 7: e12986. https://doi.org/10.1038/ncomms12986
  • Callaghan CT, Poore AGB, Major RE, Rowley JJL, Cornwell WK (2019) Optimizing future biodiversity sampling by citizen scientists. Proceedings of the Royal Society B: Biological Sciences 286: e20191487. https://doi.org/10.1098/rspb.2019.1487
  • Crall AW, Renz M, Panke BJ, Newman GJ, Chapin C, Graham J, Bargeron C (2012) Developing cost-effective early detection networks for regional invasions. Biological Invasions 14: 2461–2469. https://doi.org/10.1007/s10530-012-0256-3
  • Dietze MC, Fox A, Beck-Johnson LM, Betancourt JL, Hooten MB, Jarnevich CS, Keitt TH, Kenney MA, Laney CM, Larsen LG, Loescher HW, Lunch CK, Pijanowski BC, Randerson JT, Read EK, Tredennick AT, Vargas R, Weathers KC, White EP (2018) Iterative near-term ecological forecasting: Needs, opportunities, and challenges. Proceedings of the National Academy of Sciences 115: 1424–1432. https://doi.org/10.1073/pnas.1710231115
  • Eritja R, Ruiz-Arrondo I, Delacour-Estrella S, Schaffner F, Álvarez-Chachero J, Bengoa M, Puig M-Á, Melero-Alcíbar R, Oltra A, Bartumeus F (2019) First detection of Aedes japonicus in Spain: an unexpected finding triggered by citizen science. Parasites & Vectors 12: e53. https://doi.org/10.1186/s13071-019-3317-y
  • Fuller P, Neilson M (2015) The U.S. Geological Survey’s Nonindigenous Aquatic Species Database: over thirty years of tracking introduced aquatic species in the United States (and counting). Management of Biological Invasions 6(2): 159–170. https://doi.org/10.3391/mbi.2015.6.2.06
  • Gardiner MM, Allee LL, Brown PMJ, Losey JE, Roy HE, Smyth RR (2012) Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Frontiers in Ecology and the Environment 10(9): 471–476. https://doi.org/10.1890/110185
  • Graham EA, Henderson S, Schloss A (2011) Using mobile phones to engage citizen scientists in research. Eos, Transactions American Geophysical Union 92(38): 313–315. https://doi.org/10.1029/2011EO380002
  • Groom Q, Strubbe D, Adriaens T, Davis A, Desmet P, Oldoni D, Reyserhove L, Roy H, Vanderhoeven S (2019) Empowering citizens to inform decision-making as a way forward to support invasive alien species policy. Citizen Science: Theory and Practice 4. https://doi.org/10.5334/cstp.238
  • Janovsky RM, Larson ER (2019) Does invasive species research use more militaristic language than other ecology and conservation biology literature? NeoBiota 44: 27–38. https://doi.org/10.3897/neobiota.44.32925
  • Jarnevich C, Simpson A, Graham J, Newman G, Bargeron C (2015) Running a network on a shoestring: the Global Invasive Species Information Network. Management of Biological Invasions 6(2): 137–146. https://doi.org/10.3391/mbi.2015.6.2.04
  • Johnson BA, Mader AD, Dasgupta R, Kumar P (2020) Citizen science and invasive alien species: An analysis of citizen science initiatives using information and communications technology (ICT) to collect invasive alien species observations. Global Ecology and Conservation 21: e00812. https://doi.org/10.1016/j.gecco.2019.e00812
  • Jourdan J, O’Hara RB, Bottarin R, Huttunen K-L, Kuemmerlen M, Monteith D, Muotka T, Ozoliņš D, Paavola R, Pilotto F, Springe G, Skuja A, Sundermann A, Tonkin JD, Haase P (2018) Effects of changing climate on European stream invertebrate communities: A long-term data analysis. Science of the Total Environment 621: 588–599. https://doi.org/10.1016/j.scitotenv.2017.11.242
  • Kamenova S, Bartley TJ, Bohan DA, Boutain JR, Colautti RI, Domaizon I, Fontaine C, Lemainque A, Le Viol I, Mollot G, Perga M-E, Ravigné V, Massol F (2017) Chapter Three - Invasions Toolkit: Current Methods for Tracking the Spread and Impact of Invasive Species. In: Bohan DA, Dumbrell AJ, Massol F (Eds) Networks of Invasion: A Synthesis of Concepts. Advances in Ecological Research. Academic Press, 56: 85–182. https://doi.org/10.1016/bs.aecr.2016.10.009
  • Kress WJ, Garcia-Robledo C, Soares JVB, Jacobs D, Wilson K, Lopez IC, Belhumeur PN (2018) Citizen science and climate change: Mapping the range expansions of native and exotic plants with the mobile app Leafsnap. BioScience 68(5): 348–358. https://doi.org/10.1093/biosci/biy019
  • Laforest J, Bargeron CT (2011) EDDMapS: The common operating platform for aggregating and using invasive species distribution data. Phytopathology 101: S96–S97.
  • Larson LR, Cooper CB, Futch S, Singh D, Shipley NJ, Dale K, LeBaron GS, Takekawa JY (2020) The diverse motivations of citizen scientists: Does conservation emphasis grow as volunteer participation progresses? Biological Conservation 242: e108428. https://doi.org/10.1016/j.biocon.2020.108428
  • Latombe G, Pyšek P, Jeschke JM, Blackburn TM, Bacher S, Capinha C, Costello MJ, Fernández M, Gregory RD, Hobern D, Hui C, Jetz W, Kumschick S, McGrannachan C, Pergl J, Roy HE, Scalera R, Squires ZE, Wilson JRU, Winter M, Genovesi P, McGeoch MA (2017) A vision for global monitoring of biological invasions. Biological Conservation 213(Part 2): 295–308. https://doi.org/10.1016/j.biocon.2016.06.013
  • Lee C, Cherner T (2015) A comprehensive evaluation rubric for assessing instructional apps. Journal of Information Technology Education 14: 21–53. https://doi.org/10.28945/2097
  • Linkie M, Guillera-Arroita G, Smith J, Ario A, Bertagnolio G, Cheong F, Clements GR, Dinata Y, Duangchantrasiri S, Fredriksson G, Gumal MT, Horng LS, Kawanishi K, Khakim FR, Kinnaird MF, Kiswayadi D, Lubis AH, Lynam AJ, Maryati, Maung M, Ngoprasert D, Novarino W, O’Brien TG, Parakkasi K, Peters H, Priatna D, Rayan DM, Seuaturien N, Shwe NM, Steinmetz R, Sugesti AM, Sunarto, Sunquist ME, Umponjan M, Wibisono HT, Wong CCT (2013) Cryptic mammals caught on camera: Assessing the utility of range wide camera trap data for conserving the endangered Asian tapir. Biological Conservation 162: 107–115. https://doi.org/10.1016/j.biocon.2013.03.028
  • Luna S, Gold M, Albert A, Ceccaroni L, Claramunt B, Danylo O, Haklay M, Kottmann R, Kyba C, Piera J, Radicchi A, Schade S, Sturm U (2018) Developing mobile applications for environmental and biodiversity citizen science: Considerations and recommendations. In: Joly A, Vrochidis S, Karatzas K, Karppinen A, Bonnet P (Eds) Multimedia tools and applications for environmental {\&} biodiversity informatics. Springer International Publishing, Cham, 9–30. https://doi.org/10.1007/978-3-319-76445-0_2
  • Lyal CHC, Miller SE (2020) Capacity of United States federal government and its partners to rapidly and accurately report the identity (taxonomy) of non-native organisms intercepted in early detection programs. Biological Invasions 22: 101–127. https://doi.org/10.1007/s10530-019-02147-x
  • Mäder P, Boho D, Rzanny M, Seeland M, Wittich HC, Deggelmann A, Wäldchen J (2021) The Flora Incognita app – Interactive plant species identification. Methods in Ecology and Evolution 12(7): 1335–1342. https://doi.org/10.1111/2041-210X.13611
  • Malek R, Tattoni C, Ciolli M, Corradini S, Andreis D, Ibrahim A, Mazzoni V, Eriksson A, Anfora G (2018) Coupling traditional monitoring and citizen science to disentangle the invasion of Halyomorpha halys. ISPRS International Journal of Geo-Information 7. https://doi.org/10.3390/ijgi7050171
  • Marchante H, Morais MC, Gamela A, Marchante E (2017) Using a webmapping platform to engage volunteers to collect data on invasive plants distribution. Transactions in GIS 21(2): 238–252. https://doi.org/10.1111/tgis.12198
  • Martinez B, Reaser JK, Dehgan A, Zamft B, Baisch D, McCormick C, Giordano AJ, Aicher R, Selbe S (2020) Technology innovation: advancing capacities for the early detection of and rapid response to invasive species. Biological Invasions 22: 75–100. https://doi.org/10.1007/s10530-019-02146-y
  • Morisette JT, Reaser JK, Cook GL, Irvine KM, Roy HE (2020) Right place. Right time. Right tool: guidance for using target analysis to increase the likelihood of invasive species detection. Biological Invasions 22: 67–74. https://doi.org/10.1007/s10530-019-02145-z
  • Pimentel D, Zuniga R, Morrison D (2005) Update on the environmental and economic costs associated with alien-invasive species in the United States. Ecological Economics 52(3): 273–288. https://doi.org/10.1016/j.ecolecon.2004.10.002
  • Platenberg RJ, Raymore M, Primack A, Troutman K (2020) Monitoring vocalizing species by engaging community volunteers using cell phones. Wildlife Society Bulletin 44(4): 782–789. https://doi.org/10.1002/wsb.1141.
  • Pocock MJO, Newson SE, Henderson IG, Peyton J, Sutherland WJ, Noble DG, Ball SG, Beckmann BC, Biggs J, Brereton T, Bullock DJ, Buckland ST, Edwards M, Eaton MA, Harvey MC, Hill MO, Horlock M, Hubble DS, Julian AM, Mackey EC, Mann DJ, Marshall MJ, Medlock JM, O’Mahony EM, Pacheco M, Porter K, Prentice S, Procter DA, Roy HE, Southway SE, Shortall CR, Stewart AJA, Wembridge DE, Wright MA, Roy DB (2015) Developing and enhancing biodiversity monitoring programmes: a collaborative assessment of priorities. Journal of Applied Ecology 52(3): 686–695. https://doi.org/10.1111/1365-2664.12423
  • Reaser JK (2020) Putting a federal capacities assessment to work: blueprint for a national program for the early detection of and rapid response to invasive species (EDRR). Biological Invasions 22: 167–176. https://doi.org/10.1007/s10530-019-02177-5
  • Reaser JK, Burgiel SW, Kirkey J, Brantley KA, Veatch SD, Burgos-Rodríguez J (2020) The early detection of and rapid response (EDRR) to invasive species: a conceptual framework and federal capacities assessment. Biological Invasions 22: 1–19. https://doi.org/10.1007/s10530-019-02156-w
  • Ricciardi A, Blackburn TM, Carlton JT, Dick JTA, Hulme PE, Iacarella JC, Jeschke JM, Liebhold AM, Lockwood JL, MacIsaac HJ, Pyšek P, Richardson DM, Ruiz GM, Simberloff D, Sutherland WJ, Wardle DA, Aldridge DC (2017) Invasion science: A horizon scan of emerging challenges and opportunities. Trends in Ecology & Evolution 32(6): 464–474. https://doi.org/10.1016/j.tree.2017.03.007
  • Rowley J, Callaghan C, Cutajar T, Portway C, Potter K, Mahony S, Trembath D, Flemons P, Woods A (2019) FrogID: Citizen scientists provide validated biodiversity data on frogs of Australia. Herpetological Conservation and Biology 14: 155–170.
  • Roy HE, Peyton JM, Booy O (2020) Guiding principles for utilizing social influence within expert-elicitation to inform conservation decision-making. Global Change Biology 26(6)S: 3181–3184. https://doi.org/10.1111/gcb.15062
  • Roy HE, Rorke SL, Beckmann B, Booy O, Botham MS, Brown PMJ, Harrower C, Noble D, Sewell J, Walker K (2015) The contribution of volunteer recorders to our understanding of biological invasions. Biological Journal of the Linnean Society 115(3): 678–689. https://doi.org/10.1111/bij.12518
  • Roy-Dufresne E, Saltré F, Cooke BD, Mellin C, Mutze G, Cox T, Fordham DA (2019) Modeling the distribution of a wide-ranging invasive species using the sampling efforts of expert and citizen scientists. Ecology and Evolution 9(19): 11053–11063. https://doi.org/10.1002/ece3.5609
  • Sepulveda AJ, Nelson NM, Jerde CL, Luikart G (2020) Are environmental DNA methods ready for aquatic invasive species management? Trends in Ecology & Evolution 35(8): 668–678. https://doi.org/10.1016/j.tree.2020.03.011
  • Shackleton RT, Adriaens T, Brundu G, Dehnen-Schmutz K, Estévez RA, Fried J, Larson BMH, Liu S, Marchante E, Marchante H, Moshobane MC, Novoa A, Reed M, Richardson DM (2019) Stakeholder engagement in the study and management of invasive alien species. Journal of Environmental Management 229: 88–101.
  • Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M (2015) Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth 3(1): e27. https://doi.org/10.2196/mhealth.3422
  • Sullivan BL, Aycrigg JL, Barry JH, Bonney RE, Bruns N, Cooper CB, Damoulas T, Dhondt AA, Dietterich T, Farnsworth A, Fink D, Fitzpatrick JW, Fredericks T, Gerbracht J, Gomes C, Hochachka WM, Iliff MJ, Lagoze C, La Sorte FA (2014) The eBird enterprise: An integrated approach to development and application of citizen science. Biological Conservation 169: 31–40. https://doi.org/10.1016/j.biocon.2013.11.003
  • Terry JCD, Roy HE, August TA (2020) Thinking like a naturalist: Enhancing computer vision of citizen science images by harnessing contextual data. Methods in Ecology and Evolution 11(2): 303–315. https://doi.org/10.1111/2041-210X.13335
  • Valles SM, Strong CA, Callcott A-MA (2016) Development of a lateral flow immunoassay for rapid field detection of the red imported fire ant, Solenopsis invicta (Hymenoptera: Formicidae). Analytical and Bioanalytical Chemistry 408: 4693–4703. https://doi.org/10.1007/s00216-016-9553-5
  • van Rees CB, Hand BK, Carter SC, Bargeron C, Cline TJ, Daniel W, Ferrante JA, Gaddis K, Hunter ME, Jarnevich CS, McGeoch M, Morisette JT, Neilson ME, Roy HE, Rozance MA, Sepulveda A, Wallace RD, Whited D, Wilcox T, Kimball JS, Luikart G (in press) . A Framework to Integrate Innovations in Invasion Science for Proactive Management. Biological Reviews.
  • Vendetti JE, Lee C, LaFollette P (2018) Five new records of introduced terrestrial gastropods in southern California discovered by citizen science. American Malacological Bulletin 36(2): 232–247. https://doi.org/10.4003/006.036.0204
  • Wallace RD, Bargeron CT, Reaser JK (2020) Enabling decisions that make a difference: guidance for improving access to and analysis of invasive species information. Biological Invasions 22: 37–45. https://doi.org/10.1007/s10530-019-02142-2

Supplementary materials

Supplementary material 1 

Table S1. Search parameters

Leif Howard, Charles van Rees

Data type: metadata

Explanation note: Literature review search terms and filtering.

This dataset is made available under the Open Database License (http://opendatacommons.org/licenses/odbl/1.0/). The Open Database License (ODbL) is a license agreement intended to allow users to freely share, modify, and use this Dataset while maintaining this same freedom for others, provided that the original source and author(s) are credited.
Download file (1.35 kb)
Supplementary material 2 

Table S2. Reviewer correlation

Leif Howard, Charles van Rees

Data type: statistics

Explanation note: Reviewer correlation results.

This dataset is made available under the Open Database License (http://opendatacommons.org/licenses/odbl/1.0/). The Open Database License (ODbL) is a license agreement intended to allow users to freely share, modify, and use this Dataset while maintaining this same freedom for others, provided that the original source and author(s) are credited.
Download file (868.00 bytes)
Supplementary material 3 

Table S3. App Metadata

Leif Howard, Charles van Rees

Data type: metadata

Explanation note: Metadata, total rubric scores and scores by domain for each reviewed app.

This dataset is made available under the Open Database License (http://opendatacommons.org/licenses/odbl/1.0/). The Open Database License (ODbL) is a license agreement intended to allow users to freely share, modify, and use this Dataset while maintaining this same freedom for others, provided that the original source and author(s) are credited.
Download file (7.55 kb)
Supplementary material 4 

Table S4. Mean dimension scores by app

Leif Howard, Charles van Rees

Data type: metadata

Explanation note: Mean dimension scores by app.

This dataset is made available under the Open Database License (http://opendatacommons.org/licenses/odbl/1.0/). The Open Database License (ODbL) is a license agreement intended to allow users to freely share, modify, and use this Dataset while maintaining this same freedom for others, provided that the original source and author(s) are credited.
Download file (8.55 kb)