Invasion by non-native species can alter community structure and ecosystem functions with significant impacts on biodiversity (Wilcove et al. 1998; Grosholz 2002; Olden et al. 2004; Bruno et al. 2005; Byers 2009) and large economic (Pimentel et al. 2000), and cultural costs (Lockwood et al. 2007). Efforts to manage and mitigate invasion impacts are contingent on rigorous scientific evidence for the underlying drivers of change (Underwood 1996; Byers et al. 2002; Sutherland et al. 2004). There is debate, however, about whether non-native species cause environmental change, respond to environmental change or do both with complex feedback interactions (Gurevitch and Padilla 2004; Didham et al. 2005; Simberloff 2011). Experimental approaches, where the identity and density of non-native species are controlled, can provide much-needed causal linkages between the non-native species and ecological changes in invaded ecosystems and habitats.

The discipline of invasion ecology has grown immensely in recent decades with papers published across an increasing number of general scientific (ecology, invasion biology, conservation biology) and subject-specific (terrestrial, freshwater and marine) journals. Recently, Pyšek et al. (2008) tallied >2500 papers published on the topic of invasion ecology. Furthermore, it appears that invasion impacts have been reviewed in greater detail for terrestrial than aquatic ecosystems; e.g., recent reviews of plant invasion impact did not include examples from marine seaweeds or seagrasses (cf. reviewed reference listings, Powell et al. 2011; Vilà et al. 2011; Pyšek et al. 2012). To be able to build strong and general impact models and theories across ecosystems it is important that aquatic impact studies are similarly evaluated. Indeed, there is growing scientific evidence that invasions by non-native species are prolific in aquatic ecosystems (Carlton 1996; Hewitt and Campbell 2010; Strayer 2010). Furthermore, even if new aquatic invasions are curbed, impacts from past invaders may continue to increase as lag-phases transition into exponential phases, and ecological interaction “debts” are “collected” over time (Sax and Gaines 2008; Kuussaari et al. 2009). Existing reviews of aquatic invasion impacts have focused so far on specific taxonomic groups (e.g., Ward and Ricciardi 2007; Thomsen et al. 2009; Cucherouset and Olden 2011; Twardochleb et al. 2013), regions (e.g., Ricciardi 2006; Ruiz et al. 2011) or tested specific ecological questions (e.g., Ricciardi and Atkinson 2004; Grosholz and Ruiz 2009; Thomsen et al. 2011). However, to date there has been no synthesis of invasion impacts across non-native taxa, regions and habitats in aquatic ecosystems, thereby limiting our capability to evaluate progress and target research gaps.

Our objectives were to quantify the breadth of field-based experimental studies on invasion impacts in aquatic ecosystems, to identify what species, regions and environments have been targeted in these tests, and whether there have been any general problems relating to their experimental design and data reporting. Analogue reviews have provided important knowledge about research biases in other fields, including climate change, terrestrial ecology, conservation biology and general invasion biology (Pyšek et al. 2008; Darwall et al. 2011; Martin et al. 2012; Wernberg et al. 2012; Kimbro et al. 2013; Sorte et al. 2013).

To achieve our objectives, we reviewed field-based experiments on invasion impacts to identify what species, regions and environments have been targeted in these tests, and whether there have been any general problems relating to their experimental design and data reporting. We hope that this review will stimulate experimental research about aquatic invasion impact and this will lead to the development of new predictive tools in conservation management.

We searched for peer-reviewed, manipulative aquatic field experiments in Web of Science, Current Contents, and Google Scholar, using various combinations of ‘impact’, ‘effect’, ‘non-native’, ‘non-indigenous’, ‘exotic’, ‘invader’, ‘alien’, ‘aquatic’, ‘marine’, and ‘freshwater’. Reference lists were back-tracked from existing review papers and frequently cited impact studies (e.g. Bruno et al. 2005; Williams and Smith 2007; Byers 2009). We stopped searching on 1^st of July 2012 to allow a definitive analysis, as new impact studies are being published every month. Field conditions were defined as any outdoor setting exposed to natural cycles of temperature and light (Martin et al. 2012). We included all studies that quantified impacts on resident organisms by comparing ‘invaded treatments’ to ‘non-invaded controls’ using addition or removal of the non-native species (Thomsen et al. 2011), including whole-lake experiments where non-native species (typically trouts) had been removed or added in well-documented events to specific lakes (treatments) but not to others (controls) (Pope et al. 2009). We also included studies published in non-invasion related contexts, for example impacts of “drift algae”, “turf algae”, or “grazers”, if the manipulated organisms were entirely dominated by non-native species, such as Gracilaria vermiculophylla, Wommersly setacea, and Littorina littorea, respectively (Bertness et al. 1999; Airoldi 2000; Thomsen and McGlathery 2006).

First, we evaluated the published research in relation to the attributes of individual studies. We categorized whether the journal in which experiments were published targeted general scientists, freshwater, marine, or aquatic biologists (see Suppl. material 1 for detail). We recorded the year of publication and determined whether the experimental design was spatially pseudo-replicated (i.e. with only one ‘spatially independent’ control and/or treatment, Hurlbert 1984), and if the study reported an average invasion impact on resident organisms with an associated measure of variation (e.g., SD, SE or CL). We specifically identified community impact studies because these are few in numbers but highly valued (Parker et al. 1999; Vilà et al. 2011). For each identified community study we counted if an impact was reported using multivariate structures from graphs (e.g. nMDS plots) or inferential tests (e.g. Anosim) and on univariate biodiversity metrics including taxonomic richness, diversity and evenness.

Second, we evaluated study characteristics related to the attributes of the non-native species, that is whether the non-native species was a plant or animal, pre-dominantly occupied the pelagic or benthic realm, was mobile or sessile, whether its trophic position was a plant, filter-feeder, herbivore, omnivore or carnivore, and its taxonomic affiliation. We classified very slow-moving bivalves as sessile taxa, and grouped consumers of macroscopic or microscopic primary producers (the latter group includes sediment eaters and detrivores) together to represent herbivores.

Third, we evaluated study characteristics related to the attributes of the invaded system. Here we extracted data about the spatial location of each experiment. Studies that included nested designs or multiple experiments, where study sites were separated by >1 km, were added as independent locations. These data were also grouped into invaded continent and latitude (proxy for climate). We then noted if the invaded system was a freshwater or marine habitat, where estuaries and salt marshes were grouped as marine. This attribute represents the invaded system (instead of the non-native species) because the same non-native species (e.g., the common reed Phragmites australis) can impact both marine and freshwater habitats. These two broad habitat types were subdivided into streams, ponds, lakes, wetlands, estuaries, open coast marine sands, rocky intertidal, or rocky subtidal, where wetlands included a few riparian studies, ponds included experiments conducted in outdoor freshwater mesocosms (ponds and mesocosm experiments are both conducted in relatively small enclosed systems), estuaries included both intertidal and subtidal studies, and the rocky subtidal zone included a few coral reef experiments (too few to justify a separate analysis for this habitat, see discussion for detail).

Finally, we examined if taxa were studied in proportion to their ‘recognized occurrence as non-native species’. We used Hewitt and Campbell’s (2010) tally of marine non-native species (this is a ‘grey literature’ report, but represent the most comprehensive tally to date), i.e. we tested if some taxa are well-recognized as non-native species but never studied. This comparison was only done for marine organisms because no master list exists for freshwater non-native species. We first regrouped invaded regions into geographical units used by Hewitt and Campbell (2010; 18 marine provinces based on biological similarity) and then conducted linear regression between our data to data listed in Hewitt and Campbell (2010, Fig. 2 and 11). We used Type 1 regression because our objective was to provide simple predictive models (Quinn and Keough 2002), and we forced the line through the origin because there can be no impact study if there is no invasion (the greater the slope, the more field experiments have been conducted relative to known marine non-native species).

Our initial search suggested that more than 700 aquatic invasion impact studies have been published. However, after excluding reviews (Williams and Smith 2007; Thomsen et al. 2009), indoor lab experiments (Hoeffle et al. 2011), and correlative studies (Staehr et al. 2000; Forrest and Taylor 2003) 259 manipulative field-based impact experiments remained. These peer-reviewed papers experimentally quantified invasion impact on 101 aquatic non-native species and on 4 invasive taxa above the species level (‘Dreissena mussels’, ‘trouts’, ‘predatory fish’, and ‘crabs’, for brevity we also refer to these taxa as non-native ‘species’). A total of 20 aquatic non-native species have been experimentally manipulated in more than 5 studies (Suppl. material 1), with 5 species being studied 10 times, including Carcinus maenas (a marine crab, 18 studies), ‘trouts’ (combined effects of freshwater Oncorhynchus mykiss and Salvelinus fontinalis, 12 studies), Littorina littorea (a marine snail, 11 studies), Pacifastacus leniusculus (a freshwater crayfish, 11 studies), and Dreissena polymorpha (a freshwater mussel, 10 studies).

Evaluating research in relation to study attributes revealed that many more papers were published in journals targeting general readers (130) than marine (61), freshwater (41) or freshwater and marine (27) readers. Publications of invasion impact papers increased rapidly for all journal types (Fig. 1a). The first study was published in 1972 (Hurlbert et al.), but very few studies were published in the 1970s (total of 2 species; 3 studies) or 1980s (8; 11). However, experimental approaches to testing invasion impacts became common in the 1990s (28; 43) and increased dramatically in the 2000s (79; 158). From 2010–2012 we identified experiments on 35 species in 44 studies. This trend corresponds to a general exponential decadal increase in published aquatic field-based invasion impact studies (r² = 0.99, p = 0.0006). Of the 259 impact papers, 12 were spatially pseudo-replicated or did not replicate treatment and/or control plots (Fig. 1b) and 22 did not report any variability around reported average impacts (Fig. 1b). We found 68 studies that reported impacts on multivariate or univariate community metrics (Fig. 1c), of which 35 quantified impact with multivariate plots, 32 with multivariate inferential tests, 52 on richness, 23 on diversity and 10 on evenness. Only two of the 68 studies reported all 5 community metrics (Hejda and Pyšek 2006; Albins 2013).

Furthermore, when we evaluated study characteristics in relation to invader attributes we found that more experiments have been done on non-native animals (75 species, 185 studies) than plants (30, 76) (Fig. 2a), on benthic (83, 199) than pelagic (22, 64) invaders (Fig. 2b), and on almost the same number of sessile (54, 116) and mobile (51, 143) invaders (Fig. 2c). Subdividing non-native animals into trophic position showed that carnivores (25, 73) were more studied than filter-feeders (27, 46) and omnivores (14, 44), and that few invasive herbivores and detrivores have been studied (10, 29) (Fig. 2d). Of these 10 species, 6 were herbivores with a capacity to denude large vegetation (the mammal Myocastor coypus, the bird Cygnos olor and the four snails Littorina littorea, Bellamya chinensis, Physella acuta, Pomacea canaliculata), whereas the remaining 4 non-native species were sediment-eaters and detrivorous snails and crustaceans (Batillaria attramentaria, Ilyanassa obsoleta, Potamopyrgus antipodarum, Limnomysis benedeni). The most frequent experiments were on non-native fish (20 species, 58 studies - where two studies were from marine systems and the lionfish Pterois volitans), vascular plants (20, 38) molluscs (18, 55), crustaceans (16, 53) and algae (10, 39). By contrast, relatively few experiments were on non-native tunicates (8 species, 7 studies), annelids (3, 7), amphibians (3, 5), mammals and birds (3, 5), bryozoans (2, 3), echinoderms (1, 3) or reptiles (1, 3) (Fig. 2e).

Evaluating study characteristics in relation to the invaded system attributes showed a strong geographical pattern (Fig. 3); impacts documented in the 259 papers represent studies from 30 countries and were strongly dominated by the USA (60 species; 137 studies). Furthermore, only 9 other countries have tested for impacts of ≥5 non-native species, including Australia (13; 21), Germany (8; 7), Brazil (6; 4), Canada (8; 11), Japan (6; 5), New Zealand (6, 5), Italy (5; 12), Spain (5; 10), and France (5; 4). This geographical pattern was also evident when grouped into continental scales; invasion impacts have been tested mainly in North America (59 species, 145 studies), Europe (30; 54), and Australasia (19; 26). Far fewer non-native species have been tested from South America (13; 10), Asia (8; 11), Africa (4; 8) and various islands (4; 5) (Fig. 3, 4a).

We also found a strong geographical pattern along a latitudinal (climate-related) gradient; most studies were from the mid-latitudes of 35–46° (69 species, 171 studies), followed by 23–34° (24; 36) and 47–58° (21; 33). By contrast, few non-native species have been studied from tropical or cold/polar regions (0–22° = 12 species, 14 studies; 59–90° = 5 species, 5 studies – and these studies were all from 59–64° N) (Fig. 3, 4b). Finally, we found that more studies have been conducted in freshwater (59 species, 143 studies) than marine (48; 116) habitats (Fig. 4c), or more specifically, that most aquatic invasive species have been studied in ponds (33; 43, including mesocosm experiments), followed by estuaries (30; 53), streams (18; 43), wetlands (15; 24), rocky subtidal (16; 21, including 3 species and 4 studies from coral reefs), lakes (15; 33), rocky intertidal (12; 32), and sandy open subtidal habitats (7; 11) (Fig. 4d).

Finally, when we compared our data to Hewitt & Campbell’s compilation of marine non-native and cryptogenic species we found a significant slope, indicating that the number of marine species grouped into different invasive taxa (Fig. 5c; slope = 0.022, p = 0.0001, n = 17) and invaded biogeographical regions (Fig. 5d; slope = 0.021, p = 0.0006, n = 18) correlated with number of taxa and regions tested in manipulative field experiments. However, even though the line fit well generally, there were still over and under-represented taxa; for example, experimental studies on molluscs, angiosperms and chordates (mainly tunicates) were over-represented (well above the line), whereas fish, crustaceans and cnidarians were under-represented (well below the line). Similarly, experimental studies from the North East Pacific and the North West Atlantic were over-represented, whereas experimental studies from the Southern Pacific Ocean and the Mediterranean Sea were under-represented. Both regression slopes were small, showing that ca. 2 non-native species had been tested in field impact-experiments for every 100 known non-native marine species.

Virtually all the aquatic invasion examples presented in Charles Elton’s seminal book “The Ecology of Invasions by Animals and Plants” were based on anecdotal or observational data (Elton 1958), yet over the past half-century, intensive field experimentation evaluating the potential impact of non-native species have emerged. Today, invasion impacts have been quantified in field experiments for >100 aquatic non-native species published in at least 259 peer-reviewed papers. From this rapidly growing literature we identified taxonomic groups, regions and habitats that have been poorly studied and where additional experimental data are needed to improve our mechanistic understanding of how non-native species impact resident species, and to guide future research and management priorities. More specifically, we highlight that to provide general impact predictions, more experiments including more non-native species are needed from open coast sandy sediments, coral reefs, polar and tropical regions, the African continent, and on herbivores, as well as marine fish, crustaceans and cnidarians. Furthermore, invasion regulations and management standards are defined and set at the level of country, yet only 10 countries have tested for impacts of ≥5 different non-native species. Addressing these research gaps will allow researchers and managers to build stronger predictive models on how non-native species impact aquatic systems across taxa, habitats and biogeographical regions.

Attributes of individual studies. We found an exponential increase in experimental impact studies over the past few decades. If this trend continues, >500 experimental aquatic impact studies could be published in the next decade. It is vital that this (expensive) effort builds on previous research rather than repeats it; i.e., new experiments should be conducted in a context of the listed studies in the Suppl. material 1. For example, c. 5% of reviewed studies were spatially pseudo-replicated, a design that should be avoided whenever possible (although in some cases this may be impossible, especially for broad-scale studies) (Hargrove and Pickering 1992; Oksanen 2001; Wernberg et al. 2012). Still, this is a low percentage compared to other past ecological research fields (Hurlbert 1984; Wernberg et al. 2012), probably reflecting that experimental invasion biology is a relatively new and modern science (Fig. 1a) and that many non-native species can be manipulated in situ in small plots allowing for true replication. We also found that 14% of reviewed studies did not report any measure of variability around reported average impacts; and many more studies reported incomplete data variability (Fig. 1b). Not reporting variability around mean impacts implies that the experiments cannot be included in certain meta-analyses, thereby reducing its value in synthetic frameworks (Gurevitch and Hedges 1999). Given that it is a standard protocol to replicate randomly allocated treatments and controls, it could also be considered standard to report mean impact and associated data variability (if not in a paper then in online appendix) to provide a comparative scientific baseline and to allow for future re-analysis within synthetic frameworks.

We also noted that it was common practice to pool non-significant treatments, to not report non-significant effects, or to report only a subset of quantified responses. If future studies continue to value significance over effect size, for example only reporting significant results (the extreme case being the unknown number of studies that are never published because there are no significant results), the focus will remain on “high-impact invaders” only, thereby limiting our ability to understand contextual differences between high and low invasion impact (Pyšek et al. 2008; Ricciardi and Kipp 2008; Thomsen et al. 2011). We acknowledge that publications should be concise and sacrifice information, but suggest that average values and data variability are reported for all treatments on all responses at least in online appendices. Implementing such practices will make research more transparent and simpler to evaluate, and make it possible to extract unbiased data to build strong synthetic models across studies, ecosystems and habitats (Fernandez-Duque and Valeggia 1993). Almost half of all studies reported impacts on community metrics (Fig. 1c), thereby addressing a previously highlighted research gap (Parker et al. 1999). However, only few studies reported, in the same paper, impacts on 5 community metrics (Hejda and Pyšek 2006; Albins 2013) or showed mean impacts and associated variation from all treatments on all identified taxa (Casas et al. 2004; Ross et al. 2007). Community-impact experiments with extensive data-reporting are important to facilitate a better understanding on how non-native species affect resident species because they, within a single environmental context, can quantify direct and indirect, strong and weak, and inter vs. intra-trophic effects.

Our literature list indicates that aquatic impact studies may have been overlooked in invasion biology. As stated in the introduction, reviews of plant invasion impacts have not included examples of impacts of seaweeds or seagrasses (cf. reviewed reference listings, Powell et al. 2011; Vilà et al. 2011; Pyšek et al. 2012), even though we reviewed 42 and 5 experimental impact studies on these two groups of marine invaders, respectively (Suppl. material 1). Similarly, an exhaustive analysis on research bias in invasion biology covering both aquatic and terrestrial systems, experimental and mensurative data, invasion impact and invasion success topics, and field and laboratory methodologies, may have underestimated aquatic invasions (Pyšek et al. 2008). For example, Pyšek et al. (2008) identified the first invasion paper from 1980 (Barrows), but we found, in our more constrained/specific review, three earlier examples (Hurlbert et al. 1972; Lubchenco 1978; Lubchenco and Menge 1978) and identified 2× and 1.3× more studies on invasions from non-native tunicates and Carcinua maenas, respectively. If aquatic studies continue to be under-represented in general invasion impact reviews, scientists may miss opportunities to build strong models and theories across ecosystems, and managers, conservationist and politicians who may manage linked terrestrial and aquatic systems may receive biased information about invasion impacts in their regulatory units.

Attributes of the non-native species. More than twice as many non-native animal species than plants have been tested in field impact experiments, most likely reflecting the fact that more non-native animals than plants have been found (Pyšek et al. 2008; Hewitt and Campbell 2010). However, non-native sessile organisms (plants + sessile animals) have been experimentally studied to the same degree as invasive mobile animals, despite the latter being a more species-rich group. Over-representation of sessile species is not surprising because they are classical ecological test organisms (e.g., Connell 1961), are important in invasion biology because they are transported around the world on ship hulls (Hewitt and Campbell 2010), are relatively simple to manipulate and provide good systems to test invasion theories (Stachowicz et al. 2002; Dunstan and Johnson 2004), and can be habitat-formers and ecosystem engineers with disproportionally large effects on community structures (Crooks 2002; Ward and Ricciardi 2007; Thomsen et al. 2010). We found four times more benthic species studied than pelagic, in part reflecting that there are more of them (they include diverse marine biofouling communities) (Hewitt and Campbell 2010), in part because pelagic organisms are more difficult to control and manipulate in field experiments. It is noteworthy, however, that non-native freshwater fish, many of which are predominantly pelagic, were well-studied, which highlights that mesocosms and caging have successfully been used to quantify impacts of these species.

Analysis across trophic levels revealed that non-native herbivores have been poorly studied. Studies are clearly needed on more aquatic herbivores to better understand, predict and manage their impacts on aquatic plant communities (Lubchenco 1978; Eastwood et al. 2007) and/or provide resource subsidies for higher trophic levels (Trussell et al. 2002). Most experiments on non-native animals were conducted with filter-feeders, for the same reasons listed above for sessile species (most sessile animals are filter-feeders). The plethora of experiments conducted with sessile filter-feeders also highlights a key difference between aquatic and terrestrial invasion ecology in that terrestrial systems have no ecologically equivalent organisms.

Only 2% of recently tallied marine non-native and cryptogenic species have been tested in field impact experiments, demonstrating that basic mechanistic insight on impact is lacking for most marine non-native species. This number varied with taxonomic groups from 0% (e.g. cnidarians, porifera) to 22% (angiosperms). We found few experimental impact studies conducted on non-native cnidarians, porifera, annelids, bryozoans, echinoderms, amphibians, mammals, or reptiles, reflecting in part, that there are few species in these groups (at least for marine organisms, Fig. 5, Hewitt and Campbell 2010). Molluscs, angiosperms and tunicates were over-represented (Fig. 5c) whereas marine fish, crustacean and cnidarians were under-represented. The former taxa are sessile or slow-mobile, easy to manipulate, and typically provide important ecosystem services by controlling ecosystem fluxes and patterns of biodiversity (Crooks 2002; Thomsen et al. 2010), whereas the latter group are more difficult to manipulate; for example, manipulating many non-native marine fish and cnidaria requires large mesocosms/cages, and many crustaceans are small and cryptic. More specifically, invasion impacts of marine fish have only been tested in two studies, both on the decorative territorial lionfish on tropical reefs, even though 166 marine fish invaders have been tallied (Hewitt and Campbell 2010). By contrast, impacts on freshwater fish have been tested on 19 out of the approximately 422 (5%) non-native species that occur globally. There are several possible reasons why there are more experimental studies on freshwater than marine invasive fish. First, based on current estimates, there appear to be more freshwater than marine non-native fish species (this needs to be verified in the peer-reviewed literature). Secondly, many non-native freshwater fish are important economically, for recreational purposes, have been intentionally introduced to control diseases and weeds or to establish a fishery (Hofkin et al. 1991; Hart and Pitcher 1995; Pipalova 2006), and occur in less open and more accessible habitats than most marine species. Finally, freshwater fishes are often better adapted to survive in small-scale enclosed mesocosms (i.e. are easier to conduct experiments with) and, due to their enclosed nature, freshwater systems are amendable to whole-lake and whole-stream experiments where invasion densities are manipulated in replicated experiment at the system level (Pope et al. 2009).

Attributes of the invaded system. Not surprisingly, we found strong geographical patterns; many more aquatic invasion impact studies have been done in North America than in other countries or continents. Analysing biogeographical patterns in more detail for marine non-native species revealed a similar pattern; studies from the NE Pacific and NW Atlantic were over-represented (Fig. 4d) compared to most other marine regions. Scientific over-representation from North America, Europe and Australasia, has been documented across ecological sciences, including research on climate changes (Wernberg et al. 2012), terrestrial ecosystems (Martin et al. 2012), conservation (Clark and May 2002; Lawler et al. 2006; Darwall et al. 2011) and invasions (Pyšek et al. 2008). This geographical pattern is likely associated with historical and present-day intensive economic-related activities, such as high transport of goods and people across international borders, and therefore high propagule pressure (Pyšek et al. 2010), that many universities/researchers are located there, and that better historical and taxonomic baseline data makes it easier to identify what is native, cryptogenic and introduced. Note however, that dramatic historical invasions are still being identified also from intensively studied areas (Thomsen et al. 2006; Blakeslee and Byers 2008).

We also noted that invasion impacts have been tested with few non-native species for the majority of countries, a typical spatial unit with a specific set of management rules and regulations for invasive species. Given that invasion impacts can depend strongly on the local and regional context (Strayer et al. 2006; Thomsen et al. 2011), this low level of ‘invasion impact replication’, even at broad country scales, makes it difficult to build predictive impact models for most regions around the world. Latitudinal patterns mirrored patterns observed between continents, oceans and countries with 66% of all studies being conducted within a relatively narrow mid-latitudinal band. By contrast, few studies were conducted at low or high latitudes, and no studies were conducted beyond 65° - possible reflecting lacking baseline information, few researchers, and logistically challenging environments (Martin et al. 2012) and/or a real pattern of relatively fewer non-native species in polar and tropical regions (in particular for marine systems, Ruiz et al. 1999). However, for freshwater non-native species we expected more field experiments to be conducted in the tropics, because mensurative data suggest relatively large impacts in subtropical and tropical ponds and lakes from intentional fish and crustacean introductions (Miller 1989; Fernando 1991; Ogutu-Ohwayo 1993; Hart and Pitcher 1995) - the textbook case study being how the Nile perch Lates niloticus has contributed to extinction of endemic cichlids fishes (Ogutu-Ohwayo 1993), but supported by little experimental evidence.

Research gaps associated with invaded habitat attributes were less pronounced compared to the geographical patterns, as also noted more generally for all types of invasion studies (Pyšek et al. 2008). Thus, invaded freshwater and marine ecosystems were both represented with more than 100 studies and most aquatic habitat has been included in field based impact studies (Fig. 4d). Still, relatively few experiments have been conducted in sandy open subtidal habitats and on coral reefs. Open sandy habitats have relatively few species in low densities (Defeo et al. 2009) and probably also have few invaders (Hewitt and Campbell 2010) but coral reefs may be significantly understudied. For example, >100 marine non-native species have been registered in Hawaii alone, with many on coral reefs (Coles and Eldgredge 2002), and many species have been intentionally introduced for aquaculture, some of which have spread to adjacent reefs (Russell 1983). Alternatively, there may be few outbreaks of non-native species on coral reefs compared to more common outbreaks of native species, including diseases, seaweeds, urchins and sea stars (Lessios 1995; McCook et al. 2001; Bruno et al. 2003). Finally, we found most non-native species to be studied in freshwater ponds, largely because of common use of outdoor mesocosms in freshwater studies (Suppl. material 1), where water, plants and/or animals have been added to tanks to create semi-natural pond test systems [indeed the first identified invasion impact experiment, published in Science four decades ago, was a pond-mesocosm experiment (Hurlbert et al. 1972)].

Over the past four decades, invasion impacts have been quantified in field experiments for more than 100 aquatic non-native species published in at least 259 peer-reviewed papers. However, despite this intensive research, our ability to make generalizations and predictions remain limited because impact depends on the specific context that links attributes of the non-native species and the invaded system. Regardless of the large experimental effort, our review revealed substantial gaps in the collective knowledge. Specifically, of the 101 test species, only one was a marine fish and only six were herbivores. Similarly, of the 259 papers, only 3% were from Africa, <2% from high latitudes and only 1% from coral reefs. We therefore recommend that future experiments target these less studied non-native species, regions and habitats. We also noted that it is standard to pool or not report non-significant treatments and responses, a procedure that limits synthetic advancement by biasing meta-analysis and by making it difficult to identify invaders and environmental conditions that result in weak impacts. We therefore also recommend to report non-significant and low impact data with associated data-variability (or for continuous design show scatter-plots with explicit identification of overlapping points) and to report non-pooled impact data for all treatments and responses (even if just in online appendixes). We suggest that our ability to extrapolate impact assessments across space, time and taxa will increase significantly if these research gaps are targeted. More generally, we argue that impact experiments should manipulate new and novel combinations of different non-native species, different invaded places, different resident organisms, different abiotic conditions and different resource levels, than has already been tested in past experiments (cf. Suppl. material 1). In conclusion, the last 40 years of research activity has provided an excellent starting point to understand invasion impact mechanistically, but we are still far from being able to build generalized and predictive models of invasion impacts on local management scales of most invaders.