Communicating research with the public : evaluation of an invasive earthworm education program

Ecologists are increasingly encouraged by funding agencies and professional societies to communicate their research with the public. However, most receive relatively little training in how to do this effectively. Furthermore, evaluation of whether such an investment by ecologists actually achieves conservation objectives is rare. We created an education program, involving print, television, radio, and internet media, to increase awareness about earthworm invasions and to discourage anglers from dumping earthworm bait. Using preand post-surveys, we evaluated our program’s success in reaching its target audience and in changing knowledge and behavior. Few participants (4.1%) recalled seeing the program material and knowledge of the fact that earthworms are non-native in Alberta remained low (15.8% before, 15.1% after). Further, after being told about the negative effects of earthworms in forests, 46.7% of the anglers surveyed stated they would not change their bait disposal behavior in the future, with many commenting that they did not believe earthworms could be harmful. These results highlight the importance of evaluating education programs, rather than assuming they are successful. Given many participants’ doubts that earthworms have negative effects, both regulations and education may be needed to reduce earthworm


Introduction
Anthropogenic activities are causing unprecedented changes to the environment worldwide, leading to calls for ecologists to devote a greater amount of their time to communicating research findings with the public and policy makers (Lubchenco 1998, Holdren 2008).Many ecologists now consider such communication to be an important part of their careers (Pace et al. 2010).Public communication, and even advocacy, is viewed by some as a social responsibility of scientists (Lovejoy 1989, Noss 2007, Nelson and Vucetich 2009).This sense of duty may be a particularly important source of motivation for senior researchers, while junior scientists appear to be driven by enjoyment and personal satisfaction as well (Martin-Sempere et al. 2008).Unfortunately, regardless of their motivation for participation in public outreach, many scientists' academic training provides them with little opportunity to learn effective methods for communicating with the public.Further, scientists often do not view public communication as a core aspect of their work and may believe it has neutral or negative impacts on promotion because of the time it takes away from research (Gascoigne andMetcalfe 1997, Martin-Sempere et al. 2008).
Professional organizations such as the American Association for the Advancement of Science and the International Council for Science include enhancing communication among scientists and the public and improving science education as key goals for their organizations (AAAS 2012, ICSU 2012).Funding agencies including the National Science Foundation in the United States, and the National Science and Engineering Research Council in Canada have also attempted to promote public communication activities by including criteria relating to outreach efforts in their grant evaluation processes (Holbrook 2005).These types of initiatives should encourage public communication and education to be viewed as an integral component of scientific research.However, the high rates of failure (40-50%) estimated for public education campaigns (Ostergaard 2002) suggest that evaluation of the success of public communication efforts should be a critical part of outreach activities carried out by ecologists.Researchers' assumptions about a program's success are not sufficient.
Ideally, evaluation should include both formative evaluation, which is used to improve the program while it is being designed or run, and summative evaluation, which is used to determine if the program has achieved its objectives once it is complete (Carleton-Hug and Hug 2010).Yet, assessment of the success of public outreach efforts, whether conducted by academic scientists or others, is often overlooked.In a review of articles published from 1993 to 2008, only 20 evaluations of environmental education programs were found in the three leading environmental education journals (Carleton-Hug and Hug 2010).Lack of funding or time, inexperience with survey design, and fear of negative consequences if the program was unsuccessful are key reasons evaluations are not included in programs (Bitgood 1996, Carleton-Hug andHug 2010).
Invasive species are often the focus of public education programs (e.g., Alexander and Lee 2010, Hickey 2010, Jordan et al. 2011, Sya et al. 2009) because humans can act as key vectors for invasive spread.Our research on earthworm spread in Alberta, Canada (Cameron et al. 2008, Cameron et al. 2007) led us to work with various agencies to develop a public education program about the effects of earthworms in Alberta.Such non-charismatic species may pose particular challenges for education and management, as their invasions may receive little public attention or initially proceed undetected.In part due to the limited availability of information on earthworm invasions, "vegetation change facilitated by earthworms in North American forests" was recently identified by leading scientists as one of the top 15 global conservation issues (Sutherland et al. 2011).In northern forests, earthworms can mix organic and mineral soil horizons (Alban and Berry 1994), increase leaching of nitrate (Costello and Lamberti 2008), and alter below-and above-ground plant and animal communities (Gundale 2002, Hale et al. 2006, Eisenhauer et al. 2007).
European earthworms were accidentally introduced into North America in dry ship ballast, on plants, and in soils brought from Europe, and intentionally in agricultural areas to improve soil conditions (Gates 1970, 1982, Hendrix and Bohlen 2002).Invasions of earthworms are now occurring in remote areas across North America (Gates 1970, Gundale et al. 2005, Cameron et al. 2007), including northern forests that were previously glaciated and have no native earthworms (Gates 1970, 1982, Hendrix and Bohlen 2002).Earthworm invasion into these forests is facilitated by human activities, with dispersal of smaller earthworm species occurring along roadways in transported soil or in vehicle tire treads (Dymond et al. 1997, Cameron et al. 2007).Larger earthworm species are mainly invading near lakes, where anglers release them after using them as bait (Cameron et al. 2007, Keller et al. 2007, Kilian et al. 2012).Management of invasive earthworms in northern forests has focused on trying to prevent their introduction since there is no known method of control once populations are established (Callaham et al. 2006).
Education programs about the ecological effects of earthworms have emphasized the need to stop people from discarding live earthworm bait in areas where earthworms may establish new populations, although most efforts are on a small scale (Keller et al. 2007).The largest program in the United States, the Great Lakes Worm Watch (developed by the Natural Resources Research Institute), has done this, in part, through the distribution of more than 1500 posters and fact sheets to bait shops, nature centers, and park visitor centers (Callaham et al. 2006).Callaham et al. (2006) describe public response to this educational campaign as favorable and state that the message to stop dumping bait has been well received.However, there does not appear to have been any formal published evaluation of the success of this program, or other earthworm-related programs, at informing the general public and altering behavior.If scientists are to effectively raise public awareness, more critical and objective evaluations of the success of public education projects are required.
We developed an education program and examined its effectiveness by testing whether changes occurred in knowledge or behavior after program implementation.Using pre-and post-surveys, we evaluated whether anglers had encountered program materials, their level of knowledge about earthworm invasions, and their attitudes towards changing their method of disposing of earthworm bait.An increase in awareness of the fact that earthworms are not historically found in Alberta and a decrease in the number of anglers discarding bait on land or in water would suggest that our program was effective.

Education program
Our education program, the "Alberta Worm Invasion Project", was designed to increase public awareness of earthworm invasions in forests.We began developing this program in 2009, after conducting several years of research which indicated humans are a key vector involved in the spread of earthworms in Alberta.The key messages of the program were that earthworms are invasive and can harm plants and animals in forests, and the public can help prevent earthworm introduction by not dumping bait.The program's overall goals were thus to increase knowledge about earthworm invasions but also to change the behavior of anglers who release live earthworm bait into forests and lakes and convince them instead to save their bait or dispose of it in the garbage.It included five types of media: posters placed in bait shops, a website, two radio interviews, two televised informational clips, and two magazine articles (Table 1).
Posters were sent to 250 bait stores throughout northern Alberta, Edmonton, and Calgary by a bait distributor in 2010 (Figure 1).The main species distributed commercially in Alberta is Lumbricus terrestris, which is native to Europe but invasive in North America (Gates 1972).Bait stores included large specialized shops as well as gas stations that sell bait and approximately 100 of the stores accepted the posters and displayed them.The posters included a link to our website (http://worms.biology.ualberta.ca),which was also developed in 2010.This site can also be found if the phrase "worm invasion" or "Alberta worms" is searched in Google.On the website, five pages provide detailed information on earthworm spread, impacts on forests, ecological groups, how to avoid introduction, and an overview of our current research.
Let's Go Outdoors Radio aired two interviews about the spread of earthworms and also created two television clips in partnership with the Alberta Conservation Association (ACA).In the spring of 2010 and 2011, the radio interviews ran during a weekend  show in 16 communities across Alberta with an expected audience of 100,000 in each case.One television clip was played in 2010 and the other in 2011.Each was played 48 times over the course of two weeks in early summer, and was estimated to have an audience of 600,000 viewers.A magazine article was published in the ACA's Conservation Magazine (Cameron 2010) and in the University of Alberta's alumni magazine New Trail (Habib 2011).These magazines have readerships of approximately 40,000 and 140,000, respectively.Conservation Magazine is targeted to anglers and hunters, while New Trail reaches a larger number of Albertans but is not targeted to anglers.All interviews, television clips, and articles included the information that earthworms are not native to Alberta's boreal forest and can be spread by anglers dumping bait, as well as some discussion of the effects of earthworms in forests.

Program evaluation
In-person oral pre-and post-interviews were used to evaluate our invasive earthworm education program.All individuals surveyed were a minimum of eighteen years old.
The pre-surveys were carried out in the summer of 2009, with the ACA conducting 213 surveys and Alberta Sustainable Resource Development (ASRD) conducting 2018 surveys as part of their larger creel surveys of anglers.The post-surveys took place during the summers of 2011 and 2012.The ACA conducted 15 surveys and ASRD conducted 150 surveys in 2011, while we carried out a further 245 surveys in 2012.All surveys were performed at lakes across Alberta where participants were engaged in fishing activities.ACA conducted surveys in northern Alberta and ASRD conducted surveys in south-central Alberta, while our surveys in 2012 were carried out at previously sampled lakes across Alberta (Figure 2).In addition to these before-after surveys, we carried out a further 346 surveys in March 2011 at the Edmonton Boat and Sportsmen Show to obtain additional information on program effectiveness.Because the anglers at the show may have represented a different population of anglers and these surveys were conducted prior to the airing of the second television and radio clips, this data was not compared directly to the 2009 surveys but instead was examined qualitatively.
The 2009 pre-survey contained three questions (Table 2), which addressed participants' use of earthworm bait, how they dispose of bait, and their awareness that earthworms are not native to Alberta's forests.The surveys in 2011/12 included the same three questions and one to three additional questions (Table 2).The additional questions examined whether participants had seen material from our education program and whether they had changed their bait disposal behavior or would be willing to change their behavior in the future.If the participant was not using earthworm bait, only the questions examining if they knew earthworms were historically found in Alberta and if they had seen any information from our program were asked (questions 1 and 4 in Table 2).If they did use earthworm bait and had seen the information, they were also asked if the information caused them to change their bait use (question 6 in Table 2).If they had not seen the information but used earthworm bait, they were asked if they would change their bait use (question 7 in Table 1).In the 2012 surveys and the Edmonton Boat and Sportsmen Show surveys, additional time was available as our questions were not part of a lengthy creel survey.Therefore, we asked anglers if they would change their bait use if they ever use bait, rather than asking question 7 only to people using bait that day.

Data analysis
Chi-squared tests were used to examine whether bait use, bait disposal, and angler knowledge changed after implementation of the program.In all analyses, the independent variable was whether the survey was conducted before or after the program (i.e., 2009 vs. 2011/12).We tested whether use of earthworm bait (the proportion of anglers who used earthworm bait) changed after the program was implemented.We also examined whether the location where bait was acquired (dependent variable = proportion bringing bait from home versus digging it up at the lake) and bait disposal (dependent variable = proportion who saved bait/discarded it in the trash versus discarded it on land/water) changed after the program.Changes in awareness of earthworm invasions after the program was implemented were also examined.In this analysis, the dependent variable was the proportion of anglers who knew earthworms were not native to Alberta.Analyses were performed in Stata version 11 and were considered statistically significant at P < 0.05.

Before-after surveys
A similar proportion of anglers reported using earthworm bait on the day they were surveyed in 2009 (9.2% SE 0.61) and 2011/12 (7.6% SE 1.3) according to a chisquared test (χ 2 = 1.13,P = 0.29).In both years, most people brought their bait from home rather than digging it up at their fishing location.However, the proportion of people bringing bait from home was significantly higher in 2009 at 99% (SE 0.69) than in 2011/12 at 83.9% (SE 6.7) (χ 2 = 21.48,P < 0.001).
We used a chi-squared test to compare methods of disposing of earthworms that could contribute to their spread (releasing them in the lake or on land) versus disposal of earthworms that could prevent anglers from contributing to their spread (disposing of them in the trash or saving them for the next trip).There was a significant difference between 2009 and 2011/12 (χ 2 = 6.21,P = 0.013), with 39.2% (SE 3.4) of participants in 2009 and 62.5% (SE 8.7) in 2011/12 disposing of earthworm bait in lakes or on land (Figure 3).
Before implementation of our education program, 15.8% (SE 0.77) of anglers were aware that earthworms were not historically found in Alberta, compared to 15.1% (SE 1.8) of anglers in 2011/12.This difference was not significant (χ 2 = 0.13, P = 0.72).

Qualitative results (boat show and 2011/12 lake surveys)
When the 2011/12 survey respondents (including both those at the boat show and lakes; n = 756) were asked if they had seen any information about earthworm invasions in Alberta during the past year, only 31 people, or 4.1%, indicated that they had.Most of these participants saw an article on earthworm invasions (35.5%) or a TV clip (22.6%).The remainder received information from the website (16.1%), a poster (12.9%), another person (6.5%), the radio (3.2% -1 person), or could not remember where they saw the information (1 person).
Only eight of the participants who had seen information from our program were users of earthworm bait.Of these eight, only three indicated they had changed their behavior by reducing their use of earthworm bait or putting leftover bait in the garbage.A total of 276 participants used earthworm bait at least occasionally and had not seen the education program material.When they were asked if they would change their use of earthworms as bait knowing that they are invasive and can negatively affect plants and animals, 46.7% responded that they would not.

Program's level of success
Despite using a variety of forms of media, including print, television, radio, and internet, our program appeared to reach only a small number of anglers and to have a limited effect on knowledge and behavior.Only 4.1% of the surveyed anglers reported seeing information from our program.This low proportion suggests that our program might have benefitted from a more detailed preliminary examination of whether the media employed were likely to successfully reach our target audience.No increase in awareness of earthworm invasions was observed, with 15.8% and 15.1% of respondents before and after our program, respectively, stating correctly that earthworms were not historically found in Alberta.This level of awareness is similar to that encountered in surveys in New York, where 17% of people were aware that earthworms were exotic invasive species (Seidl and Klepeis 2011).Overall, our results emphasize the importance of conducting evaluations to assess the effectiveness of public awareness programs, rather than assuming programs have been successful based on the amount of program literature distributed or the expected audience size for various media.
It has been suggested scientists should devote one tenth of their professional time to outreach efforts and other activities intended to increase the societal benefits of science and technology research (Holdren 2008).We estimate that our program took approximately 140 hours for one of us to initiate over the course of three years, which is considerably less than this suggested amount but still represents a substantial time commitment.Given the limited change in awareness, it could be questioned whether these types of activities are a worthwhile investment for ecologists.However, it is likely that the amount of time and money (~$5000 which was spent mainly on website design, poster design, and poster printing, as we were not charged for television and radio clips) required to set up this program was less than would have been required by a conservation or government group.Although we lacked experience with public education programs, we were already familiar with research on earthworm invasions and had resources (e.g., pictures used on the website and magazine articles) available from previous work.Rather than viewing the program's limited success as an indication that academic scientists should not attempt this type of work, it could instead be seen as providing evidence of the importance of evaluating programs while carrying them out.Conservation education programs which include some type of formative evaluation are more likely to be successful (Jacobson and McDuff 1997;Norris and Jacobson 1998).

Effectiveness of media types
Some types of media were more effective than others at reaching our target audience.The articles (35.5%) and television clips (22.6%) were the forms of information most often encountered.Use of mass media such as television and radio has been identified as a factor contributing to the success of conservation programs (Jacobson and McDuff 1997).Fewer participants reported seeing our poster or website and only one mentioned the radio interview.Previous research on bait use recommended that bait stores would be an ideal location for placing signs or other sources of information about earthworm invasions (Keller et al. 2007).Our survey at the Edmonton Boat and Sportsmen Show also confirmed that many anglers do purchase their bait and thus targeting bait stores with program materials does seem to be a reasonable approach.However, signs had similarly limited effects on behavior in a study on bear-proof garbage containers (Baruch-Mordo et al. 2011).Although we consulted with other biologists and environmental program coordinators when designing our program, a more interdisciplinary approach involving greater initial consultation with other disciplines (e.g., psychology, education, business) might have improved our program's success but would also have required a substantially greater investment of our time.
Several alternative methods of delivering program material, which we have been developing through consultation with researchers in sociology and education, may be more effective than the strategies used in our program thus far.First, warning labels could be placed on bait container lids to reach anglers in bait stores.Because anglers should see these labels whenever they use their bait, stickers are less likely to be overlooked than posters on bait fridges.We conducted a small pilot study to test this idea, in which we placed stickers on 5000 containers in 2012.Because we wanted to understand whether bait disposal behavior changed when stickers were present, we enlisted the assistance of store cashiers to collect contact information of bait purchasers.We then surveyed these anglers two weeks after their purchase and 75% stated they saved their bait or put it in the trash, compared to 61% and 37% in our before-after surveys.Unfortunately, the response rate was extremely low with only 12 people responding to the survey, although a greater number left their email addresses.Nonetheless, these results suggest this strategy deserves further examination.Secondly, involvement in citizen science programs can lead to increased knowledge among the public (Jordan et al. 2011).Thus, we have begun collaborating with researchers in the Faculty of Education at the University of Alberta to develop a smartphone application that allows students and the public to participate in data collection on earthworm distributions in Alberta.Thirdly, two participants in our surveys indicated they heard about our program from other anglers, rather than by directly encountering program material.Research on behavioral choices indicates that knowing others are behaving in a particular way can strongly encourage people to conform to the same behavior, regardless of their own level of environmental awareness (Clayton and Myers 2009;Michel-Guillou and Moser 2006).Consequently, encouraging anglers who are active in anglers' associations or work in fishing stores to pass information on to other anglers might lead to greater changes in behavior.

Behavioral changes
There was no decrease in bait abandonment, with the proportion of anglers disposing of bait on land or water increasing from 39% in 2009 to 63% in 2011/12.This was driven largely by an increase in bait disposal on land, but it is not clear why such an increase would have occurred.Only three out of the eight people who used earthworm bait and also saw our program material stated that they had changed their approach to disposing of bait as a result.Furthermore, many people (46.7%) who did not see our material indicated they would not change their earthworm bait use/disposal after being told that earthworms were not native to the area and could harm plants and animals.Although we did not ask participants for comments on why they would or would not change their bait disposal, 30% of the people at the Edmonton Boat and Sportsmen Show who stated that they would not change commented that they did not believe earthworms could be a problem.Similar to this, almost 85% of residents surveyed in a study in New York moderately or strongly agreed that earthworms have a positive impact on plants (Seidl and Klepeis 2011).Many adults have likely heard or been taught about the benefits of earthworms for soil in gardens and agricultural systems.The reputation of earthworms as beneficial therefore appears to present an additional challenge for management efforts.On the other hand, even when people are aware that species are invasive, they may be unwilling to change their behavior.In a recent survey on attitudes towards invasive species, almost 30% of individuals were willing to introduce non-native species to an area if they would personally benefit (Garcia-Llorente et al. 2008).

Conclusions
The limited impact of our invasive earthworm education program highlights the importance of evaluating conservation programs.Formative evaluation is particularly essential as it allows for the improvement of programs while they are being carried out.A more interdisciplinary approach to program design may also lead to greater program success.Our survey confirmed that anglers are a source of earthworm introduction in Alberta, and therefore efforts to target anglers are needed if a reduction in spread of earthworms is desired.Increased access to artificial lures or proper disposal methods, such as labeled trash cans at boat launches, could make it easier for anglers to behave responsibly (Seidl and Klepeis 2011).However, evaluations of other education campaigns have concluded that regulations or regulations combined with education are more effective than education alone at bringing about behavioral changes (Baruch-Mordo et al. 2011).The non-charismatic nature of earthworms and the disbelief many respondents expressed upon being told earthworms could be harmful in forests suggest that regulations restricting bait dumping or bait sales, as have been implemented in other jurisdictions (Callaham et al. 2006, Kilian et al. 2012), are likely needed for a significant reduction to occur in earthworm introductions.

Figure 1 .
Figure 1.Poster distributed to bait stores as part of our earthworm education program.

Figure 2 .
Figure 2. Locations of surveys across Alberta, with • representing lakes surveyed in 2009 and 2012, ○ = lakes surveyed in 2011, □ = Edmonton Boat and Sportsman Show in 2011, and ▲ = lakes surveyed in 2011 and 2012.

Table 1 .
Educational program media and estimated audience sizes.Percentages in brackets are the audience sizes expressed as a proportion of the total population of Alberta.