Invasive species require adequate reliable detection methods to mitigate their further spread and impact. However, the reliability of molecular detection methods is often hampered by both false positives (Error type I) and false negatives (Error type II). At present, the reliability of the four published molecular detection methods for small hive beetles (SHB), Aethina tumida, has not been rigorously evaluated considering their extensive genetic diversity. Here, we performed intra- and interlaboratory comparisons of the four available methods using SHB samples representing 78 regions from 27 countries on five continents, beetles from the same genus (Aethina concolor, A. inconspicua, A. flavicollis and A. major), as well as western honey bees, Apis mellifera, and ectoparasitic mites Varroa destructor. The data show that the Idrissou et al. (2018) and Li et al. (2018) methods avoid both false positives and false negatives probably due to lower sensitivity to nucleotide mismatches on the primer and probe’s target sequences. Further, the Li et al. (2018) method can be considered more sensitive because the fluorescent amplification curve crosses the threshold at lower Cq values compared to the Idrissou et al. (2018) one. In light of our data, the Li et al. (2018) method is the most reliable molecular diagnostic tool for SHB. We therefore recommend using this method as it will contribute to management efforts of this invasive species.

Aethina tumida, inter-laboratory comparison, qPCR, ring test

The small hive beetle (SHB), Aethina tumida, is a parasite and scavenger of honey bee colonies that is continuing to invade the world since it was first noticed outside its natural distribution, in Africa, south of the Sahara, in 1996 in the USA (Hood 2000). As infestation of honey bee colonies with A. tumida did cause severe damage to apiculture in all the new areas where A. tumida has been introduced to (Neumann and Elzen 2004; Ellis and Hepburn 2006), it has been added to the lists of notifiable diseases of the World Organization for Animal Health (WOAH) and the European Union (EU). However, despite comprehensive elimination and contingency efforts, it already has established local populations on every continent except Antarctica and it is likely to continue spreading (Neumann et al. 2016; Schäfer et al. 2019).

A reliable method for the early detection of SHB specimens in places where they are not endemic provides the opportunity to have a cost-effective management of the situation which will look to prevent the initial establishment of SHB, and therefore, minimize the ecological and economical effects of this invasive species. However, despite this obvious advantage, the reliability of the different DNA-based detection methods for SHB is unknown. The molecular methods based on the detection of SHB’s DNA have the advantage of identifying not just the adults but the insect’s early developmental stages as well. Indeed, the taxonomical identification of early stages as eggs is a difficult task if it is based only on the morphology. However, a potential limitation of most molecular methods for the detection of SHB is that they were designed with limited information of SHB’s DNA variability. Actually, the primers (and probes) from most methods were designed with DNA information from specimens collected in introduced areas and few specimens from the African continent where the beetle is widely distributed and where the higher genetic diversity is expected. Therefore, it is quite important to test the reliability of the molecular methods using a much larger number of SHB specimens from their continent of origin (Idrissou et al. 2019).

Over the last 15 years several PCR methods had been developed for the detection of SHB. This study compares four genetic detection methods to evaluate their effectiveness, sensitivity and specificity, identifying the strengths and limitations of each method, aiming to identify the most accurate one. Three of those diagnostic methods were designed for using the hydrolysis probe technique (Ward et al. 2007; Li et al. 2018; Silacci et al. 2018). Those assays include a sequence-specific, oligonucleotide probe labelled with a fluorescent reporter and a quencher of fluorescence at opposite ends, in addition to the sequence-specific PCR primers. The hydrolysis method exploits the 5’ to 3’ exonuclease activity of the Taq polymerase. At the PCR extension step, once the polymerase reaches the probe, its exonuclease activity degrades the probe cleaving off the fluorescent reporter. As a result, it is separated from the quencher, resulting in a fluorescence signal. Probe-based qPCR enables the amplification of more than one target in a single reaction using different reporters with distinct fluorescent spectra. As this technique uses specific primers and probes to the target sequences, it is regarded as a technique with very high specificity. Besides the methods using the hydrolysis probe technique, also a detection method designed for conventional PCR (Idrissou et al. 2018) was evaluated. However, for this study the method was modified to test if those primers were suitable for the SYBR Green qPCR method. The qPCR non-specific detection method uses SYBR Green as fluorescent dye. This dye emits fluorescence when binds to double stranded DNA (dsDNA). Therefore, the fluorescence intensity is proportional with the concentration of dsDNA. It is considered a non-specific method because the dye binds to dsDNA, independent of the nucleotide sequence. The absent of specific-sequence fluorochrome-labelled probe make its use less expensive. However, the specificity relied entirely on the design of the primers to avoid the risk of nonspecific PCR amplifications. Commonly, this is verified assessing the melting temperature (Tm) of the amplicon by melting curve analyses that take place after the qPCR runs.

There are two error types that are of special importance to evaluate, error types I and II. Error type I, also known as a false positive, occurs when a method incorrectly identifies the presence of SHBs when they are absent. Error type II, also known as a false negative, occurs when a method fails to detect the presence of SHBs when they are actually present. For the evaluation of those parameters of the SHB qPCR detection methods, it was particularly necessary to test them using an extended collection of SHB specimens. In this study, the collection has representative specimens from the three known SHB phylogenetically clades (Liu et al. 2021) and were sampled from 4 non-endemic continents and a significant contribution from Africa (44 out of 78 total regions, and 16 out of 27 total countries) since it is the continent of the species origin holding the major genetic diversity (Idrissou et al. 2019). This is important to highlight as most of that genetic information was not available when the tested detection methods were developed, which implies that some haplotypes may have not been considered during the design of these methods, which may have consequences in their accuracy.

Finally, for further validation of the detection method comparison, we performed a blind ring test and an inter-laboratory comparison test between laboratory partners dedicated to the detection of SHB. For the ring test, the participating laboratories blindly tested selected SHB haplotypes using their own routine methods. For the inter-laboratory comparison test all participating laboratories used the selected most accurate method to tests its reproducibility and sensitivity across these laboratories.

This study compared the effectiveness and specificity of the four DNA-based detection methods for SHB, A. tumida, published over the last 15 years. Our data clearly show that Li et al. (2018) and Idrissou et al. (2018) were the only methods that accurately detected all tested samples (N = 83) including 44 regions from Africa. This is the largest diversity of SHB specimens ever tested with those methods. The Idrissou et al. (2018) method was originally designed as an end point PCR method. However, this study proves that this method can be adapted for SYBR qPCR and can be used as an alternative more economic method because it does not require labelled probes compared to a qPCR hydrolysis method. From those two methods, the one published by Li et al. (2018) was slightly less variable regarding the detection thresholds (Cq values) and more importantly, having all specimens detected lower than Cq of 25, which is a clear indicator of the robustness of the method (Figs 1, 2).

No false negative results were observed with the Li et al. (2018) and the modified Idrissou et al. (2018) methods. The Li et al. (2018) method was able to confirm the identity of all SHB specimens even on individuals with 2 (Italy: Calabria; Ethiopia: Sidama; Congo: Lume) and 3 (Burkina Faso: Bobo Dioulasso and Tenkodoge) accumulated mismatches. Similarly, the Idrissou et al. (2018) method confirmed the identity of SHB specimens even on individuals with 3 (Uganda: Busiwu), 4 (Burkina Faso: Fada N’gourma) and 5 (Burkina Faso: Bobo Dioulasso) accumulated mismatches.

Regarding the sensitivity for each method, the comparison of the Cq values from the same samples across the methods provides hints for the robustness. A method can be considered more sensitive when the fluorescent amplification curve crosses the threshold at lower Cq values. Li et al. (2018) detection Cq values range from 13.26 to 24.07, while the modified Idrissou et al. (2018) Cq values range from 10.58 to 26.32 (Figs 1, 3, 4). However, for the Ward et al. (2007) and Silacci et al. (2018) methods, the sensitivity is much lower (Cq values above 30) compared to both Li et al. (2018) and modified Idrissou et al. (2018) methods (Figs 1, 3, 4). Moreover, for the samples with higher variability, which exceed the limits of agreement across the methods (Fig. 2), the Li et al. (2018) method showed to be the most reliable as it detects the SHB samples at lower Cq values.

Regarding the false negative results, they are intrinsically linked to the nucleotide mismatches between the sequences of the primers and probes against the target genome. The Ward et al. (2007) method was unable to detect a specimen from Burkina Faso (Bobo Dioulasso). This specimen accumulated 16 nucleotide mismatches in forward, reverse primers and probe all together (Suppl. material 1: fig. S2). Additionally, several specimens that were at the limit of detection showing high Cq values (Cq > 35) also accumulated several nucleotide mismatches (i.e. Burkina Faso: Tenkodoge (15 mutations), Burundi: Rusiga (8 mutations), Burkina Faso: Fada N’gourma and Congo: Lume (7 mutations); Fig. 3; Suppl. material 1: fig. S2). The Silacci et al. (2018) method seems to be more sensitive to mismatches. Depending of the mismatch nucleotide site, a single mutation in the probe is apparently able to produce a false negative i.e. Burundi (Rusiga). However, with mismatches at different nucleotide sites, the method was able to detect samples with 2 accumulated mismatches (i.e. Portugal and Philippines).

To validate those results, a blind ring test was conducted. Overall, the results matched what was previously observed when all methods were compared (“Comparison of detection methods” section). For example, in the blind ring test, the Wageningen University & Research (WUR) Biointeractions & Plant Health laboratory used the Li et al. (2018) method, with their own modifications in the probe (van Gent-Pelzer and Cornelissen 2021; Suppl. material 1: table S2), and did not produce any false positive or false negative results. In contrast, the results of the laboratories using the Ward et al. (2007) method were consistent with the reporting of the false negatives, which were observed for this particular method. The inter-laboratory comparison using the single chosen method (Li et al. 2018) showed a uniform consistence between all laboratories again without any false positive or false negative results. This result also shows that the method can be adapted to the operating differences in each laboratory (e.g., qPCR reagents, thermocycler types, operators).

The comparison of the various genetic detection methods allowed the evaluation of their strengths and limitations. The error types I and II are of particular importance. The evaluation showed that all methods performed with ideal accuracy regarding error type I, as no false positive was detected even when including several specimens from the genus Aethina. In contrast, there were differences in their performances regarding error type II as some SHB specimens were not detected by some methods. The Idrissou et al. (2018) and Li et al. (2018) methods have less sensitivity to nucleotide mismatches on the primer and probe’s target sequences. The methods designed more recently performed better as more genetic information, in terms of more SHB specimen sequences, was available for the COI gene. This allowed for the design of primers and probes in regions with lower polymorphism or when certain polymorphic sites could not be avoided, degenerate nucleotides (W:A/T; Y:C/T) were also used (Li et al. 2018).

The evaluation of the molecular detection methods for SHB, clearly showed that both the Idrissou et al. (2018) and Li et al. (2018) methods avoid both false positives and negatives even when testing across the endemic and introduced regions. However, in view of its higher sensitivity among the tested methods, we propose to recommend the Li et al. (2018) method for the identification of SHB. Global application of such reliable molecular diagnostic tools will contribute to management and control efforts of this mandatory disease and invasive species.

We wish to express our gratitude to the honey bee research association “COLOSS” (https://coloss.org), for providing opportunity for the conception of this project.