Development of a systematic mapping review protocol for the most recent evidence on ruminant infectious disease frequency and disease-associated mortality: Ethiopia as a case study

Abstract

Almost 19% of the GDP of Ethiopia results from livestock production. Ruminants, in particular, form the majority of the national herd and are a critical source of income for smallholder farmers. Infectious diseases have been identified as a major cause of reduced livestock productivity in low- and middle-income countries (LMICs); therefore, a sound and comprehensive understanding of the relevant evidence would be beneficial in order to enable decision making on disease control policies. However, livestock disease data from sub-Saharan Africa is variable and disparate, which poses a challenge for evidence synthesis. This paper describes a protocol for a systematic mapping review of the recent available evidence on ruminant disease prevalence and associated mortality in Ethiopia. Literature sources will be identified using database search strategies. The titles, abstracts and, subsequently, full texts will be screened for inclusion based on predefined eligibility criteria. Specific data will be extracted and a preliminary qualitative assessment of the evidence will be performed using predefined indicators. The planned systematic map will be the first to provide a large-scale overview of the available ruminant disease evidence in Ethiopia; the final output will be an interactive dashboard tool to inform critical stakeholders in policy and research.

Introduction

Rationale

Ethiopia has the largest livestock population in Africa and the fifth largest cattle population in the world (Central Statistical Agency, 2017a, 2017b). Ethiopia's livestock subsector accounts for around 19% of the national GDP, representing a key contribution to the country's economy (Adem, 2019). Ruminants account for 80% of the national herd and contribute significantly to poverty reduction, with small ruminants playing a key role in improving the income status of female smallholder farmers (Shapiro et al., 2017). However, current evidence suggests that ruminant production in Ethiopia exhibits constantly low productivity and profitability rates (Shapiro et al., 2017). For example, between the years 2005 and 2015, the average mortality for cattle amounted to 7%, whilst for small ruminants it was 20% (Central Statistical Agency, 2017a, 2017b). Infectious diseases are a recognized and highly ranked cause of compromised livestock productivity in the country. In 2011, the economic impact of disease-associated losses was estimated to be US$ 150 million (Berhanu, 2002). On these grounds, it is unsurprising that disease-associated livestock morbidity and mortality are currently a highly relevant topic for international policy and research.

Systematic evidence synthesis methodologies are well developed in the fields of medicine and social sciences, as can been seen from an increasing number of published reviews. These methodologies include a rigorous, objective, and transparent process to gather and collate evidence aiming to minimize biases (Campbell Collaboration, 2015; James et al., 2016). Systematic reviews are designed to answer specific questions and often combine data from multiple studies to produce a single quantitative result using meta-analyses. However, there are instances where the poor quality of primary studies or the breadth of information covered does not allow for a systematic review synthesis and therefore, alternative methodologies have to be used. Systematic mapping is an evidence synthesis methodology that was first developed in social sciences and has been gradually adopted by other fields such as healthcare and environmental sciences, in order to identify potential research gaps to guide further primary research or subsets of evidence suitable for systematic reviews. Systematic mapping reviews share the rigorous approach of a systematic review, by aiming to provide a broad overview of a research area but without involving an evidence synthesis component or meta-analysis (Dicks et al., 2014). It should be noted that in contrast to systematic reviews, evidence quality is optionally assessed in the context of a systematic map (Haddaway and Pullin, 2014). This methodology informs policy makers, researchers, and practitioners of the best available evidence; hence, evidence-based decision-making is promoted and a more effective allocation of resources and prioritization of interventions is thus enabled (Bates et al., 2007). A variety of terms, including evidence map, scoping review and systematic map, are currently widely used under the evidence mapping review methodology (Grant and Booth, 2009; Gough et al., 2019). A number of scoping reviews has been published in the field of veterinary science and healthcare (Pham et al., 2014; Wadell et al., 2016; Rose et al., 2017). However, there is currently a dearth of scoping reviews in the area of tropical veterinary health.

According to Alonso et al. (2016), 25–40% of the published literature on specific livestock diseases in sub-Saharan Africa was of poor quality and difficult to summarize due to the diversity in study designs. Specifically, a multi-stakeholder interest in Ethiopia's livestock subsector has generated a considerable number of research initiatives in recent years, resulting in an increasing volume of evidence, which is of variable quality. Improvements in animal health data visibility and quality have been identified as key actions to increase livestock productivity in LMICs. Therefore, the application of systematic map methodology could be a valuable policy tool for LMICs, as disparate evidence poses a challenge for evidence-based decision-making.

Considering the above, the aim of the present protocol is to document the methodology that will be followed for the reviewing and mapping of the recently available evidence on ruminant infectious disease frequency and disease-associated mortality in Ethiopia. The systematic map will be the first large-scale overview of ruminant infectious disease frequency and mortality in Ethiopia and will provide a much-needed insight into the main diseases that affect livestock productivity in this country. The protocol also details the features of an interactive dashboard tool, which will be the final output and will provide a user-friendly web interface to make the evidence accessible for interested stakeholders.

Objectives

The main objectives of the systematic mapping review are: (a) to offer a broad overview of the recent available evidence on ruminant infectious disease frequency and disease-associated mortality in Ethiopia and (b) to identify any current knowledge clusters and gaps in this field. This map aims to provide an evidence base to inform livestock policy or research decisions in the Ethiopian livestock subsector and to highlight any potential need for primary research in specific knowledge areas.

Methods

Considering the lack of guidelines or standards for systematic maps in the field of Veterinary Medicine, the present protocol has been designed and written in accordance with the respective guidelines proposed by the Collaboration for Environmental Evidence (CEE) (CEE, 2018) and the PRISMA-P statement (Moher et al., 2015). These guidelines were applied with slight modifications, in order to fit better in the veterinary context. It should be noted that the CEE Evidence Syntheses guidelines and standards have been developed based on established methodologies in health sciences and have been tested through practice (Higgins and Green, 2009).

Review research questions

The primary research question that formed the basis for the systematic map was:

• • What is the most recent available evidence on ruminant infectious disease frequency and disease-associated mortality in Ethiopia?

A set of secondary research questions were formulated following the Population, Outcome (PO) scheme, in order to guide the literature searches and the data extraction. All infectious diseases that are currently known to affect animal productivity, including those that cause mortality or fertility losses, and are known to be endemic or emerging in Ethiopia will be considered for the review. The formulated questions, in particular for the ruminant population owned by smallholder farmers (i.e. cattle, sheep, and goats) in Ethiopia, are:

• • What is the current state and distribution of evidence on naturally occurring, production-limiting, infectious diseases?

• • What is the incidence and/or prevalence of naturally occurring, production-limiting, infectious diseases?

• • What is the mortality associated with naturally occurring, production-limiting, infectious diseases?

The key elements for the research questions are presented in Table 1 in reference to the PO scheme.

Table 1. Key elements of the research questions as presented in the PO acronym

Disease selection

In order to follow a standardized approach for the literature searches, a disease reference list was produced based on a well-established textbook in the field of Tropical Veterinary Medicine (Seifert, 1996) and on expert opinions (ARP, TKT). The diseases that will be eventually included in the searches are presented in Table 2. East Coast Fever, Rift Valley fever, and Nairobi Sheep Disease were purposely excluded from the searches, as confirmed cases have not so far been reported in Ethiopia (Seyoum and Teshome, 2017). Abattoir data will also be sourced for specific diseases, which are diagnosed post mortem. All reported pathogen co-infections will be recorded separately.

Table 2. List of production-limiting, infectious diseases that affect ruminants in Ethiopia [Tropical Veterinary Medicine (Seifert HSH, Tropical Animal Health, Kluwer Academic Publishers, 1996) and experts' opinion (ARP, TKT)]

Eligibility criteria

Studies that report on the incidence and/or prevalence or mortality of the aforementioned diseases affecting Ethiopian ruminants (cattle, sheep or goats) will be considered for inclusion, following the PO structure of the research questions as discussed above. As the systematic map will focus on natural infections, only observational studies with random or non-random sampling methodology will be considered for inclusion. Thus, clinical trials and other types of experimental studies (i.e. all in vitro and in vivo studies, which do not present cases of natural infections), as well as case series or case-control studies will be excluded. Additional exclusion criteria will apply for preliminary or pilot studies, studies that report on aggregated livestock data, as well as studies that cover non-infectious diseases or other conditions (e.g. drought), which can potentially affect animal productivity or cause mortality. Studies that report on other causes of ruminant mortality other than infectious diseases will also be excluded. If there are more than one report of a single study's results, the preferred piece of evidence for inclusion will adhere to the following hierarchy: (1) peer-reviewed publication, (2) peer-reviewed conference abstract, (3) non-peer-reviewed publication, (4) report, and (5) non-peer-reviewed conference abstract. Finally, the systematic literature reviews and meta-analysis studies will be considered for inclusion, provided that the data from the original studies have been made available.

Similar inclusion criteria will apply to searches on grey literature (i.e. published or unpublished pieces of evidence not subject to a peer-review process). In order to assess the consistency of the interpretation of these eligibility criteria between reviewers, the application of these criteria will be piloted in a random 5% subset of studies. In the event of a discrepancy, discussion will be initiated until a satisfactory level of agreement is reached or alternatively, a third reviewer will be assigned to facilitate consensus.

Searches will include studies published from 1 January 2010 onwards, as only recent evidence is intended to be presented. Due to time and resource constraints, only papers written exclusively in the English language will be considered for inclusion in the systematic map; we acknowledge this as a limitation of the present approach, as pieces of evidence written in Amharic, the official working language in Ethiopia, will be omitted. However, although this is expected to affect a small subset of studies, it can be a point for further consideration in future iterations.

Preliminary scoping exercise

A preliminary scoping literature search was performed in September 2018 using Google Scholar, in order to gauge the volume of the available evidence, to identify common study designs, to test search terms and develop search strings, as well as to formulate the research questions. Two reviewers performed the scoping exercise using the predefined eligibility criteria and then the search terms were re-formatted and re-trialed until a consensus was achieved.

Information sources

A variety of electronic databases will be searched, in order to comprehensively cover the relevant published and grey literature. For published literature, the databases MEDLINE, Google Scholar (first 500), Scopus (Elsevier), CAB direct, and Web of Science (Thomson Reuters) will be searched, as these have been reported to cover the most of the veterinary literature (Grindlay et al., 2012). In order to address the geographical context of the systematic map, the database African Journals Online, which includes the abstracts from more than 200 African journals and links to the full text of more than 80 titles, will also be searched (Rosenberg, 2005). For reasons of comprehensiveness, forwards and backwards citation tracking will be performed for each full-text, research paper included, in order to identify additional publications that will not have been retrieved during the initial searches.

Concerning grey literature, searches will include the research output repositories hosted by specialist institutions. This includes the International Livestock Research Institute (https://www.ilri.org/), the Consultative Group for International Agricultural Research (CGIAR) (https://www.cgiar.org/), the Food and Agriculture Organization of the United Nations (FAO, http://www.fao.org), the United States Agency for International Development (USAID, https://www.usaid.gov/), and World Organization for Animal Health (OIE, http://www.oie.int/). The selection of these institutions was based on their long-standing association with the Ethiopian livestock subsector. Additionally, the electronic repository of the Addis Ababa University (http://etd.aau.edu.et/) will be searched, as well as the databases Networked Digital Library of Theses and Dissertations, GrayLit network and Google Scholar; the latter have previously been identified as comprehensive sources for grey literature (Haddaway et al., 2015).

Search strategy

Search strings in the form of (Population) AND Ethiopia AND (Outcome) (as termed in Table 1) have been developed according to the results of the preliminary scoping exercise and the suggestions made by specialist librarians within the University of Edinburgh (Supplement 1). The aforementioned search strings will be adapted for each database searched, in order to maximize the results retrieved.

Study records

Data management

All the references (titles, abstracts, and full-texts) identified through the searches will be managed with the online reference manager software Endnote (version X8.0.1 for Windows, Thomson Reuters, New York, USA). A separate EndNote file will initially be created for each database searched; all the files will then be merged into a single file and duplicates will automatically be removed. For each search conducted, the search term, number of hits, database, and search date will be recorded to enable a rerun of each search.

Study screening and selection process

The study screening process will be conducted in two stages, according to the CEE guidelines (CEE, 2018) and by applying the defined eligibility criteria. In the first stage, all titles and abstracts will be prospectively evaluated by two independent reviewers (TKT, LMD) in terms of their relevance to the research questions. Studies, where the title and abstracts meet the eligibility criteria, will be evaluated in their full length for inclusion. At the beginning of each stage, the agreement between the two reviewers will be assessed in a random 5% subset of papers with the use of the κ statistic and in order to ensure consistency and common understanding of the eligibility criteria. If κ values indicate less that substantial agreement between the reviewers (κ < 0.5), the discrepancies will be resolved in discussion until consensus is reached; if consensus is not reached, a third senior reviewer will be assigned to resolve the discrepancies.

Data extraction process and items

From each study considered for inclusion in the systematic map, two reviewers will independently extract selected quantitative and qualitative data (meta-data) into spreadsheet forms. Each spreadsheet will be populated with specific item categories and will represent a studied disease. The data extraction tool will include items such as the year of publication, the year that the study was conducted, the state, area, ecosystem and production system of the study, the study design the total sample size used, the studied ruminant species and their ages, the type of sample and the diagnostic method used, as well as the case definition and the statistical method used. In addition, data on herd prevalence/incidence, individual prevalence/incidence, and mortality will also be extracted. Although the reporting of results does not normally fall into the scope of a systematic map, this information will be used to explore the recorded ranges. A specific coding strategy will be followed for the above data items, in order to ensure the consistency in data extraction and to facilitate data harmonization (Table 3). Primary data made available from published systematic reviews and meta-analyses will be separately extracted and will subsequently be added to the main data. The data extraction tool will be first piloted between the reviewers in a random sample of 10 included studies, in order to ensure consistency in extraction and any discrepancies will be resolved either by consensus or a third senior reviewer.

Table 3. Data extraction tool with coding categories to be used in the systematic map.

Study quality assessment

As previously mentioned, quality assessment is not typically a part of a systematic mapping review; however, as study quality is an important criterion in stakeholder decision-making, a preliminary, semi-quantitative assessment will be carried out based on defined criteria. In systematic reviews, quality criteria checklists are generally applied according to the study design (e.g. separate checklists for randomized control trials, cohort studies, prevalence studies, etc.). For the purposes of this systematic mapping review, the quality indicators proposed by The Joanna Briggs Institute Prevalence Critical Appraisal Tool (Munn et al., 2014, Table 4), which are tailored to prevalent studies, were considered appropriate. These quality criteria address issues around sample size, statistical analysis, and reliable case diagnosis, and, because they are generic, they can be applied to studies with various scopes. Disease cases were defined as ‘confirmed’, ‘suspected’, or ‘inconclusive’, based on the diagnostic test used (Supplementary 2, World Organization for Animal Health, 2012). Slight topic-specific modifications were made where necessary, and a numerical scale will be introduced for the respective answer options at a later stage, in order to enhance the objectivity of the assessment. Studies will eventually be ranked on a three-point scale of ‘high’, ‘moderate’, or ‘low’ quality, based on the sums of score. The quality assessment form will first be piloted between the reviewers (TKT, LMD) in a random 5% subset of studies, in order to ensure the clarity and the consistency in assessment. Discrepancies will be resolved by consensus or a third senior reviewer. For the purposes of the systematic map, no articles will be excluded based on quality.

Table 4. Quality criteria to be employed for the assessment of the studies (modified from The Joanna Briggs Institute Prevalence Critical Appraisal Tool, The Adelaide University, Munn et al., 2014)

^a Manual of Diagnostic Tests and Vaccines for Terrestrial Animals, World Organization for Animal Health, 7th Edition, 2012.

Evidence mapping and presentation

The final output of the systematic map will be an interactive dashboard tool. Variables such as study sample size and quality score will be encoded into disease-specific bubble plots using the visualization platform, Tableau (Washington, USA); each bubble's size and color hue will reflect study sample size and quality score, respectively. Bubble plots have previously been used for the visualization of systematic maps, as they are flexible and offer a visual summary of the available evidence in a way that highlights knowledge gaps and gluts (Petersen et al., 2008). By hovering over each bubble, users can access descriptive data for each piece of evidence, such as reference title, year of publication, disease covered, and study aspect (prevalence/incidence or mortality). The presented results will be filtered according to the disease (co-infections will be presented as a separate category), the species and production system, the study location, and the diagnostic test used. For each piece of evidence, the quality score and prevalence/incidence or mortality will be explorable by an interactive dashboard. The interactive dashboard tool will be reviewed by a group of selected stakeholders in terms of its features and usability, and will be revised according to the received feedback.

Ultimately, the interactive dashboards created will be hosted on an open source web platform and will be updated regularly in order to accommodate both newly published studies and user feedback. Knowledge clusters as well as knowledge gaps will be identified, which can in turn inform future systematic reviews or research initiatives. In the present context, no further data analysis will be pursued. A peer-reviewed systematic map report will also be produced, with a descriptive narrative on the overall methodology, the evidence base and its quality, and the current knowledge gaps. By accessing the interactive dashboard tool, critical stakeholders, such as researchers, policy makers and industrial partners, will be able to understand the recent trends in the available evidence and to inform their decisions in an evidence-based manner.