Developing a HACCP-like system for improving animal health and welfare in organic egg production – based on an expert panel analysis

Abstract

In the process of developing a generic Hazard Analysis and Critical Control Point (HACCP)-like system for securing animal health and welfare in organic egg production, an expert panel analysis was used to perform the initial hazard analysis. Eighteen advisers and researchers in organic egg production were included in the expert panel. In a series of four questionnaires, the expert panel first scored 34 health and welfare problems seen in Danish organic egg production. Based on scorings of severity and occurrence, 10 problems were selected for further analysis. The experts subsequently suggested and scored risk factors for those problems and finally suggested control points, alarm values indicating the need for corrective actions in order to control the risk factors and monitoring frequencies of these. The 10 selected problems were hunger, thirst, piling, crop impaction, blackhead, pasteurellosis, bone fractures, cannibalism, predators and red mites. A total of 154 different risk factors were suggested for these problems. The 41 risk factors which rated highest in a combined scoring of importance and occurrence were selected for further analysis. There was a high degree of consensus between experts when scoring both problems and risk factors. The level of consensus, as defined by an interquartile range ⩽1, was 79% to 100% when scoring the health and welfare problems (scale 1–5) and 77% to 95% when scoring the risk factors (scale 1–4). On average, 5.8 control points were suggested for every risk factor. Alarm values were often not detailed enough to be of practical significance and further analysis is needed in order to define these. The experts were highly diverse in their suggested monitoring frequencies and establishment of monitoring schemes should be part of developing the farm specific systems. An expert panel analysis based on questionnaires was a useful tool during the first steps of developing a HACCP plan, conducting a hazard analysis and suggesting control points. However, care should be taken regarding the panel’s size and fields of expertise in order to assure that the panel fully covers the field of interest throughout the study. A further development of the hazard analysis into a farm-specific HACCP system would include assembling an on-farm HACCP team consisting of farmers and advisors for the practical application of control points, alarm values and monitoring schemes. The results from the fourth questionnaire would be useful as input for this work.

Introduction

Organic egg production is struggling with problems compromising the health and welfare of the animals, as indicated by high rates of mortality (Hermansen et al., 2004) and high levels of feather pecking (Bestman and Maurer, 2006). The organic consumers as well as the organic organisations emphasise the importance of good animal welfare (Harper and Makatouni, 2002; International Federation of Organic Agriculture Movements, 2006). The fact that some farms have serious problems while others seem to work fine suggests that management is an area which can be improved (Hegelund et al., 2006).

The Hazard Analysis and Critical Control Point (HACCP) system is a management tool developed to secure food safety. During the last decade, the HACCP concept has gained general acceptance in the food processing industry and is presently implemented by several authorities in among others like the EU, the USA, Canada and Australia (Ropkins and Beck, 2000). HACCP systems are based on prevention, by controlling the risk factors for any given (food) hazard. The HACCP system consists of seven principles. The first three principles include (1) defining the problem (the hazard) and defining and evaluating risk factors for the problem, (2) defining critical control points and (3) choosing alarm values for the critical control points. The succeeding principles are integration of the critical control points in a HACCP system, which includes (4) setting up schemes to monitor critical control points and establishing (5) corrective actions if alarm values are exceeded, (6) verification procedures to ensure that all procedures are performed according to plans and (7) documentation procedures for keeping records of all procedures (Food and Agriculture Organization, 2001). HACCP systems can be extensive and time consuming to produce (Baird et al., 2001), making construction of generic systems within specified production types desirable for many small-scale production facilities (Food Safety and Inspection Service (FSIS), 1997). Generic systems serve as guidelines for subsequent further development of a HACCP system tailored for a specific production facility (FSIS, 1997; Canadian Food Inspection Agency, 2005).

As a consequence of the wide application of HACCP systems, interest has arisen to adapt the HACCP system to the livestock production, also focusing on key issues other than securing food safety (Noordhuizen and Frankena, 1999). The advantage of using a system based on prevention rather than cure seems obvious when dealing with animal health and welfare. HACCP-like systems have recently been developed with the purpose of securing animal welfare and managing diseases, as for instance pre-slaughter stress in pigs (Von Borell and Schäffer, 2005), calf-rearing management (Boersema, 2006), quality assurance on dairy farms (Silva et al., 2006) and herd health management in organic pig production (Bonde and Sørensen, 2004). However, using the HACCP system to secure animal health and welfare poses different challenges than applying HACCP for food safety purposes. In the food industry, the risk factors (i.e. disease causing agents) are clearly defined physical entities (bacteria, metal pieces, etc.) which can be traced through the production chain. In contrast, risk factors for the development of welfare problems often comprise specific housing conditions or animal interactions which cannot be treated as yes/no entities. This leads to a central point of the HACCP development: defining and evaluating risk factors for the chosen problem. Often the lack of empirical knowledge of the importance of each risk factor reduces the possibility completely to control the problem within the frames of the farming system (Noordhuizen and Frankena, 1999). This is especially true for multicausal problems such as tail biting in pigs or cannibalism in poultry. Expert panel analyses can be used to analyse situations where empirical data are too scarce to perform traditional statistical analyses (Vose, 2000).

The purpose of this paper is to describe and discuss the development of a generic HACCP-like system, aimed at securing animal health and welfare in commercial organic egg production. The HACCP-like system is developed on the basis of an expert panel analysis.

Methods

Experts familiar with organic egg production systems were used to perform the first steps of an HACCP analysis, including defining the health and welfare problems, defining and evaluating risk factors for the problem, defining critical control points and choosing alarm values for the critical control points. The experts filled out a series of four questionnaires, where the design of each questionnaire was dependent on the answers of the previous questionnaire. The method has many similarities with a Delphi approach, which is frequently used in the human health-care sector (e.g. Keeney et al., 2001) but which has also been applied within animal husbandry and management, for a quantification of expert knowledge (Van der Fels-Klerx et al., 2000; Sørensen et al., 2002).

Although Delphi analyses are widely used, there are no standard definitions for consensus (Powell, 2003). In the present study, the level of consensus is measured using the interquartile range (IQR), with the following definitions: IQR = < 0.5: strong consensus, 0.5 < IQR < = 1: moderate consensus, IQR > 1: no consensus.

Recruitment

As a result of the expert panel, analysis is an aggregation of the experts’ assessment of a given problem, where the choice of experts becomes highly important. To ensure that experts had the same field of expertise and the same cultural and legislative background, experts were included on the basis of two criteria. (1) Experts should be researchers and/or advisors with experience in on-farm welfare in commercial free-range or organic egg production systems. (2) Experts should be from EU countries geographically close to Denmark. To avoid bias, only one expert could be included per research institute. The aim was to include 15–20 experts in the panel. All correspondence was performed via e-mail and the experts were anonymous to each other.

Six key persons in Denmark, Sweden, Germany, The Netherlands and The UK were selected. The key persons suggested candidates for the expert panel and, in addition, the contacted panel candidates were asked to suggest other relevant candidates.

Questionnaire sequence

As explained below, the process of the expert panel analysis was a repeated sequence of preparing a questionnaire (Q), sending it to the expert panel, collecting replies and preparing the next questionnaire based on these replies.

Questionnaire 1 (Q1)

A list aimed to include the health and welfare problems seen in Danish organic egg production was assembled. Initially, the list included the primary causes of death seen in Danish organic egg production and behavioural deprivations caused by inability to perform natural behaviours (Johansen, 2003). Subsequently, four Danish researchers with experience in practical management and two poultry advisors from the Danish Agricultural Advisory Service were asked to add suggestions and comment on the list. This resulted in a list of 34 health and welfare problems seen in Danish organic egg production (Appendix A).

In Q1, the experts were asked to score each of the 34 problems according to how severe is it for the animal’s welfare to be subjected to the problem (‘severity’) and how often an animal in the organic egg production is subjected to the problem (‘frequency’). Severity and frequency were scored on a five-point scale: 5 = very severe/very frequent, 4 = severe/frequent, 3 = moderate/regular, 2 = less severe/rare, 1 = not severe/very rare.

Questionnaire 2 (Q2)

The median values of severity and frequency were calculated for each problem and based on these scorings, the most important health and welfare problems were selected. The most severe (scoring 5) and most frequent (scoring 5) problems were selected first and then the most severe (scoring 5) and less frequent (from 4.5 to 1) problems were selected. To reduce the scope of the expert panel analysis, a maximum of 10 health and welfare problems were set to be the limit for the subsequent analysis.

In Q2, all experts received a spreadsheet with the selected health and welfare problems listed in the same chronological order and were asked to suggest risk factors for these problems.

Questionnaire 3 (Q3)

Based on the replies to Q2, lists of risk factors for each problem were generated. The lists were edited with the purpose of ensuring that all risk factors were equally detailed. Risk factors with a high degree of similarity were merged and risk factors not compatible with organic regulations were deleted, e.g. beak trimming is prohibited according to organic regulations, so ‘no beak trimming’ is not accepted as a risk factor for ‘cannibalism’.

To check for problems with response-fatigue when filling out the comprehensive spreadsheet, the number of suggested risk factor per problem was examined.

In Q3, experts were asked to score each risk factor with respect to how severe the risk factor is for developing the welfare problem (severity) and how frequent the risk factor occurs in the production (frequency). Severity and frequency were scored on a four-point scale: 4 = severe/frequent, 3 = moderate/regular, 2 = less severe/rare, 1 = not severe/very rare.

Questionnaire 4 (Q4)

After the panel had rated the risk factors, the median values of severity and frequency were calculated and risk factors were chosen based on their scorings. The aim was to include an average of 3–4 risk factors per problem, but imposing the criterion that all hazards have a minimum of one risk factor. Risk factors with the following scorings were selected: severity 4 and frequency 2–4; severity 3.5 and frequency 3–4; severity 3 and frequency 4.

In Q4, the experts were asked to list all possible control points in as much detail as possible for each risk factor and also suggest alarm values for each control point. Control points were defined as ‘elements of the production that needed to be monitored (controlled) in order to prevent the risk factor from appearing’, and alarm values as ‘the values that separate good conditions from bad conditions’. Examples were given to the experts in order to clarify the explanation. For example, the risk factor for intestinal parasites ‘insufficient cleaning of the outdoor area near the house, between flocks’ could have the following control point and alarm values: change of top soil between flock: change of top 10 cm of earth in 0–15 m distance from pop holes.

Results

Recruitment

Thirty-two experts were contacted, and 18 accepted participation in the expert panel. They were from Denmark (4), Germany (3), the UK (3), Sweden (2), The Netherlands (2), Austria (1), Norway (1), Finland (1) and Switzerland (1). One expert was included after preparation of Q2, and therefore only participated in Q2 to Q4. While in one case, two experts cooperated in answering; they returned only one set of questionnaires and were registered as only one expert in the panel.

Questionnaires were prepared and sent out in May (Q1), June (Q2), July (Q3) and September (Q4) 2005. As the expert panel analysis progressed, the response rate dropped from 17 in the first two questionnaires to 12 in the third and 13 in the final questionnaire. Reminders were sent to experts not complying with the deadlines and the number of reminders increased from 3 in Q1, to 5 in Q2, 7 in Q3 and reached 19 in Q4. In Q4, six experts responded without reminders, four responded after the first reminder, and three responded after two reminders.

Questionnaire 1

Ten problems were selected for further analysis: hunger, thirst, piling, crop impaction, blackhead, pasteurellosis, bone fractures, cannibalism, predators and red mites (Table 1). Seventeen problems scored 4 in severity and 2 to 3 in frequency, and the remaining seven problems scored 2 to 3 in severity and 2 to 5 in frequency.

Table 1 Scorings of severity and frequency of the 10 selected health and welfare problems

n = number of experts scoring the problem; median = median scoring; IQR = interquartile range.

Consensus was reached in all selected problems when scoring severity (20% strong and 80% moderate) and in eight of the selected problems when scoring frequency (20% strong and 60% moderate). Among the 24 problems that were excluded, there was consensus in 21 problems when scoring severity (12.5% strong and 75% moderate) and in 19 when scoring frequency (20.8% strong and 58.3% moderate).

Questionnaire 2

Q2 resulted in a total of 412 suggested risk factors including several doublets. After editing, the list of risk factors was reduced to 154 distributed on the following 10 health and welfare problems: predators (9), pasteurellosis (18), piling (13), hunger (20), blackhead (16), crop impaction (12), cannibalism (21), bone fractures (12), thirst (16), and red mites (17). On average, each risk factor was suggested 2.6 times. Sixty-nine of the 154 risk factors were suggested by one expert only (Figure 1).

Figure 1 Distribution of number of risk factors suggested by between one and 13 experts in the panel.

Apparently, there were no problems with response-fatigue as the health and welfare problems getting the fewest suggestions from the experts were numbers four and six in the spreadsheet, while the first and ninth got the most suggestions.

Questionnaire 3

The list of 154 risk factors was scored by the expert panel and 41 risk factors fell within the selection criteria (Table 2): predators (3), pasteurellosis (3), piling (3), hunger (4), blackhead (2), crop impaction (1), cannibalism (8), bone fractures (4), thirst (7) and red mites (6). Seven of the selected 41 risk factors were suggested by only one expert in Q2.

Table 2 Selected risk factors based on scorings in questionnaire 3

n = number of experts scoring the risk factor; median = median scorings; IQR = interquartile range of scorings.

There was consensus in 119 of the 154 risk factors when scoring severity (28.6% strong and 48.7% moderate) and in 137 when scoring frequency (34.4% strong and 54.5% moderate). Among the 41 selected risk factors, consensus was reached in 39 when scoring severity (39% strong and 56.1% moderate) and in 37 when scoring frequency (29.3% strong and 61% moderate).

Questionnaire 4

Thirteen experts returned the questionnaire and the average response rate for the 41 risk factors was 86.9 (Table 3). A total of 390 suggestions referred to control points, resulting in 99 different suggestions when eliminating doublets (within and between risk factors) and control points not practically applicable by the producers in general management procedures. On average, 5.8 different control points were suggested for every risk factor. Alarm values were suggested in 381 of the answers and, of these, 44.6% were specific enough to be practically applicable. The remaining alarm values were less precise, including terms such as ‘high’ or ‘good’. A monitoring frequency was suggested in 260 answers and 88.1% of these were detailed enough to be of practical significance, while the remaining answers included terms such as ‘frequent’ or ‘regular’.

Table 3 Response rate and number of specific answers from the experts in the panel, in questionnaire 4

A wide range of control points were suggested, relating to different aspects of the production including caretaker routines/training (15), rearing conditions (7), behaviour (14), production (4), system (27), pasture (6), diet (11), diseases (7) and cleaning procedures (8), as exemplified in Table 4. The large number of control points reflects that some risk factors can be monitored using different types of control points, including both animal-based indicators and system-based indicators. For example, the risk factor for thirst ‘animals too small to reach drinkers’ can be controlled by checking either drinker height (at level with bird’s back) or bird’s posture when drinking (stretching neck). In this case, the exact choice of control points should be selected to suit the management in the best possible way when developing the farm-specific systems. Also, some risk factors must be monitored using information from more than one control point, for instance, the risk factor for cannibalism ‘physiological stress at onset of lay’ would require monitoring of ‘bird weight’, ‘egg production’, ‘feed plan’ and ‘light programme’. Other risk factors were easier to monitor, resulting in uniform suggestions from the panel, e.g. the risk factor for predators: ‘insufficient closure of pop holes’, where all experts agreed on ‘check closure of pop holes every evening’.

Table 4 Examples of problems with associated risk factors and control points

In some cases, a considerable variation was seen in the suggested alarm values for the same control point. It was most pronounced in the suggested alarm values for control point ‘stocking density’ in order to avoid ‘piling’ which were 4, 5 and 6 hens/m². However, most differences were seen in monitoring frequencies, e.g. in the control point ‘checking the house for holes’ to avoid access of ‘predators’, suggested monitoring frequency included ‘daily, ‘weekly’, ‘every 6 months’ and ‘every year’.

Discussion

The health and welfare problems used as a starting point for the questionnaire series correspond quite well to the health problems seen in European organic egg production, as reviewed by Thamsborg et al. (2004) and Bestman (2004), and problems associated with performing natural behaviour as described by Waiblinger et al. (2004); however, some differences are noticeable. Some diseases included in the reviews were not included in Q1 and although the agreement between the two reviews is limited to only five diseases, one of those, coccidiosis, is not included in Q1. Coccidiosis was omitted because it is consistently controlled by vaccines and is predominately considered a problem in the rearing units (Johansen, 2003), while the present system is developed for egg production. In addition, feather pecking was excluded from the list although it is probably one of the most used indicators of welfare problems in organic egg production (e.g. Bestman and Maurer, 2006). This decision was taken because feather pecking, sometimes being a precursor of cannibalism (Kjaer, 1999; Thamsborg et al., 2004), would be controlled as part of preventing cannibalism. On the other hand, 12 health and welfare problems not mentioned as health or welfare problems in the reviews were included in the questionnaire. Three of these (hunger, thirst and crop impaction) were subsequently selected for further analysis. In conclusion, the risk of missing problems such as thirst, which is related to impaired system maintenance and therefore unlikely to gain attention in research papers, and the lack of empirical evidence on the severity of the problems, emphasises the value of an expert panel for scoring and thereby selecting problems for further analysis instead of basing the selection solely on literature reviews.

Analysis of the responses indicates that the expert panel might be undersized. Seventeen percent of the selected risk factors in Q3 were suggested by only one expert in Q2. This appears to be a relatively low degree of overlap between experts and a possible consequence is an increased risk of missing important risk factors. The problem is most obvious in the third and fourth questionnaire where the response rate dropped to 30%. A considerable drop in response rate is not unusual in studies comprising a series of questionnaires (Mullen, 2003). It is probably caused by the long duration of the analyses and the anonymity of the experts, as they consequently feel less motivated or obligated. Studies show that a very close follow-up on responses can improve the response rate (Keeney et al., 2006). Consequently, including more experts in the panel may have resulted in a more comprehensive analysis of the problem areas and compensated for missing expertise due to falling response rates.

Due to the considerable scope of the topic ‘health and welfare in organic egg production’, the experts need to cover many different subjects which increases the risk of lacking knowledge in certain areas. However, throughout the questionnaire there is generally a high level of consensus between the experts both regarding scorings of problems and scorings of risk factors. The high level of consensus indicates that the choice of experts meets the requirements set for the expert panel and sufficiently fulfils the first part of the analysis. The experts base their knowledge on comparable organic systems and they focus on the same risk factors for controlling health and welfare parameters in productions. Still, there is a tendency for the total percentage of consensus (including strong and moderate) to be lower when scoring severity in Q3 compared with Q1, indicating more disagreement when scoring risk factors; especially, when considering that changing the scale from Q1 to Q3 increases the chance of a higher level of consensus in Q3.

In the fourth questionnaire, the experts suggested many different control points. Some of this variation is caused by the multicausal nature of the health and welfare problems, resulting in a need for multiple control points when managing the problems. But also the experts’ preferences for system-based v. animal-based indicators would result in different control points. The experts were also diverse in their suggested alarm values, but whether this apparent diversity would lead to low consensus is not investigated in a rating questionnaire. The most obvious problem was the lack of specific alarm values to some of the control points. A general reluctance to set alarm values probably reflects the fact that many control points in the production have a continuous range of values, ranging from ‘perfect condition’ to ‘unacceptable’, and setting a specific value separating ‘good’ from ‘bad’ is very difficult. Consequently, the stated alarm values do not reflect a thorough evaluation of critical limits, emphasising the need for a separate analysis to achieve specific alarm values for the HACCP system. It is unlikely that simply rephrasing the questions would result in more specific answers, as all experts were able to supply specific alarm values to some of the control points. Still, the suggested control points and alarm values can be used as input for developing the farm-specific HACCP systems.

The combination of lack of practical management experience by the experts and the lack of information of the specific housing conditions probably made it difficult for the expert panel to agree on monitoring frequencies. Consequently, the specification of monitoring frequencies should be a part of developing the farm-specific systems.

In conclusion, using an expert analysis to develop a generic HACCP system has led to selection of important problems and risk factors. In addition, a number of possible control points with related alarm values and monitoring schemes were suggested. A further development of the hazard analysis into a farm-specific HACCP system would include assembling an on-farm HACCP team consisting of farmers and advisors for the practical application of control points, alarm values and monitoring schemes. Attention should be focused on the selected risk factors and the results from the fourth questionnaire can be used as input for selecting appropriate control points. The alarm values should ensure that the value targeted as the appropriate level of each risk factor is not underestimated, however, a separate analysis is necessary to define this level.