Experimental and Epidemic Risks: Matters of Science and Judgement

Abstract

I wish to caution against undue weight being given to received wisdom and false assumptions of ‘expertise’, particularly in the context of new epidemics. I argue that, in some cases, reliance on received wisdom can lead to poor decision-making in public health terms, whether we are talking about Mad Cow Disease, the likelihood of a novel strain of influenza causing many fatalities, or the best way to reduce overdose deaths soon after release from prison. I shall also stress the importance of well-designed data-acquisition which is, or should be, the forte of statistician-scientists.

Mad Cow Disease

My first example concerns prion diseases. In the mid-1980s, after Bovine Spongiform Encephalopathy (BSE) was diagnosed in cattle, the UK introduced a series of measures to safeguard the feed and food chains from BSE contamination. Unfortunately, controls at abattoirs that were designed to safeguard human health were honoured in the breach so that spinal cord was imperfectly removed and there was brain contamination of head meat, which did not cease until the mid-1990s.

On 20 March 1996, the Daily Mirror anticipated the announcement in Parliament that afternoon about ten UK cases of a new variant of Creutzfeldt-Jakob disease (vCJD). (This was an example of inspired journalism: the paper had been alerted by the Government's commissioning of advertisements to promote British beef and put two and two together.)

After the announcement in Parliament of vCJD, in effect BSE in humans, and subsequent experimental evidence that BSE was transmissible in blood, we needed to know about the subclinical carriage of vCJD by birth-cohort, sex and genotype. Birth-cohort is important because of the young age of vCJD cases. Indeed, Cooper and Bird¹ showed that the occurrence of clinical vCJD in people born in 1970–1995 was disproportionately high relative to their BSE consumption. See later for why genotype matters.

The first major paper on the transmission dynamics of BSE in British cattle was published remarkably quickly, in August 1996, by Anderson et al. in Nature.² The study estimated that the mean BSE incubation time was 5 years. But other assumptions – first, no exposure of cattle to BSE contamination after August 1996 and, secondly, no survival detriment for BSE-infected but not affected cattle – proved incorrect.

The expert opinion of the Spongiform Encephalopathy Advisory Committee (SEAC) in 1996 was that the majority of dietary exposure to BSE had occurred prior to 1989 (when the Specified Bovine Offals regulations were implemented) and had been mainly due to mechanically recovered meat. But later studies disproved both opinions. In particular, Cooper and Bird¹ showed that, for each of three broad birth-cohorts, BSE dietary exposure was greater in 1990–1996 than it had been in 1980–1989, see Figure 1. It was also shown that the 1940–1969 birth-cohort had consumed more BSE-contaminated meat than those born in 1970–1995.

Figure 1 UK dietary exposure to BSE: infectivity from beef mechanically recovered meat and head meat¹.

From January 2001, the European Union instituted post-mortem surveillance for late-stage BSE in all apparently healthy adult cattle (aged 30+ months) that came to abattoirs for slaughter and also in all at-risk-animals (aged 24+ months), which included cattle that had died on the farm.⁹

Figure 2 shows that, even in 2002, the UK had twice as many clinical BSE cases (467) as in the other 14 EU member-states combined (206). However, across Europe, excluding the UK, there were at least as many surveillance-detected late-stage BSE cases at slaughter (273) as clinical cases (206) and 2.5 times as many were detected in risk-cattle (507).

Figure 2 European Union (EU) active BSE surveillance in 2002 by 15 member-states: BSE rate per 1 million tested (pmt).

Late-stage BSE was detected in risk-stock at broadly similar rates of 2710 and 2390 per million risk-cattle tested in UK and Ireland; and at a rate of 875 per million risk-cattle tested in Spain, where the late-stage BSE rate in risk-stock was 10 times that at normal slaughter (rather than closer to 15 times for all¹⁵ EU member-states excluding the UK: 500 versus 30 per million cattle tested).

Testing for subclinical vCJD in humans

There is, as yet, no blood test for vCJD – although vCJD is transmissible in blood.

We also know that genotype matters when it comes to prion diseases (for example: scrapie in sheep; kuru in humans; CJD and vCJD in humans). Genotype may affect the likelihood (or rate) of progression to clinical disease and so the prevalence of subclinical vCJD by genotype is of specific interest.

Both human appendix tissue and the spleen may test positive for abnormal prions some years before the onset of clinical vCJD. However, we do not have empirical evidence that abnormal prions are detectable in tonsils before clinical symptoms of vCJD have appeared.

Ethical approval was given for non-attributable testing of spleen tissue at forensic autopsy in the UK, provided that relatives had given their consent. Analysis by birth-cohort, sex and genotype was planned. However, the study did not proceed due to objections from coroners who were in dispute with the government about the funding and organization of coronial services more generally.

See www.hpa.org.uk/hpr/archives/2012/hpr3212.pdf for the results of the National Appendix Tissue Survey that were released in August 2012.

The Health Protection Agency had established a National Appendix Survey that aimed to test at least 30,000 appendices for the presence of abnormal prisons. Based on the interim results from the National Appendix Survey (above) together with the earlier results by Hilton et al.,³ we observe in Figure 3 that the centrally-estimated abnormal prior positive rate per million people tested is twice as high in the 1941–1960 birth-cohort as for those born during 1961–1985.

Figure 3 Human surveillance in appendices for abnormal prion: UK studies (VV denotes that codon 129 genotype was valine-valine).

The central prevalence estimates are consistent with the BSE-consumption results by Cooper and Bird¹ but the 95% confidence interval is particularly wide for the 1941–1960 birth-cohort so that further testing in this birth-cohort, in particular, will be needed for more precise estimation.

Elsewhere, I have argued that permission (for vCJD-informative testing at autopsy) should be sought in-life from those individuals whose vCJD-risk has been estimated at 1% or higher (for example, on account of multiple transfusions; blood donation to, or from, a vCJD case; or surgically).

Let me explain why I think post-mortem testing matters for those who are at higher risk of developing vCJD (see http://www.hpa.org.uk/hpr/archives/2012/hpr3212_cjd.pdf).

In Figure 4, I consider the plight of B and C, who are about the same age as A, and who each donated 1 unit of blood to A.

Figure 4 Why permission in life matters.

A later developed vCJD.

Either A's BSE exposure was dietary, or indirectly-dietary in the sense that either B's dietary risk was transferred to A by transfusion or C's was.

If the indirect-transfer-rate were 100%, then the probability that B caused A's vCJD would be:

B's dietary-risk/{A's dietary risk + B's dietary risk + C's dietary risk} = 1/3

Even if the indirect-transfer-rate by transfusion was only 10%, the chance that B is vCJD carrier would be:

0.1 × B's dietary risk/{A's dietary risk + 0.1 × [B's dietary risk+C's dietary risk]} = 1/12

Suppose now that C dies without having given prior permission for vCJD-informative tests at autopsy. Then B continues to have to take surgical precautions for the rest of his or her lifetime whereas, had C been a vCJD-carrier, B could have been released from these obligations.

Novel epidemics expose the limits of empiricism

We are familiar with the theme that absence of evidence is not the same as evidence of absence. Rather, absence of evidence is a cue for bespoke data-acquisition, which is well-designed.

Low counts can be informative. In the H1N1 pandemic, comparison of actual H1N1 deaths versus the assumptions made by the authorities when planning for epidemics like this showed very early in the pandemic that H1N1 might be a ‘wild thing’ but it was also a ‘mild thing’.

Press furore is not a good measure of risk. For example, press coverage of Ecstasy deaths in the 1990s diverted attention from Ecstasy's in-reality low lethality for its many hundreds of thousands of users.

Likewise, mephedrone was highlighted in the UK press in 2009/10. But we obtained persuasive circumstantial evidence from the British Army's compulsory drugs testing (CDT) regime that the number of soldiers taking cocaine and ecstasy fell dramatically from the last quarter in 2008. (Mephedrone was then legal, and only illegal drugs are tested for in CDTs.) The mephedrone epidemic had in all likelihood started in late 2008. Subsequently, we were able to demonstrate that the 30% fall in cocaine-related deaths in 2009 coincided with, and was commensurate with, the decrease in declared use of cocaine by respondents to the British Crime Survey, see Straight Statistics (search on ‘mephedrone’).

Evidence-base for Policy Change

I wish now to focus on the strengths (and limitations) of randomized controlled trials (RCTs) when it comes to providing an evidence-base for policy change.

Some policies, such as presumed consent for organ donation, have to be adopted (if at all) on a national basis. In other jurisdictions than medicine, such as criminal justice, there is little adherence to formal experimentation with, for example, court-based randomization. As a consequence, judges prescribe sentence on lesser evidence than doctors prescribe medicines.⁴

Arguments against the use of RCTs to determine public policy are as follows. The first, as above, is cultural (despite the principle of randomization being universal). Secondly, RCT-costs are said to be greater, not least by dint of having control subjects. And the perception may be that the policy-change is unlikely to harm so that experimentation is therefore unnecessary as policy-change then comes down to affordability.

Timeliness, or rather the lack of it, is another counter-argument: it will take at least three years to know if reconviction-rates in the first 2-years after sentencing have reduced by x%, but Ministers want to be seen to be making decisions before three years are up.

Fourth is the argument that ministers have a democratic mandate to determine policy (irrespective of other evidence) and the uncertainty that formal experimentation conveys undermines political authority.

Pressure-groups may argue against RCTs because, in the absence of robust evidence, opinions (including theirs) hold sway. In addition, practicalities may dissuade officials from venturing into RCTs, as it appears easier to roll-out a changed policy than to seek individuals’ informed consent to be managed by policy A versus B. Compared with the rigours of peer-review that precede the set-up of RCTs, political fiat is fast and funded.

Realistically, equipoise may be more phantom than factual – at least on the part of those who design RCTs. I was part of the Cambridge team who instigated the British Randomized Evaluation of ALEC THErapy (BREATHE). The Cambridge team's prior belief was that Artificial Lung Expanding Compound (ALEC) would reduce by one third the mortality of very premature babies (from 36% to 24%), whereas non-Cambridge paediatricians’ prior belief centred on a quarter reduction, which is what BREATHE was powered to detect.

Roy Calne, professor of surgery at Cambridge, once remarked to me that he had done the operation in humans and so it was now ethical to randomize in dogs. And Michael Rawlins, chairman of NICE, that he always ‘knew’ what he expected to find; the purpose of the RCT was to convince others, see Figure 5. And, indeed, RCTs protect participants lest the instigators’ ideas are less good than the ideas of those I've mentioned!

Figure 5 Randomized Controlled Trial (RCT) versus before/after policy change.

We cannot afford to subject all changes to formal experimentation and so it makes sense to take a risk-based approach – to benefits, harms, and treatment-costs – and to invest in formal experimentation when it is most justified. If prior belief in benefit is nearly overwhelming, the likely harm minimal, and the cost of making the change affordable or cost-saving, there is a sound case for decision-making on the basis of expert opinion rather than expenditure of time and resource on RCTs.

Which brings me to my last example, which concerns RCT or policy-change with before/after evaluation.

I declare my interest as MRC co-grantholder for the pilot phase of the N-ALIVE Trial of the effectiveness of naloxone-on-release for reducing (by 30%) opiate-overdose deaths soon after release for prisoners who ever injected heroin. The story behind the N-ALIVE Trial began in Edinburgh Prison.

Seaman et al.⁵ were the first to quantify an eight-fold higher risk of drugs-related death in the first fortnight after release from Edinburgh Prison in 1983–1994 for HIV-diagnosed male injectors.

Bird and Hutchinson⁶ studied nearly 19,500 male-releases and demonstrated that this high risk persisted for men aged 15–35 years when released from Scottish prisons in 1996–1999, see Figure 6. We estimated that, even in the late 1990s, 1 in 200 men who had ever injected heroin died within a fortnight of release from Scottish prisons.

Figure 6 Drugs-related deaths in fortnight after prison: 19,486 male ex-prisoners, aged 15–35 years, released after 14+ days’ incarceration.

We had corroborated the high-risk identified by Seaman et al. ⁵ and we had narrowed the uncertainty about the estimated relative risk of 7 (95% CI: 3 to 16) in the first 2-weeks compared with the per-fortnight's risk in the subsequent 10 weeks. We also proposed that, for prisoners with a history of heroin injection, there should be a prison-based RCT of the effectiveness of naloxone (opiate antagonist that is administered intramuscularly) for reducing drugs-related deaths soon after release.

In accordance with advice from the Advisory Council on the Misuse of Drugs in its report on Drugs-related Deaths,⁷ naloxone was added, in 2005, to the exempt list of Prescription Only Medicines for administration by anyone in an emergency to save life. This change in UK's licensing of naloxone made possible the RCT that we had proposed.

Parmar (Director of MRC Clinical Trials Unit), Bird and Strang (Director of National Addiction Centre) had obtained MRC funding for the pilot phase (the first 5600 randomizations) of the N-ALIVE Trial, which aimed to randomize, in total, 56,000 prisoners aged 18–44 years with a history of heroin injection who had been incarcerated for 7+ days. The design assumptions for the N-ALIVE Trial are as follows (see Figure 7):

Figure 7 Design assumptions: N-ALIVE.

1. (i) at 80% of opiate overdoses, someone else is present {evidence-based};

2. (ii) there is a 75% chance that the ex-prisoner carries his/her naloxone in the first four weeks post-release (but this chance drops to 50% in the next eight weeks) {plausible a priori assumption};

3. (iii) there is a 50% chance that the present other locates the naloxone and has the presence of mind to administer it intramuscularly (as instructed in prison or on YouTube when searching for N-ALIVE) {conservative prior belief}.

Together, assumptions (i) to (iii) suggest that efficacy of naloxone, which is near to 100%, reduces to an effectiveness of 80%×75%×50%=30% in the first 4-weeks post-release.

Finally, in the twenty-first century, we are assuming one opiate overdose death in the first 4-weeks per 200 eligible ever-injectors who are randomized to the control group in the N-ALIVE Trial, in which we expect drugs-related deaths (DRDs) as given in Table 9.

The expected drug-related deaths in N-ALIVE Trial are given in Figure 8.

Figure 8 Prison-based, with consent* RCT for 56,000 pre-release adult prisoners who ever injected heroin (IDUs = injection drug users).

Together with colleagues from the MRC Clinical Trials Unit, I spent a lot of time in jails in 2011 so that the N-ALIVE Trial's first randomizations took place at Nottingham Prison in May 2012 – already 14 years after Seaman et al. ⁵

In the meantime, there has been a 1-year evaluation of take-home naloxone (THN) in half of Wales by Bennett and Holloway,⁸ which reported in May 2011. The Welsh evaluation recorded more ‘saves’ than there are heroin-related deaths annually in half of Wales, which indicates that THN-administration is not synonymous with ‘saving a life that would otherwise have been lost to opiate-overdose’. The percentage of opiate overdoses that are fatal is rather poorly documented but may be as low as 5%.

Scotland – with a population of 5 million and over 500 drugs-related deaths per annum, three-quarters of them opiate-related – made a breakthrough in 2011 by becoming the first country to adopt THN as a public health policy.

Take-home Naloxone has been funded in Scotland for three years (2011–2013) for prescription by its drug action teams, doctors and in prisons: with reimbursement of 6000 THN-kits annually in the community and 5000 to be issued by prisons.

England has not adopted THN as a public-health policy; but England has the N-ALIVE Trial, which cannot now randomize in either Scotland or Wales due to their THN policies.

The delayed registration of inquest-deaths in England, see Figure 9, poses a problem for the N-ALIVE Trial because we may have to wait a year or more to know if a participant ex-prisoner has indeed survived throughout his/her first 12 weeks post-release.

Figure 9 Registration delay exceeds six months for half of all drugs-related deaths in England and Wales.

In Scotland, by law, all deaths must be registered within eight days of the death having been ascertained. The Royal Statistical Society is calling for similar legislation to apply in England and Wales.

In summary, assumptions and expert opinion about the BSE and vCJD epidemics were overturned by well-designed BSE surveillance in cattle, by synthesizing evidence on the UK's BSE consumption by birth-cohort and era, and by abnormal prion surveillance in human appendices. Low counts can be informative. For example, from the outset, H1N1 deaths were substantially below planning assumptions, and press furore is no substitute for proper insight on epidemics (as ecstasy and mephedrone attest). A number of arguments against randomized controlled trials for policy-evaluation include a naive belief that policy-change is unlikely to harm and the authority of a democratic mandate. This chapter of the story ends with juxtaposition of the N-ALIVE Trial of naloxone-on-release in England and before/after policy evaluation in Scotland where take-home naloxone has been introduced as a funded public health policy for 2011–2013.