30 - The Challenge of ‘Evidence’

Summary

Governments and stakeholders have struggled to find a common ground on how to regulate research for different (‘proven’ or ‘unproven’) practices. Research on traditional, alternative and complementary medicines is often characterised as following weak research protocols and as producing evidence too poor to stand the test of systematic reviews, thus rendering individual case studies results insignificant. Although millions of people rely on traditional and alternative medicine for their primary care needs, the regulation of research into, and practice of, these therapies is governed by biomedical parameters. This chapter asks how, despite efforts to accommodate other forms of evidence, regulation of research concerning traditional and alternative medicines is ambiguous as to what sort of evidence – and therefore what sort of research – can be used by regulators when deciding how to deal with practices that are not based on biomedical epistemologies. Building on ideas from science and technology studies (STS), in this chapter we analyse different approaches to the regulation of traditional and non-conventional medicines adopted by national, regional and global governmental bodies and authorities, and we identify challenges to the inclusion of other modes of ‘evidence’ based on traditional and hybrid epistemologies.

30.1 Introduction

Governments and stakeholders have struggled to find a common ground on how to regulate research for different (‘proven’ or ‘unproven’) practices. Research on traditional, alternative and complementary medicines is often characterised as following weak research protocols and as producing evidence too poor to stand the test of systematic reviews, thus rendering individual case studies results insignificant. Although millions of people rely on traditional and alternative medicine for their primary care needs, the regulation of research into, and practice of, these therapies is governed by biomedical parameters. This chapter asks how, despite efforts to accommodate other forms of evidence, regulation of research concerning traditional and alternative medicines is ambiguous as to what sort of evidence – and therefore what sort of research – can be used by regulators when deciding how to deal with practices that are not based on biomedical epistemologies. Building on ideas from science and technology studies (STS), in this chapter we analyse different approaches to the regulation of traditional and non-conventional medicines adopted by national, regional and global governmental bodies and authorities, and we identify challenges to the inclusion of other modes of ‘evidence’ based on traditional and hybrid epistemologies.

30.2 Background

Non-conventional medicines are treatments that are not integrated to conventional medicine and are not necessarily delivered by a person with a degree in medical science. This may include complementary, alternative and traditional healers who may derive their knowledge from local or foreign knowledges, skills or practices.¹ For the World Health Organization (WHO), traditional medicine may be based on explicable or non-explicable theories, beliefs and experiences of different indigenous cultures.² That being said, traditional medicine is often included within the umbrella term of ‘non-conventional medicine’ in countries where biomedicine is the norm. However, this is often considered a misnomer insofar as traditional medicine may be the main source of healthcare in many countries, independent of its legitimate or illegitimate status. Given the high demand for traditional and non-conventional therapies, governments have sought to bring these therapies into the fold of regulation, yet, the processes involved to accomplish this task have been complicated by the tendency to rely on biomedicine’s standards of practice as a baseline. For example, the absence of and/or limited data produced by traditional and non-conventional medicine research and the unsatisfactory methodologies that do not stand the test of internationally recognised norms and standards for research involving human subjects have been cited as common barriers to the development of legislation and regulation of traditional and non-conventional medicine.³ In 2019, the WHO reported that 99 out of 133 countries considered the absence of research as one of the main challenges to regulating these fields.⁴ At the same time, governments have been reluctant to integrate traditional and non-conventional medicines as legitimate healthcare providers because their research is not based on the ‘gold standard’, namely multi-phase clinical trials.⁵ Without evidence produced through conventional research methodologies, it is argued that people are at risk of falling prey to charlatans who peddle magical cures – namely placebos without any concrete therapeutic value – or that money is wasted on therapies and products based on outdated or disparate bodies of knowledge rather than systematic clinical research.⁶ While governments have recognised to some extent the need to accommodate traditional and non-conventional medicines for a variety of reasons⁷ – including the protection of cultural rights, consumer rights, health rights, intellectual property and biodiversity⁸ – critics suggest that there is no reason why these modalities of medicine should be exempted from providing quality evidence.⁹

Picking up on some of these debates, this chapter charts the challenges arising from attempts to regulate issues relevant to research in the context of traditional and alternative medicine. From the outset, it explores what kinds of evidence and what kinds of research are accepted in the contemporary regulatory environment. It outlines some of the sticky points arising out of debates about research of traditional and non-conventional medicines, in particular, the role of placebo effects and evidence. Section 30.4 explores two examples of research regulation: WHO’s Guidelines for Methodologies on Research and Evaluation of Traditional Medicine and the European Directive on Traditional Herbal Medicine Products (THMPD). Both incorporate mixed methodologies into research protocols and allow the use of historical data as evidence of efficacy, thus recognising the specificity of traditional medicine and non-conventional medicine. However, we argue that these strategies may themselves become subordinated to the biomedical logics, calling into question the extent to which other epistemologies or processes are allowed to shape what is considered as acceptable evidence. Section 30.5 focuses on the UK, as an example of how other processes and rationalities, namely economic governmentalities, shape the spaces that non-conventional medicine can inhabit. Section 30.6 untangles and critically analyses the assumptions and effects arising out of the process of deciding what counts as evidence in healthcare research regulations. It suggests that despite attempts to include different modalities, ambiguities persist due to acknowledged and unacknowledged hierarchies of knowledge-production explored in this chapter. The last section opens up a conversation about what is at stake when the logic underpinning the regulation of research creates a space for difference, including different medical traditions and what counts as evidence.¹⁰

30.3 Evidence-Based Medicine and Placebo Controls

Evidence-based medicine (EBM) stands for the movement which suggests that the scientific method allows researchers to find the best evidence available in order to make informed decisions about patient care. To find the best evidence possible, which essentially means that the many is more significant than the particular, EBM relies on multiple randomised controlled trials (RCTs) and evidence from these is eventually aggregated and compared.¹¹ Evidence is hierarchically organised, whereby meta-reviews and systematic reviews based on RCTs stand at the top, followed by non-randomised controlled trials, observational studies with comparison groups, case series and reports, single case studies, expert opinion, community evidence and individual testimonies at the bottom. In addition to reliance on quantity, the quality of the research matters. Overall, it means that the best evidence is based on data from blinded trials, which show a causal relation between therapeutic interventions and the effect, and isolates results from placebo-effects.

From a historical perspective, the turn to blinded tests represented a significant shift in medical practice insofar as it diminished the relevance of expert opinion, which was itself based on a hierarchy of knowledge that tended to value authority and theory over empirical evidence. Physicians used to prescribe substances, such as mercury, that although believed to be effective for many ailments, were later found to be highly toxic.¹² Thus, the notion of evidence arising out of blinded trials closed the gap between science and practice, and also partially displaced physicians’ authority. Blinded trials and placebo controls had other effects: they became a tool to demarcate ‘real’ medicine from ‘fake’ medicine, proper doctors from ‘quacks’ and ‘snake-oil’ peddlers. By exposing the absence of a causal relationships between the therapy and the physical effect, some therapies and knowledges associated with them were rebranded as fraudulent or as superstitions. While the placebo effect might retrospectively explain why some of these discarded therapies were seen as effective, in practice, EBM’s hierarchy of evidence dismisses patients’ subjective accounts.¹³ While explanations about the placebo effect side-lined the role of autosuggestion in therapeutic interventions, they did not clarify either the source or the benefits of self-suggestion.

Social studies suggest that the role of imagination has been overlooked as a key element mediating therapeutic interactions. Phoebe Friesen argues that, rather than being an ‘obstacle’ that modern medicine needed to overcome, imagination ‘is a powerful instrument of healing that can, and ought to be, subjected to experimental investigations.’¹⁴ At the same time, when the positive role of the placebo effect and self-suggestion has been raised, scholarship research has pointed out dilemmas that remain unsolved, for example: Is it ethical to give a person a placebo in the conduct of research on non-orthodox therapies, and when is it justifiable, and for which conditions? Or, could public authorities justify the use of tax-payers money for so-called ‘sham’ treatments when people themselves, empowered by consumer choice rhetoric and patient autonomy, demand it? As elaborated in this chapter, some governments have been challenged for using public money to fund therapies deemed to be ‘unscientific’, while others have tightened control, fearing that self-help gurus, regarded as ‘cultish’ sect-leaders, are exploiting vulnerable patients.

To the extent that physiological mechanisms of both placebo and nocebo effects are still unclear, there does not seem to be a place in mainstream public healthcare for therapies that do not fit the EBM model because it is difficult to justify them politically and judicially, especially as healthcare regulations rely heavily on science to demonstrate public accountability.¹⁵ And yet, while the importance of safety, quality and efficacy of therapeutic practices cannot be easily dismissed, the reliance on EBM as a method to demarcate effective from non-effective therapies dismisses too quickly the reasons why people are attracted to these therapies. When it comes to non-conventional medicines, biomedicine and the scientific method do not factor in issues such as patient choice or the social dimension of medical practice.¹⁶ In that respect, questions as to how non-conventional medicine knowledges can demonstrate whether they are effective or not signal broader concerns. First, is it possible to disentangle the reliance of public accountability from science in order to solve the ethical, political, social and cultural dilemmas embedded in the practice of traditional and alternative medicine? Second, if we are to broaden the scope of how evidence is assessed, are there other processes or actors that shape what is considered effective from the perspective of healthcare regulation, for example, patient choice or consumer rights? And, finally, if science is not to be considered as the sole arbiter of healing, what are the spaces afforded for other epistemologies of healing? Without necessarily answering all of these questions, the aim of this chapter is to signpost a few sticky points in these debates. The next section explores three examples, at different jurisdictional levels – national, regional and international – of how healthcare regulators have sought to provide guidelines on how to incorporate other types of evidence into research dealing with traditional and non-conventional medicine.

30.4 Integration as Subordination: Guidelines and Regulations on Evidence and Research Methodologies

Traditional medicine has been part of the WHO’s political declarations and strategies born in the lead up to the 1978 Declaration of Alma Ata.¹⁷ Since then, the WHO has been at the forefront of developing regulations aimed at carving out spaces for traditional medicines. However, the organisation has moved away from its original understanding of health, which was more holistic and focused on social practices of healing. Regional political mobilisations underpinned by postcolonial critiques of scientific universalism were gradually replaced again by biomedical logics of health from the 1980s onwards.¹⁸ This approach, favouring biomedical standards of practice, can be appreciated to some extent in the ‘General Guidelines for the Research of Non-Conventional Medicines’, which is prefaced by the need to improve research data and methodologies with a view of furthering the regulation and integration of traditional herbal medicines and procedure-based therapies.¹⁹ The guidelines state that conventional methodologies should not hamper people’s access to traditional therapies; and instead, reaffirms the plurality of non-orthodox practices.²⁰ Noting the great diversity of practices and epistemologies framing traditional medicine, the guidelines re-organised them around two broad classifications – medicines and procedure-based therapies.

Based on these categories, the guidelines suggest that efficacy can be demonstrated through different research methodologies and types of evidence, including historical evidence of traditional use. To ensure safety and efficacy standards are met, herbal medicines ought to be first differentiated through botanical identification based on scientific Latin plant names. Meanwhile, the guidelines leave some room for the use of historical records of traditional evidence of efficacy and safety, which should be demonstrated through a variety of sources including literature reviews, theories and concepts of system of traditional medicine, as well as clinical trials. It also affirms that post-marketing surveillance systems used for conventional medicines are relevant in monitoring, reporting and evaluating adverse effects of traditional medicine.

More importantly, the guidelines contemplate the use of mixed methodologies, whereby EBM can make up for the gaps of evidence of efficacy in traditional medicine. And, where claims are based on different traditions, for example, Traditional Chinese Medicine (TCM) and Western Herbalism, the guidelines require evidence linking them together; and where there is none, scientific evidence should be the basis. If there are any contradictions between them, ‘the claim used must reflect the truth, on balance of the evidence available’.²¹ Although these research methodologies give the impression of integrating traditional medicine into the mainstream, the guidelines reflect policy transformations since the late 1980s, when plants appeared more clearly as medical objects in the Declaration of Chiang Mai.²² Drawing on good manufacturing practice guidelines as tools to assess the safety and quality of medicines, WHO’s guidelines and declarations between 1990 and 2000 increasingly framed herbal medicines as an object of both pharmacological research and healthcare governance.²³

WHO’s approach resonates with contemporary European Union legislation, namely the Directive 2004/24/EC on the registration of traditional herbal medicines.²⁴ This Directive also appears to be more open to qualitative evidence based on historical sources, but ultimately subordinates evidence to the biomedical mantra of safety and quality that characterises the regulation of conventional medicines. Traditional herbal medicine applications should demonstrate thirty years of traditional use of the herbal substances or combination thereof, of which fifteen years should be in the European Union (EU). In comparison with conventional medicines requiring multiphase clinical trials in humans, the Directive simplifies the authorisation procedure by admitting bibliographic evidence of efficacy. However, applications must be supplemented with non-clinical studies – namely, toxicology studies – especially if the herbal substance or preparation is not listed in the Community Pharmacopeia.²⁵ In the end, these regulations subordinate traditional knowledges to the research concepts and methodologies of conventional medicine. Research centres of non-conventional medicines in the EU also align mission statements to integration-based approaches, whereby inclusion of traditional and non-conventional medicine is premised on their modernisation through science.²⁶ However, as we argue in the next section, science is not the sole arbiter of what comes to be excluded or not in the pursuit of evidence. Indeed, drawing on the UK as a case study, we argue that economic rationalities are part of the regulatory environment shaping what is or is not included as evidence in healthcare research.

30.5 Beyond Evidence: The Economic Reasoning of Clinical Guidelines

Despite there being no specific restrictions preventing the use of non-conventional treatments within the National Health Service (NHS), authorities involved in the procurement of health or social care work have been under increasing pressure to define the hierarchy of scientific evidence in public affairs. For example, under pressure of being judicially reviewed, the Charities Commission opened up a consultation that produced new guidance for legal caseworkers assessing applications from charities promoting the use of complementary and alternative medicine. Charities have to define their purpose and how this benefits publics. For example, if the declared purpose is to cure cancer through yoga, it will have to demonstrate evidence of public benefit, based on accepted sources of evidence and EBM’s ‘recognised scales of evidence’. Although observations, personal testimonies or expert opinion are not excluded per se, they cannot substitute scientific medical explanation.²⁷ For the Commission, claims that fail the scientifically-based standard are meant to be regarded as cultural or religious beliefs.

There have also been more conspicuous ways in which evidence, as understood through a ‘scientific-bureaucratic-medicine’ model, has been used to limit the space for non-conventional medicines.²⁸ Clinical guidelines are a key feature of this regulatory model – increasingly institutionalised in the UK since the 1980s. The main body charged with this task is the National Institute for Health and Care Excellence (NICE), a non-departmental public body with statutory footing through the Health and Social Care Act 2012. The purpose of NICE clinical guidelines is to reduce variability in both quality and availability in the delivery of treatments and care and to confirm an intervention’s effectiveness. Although not compulsory, compliance with the guidelines is the norm and exceptions are ‘both rare and carefully documented’²⁹ because institutional performance is tied to their implementation and non-adherence may have a financial impact.³⁰ Following a campaign by ‘The Good Thinking Society’, an anti-pseudoscience charity, NHS bodies across London, Wales and the North of England have stopped funding homeopathic services.³¹ Meanwhile, an NHS England consultation also led to the ban of the prescription of products considered to be of ‘low clinical value’, such as homeopathic and herbal products. Responding to critics, the Department of Health defended its decision to defund non-conventional medicine products stating they were neither clinically nor cost effective.³² However, it is also worth noting that outside of the remit of publicly funded institutions, traditional and non-conventional medicines have been tolerated, or even encouraged, as a solution to relieve the pressure from austerity healthcare policies. For example, the Professional Standards Authority (PSA) has noted that accredited registered health and social care practitioners – which include acupuncturists, sports therapists, aromatherapy practitioners, etc. – could help relieve critical demand for NHS services.³³ This raises questions about what counts as evidence and how different regulators respond to specific practices that are not based on biomedical epistemologies, particularly what sort of research is acceptable in healthcare policy-making. What we have sought to demonstrate in this section is the extent to which, under the current regulatory landscape, the production of knowledge has become increasingly enmeshed with various layers of laws and regulations drafted by state and non-state actors.³⁴ Although the discourse has focused on problems with the kind of evidence and research methodologies used by advocates of non-conventional medicine, a bureaucratic application of EBM in the UK has limited access to traditional and non-conventional medicines in the public healthcare sector. In addition to policing the boundaries between ‘fake’ and ‘real’ medicines, clinical guidelines also delimit which therapies should be funded or not by the state. Thus, this chapter has sketched the links between evidence-based medicine and law, and the processes that influence what kind of research and what kind of evidence are appropriate for the purpose of delivering healthcare. Regulation, whether through laws implementing the EU Directives on the registration of traditional herbal medicines, or clinical guidelines produced by NICE, can be seen as operating as normative forces shaping healthcare knowledge production. The final section analyses the social and cultural dimensions of knowledge production and it argues that contemporary regulatory approaches discussed in the preceding sections assume non-conventional knowledges follow a linear development. Premised upon notions of scientific progress and modernity, this view ultimately fails to grasp the complexity of knowledge-production and the hybrid nature of healing practices.

30.6 Regulating for Uncertainty: Messy Knowledges and Practices

Hope for a cure, dissatisfaction with medical authority, highly bureaucratised healthcare systems or limited access to primary healthcare, are among some of the many reasons that drive people to try untested as well as the unregulated pills and practice-based therapies from traditional and non-conventional medicines. While EBM encourages a regulatory environment averse to the miracle medicines or testimonies of overnight cures and home-made remedies, Lucas Richert argues ‘unknown unknowns fail to dissuade the sick, dying or curious from experimenting with drugs’.³⁵ The problem, however, is the assumption that medicines, and also law, progress in a linear trajectory. In other words, that unregulated drugs became regulated through standardised testing and licensing regulations that carefully assess medicines quality, safety and efficacy before and after they are approved into the market.³⁶

Instead, medicines’ legal status may not always follow this linear evolution. We have argued so far that the regulatory environment of biomedicine demarcates boundaries between legitimate knowledge-makers/objects and illegitimate ones, such as street/home laboratories and self-experimenting patients.³⁷ But ‘evidence’ also acts as a signpost for a myriad of battles to secure some kind of authority over what is legitimate or not between different stakeholders (patient groups, doctors, regulators, industry, etc.).³⁸ Thus, by looking beyond laboratories and clinical settings, and expanding the scope of research to the social history of drugs, STS scholarship suggests that the legal regulation of research and medicines is based on more fragmented and dislocated encounters between different social spaces where experimentation happens.³⁹ For example, Mei Zhan argues that knowledge is ‘always already impure, tenuously modern, and permanently entangled in the networks of people, institutions, histories, and discourses within which they are produced’.⁴⁰ This means neither ‘Western’ biomedical science or ‘traditional’ medicines have ever been static and hermeneutically sealed spaces. Instead, therapeutic interventions and encounters are often ‘uneven’ and messy, linking dissimilar traditions and bringing together local and global healing practices, to the point that they constantly disturb assumptions about ‘the Great Divides’ in medicine. For example, acupuncture’s commodification and marketisation in Western countries reflects how Traditional Chinese Medicine has been transformed through circulation across time and space, enlisting various types of actors from different professional healthcare backgrounds – such as legitimate physicians, physiotherapists, nurses, etc. – as well as lay people who have not received formal training in a biomedical profession. New actors with different backgrounds take part in the negotiations for medical legitimacy and authority that are central to the reinvention of traditional and non-conventional medicine. These are processes of ‘translocation’ – understood as the circulation of knowledges across different circuits of exchange value – which reconfigure healing communities worldwide.⁴¹

So, in the process of making guidelines, decisions and norms about research on traditional and non-conventional medicines, the notion of ‘evidence’ could also signify a somewhat impermanent conclusion to a struggle between different actors. As a social and political space, the integration of traditional medicine and non-conventional medicine is not merely a procedural matter dictated by the logic of medical sciences. Instead, what is accepted or not as legitimate is constantly ‘remodelled’ by political, economic and social circumstances.⁴² In that sense, Stacey Langwick argues that evidence stands at the centre of ontological struggles rather than simply being contestations of authority insofar it is a ‘highly politicized and deeply intimate battle over who and what has the right to exist’.⁴³ For her, determination of what counts as evidence is at the heart of struggles of postcoloniality. When regulations based on EBM discard indigenous epistemologies of healing or the hybrid practices of individuals and communities who pick up knowledge in fragmented fashion, they also categorise their experiences, histories and effects as non-events. This denial compounds the political and economic vulnerability of traditional and non-conventional healers insofar as their survival depends on their ability to adapt their practice to conventional medicine, by mimicking biomedical practices and norms.⁴⁴ Hence, as Marie Andree Jacobs argues, the challenge for traditional and non-conventional medicines lies in translating ‘the alternativeness of its knowledge into genuinely alternative research practices’ and contributes to reimagining alternative models of regulation.⁴⁵

30.7 Conclusion

This chapter analysed how regulators respond to questions of evidence of traditional and non-conventional medicines. It argued that these strategies tend to subordinate data that is not based on EBM’s hierarchies of evidence, allowing regulators to demarcate the boundaries of legitimate research as well as situating the ‘oddities’ of non-conventional medicines outside of science (e.g. as ‘cultural’ or ‘religious’ issues in the UK’s case). In order to gain legitimacy and authority, as exemplified through the analysis of specific guidelines and regulations of research of traditional and non-conventional medicines, the regulatory environment favours the translation and transformation of traditional and non-conventional medicines into scientised and commercial versions of themselves. Drawing on STS scholarship, we suggested understanding these debates as political and social struggles reflecting changes about how people heal themselves and others in social communities that are in constant flux. More importantly, they reflect struggles of healing communities seeking to establish their own viability and right to exist within the dominant scientific-bureaucratic model of biomedicine. This chapter teased out limits of research regulation on non-conventional medicines, insofar practices and knowledges are already immersed in constantly shifting processes, transformed by the very efforts to pin them down into coherent and artificially closed-off systems. By pointing out the messy configurations of social healing spaces, we hope to open up a space of discussion with the chapters in this section. Indeed, how can we widen the lens of research regulation, and accommodate non-conventional medicines, without compromising the safety and quality of healthcare interventions? At the very minimum, research on regulation could engage with the social and political context of medicine-taking, and further the understanding of how and why patients seek one therapy over another.