Evaluating traditional and complementary medicines: Where do we go from here?

Abstract

Traditional and complementary medicines are increasingly considered possible options for prevention and symptomatic treatment of the novel coronavirus, COVID-19. With renewed attention on these therapies from researchers and policy makers alike, the well-documented challenges of evaluating their safety and efficacy are once again of global concern. Between 2005 and 2018, the World Health Organization conducted a series of surveys, in which 88 percent of responding member states confirmed that their biggest challenge in traditional medicine was the need for technical guidance on research and evaluation. As a first step in pursuing this need, our commentary summarizes thirteen international and regional guidance documents by three broad categories on evaluating safety, efficacy, and product quality for market-based approval and distribution of these treatments. We highlight the paucity of updated international recommendations on these subjects and identify gaps that could inform the current evidence base. All available guidance note the need for evidence surrounding the efficacy of these treatments and practices but are also quick to caution against methodological difficulties in the conduct of such evaluations. Evidence suggests that improved evaluation methods on efficacy and effectiveness are crucial toward expanding future research into establishing the cost-effectiveness of these therapies, in the context of shifting acceptance, interest, and integration of traditional medicines into health systems, and as another step toward Universal Health Coverage.

Background

Globally, countries are under pressure to develop methods to prevent and treat symptoms of the novel coronavirus (COVID-19) and a few, like China and India, have indicated the need to consider traditional medicine as a viable option. Most recently, the General Office of the National Health and Health Commission of China supported the wider application of a Traditional Chinese herbal medicine treatment program for COVID-19 pneumonia and released an update to the regional outbreak treatment guidelines facilitating inclusion of these treatments on the frontline (1;2). Meanwhile, India has initiated a national-level clinical trial on three herbs to improve immune response (3), whereas in the southern state of Tamil Nadu, officials have begun community-level distribution of a herbal concoction that has yet to undergo any evaluations on safety or effectiveness regarding its immunity-boosting claims (4). The national Ministry of Ayurveda, Yoga & Naturopathy, Unani, Siddha and Homoeopathy (AYUSH), the traditional medicine authority in the country, continues to promote several “self-care” prevention tips using herbal remedies for symptoms of COVID-19 and has recently also opened a call for proposals to study possible prophylaxis, although providing little guidance on the types of supporting evidence required as a component of these studies. With renewed attention on these therapies from researchers and policy makers alike during the current viral outbreak, the well-documented challenges of evaluating their safety and efficacy are once again of global concern.

In a series of three World Health Organization (WHO) global surveys distributed between 2005 and 2018, 99 of 113 (88 percent) responding member states confirmed that their biggest challenge in traditional medicine was the need for more technical guidance on research and evaluation relating to the safety, efficacy, and quality of these treatments (5), given that the WHO's last comprehensive global guidance on these topics was published almost 20 years ago in 2000 (6). The increasing demand for research guidance has also been catalyzed by the adoption of the 2009 World Health Assembly resolution (WHA62.13) to encourage the appropriate development of traditional systems of medicine and examine the therapeutic properties of such remedies (7). Most recently, in 2019, the inadequacy of evidence concerning the safety and efficacy of Traditional Chinese medicine has been raised in critique of its inclusion into the 11th revision of the WHO International Code of Disease (WHO-ICD) (8), the first instance of formal recognition of these therapies.

Traditional medicine is defined by the WHO as the “sum total of the knowledge, skill and practices based on the theories, beliefs and experiences indigenous to different cultures, whether explicable or not, used in the maintenance of health as well as in the prevention, diagnosis, improvement or treatment of physical and mental illness” (9). Complementary/alternative medicine is a related term for practices that exist but are not indigenous to the country, and it is often used interchangeably with the term traditional medicine (5). Combined, traditional, and other complementary medicines (T&CM) comprise products, practices, and practitioners; products are specific to herbs and herbal materials, whereas examples of practices include medication and procedure-based therapies, such as herbal medicines, acupuncture, chiropractic, osteopathy, yoga, naturopathy, and thermal medicine (9). 171 of 194 (88 percent) WHO member states have acknowledged the formal use of T&CM within their health systems (5), establishing offices, national policies, laws, and regulations for T&CM uptake. At the same time, these therapies have been variably conceptualized, defined, and categorized in terms of their integration within the health system, service delivery, and consumer levels (10).

In this commentary, we put together insights from peer-reviewed articles, gray literature sources, as well as regional and international guidance on the evaluation of herbal products and T&CM practices specifically enlisted for market-based approval and distribution, These are active guidance still endorsed by and published on the Web sites of organizations (e.g., WHO, EU). We also highlight the paucity of updated international guidance and identify gaps that could help inform and contribute to the current evidence base for an increased uptake of these therapies in future.

Safety Evaluation

Across thirteen guidelines (see Table 1 for the list of documents), a central guiding principle on evaluating the safety of T&CM is the distinction in evidence requirements between long-standing use of these medicines for generations, as compared to those considered more recent. Herbal medicines of widespread use are exempt from providing additional safety evidence for market approval (11). Although global guidance defines “traditional use” to indicate those used over a long period with no demonstrable harm, regional guidance specifies certain market-based regulatory criteria such as length of use, individual product components, and other categories to define these drugs (6;11–17). For instance, in the European Union (EU), a product qualifies as “traditional use” if it has been used for at least 30 years, including at least 15 years within the EU (11).

Table 1. Guidance documents on safety, efficacy, and quality evaluation of T&CM, organized by region and year published (in alphabetical order)

ASEAN, Association of Southeast Asian Nations; FDA, Food and Drug Administration (US); EMA, European Medicines Agency; EU, European Union; NDI, New Dietary Ingredient; OECD, Organization for Economic Co-operation and Development; WHO, World Health Organization.

That said, even in cases where drugs have been used over a long period, chronic toxicological risks may have occurred but not recognized, raising concerns regarding safety evaluations of such treatments. The 2000 guidance outlines specific conditions where new safety evidence is required such as in cases where unrecognized chronic toxicology risks are likely, even in established, long-term use drugs, and also emphasizes the need for consistent monitoring of side-effects in line with normal pharmacovigilance practices (6;12;14;15;18). In guidelines applicable to member states of the Association of Southeast Asian Nations (ASEAN), four types of products require evidence to substantiate their safety: (i) products with new ingredients, (ii) products with ingredients derived from new methods of purification, extraction, or manufacturing, (iii) existing products with new combinations, new dosage, new method of delivery, or new indication, and (iv) existing products with emerging safety concerns (19). Across these guidelines, standard methods of nonclinical toxicological studies to assess safety include carcinogenicity tests as well as a broad umbrella of toxicological studies such as immunotoxicity (e.g., tests for allergic reactions), genotoxicity, reproductive toxicity, and others to measure specific, acute, chronic, and long-term toxicity, as suggested by the WHO's global guidance in 1993 and 2000.

In 2004, the WHO further proposed the inclusion of herbal medicines into its existing WHO International Drug Monitoring Programme, a pharmacovigilance initiative established in the 1970s for exchanging safety information among member states and maintaining the global WHO database of adverse drug reaction (ADR) reports (20). A licensed T&CM product undergoes similar postmarketing safety monitoring as all other drugs in Western medicine. For countries in the Organization for Economic Co-operation and Development (OECD), the establishment of safety standards for component chemicals in T&CM products in accordance with OECD test guidelines and principles of good laboratory practices allows acceptance of these results by all members under the OECD Mutual Acceptance of Data (MAD) system (15). Similar regional harmonization methods can be observed in the set of ten guidelines developed for the ASEAN region; for example, establishing negative lists for substances and limits on contaminants in herbal medicine as well as recommendations for member states to ensure the availability of postmarket surveillance to handle early signals of adverse drug events and/or product safety issues (21). National monographs or pharmacopoeias alongside good manufacturing practices for countries/regions also aid evaluations of safety using shared frameworks as in the Western Pacific region as well as the EU (16;22).

To assess the safety of T&CM practice and procedure-based therapies, effective measures include ensuring that equipment in use is of good quality and practitioners have sound theoretical and practical training. To maintain both efficacy and safety in T&CM practices, some countries such as China and Hong Kong have professional licensing bodies that require the completion of an exam in order to be a registered practitioner. In Singapore and Thailand, licensing bodies for different T&CM practices and procedures approve recognized international accreditations and undergraduate degrees in T&CM from local institutions (5). In both high- and low-income countries, T&CM practitioners are required to complete professional trainings and programs, with countries in North America, India, Republic of Korea, and Vietnam mandating that T&CM practitioners must be university graduates (5). However, in many lower-income countries, traditional medicine practices are passed on intergenerationally via informal channels, which is a challenge in ensuring uniform patient safety, despite existing guidelines (7).

Specifically, the lack of evidence concerning adverse effects from procedure-based therapies is well recognized, calling for a more systematic approach to record these and develop specific guidelines for the safe conduct of these procedures (6;23).

All guidelines indicate that the primary source of evidence regarding drug safety can be obtained from published data sources, such as peer-reviewed scientific literature, with a focus on conducting independent trials in cases where possible and mandated, such as “stand-alone applications” in the EU that require safety data to be supplemented with data from independent studies conducted by the company seeking approval (6;14;17;20;24). Similarly, the United States Food and Drug Administration (FDA) also stipulates the need for all products filing for commercialization to provide evidence of a history of safe use; other evidence including clinical testing, animal testing, or both; or some combination of history of use and other evidence of safety (14). However, support for conducting tests in vivo (in live specimens) by the FDA is in contrast to WHO guidelines, which give preference to in vitro techniques (outside of live specimens) (6). Other accepted sources of evidence include reports from authoritative bodies, data on food or nutrient composition/consumption, as well as case reports of serious ADR and must be shown to be weighed against appropriate additional potential benefit (13;14;21).

Many of these safety guidelines for products and practices are integrated into regional and national regulatory processes, where inadequate compliance to safety reporting and monitoring standards implicates approval being refused or re-evaluated. In 2018, the AYUSH Ministry in India released the first edition of guidelines for safety and toxicity evaluations of traditional drug formulations drawing from international guidance on the topics from the WHO, the OECD, and the US FDA, among others (25). A Web portal for pharmacovigilance of Ayurvedic formulations, “AyushSuraksha,” has also been set up for consumers to confidentially log any ADRs from these drugs. However, this is not available for T&CM practices such as yoga and naturopathy and is limited to drugs alone.

Efficacy Evaluation

Guidance on efficacy assessment frequently emphasizes the need for different considerations to that of conventional medicine because T&CM has to be evaluated in an integrated manner, taking into account its holistic practice and approach, as well as keeping in mind the varying methods of individual practice and cultural background of the user (6;25). For instance, many herbal remedies consist of a combination of several active ingredients, and the assessment should differentiate between old and new combination products (6;12;24). The WHO views well-established, randomized controlled clinical trials (RCTs) as the highest level of evidence for efficacy in its 1993 and 2000 guidelines (6;12). An explanation of a new combination of well-known substances, including effective dose ranges and compatibility, is required in addition to documentation of traditional knowledge of single ingredients (6;24). Clinical studies are required to justify the efficacy of each new ingredient and its positive effect on the total combination, as well as facilitate the acceptance of herbal medicines in different regions and in people with different cultural traditions. However, in cases of T&CM treatment of minor disorders, nonserious indications, and in use as prophylaxis, efficacy can be adequately evaluated using observational studies (6).

Where traditional use of a specific herb in a country or region has not been established, additional evidence on clinical efficacy is required. More specific to countries in the EU, traditional herbal medicinal products that satisfy the criteria of “traditional use” do not require clinical trials on efficacy (11;18;24). If the product meets the conditions of “well-established use” (active substances that have been in medicinal use in the EU for at least 10 years), registration should include strong evidence in scientific literature establishing the efficacy of active substances. Candidates for both types can refer to the EU Herbal Monographs for established medicinal products/active substances of the committee approved level of safety and efficacy (18;22). “Stand-alone applications” require clinical data on efficacy to be supplemented by the company seeking approval, using their own studies. Key elements of clinical data include a combination of: systematic reviews, critical assessment of clinical data for credibility and methodological rigor, clinical data reports that meet reporting standards, assessment of clinical relevance of putative efficacy, and at least one controlled clinical study of good quality as evidence of efficacy (22).

In assessing the efficacy of traditional procedure-based therapies and practice, researchers should consider the holistic contribution of practitioner proficiency and equipment quality, which is often impacted by varying standards of manufacturing quality. This may explain reports of disparity or inconsistency in results, even though the methodologies in different studies have been equally sound (5). Clinical trials are key in efficacy evaluations of traditional procedure-based therapies.

The general principles and requirements for conventional drugs [e.g., the WHO's Good Clinical Research Practice (GCP) Protocols] (26) should still apply to new herbal medicines, or new indications, dosage, or the method of administration for existing medicines. Trials involving human subjects should be conducted in accordance with international ethical guidelines for biomedical research, for example, the Declaration of Helsinki by the World Medical Association (12).

Product Quality Evaluation

Quality evaluation guidance by the WHO and the European Medicines Agency (EMA) focuses on the quality aspects of natural herbal medicinal products in the market, which excludes any product containing chemically isolated constituents or mixtures (12;27). Within this scope, reliability of the identity and quality of plant material and preparation should be ensured by stipulating how to identify, label, collect, and process the herbal material, as well as checks for authenticity and purity (12). In preparation of the material for manufacturing, guidance documents define qualitative and quantitative specifications of active ingredients, a description of the method of preparation, control tests of starting materials, manufacturing, and stability tests. The European set of guidelines should be read in conjunction with the EU's Good Manufacturing Practice (GMP) for Medicinal Products, Volume 4, Rules Governing Medicinal Products, and all GMP recommendations should be followed (27). Discussion of quality assurance or evaluation does not extend beyond the scope of herbal medicine and products in the literature. In Thailand, Indonesia, and India, GMP regulatory mechanisms for T&CM drugs are considered different from those for conventional pharmaceutical drugs due to the nature of the raw materials involved. They undergo regular inspections and government-authorized laboratory tests and require safety and toxicity assessments, among other regional stipulations, before registration and are governed by special clauses under the Drug legislations (5). In contrast, in countries like Sri Lanka, Republic of Korea, Myanmar and Nepal, these drugs are treated the same as pharmaceuticals in their registration processes, and some countries like Nepal have yet to follow GMP as well (5).

Gaps in Current Guidance on T&CM Evaluation

A significant finding is the absence of an updated global guidance on the evaluation of T&CM; the last comprehensive guidance from the WHO on this topic dates back two decades, although other regional resources on the subject have since been developed. This is key in setting the direction for research, regulation, and policy mechanisms, both internationally and regionally, as the WHO plays a key advisory role in the development of national and regional guidance documents. For instance, the WHO African Region emphasizes the role of national governments in establishing formal safety standards for T&CM in consultation with WHO protocols on these subjects (17). In further recognition of the importance of international guidance, we also identify challenges and gaps in the evidence base for T&CM, suggesting that improved standardized methods and data on efficacy and effectiveness are crucial in expanding future research into establishing cost-effectiveness of these therapies.

Effectiveness

All available guidance notes the need for evidence surrounding the efficacy of these treatments and practices but are also quick to caution against methodological difficulties in evaluation. For example, RCTs could be challenging on technical and ethical fronts. Placebo control is difficult when herbal medicines have a strong flavor—sometimes low dosages of the same herbal medicine of similar color, taste, and weight must be used as a control (6). Sham acupuncture interventions used as placebo are often not disclosed to participants (23), and such deceptive practices do not correspond to ethical standards for clinical research. Moreover, as the duration of clinical research studies is usually short, RCTs can reveal only the short- and mid-term clinical efficacy of interventions, whereas the effectiveness of T&CM techniques typically manifests over the long term (28).

In a review focusing on Traditional Chinese medicine from 2019, Yang et al. (28) suggest that systematic and standardized evaluation indicators based on basic theories such as the Concept of Holism, Yin-Yang, and Meridian have not yet been established. Traditionally, clinical efficacy evaluation of Traditional Chinese medicine techniques is based on biochemical examination indicators and some subjective observation indicators. According to the authors, its working principle and multiple-link comprehensive effect are difficult to accurately evaluate by existing research methods (28), and RCT findings cannot objectively reflect the clinical value of Traditional Chinese medicine diagnosis and treatment.

These problems point to whether measuring effectiveness would better suit the concept of T&CM. Effectiveness refers to the benefit of using a technology, program, or intervention to address a specific problem under general or routine clinical conditions, rather than under controlled conditions in an RCT (29). Guidance on effectiveness evaluations is rather limited. Observational studies involving large numbers of patients could be a valuable tool for the evaluation of herbal medicines and its effectiveness over the long term, alongside other study designs such as single-case design and black-box design (6), but there are no frameworks for adapting these to T&CM. There is no universally valid method for grading observational studies, but a list of categories has been offered by the WHO, notably cohort (longitudinal) studies, case-control studies, cross-sectional studies, uncontrolled case series or cohort studies, and time-series studies (6). Studying the effectiveness of T&CM is particularly challenging as they are often combined with Western medicine treatments in clinical scenarios, and the true effects of traditional medicine treatments are difficult to measure (28). As such, studies that attempt to provide evidence for either efficacy or effectiveness of T&CM will have to mitigate the issues of confounding, blinding, and randomization. In an attempt to recognize and address these evaluation challenges, Liu et al. (30) have developed guidelines on using different types of research methods for evaluating the therapeutic effect of Traditional Chinese medicine, including a variety of clinical trial–based techniques; these hold valuable insights for other T&CM as well. Examples include pragmatic clinical trials that are closer to the real-world conditions but retain the elements of randomization essential to comparative effectiveness research (30).

Cost-Effectiveness

Essential medicines are defined as those that satisfy priority healthcare needs of the population, and the inclusion of T&CM in the lists of countries’ national essential medicines (NEML) signals strong public recognition for these treatments. Thirty-four WHO member states (10 in Asia) have herbal medicines on their NEML (5). Additionally, several governments provide national insurance coverage for T&CM, including Bhutan, where full coverage is available for gSo-ba Rig-pa (traditional Bhutanese medicine). In China, government and commercial insurance (both state-owned and private insurance companies) cover indigenous traditional medicine but only part-cover T&CM practices of acupuncture, herbal medicines, osteopathy, and Traditional Chinese medicine. Thailand includes herbal medicines in its national list of essential medicines and T&CM services are reimbursed by public insurance as of 2018 (5). In countries that now provide partial coverage, such as Japan, Korea, and others in the Western Pacific, there is an impetus to conduct cost-effective analyses for T&CM that have proven safety and efficacy (a process more generally known as health technology assessments or HTA), as a means to formalize their coverage through national insurance schemes (16).

In theory, T&CM appear cost-effective and good candidates for HTA as studied under a payer or societal perspective because of their impact on national budgets, especially in countries where they are subsidized for large populations; avoid technology; offer inexpensive remedies; and harness the power of the body's natural ability to heal itself (27). They are also likely to be the most easily available options for remote populations (10). HTAs of specific therapies [e.g., acupuncture (28)] have been previously conducted, but there is, to our knowledge, no organized framework or the adaptation of economic evaluation methods to the unique features of T&CM. The collection of costs and patient-reported/health-related quality-of-life outcomes remains uncommon in T&CM evaluation, despite their impacts on national budgets, as they are heavily subsidized for populations.

Conclusion

Globally, many countries and regions have established regulatory systems for T&CM as well as networks and initiatives for cooperation between member states to develop, share, and adopt regulatory standards as can be seen from guidelines in the EU, OECD, US FDA, ASEAN as well as overarching guidance from the WHO (6;12–15). Africa and the Western Pacific have also shown thought leadership, with frameworks, tools, and concrete objectives for integration of T&CM into their health systems (17). Specifically, Asia shows great promise for uptake of these products and procedures as affordable, available, and accessible medical care on the path to Universal Health Coverage (UHC), especially through primary health systems. China, Japan, Thailand, and the Republic of Korea have stringent regulation and integration of these treatments into their health systems; Malaysia and Cambodia too are in the process of developing similar ones (10;16). Twelve WHO member states, eleven of whom were from the global South, affirmed that they were using public or government research funding toward T&CM (5).

As we write this commentary, the interest in traditional medicinal products as a potential supplement or alternative to Western medicine is strong, as has been witnessed during this pandemic outbreak; however, global guidance lags behind the demand for it (6). In this context, we highlight some of the pertinent challenges that future guidance should explicitly make recommendations on, which include: (i) the lack of a system to evaluate efficacy and effectiveness that is based on T&CM concepts, (ii) technical problems with studies (placebo control is difficult when herbal medicines have a strong flavor; difficulties blinding practitioners; variation in practitioner skill) (6), and (iii) confounding issues, for example where T&CM treatments combine multiple T&CM products, practices, practitioners; and often also combined with Western medicine in clinical scenarios, making the true effect of T&CM treatments difficult to measure. We recognize that our commentary has been limited to the content of all guidance documents in effect and is unable to account for the adherence or application of the same guidance in practice. It is also restricted to documents available in the English language up until May 2020 for a topic such as this that demands a broader Asian perspective, although several of our English sources from the ASEAN, the WHO, and peer-reviewed articles had considered country-level safety, quality, and efficacy evaluation requirements for market entry in many of the Asian countries for T&CM products. In addition, the ubiquitous nature of T&CM as a system of healing has ancient origins with treatments and practices extensively conducted at the individual, household, or community levels. Home- or community-based remedies offer great potential as a method of accessible, affordable, and available care; however, their means of distribution and practice have not been included and, therefore, offer a crucial opportunity for future research.

The shift since 2000 in global acceptance and integration of T&CM into health systems also raises questions on how to evaluate their new and important aspects, including effectiveness and cost-effectiveness. One of the most significant policy-level decisions for governments has been whether to include traditional medicines into a scheme of UHC, and if accepted, what this decision would look like in practice (16). Forty-five of the WHO member states currently have health insurance coverage for traditional medicine and T&CM practices (e.g., acupuncture and chiropractic) that are provided by the government and/or private sectors (5). In many cases, this coverage is partial and is specifically designed to regional conditions; for instance touching only specific areas such as acupuncture, chiropractic, and herbal medicine or, in other cases, is only specific to government-approved products and practices as in the Western Pacific region (16). Should countries want to take a step in the direction of UHC for traditional medicines, evidence-based assessment of its budget impact and value for money will become a matter of growing importance.

Keeping these current challenges in mind, future research must focus on developing better frameworks for evaluating T&CM. Such guidance must align with incentives for compliance in order to obtain regulatory approval and/or reimbursement. Given the trend of supranational monitoring and current interest and development of these products and practices in Asia, the region offers wide opportunities to systematically study these treatments in detail and for evidence-informed policymaking in T&CM.