Introduction
Bhutan is a small landlocked Himalayan Kingdom situated between China and India, with a population of less than 800,000 (National Statistics Bureau 2024). Despite relatively well-developed health services, the healthcare system faces challenges from the country’s extremely remote and rugged terrain and sparsely distributed population (National Statistics Bureau 2024). The national publicly funded healthcare system provides healthcare to all citizens; however, palliative care has not been included in a substantial way within this system (Thinley et al. Reference Thinley, Tshering and Wangmo2017).
There are an estimated 700 children affected by cancer and other serious illnesses including heart diseases, neurological conditions, premature births, congenital anomalies, and severe birth asphyxia each year in Bhutan; these children and their families require palliative care to improve their quality of life and address serious health-related suffering (Ministry of Health, Bhutan 2022). Palliative care is in a very early stage of development in Bhutan, with only 1 palliative care service at the present time, which is a home-based service operating from National Referral Hospital (Laabar et al. Reference Laabar, Saunders and Auret2021). This service focuses on the needs of adult patients in the region of the capital city of Thimphu, and there are currently no established children’s palliative care programs in Bhutan.
Recent studies have explored the palliative care needs of adult patients and their families in Bhutan; however, the needs of children have not yet been formally explored (Laabar et al. Reference Laabar, Saunders and Auret2021). Palliative care has yet to be incorporated into the professional training programs of doctors, nurses or other types of healthcare professionals (HCP). Without education, HCPs are often unaware of which children can benefit from palliative care and wrongly equate palliative care with end-of-life care, believing that palliative care is only needed for children who are in the last days or weeks of their illness (Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024).
Given the challenges of Bhutan’s remote geography, online education suggests an innovative solution to address the gaps in HCP knowledge and skills related to palliative care. Linking HCP in remote areas with specialists to support the provision of palliative care has been described in Nepal and other parts of South Asia (Doherty et al. Reference Doherty, Rayala and Evans2021b; Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). Online learning offers additional benefits, since it can reduce the costs associated with training, by reducing the need to travel to training programs in urban centers. Despite these benefits, there are challenges associated with online training; the lack of face-to-face interaction can make it more difficult for participants to stay engaged in the learning. Additionally, learners note that without opportunities to apply their newly acquired knowledge under supervision, participants may lack confidence to apply this new knowledge in clinical practice (Frehywot et al. Reference Frehywot, Vovides and Talib2013). Additionally, the treatment recommendations provided by remote experts may not be feasible in the local healthcare context, particularly in rural areas where resource constraints are common (Frehywot et al. Reference Frehywot, Vovides and Talib2013).
Project ECHO is a model of online learning, which is being used around the world to connect providers in underserved or remote areas with clinical experts for teaching and mentorship (Arora et al. Reference Arora, Snead and Zalud-Cerrato2017). Using a “hub-and-spoke” model, experts at a “hub” site connecting regularly with providers (“spokes”), and learning occurs through case discussions and formal teaching (Figure 1), which leads to improved care for patients.
Figure 1. Hub-and-spoke model of Project ECHO.
Several recent studies described the use of the Project ECHO model for palliative care in South Asia, with studies including HCP in India, Nepal, and Bangladesh (Doherty et al. Reference Doherty, Lynch-Godrei and Azad2022, Reference Doherty, Rayala and Evans2022; Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). These studies identified improvements in palliative care knowledge and attitudes among participants, while noting challenges related to language barriers and finding the time to join training sessions (Doherty et al. Reference Doherty, Lynch-Godrei and Azad2022; Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). The use of online learning for palliative care has not been described in Bhutan, where Project ECHO suggests an innovative solution to address gaps in HCP knowledge and skills related to children’s palliative care. Given the country’s unique health system and culture, exploring the use of online training for HCP will help adapt such training programs to the local context.
The aim of this study is to describe the impact of an online learning program (Project ECHO) for HCP in Bhutan, exploring the impact on participants’ knowledge, confidence and attitudes toward children’s palliative care through pre- and post-program surveys. The study also aims to explore the acceptability and feasibility of using online learning in Bhutan. These findings will guide best practices for the implementation of online training programs, supporting the contextualization of learning programs in Bhutan.
Methods
Learning program description
The development of the ECHO program followed the Project ECHO model with contextualization and adaptions as we previously described (Doherty et al. Reference Doherty, Rayala and Evans2021b; Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). Key steps included bringing together a course leadership team, which include local stakeholders and regional palliative care experts from Bhutan, India, and Canada. Involving regional experts allowed sharing of experiences from similar palliative care training in Nepal, India, and Bangladesh. The teaching topics, timing and duration of sessions were selected via group consensus with input from potential participants (Table 1).
Table 1. Teaching topics included in children’s palliative care Project ECHO in Bhutan
Course participants were recruited via email, social media, and word of mouth through the leadership team’s networks. The program was offered free of cost and was open to all HCP and health administrators. Learners were advised of the need for a mobile device or computer and a stable internet connection and the program time commitment. The program included 6 weekly sessions of 75 minutes duration (July–Aug 2023), which were conducted via Zoom videoconferencing software. The sessions included didactic teaching from a palliative care expert (10–12 minutes), followed by virtual small group discussions, case presentation, and then discussion. Each session was led by an engaging facilitator who encouraged active participation from learners. Facilitators were provided with a training session and guidelines for best practices to encourage learner participation.
The first session included an opening ceremony, with the president of the Bhutanese National Health Service and program partner organizations including Hyderabad Centre for Palliative Care (India) and Two Worlds Cancer Collaboration (Canada). This provided an opportunity to raise awareness about the course with a broader audience both locally and regionally. These stakeholders’ support was anticipated to be important for future implementation of palliative care locally following completion of the ECHO program. The structure of the weekly sessions followed the schedule outlined in Table 2.
Table 2. Structure of weekly ECHO sessions
Case presentation
A clinical case was prepared and presented by 1 participant following a structured template each week. The case presentation was reviewed by a faculty member prior to presentation to ensure clarity and that the case was focused on palliative care considerations. A sample case presentation is shown in Table 3.
Table 3. Sample of a case presentation presented during the Bhutan ECHO program
y.o. = years old, ARDS = acute respiratory distress syndrome, CPR = cardiopulmonary resuscitation.
Small group discussion
Weekly session small group discussion were conducted in virtual “breakout rooms,” and included 6–8 learners and 1 faculty member, who moderated the discussion. Several structured questions were used in the small group discussion to stimulate deeper engagement with the topic. After small group discussion, learners reconvened with the larger group, and 1 learner summarized 2–3 key points from their discussion. Faculty were provided with breakout room questions and model answers before each session, and the answers were provided to all participants after the session.
Post-session learning resources
Following each session, participants were provided with the key learning points, didactic slides, answers to the discussion questions, the session recording link, and additional relevant reference documents (e.g. articles, guidelines, video links, etc.) by email and private group chat (social media).
Study recruitment
All leaners who were registered and participated in the ECHO program were invited by email and social media to participate in the pre- and post-ECHO surveys.
Survey development
The survey was developed by reviewing and adapting similar study instruments used for a pediatric palliative care Project ECHO in Nepal and other countries in the region (Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). The survey was reviewed by local leadership team members, and question wording was adapted to the local context. The survey included questions exploring palliative care knowledge, as well as self-reported comfort and attitudes related to palliative care. Participants were asked to rate their comfort and attitudes both before and after program participation and were provided an opportunity to revise their responses from the pre-survey at the time of the post-survey. This is particularly relevant, as previous studies describe that learners may initially overestimated their abilities (Lalloo et al. Reference Lalloo, Osei-Twum and Rapoport2020). The post-program survey also explored course satisfaction, barriers, and enablers of participation and practice change. Demographic data, including years of experience, profession, areas of specialization, and number of children with serious illness which the participant cared for monthly, were collected.
Ten multiple choice knowledge questions which had been piloted, tested, and refined by the leadership team were used to assess learners’ knowledge before and after the program. Each question was mapped to a course learning objective. To assess comfort, 8questions with the stem “for my scope of practice, I am confident in my ability to …,” were used, with each item linked to a course learning objective. To assess attitudes, participants were asked to rate their level of agreement with 10 statements related to palliative care. Comfort and attitude responses used a 7-point Likert scale with response options ranging from [1] “strongly agree” to [7] “strongly disagree.” Other questions included response as multiple choice or free text, depending on the question. The full questionnaire is available in Data Supplement 1.
Statistical analysis
Data were analyzed using Microsoft Excel to obtain descriptive statistics. Mann–Whitney U-tests were performed to evaluate differences between comfort and attitude scores of survey participants before and after participating in the ECHO program. An a-priori statistical significance was set to p = 0.05. Likert scales were reverse coded to improve clarity of results in presentation for applicable questions. Effect size was calculated using Cohen’s D S, with an effect size 0.80 or more suggesting a large effect.
Results
Participant characteristics
Study participants included nurses (n = 26, 49%), physicians (n = 18, 34%), and medical/nursing educators (n = 5, 9%). Most participants were working in pediatric and/or neonatal care (n = 35, 68%) or general pediatric practice (n = 7, 13%). The average duration of clinical practice was 9.3 years. Most participants (n = 28, 53%) reported caring for between 1 and 10 children needing pediatric palliative care per month. Further demographic details are shown in Table 4.
Table 4. Project ECHO in Bhutan (n = 53)
Palliative care knowledge, comfort, and attitudes
The average pre-course knowledge score was 55%, with average scores increasing to 87% on the post-test, with significant improvements in all areas of knowledge. Participants most often described low initial levels of comfort (responses of “strongly disagree,” “disagree,” and “somewhat disagree”) in identifying patients who are approaching end of life (n = 9, 24%), explaining prognosis to parents (n = 13, 34%), and caring for distressing symptoms (n = 9, 24%). There were statistically significant (p < 0.05) improvements in all 8 areas of comfort, with effect sizes of 0.8 or higher.
Participants’ initial attitudes related to palliative care were most likely to be low (responses of “strongly agree,” “agree,” and “somewhat agree”) for the following statements: early initiation of palliative care will increase anxiety (n = 16, 42%), palliative care is the same as “end-of-life” care (n = 25, 66%), and treatment with placebo is an appropriate treatment for pain (n = 25, 67%). There were statistically significant (p < 0.05) improvements in 7 out of 10 areas of attitudes assessed. Table 5 shows more detailed changes in comfort and attitudes, including effect sizes. The changes are considered large, defined as Cohen’s Ds of 0.80 or higher for 3 attitude statements (30%).
Table 5. Changes in Likert scale scores of participants’ self-rated comfort, and attitudes about palliative care at baseline and the end of ECHO program. (n ranged from 37 to 39 for individual questions, full data set available on request) item response options ranged from strongly agree to strongly disagree, (1 = strongly disagree to 7 = strongly agree)
a Effect size (Cohen’s D s); Effects of ≥0.8 generally suggest a large effect.
Program acceptability and satisfaction
Participants were highly satisfied with the training program, with 100% agreeing that the training was applicable to their clinical practice and stating that they would recommend the program to their colleagues. Most participants identified that the program increased their awareness of their own learning needs (86%), increased their motivation to learn more about palliative care (84%), and increased their confidence in the management of children with serious illness (81%). More than half of participants (56%) noted that they needed clinical training to be able to implement palliative care in their setting. The most common barriers to attending sessions included time pressures (n = 30, 70%) and technical challenges (n = 15, 35%). Table 6 shows additional details of participants’ experiences and challenges.
Table 6. Participants’ learning experience of ECHO program
a Includes strongly agree, agree, or somewhat agree responses.
b Multiple responses permitted.
Further palliative care training following Project ECHO
After completion of this ECHO program, many learners expressed strong interest in continuing to learn about children’s palliative care, which lead the leadership team to develop a longitudinal Community of Practice ECHO program consisting of 10 monthly online sessions on children’s palliative care. The topics for these monthly sessions were based on topics which learners had identified (Table 7). The sessions were conducted between November 2023 and July 2024.
Table 7. Topics for the monthly children’s palliative care community of practice ECHO (2023–2024), conducted following the initial 6-week ECHO course
Discussion
There is an urgent global need to address serious health-related suffering for children with life-threatening illnesses; however, palliative care which can address this suffering, remains unavailable for the majority of children in Bhutan and globally (Caruso Brown et al. Reference Caruso Brown, Howard and Baker2014; Connor et al. Reference Connor, Downing and Marston2017; Kenneson et al. Reference Kenneson, Hughes-Visentin and Wrigley2023). A lack of specialists and limited access to training remain significant barriers to the development of palliative care for children, particularly in LMICs. This study describes the successful design and implementation of an online palliative care training and mentorship program for pediatric clinicians in Bhutan, suggesting a novel approach to addressing barriers to training HCP in children’s palliative care.
The program focused on key introductory topics in children’s palliative care and was adapted to the local context. Following program completion, participants’ knowledge of children’s palliative care improved, including their understanding of how to manage pain with opioids, how to introduce palliative care, and how to explain palliative care to parents of a seriously ill child. This short training program, which was the first of its kind in Bhutan, led to further training activities in children’s palliative care, including a year-long online community of practice ECHO program, in-person training workshops, and the identification and capacity building for local palliative care champions (Laabar et al. Reference Laabar, Rayala and Doherty2024).
Previous studies have identified the positive impacts of Project ECHO training for HCPs in India, Nepal, and Bangladesh, with learners reporting increased comfort identifying children who need palliative care, managing symptoms of nausea and dyspnea, and providing end-of-life care (Doherty et al. Reference Doherty, Modanloo and Evans2021a, Reference Doherty, Rayala and Evans2021b; Lynch-Godrei et al. Reference Lynch-Godrei, Sapkota and Rowe2024). The present study identifies similar findings for participants in Bhutan, demonstrating improvements in knowledge and self-perceived comfort and attitudes toward palliative care. This study’s ECHO program was much shorter than other previously described programs (weekly sessions over 6 sessions), while previous programs had between 10 and 26 sessions. The shorter duration may explain why attitude scores improved less than in previous studies. Overall, the study findings support the use of short online program, using the Project ECHO format, to provide palliative care training in LMICs, providing clinicians with basic skills to support children with serious illnesses and their families.
Project ECHO provides didactic teaching as well as a supportive learning community where participants can share ideas and experiences about how to improve care in their setting. Participants also benefit from mentoring relationships which develop with experts from the “hub” and support changes in clinical care (Doherty and Abdullah Reference Doherty and Abdullah2024). A recent Project ECHO study in India and Bangladesh described the use of small group discussions, which occurred via online “breakout rooms,” to provide learners with more direct contact with experts, thereby consolidating their knowledge (Doherty and Abdullah Reference Doherty and Abdullah2024). These findings suggest that while online training can be beneficial, it requires particular attention to ensuring that participants have opportunities for discussion and direct interactions with faculty members.
This introductory training program was intended to raise awareness and sensitize HCP to the specific needs of children with life-threatening or life-limiting conditions, while recognizing that further training and mentorship would be necessary to support clinicians providing palliative care to children. Following this educational program, the same team developed a longitudinal mentorship and learning program intended to foster ongoing learning and practice change among participants, through a “Community of Practice” ECHO consisting of 10 monthly sessions. The Community of Practice sessions were designed to provide participants with more detailed information on topics within children’s palliative care and reinforce key practices from the introductory program.
Developing national champions to lead work on inclusion of palliative care has been identified as critical for successful efforts to develop palliative care (Callaway et al. Reference Callaway, Connor and Foley2018). Through the study’s ECHO program, 2 children’s palliative care champions, a nurse and a pediatrician, were identified. These champions have been encouraged to develop their clinical skills in palliative care, as well as understanding of other aspects critical to palliative care development, including resources and health system planning, essential medicines, health policy and financing, and education. Following the model described by Callaway et al. (Reference Callaway, Connor and Foley2018), these champions were supported to undertake more advanced clinical and leadership training (Downing Reference Downing2024; Palat et al. Reference Palat, Doherty and Brown2023). The local champions have also facilitated further training at regional referral hospitals in Bhutan and have been actively meeting with government health officials to raise awareness of the need for palliative care for children (Laabar et al. Reference Laabar, Rayala and Doherty2024). Building on the work of the national champions, a site visit from a palliative care mentor from India further supported local advocacy and training goals, meet with key government officials and conducted training seminars for local pediatric clinicians.
The present study was limited, in that it was designed to assess clinical impacts for children and their families. The study assessed learners’ outcomes at a single timepoint, immediately after course completion; however, further studies should explore the impact of the community of practice sessions and whether the changes in knowledge and attitudes are sustained over time and lead to practice change. Despite these limitations, the study presents important knowledge about the role of virtual training on clinician knowledge, comfort, and attitudes related to children’s palliative care in Bhutan, which will be useful in informing future work developing palliative care in settings where there are very few services for children at the present time.
The Project ECHO Paediatric Palliative Care Program describes a short online training program which is adapted to the local healthcare context. The course design follows a stepwise approach described in previous studies in South Asia. Notably, this educational intervention requires minimal resources and can be implemented by a small local team supported by a wider group of regional and global experts. This model can be used to leverage impact in settings where access to local palliative care training courses and experts are limited.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1478951525000033.
Availability of data and materials
Available upon request.
Acknowledgments
The authors would like to acknowledge the support of Two Worlds Cancer Collaboration Foundation, and ECHO India for their support with the implementation of this Project ECHO.
Author contributions
Conception and design: MD, ALG, SR. Provision of study materials or patients: MD, SR, TDL, PB. Collection and assembly of data: MD, ALG, SR, TDL. Data analysis and interpretation: ALG, MD, SR, PB, TDL. Drafting of the article or revising it for intellectual content: all authors. Final approval of manuscript: all authors. Accountable for all aspects of the work: all authors.
Funding
No funding was obtained for this study.
Competing interests
All authors have no conflicts to declare.
Ethical approval
This research was conducted in accordance with the World Medical Association Declaration of Helsinki. Ethical approval for the study was received from the research ethics board of the Children’s Hospital of Eastern Ontario (Approval Number 17/201X, date of approval: Dec 15, 2020).
Open access
