Measuring the Collective Community Capacity of a Network to Address Health Inequities during a Public Health Emergency: Findings from the National COVID-19 Resiliency Network

Abstract

Objective

This study assesses the collective community capacity of the National COVID-19 Resiliency Network (NCRN), a multisectoral network mitigating the disproportionate impact of COVID-19 on minoritized populations.

Methods

From January to April 2022, we used two concurrent data collection methods: a Collective Community Capacity (C3) survey (n=65) and key informant interviews (KIIs) (n=26). The C3 assessed capacity for creation of a shared vision, engagement in community change, and distributive leadership. KIIs assessed perspectives on network formation and implementation. We used a convergent design and triangulation for interpretation.

Results

NCRN has growing collective community capacity. The C3 survey found high capacity for establishing a shared mission and evidence of mutual commitment, trust, and accountability. About three-quarters of respondents strongly agreed that partners addressed social, economic, and cultural barriers related to COVID-19. Interviewees valued NCRN leaders’ openness, availability, and willingness to listen. Partners learned from one another, increased their health communication capacity, and supported sustainability. They sought greater opportunities to partner and support decision-making.

Conclusions

NCRN developed a collaborative network with a shared vision of improving health equity during and beyond the COVID-19 pandemic, while identifying areas for improvement in distributive leadership. Findings can support other organizations seeking to build collective community capacity to address equity in public health emergencies.

Racial and ethnic disparities in health outcomes during public health emergencies have been long recognized, including the disproportionate burden of illness, hospitalization, and death on minoritized populations during the 2009 H1N1 influenza pandemic and the 2015–2016 Zika virus epidemic.^1-5 Hurricane Katrina in 2005 also laid bare racial and ethnic disparities; the areas most damaged by the hurricane were largely populated by Black persons with low incomes.⁶ Since the outset of the COVID-19 pandemic in 2020, racial and ethnic minoritized groups in the U.S. have experienced significantly higher disease prevalence, hospitalization, and mortality as compared to their White counterparts.^7-10 Advocates and scholars raised awareness about the dual pandemics of COVID-19 and structural racism.^{11, 12}

The COVID-19 pandemic exposed gaps in health systems’ capacity to adequately respond to and collaborate with affected communities.¹³ This predicament surfaced despite over two decades of federal investment in healthcare coalitions (HCCs) as a cornerstone of public health emergency preparedness. HCCs involve intersectoral members—including public health agencies, hospitals, emergency medical services, and emergency management agencies—that “coordinate activities…to ensure each member has what it needs to respond to emergencies and planned events, including medical equipment and supplies, real-time information, communication systems, and educated and trained healthcare personnel.”¹³ HCCs distribute critical core functions among their members, such as governance, geographic reach, membership, training, communications, situational awareness, surge capacity, allocation of scarce resources, and alternative care facilities.¹⁴ Further, the Federal Emergency Management Agency (FEMA)’s “Whole Community” approach has long called on individuals, businesses, faith-based and community organizations, academia, nonprofit groups, and others to share responsibility in preparedness efforts, shifting focus from hospitals to communities.¹⁵

Community-driven action is considered central to promoting health equity.¹⁶ Coalitions can build collective community capacity to address health concerns and create community change.¹⁷ Positive evidence of health outcomes associated with coalitions is modest.¹⁸ On community coalition-driven interventions to reduce health disparities among racial and ethnic minoritized populations, a systematic review of 58 intervention studies published between 1990 and 2014 found that “interventions led by community coalitions may connect health and human service providers with ethnic and racial minority communities in ways that benefit individual health outcomes and behaviors, as well as care delivery systems.”¹⁸ The studies mostly focused on disparities related to chronic conditions or their risk factors (e.g., cardiovascular disease and diet or exercise), cancer, and human immunodeficiency virus (HIV); academic-community partnership was the most common type of coalition (34 studies).

There is growing evidence about the process of building collective community capacity itself. This includes the extent to which coalitions can create a shared vision and set of values, foster multisectoral collaboration, and increase communities’ capacity to shape outcomes.¹⁹ A qualitative assessment funded by the Administration for Strategic Preparedness and Response (ASPR; formerly Assistant Secretary for Preparedness and Response) characterized highly functioning HCCs as having a variety of ways of engaging partners, governing, and sustaining their efforts as they evolved and grew over time.²² Literature on the effectiveness of coalitions or networks (often used interchangeably²⁰) indicates that leadership, defined governance structures, and members’ degree of commitment are critical to their success.¹⁴ Successful coalitions require ongoing attention to relationships,²¹ and individuals or organizations involved must go beyond contractual agreements to actively drive efforts.¹⁴

However, research on coalitions addressing inequities during the COVID-19 public health emergency in the U.S. is limited. A 2022 analysis of five coalitions working to advance health equity—including the Latinx Advocacy Team and Interdisciplinary Network for COVID-19 (LATIN-19) and Black Coalition Against COVID (BCAC)—found that their success was based upon “understanding of, commitment to, and entrenchment in their population of interest.”²³ It also found that each exemplar coalition assessed their population’s needs, met their needs in innovative, relationship-building ways, and emphasized the importance and need for collaboration.

This paper assesses the collective community capacity of a multiethnic, intersectoral initiative to address COVID-19 health disparities experienced by diverse racial, ethnic, and other priority populations in the U.S.: the National COVID-19 Resiliency Network (NCRN).

The National COVID-19 Resiliency Network

In July 2020, the Morehouse School of Medicine (MSM) launched a 3-year initiative called NCRN.¹ Its intent was to build community resilience, the sustained ability of a community to withstand and recover from adversity, a cornerstone of disaster risk reduction, response, and recovery.²⁴

MSM, a Historically Black College and University, drew on decades of experience, expertise, and partnerships in community-engaged implementation, disaster response work, and health equity research and programming to form NCRN. MSM rapidly developed and launched a multiethnic/racial, multisectoral network of national, state/territorial/tribal (STT), and community-level organizations to mitigate the impact of COVID-19 on racial and ethnic minoritized, rural, and other disproportionately impacted populations, otherwise referred to as “priority populations.” Priority populations, which are intersecting, included people with disabilities, migrant and meatpacking workers, justice-involved populations, and immigrant and refugee populations.

NCRN’s goal was to disseminate culturally and linguistically appropriate information on COVID-19 and other health-care and social services to

• ▪ improve the reach of COVID-19-related public health messaging.

• ▪ increase the connection to healthcare and social services.

• ▪ decrease disparities in COVID-19 testing and vaccination rates among disproportionately impacted populations.

• ▪ enhance STT and community-level capacity and infrastructure to support response, recovery, and resilience.

NCRN’s purview was both national and local. It engaged 44 contracted partners and over 250 voluntary partners throughout the U.S.²⁵ Partners ranged from universities, national healthcare associations, and national, regional, and local nonprofits serving or representing priority populations to state and local health departments, health-care systems, quality improvement organizations, technology designers and vendors, and consultants. The network’s participatory governance structure consisted of multiple virtual (i.e., Zoom-based) advisory boards comprised of partner members. Its National Advisory Board informed overall strategy, execution, and alignment of activities. The National Community Coalition Board ensured data on community assets for COVID-19 testing, vaccinations, and other healthcare and social services were shared among network members. The Regional Community Coalition ensured partners in particular geographic areas had an opportunity to network and engage with each other. Finally, quarterly Data Partners Meeting allowed partners to discuss epidemiological trends, approaches to analyzing data, and new data platforms.

NCRN activities were wide-ranging. The network offered 37 microgrants to community-based organizations to support COVID-19 mitigation efforts in their areas and built partners’ capacity to engage in social marketing campaigns related to COVID-19 and beyond. The network developed and launched a public facing multilingual website and mobile application with search engines for health and social services, COVID-19 risk indicators, and information about NCRN and its activities and products. NCRN also developed a data repository and aggregator for registered users—namely, researchers and policy-makers. Finally, NCRN and its members conducted engagement, outreach, and education to communities around COVID-19 and related health and social services. Although NCRN was created in response to the COVID-19 pandemic, partners set the goal of it being a sustainable network that reduces health inequities beyond the 3-year grant period.

Methods

A year and a half after the network’s launch, NORC at the University of Chicago (NORC), the external evaluator for NCRN, engaged in two primary data collection efforts to better understand formation and implementation of NCRN as a participatory coalition and its outcomes: a Collective Community Capacity (C3) survey and key informant interviews (KIIs). The study focuses on the process of implementing NCRN. The primary outcome of interest is the degree of collaboration among partners and their perspectives on implementing the activities.

C3 Survey

The C3 survey assessed six composite domains of collective capacity: 1) collaboration to create and practice a shared vision, 2) measurement and use of data to guide community change efforts, 3) active engagement in community change efforts, 4) distributed leadership in equity-focused community change efforts, 5) effective and innovative community change programs, policies, and practices, and 6) infrastructure to support, sustain, and spread community change. C3 assessed each domain using 5-point Likert scales (“not at all” to “completely”) and open-ended questions. The survey was tailored to assess partners’ perspectives on NCRN-specific goals, activities, and outcomes.

From January 11 to March 1, 2022, NORC distributed the C3 Survey to 114 individuals who were part of 44 contractually involved partner organizations and 310 individuals in 251 other affiliated (not contracted) organizations or groups that participated in the network; the goal was to receive one response per organization. The overall response rate was 59% (n=26) for contracted partner organizations and 16% (n=39) for affiliated organizations, resulting in a total of 65 organizations responding to the survey.

KIIs

Concurrently, between January and April 2022, NORC conducted 60-minute, virtual, semi-structured KIIs with 38 individuals representing 26 contracted organizations. The purpose of the KIIs was to understand partners’ perspectives on the formation and implementation of the network. Domains of the RE-AIM Model,²⁶ an implementation science framework to assess reach, effectiveness, adoption, implementation, and maintenance, informed the discussion guide.

Analysis

We conducted a descriptive analysis of survey respondent characteristics, each survey domain, and its subsections. For open-ended questions, we analyzed and grouped responses according to common themes. As a supplement to the domain analysis, respondents described their frequency of collaboration with other organizations in the network, which informed a social network analysis to understand the network’s structure. For the KIIs, we used NVivo software (QSR International Pty Ltd., Melbourne, Australia) to code interview transcripts and synthesize findings based on thematic analysis. We developed an initial codebook based on the interview guide and evaluation questions and then updated it inductively based on emergent themes from reviews of the transcripts.

We used triangulation and a convergent design to conduct a mixed-methods analysis of qualitative KII findings in relation to the quantitative C3 findings.^{27, 28} The NORC analysis team considered the same topic or subject (e.g., NCRN structure and leadership, engagement in NCRN activities) from multiple data sources simultaneously.²⁹ Four study team members compared findings related to similar themes across data sources. NORC then engaged NCRN leadership and the Evaluation Advisory Board―nine individuals from NCRN partner organizations convened annually to inform the evaluation―in interpreting findings through two group discussions in May 2022.

NORC’s Institutional Review Board reviewed the study and found it to not be human subject research.

Strengths and Limitations

A strength of our study is its mixed-methods study design. We used a validated, multidimensional C3 instrument to quantify collective community capacity and understand the structure of the network. The qualitative research helped us understand the facilitators and barriers in implementing this type of network. Though the evaluation did not employ an experimental design, it was purposeful in its collection of data to answer specific questions about network implementation and its perceived outcomes.

However, there are several limitations. First, the network launched rapidly in the first months of the global pandemic that continued to evolve over time. When NCRN launched, COVID-19 vaccines did not exist, and antibody tests were limited; the primary health intervention was to deliver information about risk reduction behaviors such as masking and social isolation. As this evolved, so did NCRN messaging and activities. Second, NCRN launched amid numerous other state and federal programs aimed at similar populations with documented disparities in COVID-19; partners were often involved in parallel initiatives focused on one or more populations. Therefore, it is difficult to isolate the effects of NCRN from that of other COVID-19 initiatives.

Finally, there is potential for respondent biases; many study participants were contractually obligated to the network and reported involvement from their own perspectives. Though the C3 survey captured responses from a majority of partners (59%), findings should be interpreted with caution because not all organizations described their collaboration with others in the network. We also acknowledge nonresponse bias, as the people who did not participate in the survey or interviews were likely to be different than those who did.

Results

Partner Composition

About 40% of C3 survey respondents reported that their geographic scope was national, 32% worked within a single state or region (multistate), and 28% worked within a single or multiple local communities. C3 respondents also served a variety of priority populations, with many reaching Hispanic/Latinx, Black/African American, and rural populations (Table 1). Interviewees indicated that many local, regional, and national partners focused on particular population groups defined by race or ethnicity (e.g., Black/African American, Latinx, or Asian and Pacific Islander populations), condition (e.g., people with intellectual and developmental disabilities), or occupation (migrant agricultural workers).

Table 1. Priority populations of focus for NCRN-related work, C3 survey

NCRN Leadership

NCRN was a highly centralized network, with a degree centralization score of 0.8573, based on survey respondents’ reported frequency of collaboration with other members. A degree centralization near 1 indicates more hierarchy and less variation in the number of connections per organization. A degree of centralization near 0 implies more uniformity in the number of connections per node, and consequently, collaborative relationships are shared throughout the network. MSM served as the central connection to the partners, and the network had not formed collaborative subgroups of three or more organizations.

Interviewees offered perspectives on MSM’s role as the central connector. They noted that they valued MSM’s openness, availability, and willingness to listen and learn. They commended MSM on their transparency and approachability, from their willingness to serve as medical subject matter experts for community-facing webinars to their openness in negotiating partners’ scope of work in ways that met partners’ needs and capacities.

Degree of Collaboration

Though centralized, the C3 survey found that NCRN had a great deal of capacity for collaboration to create and practice a shared vision. A majority of survey respondents believed that network members can make a difference together (80%), believed that community members supported their efforts (68%), and trusted each other to work together (59%). They also felt that NCRN members held each other accountable (56%).

The C3 also measured distributed leadership, a key feature of creating collective community capacity that views leadership as a collaborative, shared process among partners (Figure 1).³⁰ Respondents reported that NCRN strongly (i.e., a “great deal” or “completely”) upheld many aspects of distributed leadership, including sharing leadership of network resources management (35%), NCRN meetings (35%), internal and external communications (34%), member recruitment and coordination (33%), and decision-making processes (30%). Over a third of respondents reported that they were unsure about whether NCRN had these characteristics for each option. A small portion (around 12%) reported that NCRN hardly had them, indicating room for growth in distributed leadership.

Figure 1. Perspectives on distributed leadership, C3 survey (N=64).

In terms of communicating with members, over a third (39%) of C3 survey respondents did not feel like NCRN members were well-informed about what was going on with NCRN or that NCRN members communicated openly with each other about work to address the disproportionate impact of COVID-19 on priority populations.

Diversity and Inclusion

A majority of C3 survey respondents (68%) agreed that network meetings were inclusive and that people at the community level can take advantage of network leadership opportunities. However, though NCRN attracted a diverse array of partners, several interviewees reported that NCRN was not inclusive enough in its first year. In interviews, partners explained that they were glad to learn about the work and experiences of other partner organizations, but some found it difficult to be the “only ones” of a particular racial/ethnic or other type of group. In addition, interviewees expressed they would like more opportunities to participate in events in languages other than English or required closed-captioning or sign language interpretation to participate in events.

Some interviewees appreciated MSM for learning to work with diverse partners with intersecting identities (e.g., race/ethnicity, disability, immigration status), given this was the largest and most diverse network that MSM had ever convened. One interviewee said, “Their openness to learning… I think has opened some doors that there may be some partnerships in the future…now [that] we understand each other in a different way.” However, a few interviewees called on the network to foster greater understanding of the intersectionality between priority populations. Partners also sought more meaningful engagement with other partners and additional opportunities for peer-to-peer learning.

Engagement in NCRN-Related Activities

NCRN partners were involved in partner and community engagement and in communication and dissemination of information related to COVID-19 (Figure 2). Forty-two percent of C3 survey respondents reported being involved in partner and community engagement activities. Interviewees reported conducting outreach in public spaces and door-to-door, as well as hosting workshops, trainings, and other events for community members and community health workers (CHWs). Nearly a third (30%) of survey respondents reported communications and dissemination of new and existing COVID-19-related resources that leveraged television and radio, podcasts, websites, YouTube, written materials, and social media. Interviewees noted that these communications efforts and campaigns consisted of providing culturally and linguistically appropriate messaging on COVID-19 testing, prevention, and vaccination. Sixteen percent of survey respondents reported conducting capacity-building activities like leveraging funding and resources for communication and dissemination.

Figure 2. Partners’ NCRN-related activities, C3 survey (N=65).

Research activity like conducting community needs assessments was one of the least reported NCRN activities (6%). Yet nearly 60% of survey respondents strongly agreed that NCRN had enough capacity and expertise to analyze and use data for decision-making. Several interviewees described using research-related activities, including conducting needs assessments, environmental scans, surveys, polls, and focus groups, to try to understand their communities’ needs. A few conducted their own research using NCRN data or developed manuscripts or reports to share what they were learning about COVID-19 and their NCRN efforts.

Five percent of C3 survey respondents reported conducting technology-related activities (e.g., the development of NCRN’s website or data platforms). NCRN leadership engaged partners to develop and enhance an NCRN website and mobile application to support health communications and gather information on community needs. An interviewee emphasized the importance of the network building the website’s presence, noting,

The platform, and the technology itself, is equally as important as all the partnerships and the people that are involved… And it takes the partnerships and the people to create the trust that folks will engage with this [platform]. But it’s going to take the platform to really scale all the dissemination and [get] the information out there.

Impacts and Outcomes of NCRN

NRCN’s primary goal was to disseminate culturally and linguistically appropriate COVID-19 information to priority populations. About three-quarters of C3 survey respondents (74%) strongly agreed that NCRN and its partners worked to address social, economic, and cultural barriers related to the disproportionate impact of COVID-19 on priority populations. C3 survey respondents and interviewees noted that their organizations’ community outreach, education, and dissemination efforts had broad reach among priority populations. They highlighted that NCRN connected them and their communities to reliable COVID-19 educational resources, information on health services, and research. Interviewees also mentioned that NCRN increased their own staff’s knowledge and understanding about COVID-19, vaccines, and health equity, thereby increasing their ability to educate their community members. As one interviewee noted,

[NCRN] has made us look at health equity [and social determinants of health] differently… we look at how we will address health equity, making sure everybody understands that we have a working definition of it… and what are the strategies we’ll use to help resolve or address it in our communities and in the areas that we work.

Some interviewees mentioned that NCRN’s resources enabled them to be trusted messengers within their communities. In some instances, being part of the network allowed organizations to broaden their reach or engage with priority populations in new ways. One interviewee noted

A population that we want to serve [is] really getting to know us because there’s someone that can go out to meet them and go into their space, go to places of worship, and talk about the services that we provide. That has really been helpful. It has also allowed us to have a voice in the public health space… And of course, we’re having an impact on clients.

However, though the NCRN website is available in 12 languages, interviewees noted that available materials did not meet all of their language needs and that there was a general lack of translated resources for their communities.

C3 survey respondents believed that NCRN had “a great deal” of capacity for effective, innovative community change programs, policies, and practices, though relatively less capacity to support, sustain, and spread community change compared to other C3 measures. Nearly half of C3 survey respondents (47%) agreed that the network mobilized allies successfully to advocate for policy changes. Forty-four percent of C3 survey respondents noted they had enough training and technical assistance to address the disproportionate impact of COVID-19 on priority populations. A few interviewees noted that their involvement with NCRN allowed them to expand their organizational and staff capacity and skills, particularly for developing and disseminating culturally and linguistically appropriate educational materials. However, both interviewees and C3 survey respondents reported challenges in hiring new staff and general workforce shortages within their organizations.

Some interviewees suggested that their affiliation with NCRN strengthened their credibility, increased the name recognition of their organization, and increased their visibility within communities. For example, one interviewee stated,

Because of [NCRN], we are able to have conversations about healthcare with our clients and with the community and be taken seriously… It gives us leverage for people to be able to hear us and understand and really buy into what we have to share.

Discussion

C3 survey and KII findings describe the growing collective community capacity of NCRN, a coalition formed to address both emergent and long-standing health inequities among racial and ethnic minority communities and other populations in the U.S. during the COVID-19 public health emergency. Findings show that NCRN members have the capacity to develop a shared vision, a growing interest in multisectoral collaboration among partners, and increasing capacity to shape outcomes through health communications.

NCRN actively sought to engage communities and organizations that served priority populations through multiple advisory meetings like their National Community Coalition Board. Given long-standing racial and ethnic disparities in health outcomes during public health emergencies, creating a national coalition that seeks bidirectional communication with the populations it served using a participatory approach was a novel structure in addressing public health emergencies; this approach can serve as a model for future equity-focused disaster responses and preparedness efforts.

These findings helped inform planning for subsequent years of grant funding, including further developing relationships among NCRN partners. Recognizing the importance of building collective community capacity of the network and its partners, NCRN used the results of the C3 survey and KIIs to try to enhance opportunities for cross-collaboration among partners. As it moved into its third year, NCRN sponsored meetings between partners and dedicated time for sharing lessons about how partners could collaborate, offered closed-captioning and Spanish translations for meeting participants, and dedicated more time for partners to share their work.

While this evidence indicates success in building collective community capacity in this short time frame, like most coalitions, NCRN is challenged to sustain the network to see and measure changes in health equity in disproportionately affected communities over time.¹⁷ With the federal government having ended the official public health emergency, communities of color are expected to bear the disproportionate burden of related policy changes, such as the elimination of continuous Medicaid enrollment and reduced availability of free testing, vaccination, and access to COVID-19 outpatient treatment.³¹ These changes will limit mitigation strategies for COVID-19 hospitalization and death even as transmission of disease remains high.³²

While there is promising evidence that multifaceted and multisectoral pandemic-related policies and programs could address social needs and social determinants of health, achieving health equity requires learning from specific interventions for long-standing solutions.³³ Communities will need trusted partners to help them navigate the shifting policy and service environment. More evidence is needed to account for how building community capacity in environments with overlapping interventions among populations most at risk can or do advance health equity.