Collaborative quality improvement in the cardiac intensive care unit: development of the Paediatric Cardiac Critical Care Consortium (PC⁴)

Abstract

Despite many advances in recent years for patients with critical paediatric and congenital cardiac disease, significant variation in outcomes remains across hospitals. Collaborative quality improvement has enhanced the quality and value of health care across specialties, partly by determining the reasons for variation and targeting strategies to reduce it. Developing an infrastructure for collaborative quality improvement in paediatric cardiac critical care holds promise for developing benchmarks of quality, to reduce preventable mortality and morbidity, optimise the long-term health of patients with critical congenital cardiovascular disease, and reduce unnecessary resource utilisation in the cardiac intensive care unit environment. The Pediatric Cardiac Critical Care Consortium (PC⁴) has been modelled after successful collaborative quality improvement initiatives, and is positioned to provide the data platform necessary to realise these objectives. We describe the development of PC⁴ including the philosophical, organisational, and infrastructural components that will facilitate collaborative quality improvement in paediatric cardiac critical care.

High-quality paediatric cardiac critical care remains a crucial component of efforts to optimise outcomes for congenital heart surgery patients and others with critical cardiovascular disease. To continue progress made to date with these populations, we must enhance our understanding of how critical care interventions and morbidities impact short- and long-term outcomes. Paediatric cardiac critical care practitioners and investigators need a platform to measure variability in practice, test new initiatives aimed at improving quality, and to assess cardiac intensive care unit performance through rigorous analysis of outcomes. The ideal platform would generate risk- and reliability-adjusted outcome data specific to the paediatric cardiac population that can be used to identify variability in quality and cost outcomes between hospitals, and should be accessible in real-time so that institutions can monitor their own performance. These data offer the opportunity to perform rigorous empirical science to identify the key structures and processes that underlie high-quality care at top-performing hospitals. Finally, for science to be translated into improved care, a mechanism is required to disseminate and implement evidence-based practices. The Pediatric Cardiac Critical Care Consortium (PC⁴), a quality improvement collaborative for the paediatric cardiac intensive care community, aims to provide the necessary infrastructure to achieve these objectives.

History of PC⁴

PC⁴ began in 2009 when leaders from five hospitals with cardiac critical care programmes – University of Michigan C.S. Mott Children’s Hospital, Boston Children’s Hospital, University of California-San Francisco Benioff Children’s Hospital, Lucile Packard Children’s Hospital, and Children’s Hospital of Wisconsin – collaborated to form a consortium focused on standardising data collection for cardiac critical care patients across institutions and defining quality metrics for clinical practice. The National Institute of Child Health and Human Development and the National Center for Research Resources jointly funded this effort (UL1 RR024986) through the Clinical & Translational Sciences Award network. PC⁴ initially partnered with the Virtual PICU Systems (Los Angeles, California, United States of America) who provided the database platform for this endeavour. This initial group of institutions, which eventually included Seattle Children’s Hospital, developed a new set of data variables to capture clinically relevant information on cardiac intensive care unit practice and outcomes. Over the course of the 2-year grant period, this data set was added to the Virtual PICU system platform, pilot-tested, and launched.

In 2012, the leaders of PC⁴ saw an opportunity to grow beyond its original goal to simply produce a clinical registry; a total of seven new centres were recruited to join this developmental process, bringing the total to 12. Participants believed that patients and hospitals should receive a greater return on investment from the resources devoted to data collection. As such, PC⁴ refocused on the organisation philosophically, scientifically, and operationally to become a data-driven quality improvement collaborative in the model of other successful programmes in adult cardiac surgery and inpatient specialty care. To do so, the PC⁴ leadership recognised that a new and innovative data platform was necessary to achieve these objectives. Subsequently, the clinical leaders from the core centres voted unanimously to form a platform de novo in a public–private partnership with two health-care software vendors.

This new database, created by CardioAccess, Incorporated (Fort Lauderdale, Florida, United States of America), and ArborMetrix, Incorporated (Ann Arbor, Michigan, United States of America), was unveiled in December, 2012, and after an extensive testing phase hospitals began submitting data in July of 2013. Currently, 19 institutions participate in PC⁴, with several others set to join in 2014 (Fig 1). Centres currently pay ~$13,000 (United States dollars) per year for the software to submit data and access the analytic and reporting framework. Figure 2 shows the rate of case accrual in the registry at the time of manuscript submission.

Figure 1 Map showing location of current PC⁴ participants.

Figure 2 Cardiac intensive care encounters accrued by month.

Quality improvement philosophy

PC⁴ was developed to follow the blueprint of successful collaborative quality improvement pioneers. The Northern New England Cardiovascular Disease Study Group (adult cardiac surgery) and several statewide Michigan collaboratives for surgery and inpatient acute care ^{1 , 2} have demonstrated the incredible potential of collaborative quality improvement to improve quality and value of care delivery. The key components of these successful collaborative quality improvement initiatives include:

• ∙ Maintenance of a detailed and relevant clinical registry tracking practice and outcomes.

• ∙ Timely performance feedback to clinicians.

• ∙ Collaborative learning among participants with transparent data sharing.

Several studies have demonstrated the benefits of this approach. In 1996, the Northern New England Cardiovascular Disease Study Group published results showing that mortality after coronary artery bypass graft surgery decreased almost 60% over a decade for hospitals participating in their cardiac surgery collaborative. ¹ The Michigan collaboratives, a joint venture between hospitals and Blue Cross/Blue Shield of Michigan, replicated the Northern New England group’s results and successfully reduced morbidity, mortality, and health-care costs across a different region of the country, and across other medical and surgical specialties. ² There are also examples of successful collaborative quality improvement in paediatric subspecialty care, ³ and new programmes in paediatric cardiac care have been recently established, ⁴ though none focusing primarily on cardiac critical care. In this paper, we describe the key elements of PC⁴ that have been carefully planned to apply the lessons learned from these successful collaborative quality improvement initiatives to patients with critical cardiovascular disease (Box 1).

Box 1 PC⁴ mission statement

PC⁴ is a multi-institutional collaboration committed to investigation through transparent sharing of data. We will partner with professional organisations across geographic and subspecialty boundaries to integrate with existing databases and harmonise our common efforts. We seek to advance paediatric cardiac intensive care medicine through critical evaluation of data, identification of evidence-based practices, and public dissemination of this information.

PC⁴ organisational structure

Executive committee and sub-committees

The PC⁴ organisational structure consists of an Executive Committee, sub-committees, and the data-coordinating centre. The Executive Committee is made up of nine members. In addition to the Executive Director, there are four members who serve as committee chairpersons, and four members at-large. Each of the following organisations/groups is represented by at least one Executive Committee member:

• ∙ The Pediatric Cardiac Intensive Care Society Board of Directors.

• ∙ The American College of Cardiology Quality Work Group cardiac intensive care unit subcommittee.

• ∙ The Congenital Heart Surgeon’s Society Quality Committee.

• ∙ The Society of Thoracic Surgeons Congenital Heart Surgery Database Task Force.

There are four primary sub-committees: Database, Scientific Review, Finance, and Communications (Fig 3). In addition, there is a Publications and Presentations committee, as well as one for Operations and Procedures that works under the auspices of the data-coordinating centre.

Figure 3 PC⁴ organisational structure.

Data-coordinating centre

The PC⁴ data-coordinating centre is housed at the University of Michigan and supported by resources from the Michigan Congenital Heart Outcomes Research and Discovery Program. There is a clinician-scientist Director of the data-coordinating centre, a person with advanced research training must fill this position. Under the Director’s leadership are a dedicated project manager, database manager, and a dedicated analyst. The data-coordinating centre directs the Operations and Procedures Committee, with conceptual oversight from the Executive Committee, to set the policies governing data submission and management, privileges of participants to use the data, and several other Consortium activities. Funding for the data-coordinating centre comes from the University of Michigan Department of Pediatrics and Communicable Diseases and philanthropic donations.

Organisation of the PC⁴ registry

Overview

PC⁴ has created a comprehensive relational database to capture characteristics and outcomes of patients cared for in the cardiac intensive care unit and populate the clinical registry. The content of the database was created through an iterative process by a multi-institutional expert panel of cardiac intensivists and surgeons. The purpose of the registry is to allow hospitals to monitor performance benchmarked to peer institutions within the consortium, to serve as a foundation for quality improvement science, and to function as a platform with which to conduct hypothesis-driven research projects.

Patient population

The PC⁴ registry is designed to capture data on all patients under the care of the cardiac critical care team, including patients physically located within a cardiac intensive care unit as well as those in other locations (neonatal ICU, general paediatric ICU, etc.), where the cardiac critical care attending physician is primarily responsible for a patient’s care. Data are captured on both medical and surgical patients, and information is obtained on all cardiac intensive care unit encounters during the hospitalisation in the event where a patient has multiple cardiac intensive care unit admissions during the same hospital stay. The inclusion of medical patients and the detailed data collected on the postoperative care during surgical admissions are important complements to congenital cardiac surgical databases.

Data collection

Demographics, patient characteristics, diagnoses and comorbidities, procedures performed, and outcome data including complications, length of stay, and mortality are collected in the PC⁴ registry. Definitions for variables are specified within the PC⁴ Data Definitions Manual, and only trained data managers who have completed a certification examination enter data. New definitions were created for variables unique to the PC⁴ registry, and existing definitions from related database projects were adopted for shared fields; PC⁴ shares common International Pediatric and Congenital Cardiac Code nomenclature, ⁵ as well as definitions utilised by the Society of Thoracic Surgeons Congenital Heart Surgery Database and American College of Cardiology IMPACT Registry^®.

Data are entered into a commercially available online data entry platform designed specifically for PC⁴ by certified vendors. The current software provides an integrated data entry platform for multiple congenital heart disease registries, such that the variables shared across multiple registries need to be entered only once at the participating sites. This reduces the data entry burden and maximises accuracy across registries. To enhance the value of the real-time feedback provided by PC⁴ (see below), the sites are required to submit cases within 30 days of discharge.

The registry is primarily for quality improvement activities locally and collaborative-wide; each participating hospital’s institutional review board determines the degree to which it will oversee data collection. Written informed consent on the part of patients or families is not required by PC⁴, but is at the discretion of the local institutional review board.

Risk and reliability adjustment

An accurate risk adjustment method specific to the cardiac critical care patient is necessary for benchmarking outcomes, highlighting areas in need of quality improvement and comparative outcomes for research purposes. Current risk stratification systems used primarily for general, non-cardiac populations of critically ill children have yielded unsatisfying results when applied to a cardiac intensive care unit population, particularly for surgical patients. ⁶ Therefore, PC⁴ has elected to develop and test new methodologies. Specific variables are collected for medical and surgical patients and separate models will be constructed for these two populations. When the risk adjustment methodology has been validated after an adequate number of cases are accrued, the PC⁴ platform will report real-time, risk-adjusted data, allowing sites to assess their outcomes on a continual basis. In addition, estimates will be reliability adjusted, which is a technique that allows for stabilisation of estimates particularly in situations where there are small sample sizes. ⁷

PC⁴ audit methodology: ensuring data quality and accuracy

PC⁴ member institutions and the data-coordinating centre are committed to providing the highest-quality data for comparative reporting and scientific discovery. The philosophy of the collaborative is that inter-reliability testing, while certainly valuable, is insufficient alone for ensuring data accuracy. Therefore, PC⁴ has developed an audit process to perform source data verification with data collection teams and clinicians across institutions. Every PC⁴ site will be audited within the first year of data collection, and at least every 2 years thereafter; four hospitals have already been audited at the time of press. Random cases are selected by the data-coordinating centre and reviewed using a mixed methodology of secondary chart abstraction by the coordinating centre personnel and source data verification with local collection teams. Reports are provided back to the participating sites, and if the data do not meet pre-specified standards for accuracy they are removed from the comparative reporting platform until cleaned and resubmitted. Most importantly, these audits include a review of third-party information, for example hospital billing logs, to verify that all eligible cases have been submitted to the registry.

Feedback to participating sites and benchmarking

A unique aspect of PC⁴ is that the participating sites have 24/7 access to real–time information on patient characteristics and outcome data, including comparative analytics showing benchmark data from the consortium for multiple outcome metrics. PC⁴ has partnered with ArborMetrix Inc. to provide the participating sites with this enhanced access to information, which is available on the PC⁴ web site (www.pc4quality.org). Data are updated on a continual basis as cases are closed and submitted at the site. This allows the sites to better track current outcomes and quality metrics, rather than receiving data 6–12 months after data submission, as is common with many registries. In addition, the PC⁴ dashboard is customisable and allows stratification by time period, age group, and diagnostic and procedural categories. In addition to receiving comparative analytics at the hospital level, the platform allows users to “drill down” on their centre’s individual patients, as well as groups of patients with a specific procedure, diagnosis, or complication over a specified time period.

PC⁴ leadership believes that transparent sharing of data is essential to facilitate quality improvement and collaborative learning. Therefore, within the consortium, the participating sites’ centre-level data are unblinded once an appropriate threshold number of cases are accrued and submitted. Centre identity will be accessible to specified clinicians and administrators within each institution, discussed annually at the in-person, collaborative-wide meeting, and in relation to any proposed or ongoing quality improvement initiatives. The purpose of allowing centres within the consortium to identify one another is to stimulate local-quality improvement: participants who desire to improve on a particular quality metric can ascertain which hospitals are achieving the best outcomes on that metric and initiate contact to exchange ideas. Unblinded data may not be shared outside the consortium, and sites that do not meet audit requirements for accuracy or timeliness of data submission will have access to unblinded data suspended.

Early quality improvement and research initiatives

Mechanisms underlying variation in outcome across hospitals

Although several studies have documented that mortality following congenital heart surgery vary widely across paediatric heart centres, ^{8 – 11} little is known regarding underlying mechanisms. Moreover, less is known about outcomes in non-surgical patients. The Northern New England Cardiovascular Disease Study Group’s effort to reduce mortality after coronary artery bypass graft surgery illustrates how understanding clinical epidemiology drives quality improvement. They developed a unique approach to classifying the reason for a patient’s death, which they termed the “mode” of death, by determining the seminal complication that either directly caused mortality or started a sequence of complications – the first “domino” to fall – that ultimately led to the patient’s demise (Fig 4). This approach differs from the traditional cause of death analyses that typically evaluate the last complication in a series of events, and has been explored previously in the congenital heart surgery literature. ¹² Contrary to the final cause of death, identifying common seminal complications creates opportunities for prevention, recognition, and intervention quality improvement strategies.

Figure 4 Conceptual model of postoperative complications leading to mortality.

The Northern New England collaborative described large variation in risk-adjusted mortality rates after coronary artery bypass graft surgery between the 18 surgeons at five centres, and determined that 80% of overall variation in surgeon performance was attributable to a single mode of death: low-output cardiac failure. ¹³ These results informed a successful search for the important structures and processes of care linked to this seminal complication, and served as the foundation for quality improvement initiatives that were successful in reducing morbidity and mortality within the consortium.

Within PC⁴, an ongoing project is focused on delineating the sequence of events and seminal complications leading to death in our patient population through analysis of all surgical mortalities at two PC⁴ centres over a 5-year time period. PC⁴ investigators have developed a method for identifying the mode of death from chart abstraction using the Northern New England collaborative’s framework. For future multi-centre analyses, complications collected in the PC⁴ database are all date and time stamped; this allows recreation of the timeline of clinical events on all patients in the registry. These data have the potential to elucidate the epidemiology of variation across hospitals, and will provide insight on how low-mortality centres achieve their results. For instance, they may shed light on whether the highest-performing hospitals prevent seminal complications, or more effectively rescue patients who have that complication, or both. These data will hopefully lay the foundation for subsequent strategies aimed at improving quality among all participants.

Facilitating observational research

The richness of the PC⁴ core registry elements combined with risk- and reliability-adjusted outcomes make the database ideal for multi-institutional observational research. However, no database can capture all the necessary variables to answer every research question that may arise. Therefore, PC⁴ has developed tools to facilitate these types of research studies within the consortium, building on the success of these methods in other fields. The PC⁴ data-coordinating centre is able to create and implement separate research modules for a finite period of time that allow collection of additional data elements by all or a subset of PC⁴ sites related to a specific research study. These data are submitted directly to the data-coordinating centre by the participating sites and can then be combined with the core registry data on patient characteristics, practices, and outcome for subsequent analysis. Data collection resources can be focused on those elements specific to the research question under study, without having to collect basic demographic and outcome data on patients eligible for the study.

PC⁴ demonstrated the efficiency of such an approach with its first research study. ¹⁴ In all, four institutions collaborated to collect detailed data in a module on pharmacologic cardiovascular support in an effort to validate the vasoactive–inotropic score as an intermediate predictor of eventual clinical outcome. Data were collected and combined as described above. This module was created in REDCap ¹⁵ and managed by the data-coordinating centre, whereas regular registry elements were collected by the usual methods according to standard operations and procedures. Patients were accrued quickly with few additional human resources needed to complete the data collection for the study, thus minimising the costs of the study as well; a total of 391 patients were entered into the cohort over 6 months. This efficient approach should allow multiple prospective observational studies to be conducted at once without excessively burdening each participating institution.

Summary

Improving the quality of care to paediatric and young adult patients with critical cardiovascular disease depends on collaborative efforts among institutions working towards common objectives. PC⁴ has been developed philosophically and organisationally to follow the roadmap previously laid out by successful collaborative quality improvement pioneers. Transparency, introspection, and a commitment to rigorous science are the philosophical cornerstones of our approach, and the participating institutions have committed to these principles completely. PC⁴ is designed to facilitate discovery that will identify high performance and uncover how that level of performance is achieved using innovative research methods and collaborative learning.