A comparison of ECG-based home monitoring devices in adults with CHD

Abstract

Background:

Various electrocardiogram (ECG)-based devices are available for home monitoring, but the reliability in adults with CHD is unknown. Therefore, we determined the accuracy of different ECG-based devices compared to the standard 12-lead ECG in adult CHD.

Methods and results:

This is a single-centre, prospective, cross-sectional study in 176 consecutive adults with CHD (54% male, age 40 ± 16.6 years, 24% severe CHD, 84% previous surgery, 3% atrial fibrillation (AF), 24% right bundle branch block). Diagnostic accuracy of the Withings Scanwatch (lead I), Eko DUO (precordial lead), and Kardia 6L (six leads) was determined in comparison to the standard 12-lead ECG on several tasks: 1) AF classification (percentage correct), 2) QRS-morphology classification (percentage correct), and 3) ECG intervals calculation (QTc time ≤ 40 ms difference). Both tested AF algorithms had high accuracy (Withings: 100%, Kardia 6L: 97%) in ECGs that were classified. However, the Withings algorithm classified fewer ECGs as inconclusive (5%) compared to 31% of Kardia (p < 0.001). Physician evaluation of Kardia correctly classified QRS morphology more frequently (90% accuracy) compared to Eko DUO (84% accuracy) (p = 0.03). QTc was underestimated on all ECG-based devices (p < 0.01). QTc duration accuracy was acceptable in only 51% of Withings versus 70% Eko and 74% Kardia (p < 0.001 for both comparisons).

Conclusions:

Although all devices demonstrated high accuracy in AF detection, the Withings automatic algorithm had fewest uninterpretable results. Kardia 6L was most accurate in overall evaluation such as QRS morphology and QTc duration. These findings can inform both patients and caregivers for optimal choice of home monitoring.

CHD is the most common congenital disorder, affecting about 0.8% of newborns. ¹ Over the past decades, life expectancy of children born with CHD has improved considerably. Nonetheless, patients are at risk for arrhythmias such as atrial fibrillation due to both residual lesions after surgical repair and acquired heart disease. ² Arrhythmias impact quality of life and are a major cause of hospital admission and mortality in ageing adults with CHD. ^3–5 As silent atrial fibrillation in adult CHD patients may precede stroke, early diagnosis, and optimal treatment is warranted. ⁶ However, atrial fibrillation is often undiagnosed and untreated as the heart rhythm is not regularly monitored and many patients remain asymptomatic. ^1,2,7–9 Therefore, continuous or intermittent rhythm monitoring in asymptomatic patients may facilitate early diagnosis. ^10–13

In recent years, the sale of wearable devices has increased and is expected to reach 520 million units by 2025. ¹³ Many devices are able to record an ECG, using different configurations such as lead I ECG, precordial ECG, and six-lead ECG. In particular, young adults with CHD are interested in the use of these devices and are in need for lifelong follow-up. ^7,14 In healthy individuals and in patients with acquired heart disease, wearable devices that monitor and classify the heart rhythm are becoming mainstream diagnostic tools. However, in the adult CHD population, ECG interpretation may be more difficult due to the wide variation in both heart rhythm, QRS morphology such as right bundle branch block (RBBB), and ECG intervals. Moreover, ECG-based devices use algorithms for AF detection that were only validated in the general non-CHD population. ¹⁵ Assessment of reliability of ECG-based devices in the adult CHD population is needed in order to inform both patients and caregivers in options and reliability of home-monitoring devices. ^16,17

Therefore, we aimed to determine the accuracy of different ECG-based devices (lead I, precordial lead, six leads) to detect atrial fibrillation, QRS morphology, and ECG intervals compared to the standard 12-lead ECG in adults with CHD.

Materials and method

Study design and population

In this prospective, cross-sectional study, consecutive patients from the outpatient adults CHD clinic at the Amsterdam University Medical Center (UMC), location Academic Medical Center, were invited to participate. Inclusion criteria were: 1) presence of CHD (classified into simple, moderate, or complex lesions ¹⁸ ), 2) age ≥ 16 years old (patients transferred from paediatric to adult care were eligible), and 3) standard 12-lead ECG acquired at day of visit. Patients with cognitive impairment who were not capable to give informed consent were excluded. The conduct of this study was approved by the medical ethical board of Amsterdam University Medical Center and complies with the declaration of Helsinki.

All patients were instructed to use the three devices in accordance with manufacturer recommendations by a research coordinator (LP), complying with local Covid-19 restrictions. Briefly, the Withings Scanwatch (Withings, Issy les Moulineaux, France) was used to acquire a 30 s lead I ECG while worn around the left wrist. The Eko DUO (Eko Health, Oakland, United States of America) was used to acquire a 30 s precordial ECG, using a mid-sternal position (level of fourth intercostal space, deferring to left if insufficient ECG quality). Finally, patients were instructed to hold the Kardia Mobile 6L (Kardia 6L) (AliveCor, Mountain View, United Statesof America) on the skin of left knee or alternatively left ankle with both thumbs to acquire a six lead (standard limb leads: I, II, III, aVF, aVR, aVL) ECG. Each device recorded an ECG for 30 s with an iPhone X (Apple Inc., Cupertino, United Statesof America) using the standard manufacturer applications for each of the devices. A maximum of three attempts were performed to acquire the ECG by each device. In patients using the Kardia 6L a final attempt was performed using it as lead I (using both hands) ECG if the six leads were not recorded.

The standard 12-lead ECG was acquired in supine position with CardioSoft V6.73 ECG system (GE Healthcare). The 12-lead ECG was evaluated by a single experienced observer (JPB) in collaboration with treating adult CHD specialists. All device ECGs were anonymised and interpreted blinded to other ECGs or clinical data by the same observer (JPB).

For Withings and Kardia 6L, the automatic rhythm evaluation was classified into normal, atrial fibrillation, or uninterpretable (algorithm interpretation: uninterpretable, high heart rate, or low heart rate). The Eko DUO automatic algorithm was not available in our study. In addition, all device ECGs were classified for rhythm (atrial fibrillation, normal/other) by the single observer (JPB) whenever QRS complexes were identified. The QRS morphology was classified into normal (<120 ms), right bundle branch block (RBBB) (≥120 ms), and non-RBBB wide complex (≥120 ms, not meeting RBBB criteria). RBBB was defined as ≥120 ms and terminal negative S-wave in lateral leads (lead I or aVL, when available) and/or RSR’ pattern in lead V1, aVR, or the precordial ECG (when available). Furthermore, all device ECGs were manually evaluated on ECG intervals, whenever PR, QRS, and QT were identified. The percentage of identified PR, QRS, and QT intervals was reported. The QT corrected (QTc) for heart rate was calculated using Bazett formula. The device ECGs were rated in quality using a 5-point Likert Scale (very poor, poor, moderate, good, and excellent ¹⁹ ). Finally, the participants were asked two survey questions on device preference and willingness to use devices in daily life using a 5-point Likert Scale. All included patients provided informed consent for the anonymised use of their data. Additional clinical information was obtained from electronic hospital charts.

Definition of outcomes

The co-primary outcomes were accuracy of the devices compared to the 12-lead ECG on: 1) atrial fibrillation (AF) detection (both device algorithm and independent evaluation), 2) QRS morphology (normal, complete right bundle branch block (RBBB), non-RBBB wide complex), and 3) ECG intervals (PR, QRS, QTc in milliseconds). For QTc, a difference of >40 ms compared to the gold standard was considered clinically unacceptable, as QTc prolongation of >40 ms constitutes a serious arrhythmic risk and is beyond normal variation. ²⁰ The secondary outcome measures were to determine the quality of the ECGs from the devices on a 5-point scale and to determine if patients were willing to use devices in daily life.

Statistical Analyses

Demographic variables are presented using descriptive statistics, using number with percentage, mean with standard deviation, or median with interquartile range, as appropriate. Sensitivity and specificity of AF detection compared to the gold standard 12-lead ECG were calculated using 2 × 2 contingency tables. McNemar test for paired dichotomous data was used to compare the frequency of uninterpretable algorithm results for Withings and Kardia 6L. In addition, McNemar test was used to compare the proportion of correct QRS morphology classification (correct/incorrect) and identification (yes/no) of ECG intervals of different devices. Bland-Altman plots and Pearson correlation were used to compare the devices with the 12-lead ECG for ECG intervals (normally distributed). Independent samples T-test was used to compare ECG intervals between devices and gold standard. Wilcoxon signed rank test was used to compare ECG quality on the 5-point Likert scale. Head-to-head 1:1 comparisons between device preference (non-paired dichotomous) were made using the Chi-square test. A p-values of <0.05 was considered statistically significant. Analysis was performed using SPSS software (version 26).

Results

A total of 222 patients were screened for eligibility from October through December 2020 and 176 patients (54% male, age 40 ± 17 years) enrolled in the study (Supplementary Figure 1). Of all patients, 32% were classified as simple CHD, 43% moderate CHD, and 24% severe CHD. A total of 84% patients had previous CHD-related interventions and 11% had a pacemaker or implantable cardioverter defibrillator (ICD). About three-quarter of the patients were in NYHA class I. Demographics and clinical characteristics are summarised in Table 1. In all patients, the Withings Scanwatch and the Eko DUO ECG were recorded. In one patient, Kardia 6L failed to record an ECG, and in 15 patients, the Kardia was used to acquire a single lead ECG because the 6 lead ECG could not be recorded. A global overview of the study results is provided in Figure 1.

Figure 1. Overview of the study results on different study outcomes. Statistical significance is indicated based on P-value < 0.05 of head-to-head comparisons between devices (see results for detailed comparisons). P-value < 0.05 for Eko DUO compared to Kardia 6L for QRS morphology although no statistically significant difference of both devices compared to Withings. The AF algorithm of Eko DUO was not available in this study. For the other devices, the percentage of rhythm assessed is reported in addition to sensitivity and specificity for AF detection. P-wave detection (%) indicates the assessment of PR-interval by physician evaluation irrespective of accuracy of PR-interval. QTc difference (%) indicates the proportion of device ECGs with deviation of more than 40 ms in QTc (assessed by physician) compared to the gold standard 12-lead ECG. QRS morphology correct (%) indicates the correct classification of QRS morphology (normal (<120 ms), right bundle branch block (≥120 ms), and non-RBBB wide complex (≥120 ms)). Patient preference for device was asked to all patients, the remainder had no preference. Good or excellent quality of devices indicates score 4 or 5 on the Likert-scale (1–5). P-value < 0.05 for Eko DUO compared to Kardia 6L for QRS morphology. Abbreviations: AF: atrial fibrillation, ECG: electrocardiogram.

Table 1. Baseline characteristics

Data are described as number with frequency or mean (± standard deviation). Anti-arrhythmic include: sotalol, amiodarone and digoxin. Abbreviations: CHD: congenital heart disease, NYHA: New York Heart Association, ASD: atrium septal defect, VSD: ventricular septal defect, AOV: aortic valve, ICD: implantable cardioverter defibrillator, SVT: supraventricular tachycardia

Atrial fibrillation detection

A minority of six patients (3.4%) had AF on the 12-lead ECG. Overall, 54 (31%) of Kardia 6L ECGs were inconclusive by the algorithm compared to only 9 (5%) of Withings (p < 0.001) (Table 2). The uninterpretable ECGs by the Kardia algorithm had longer QRS duration (mean = 137 ± 34 ms) compared to interpretable ECGs (mean = 104 ± 24 ms, p < 0.001).

Table 2. Device heart rhythm classification compared to 12-lead ECG

Overview of hearth rhythm detection of device algorithm and independent reviewer interpretations from devices compared to readings interpreted from 12-lead ECG. Possible non-AF heart rhythms interpreted from the 12-lead ECG: sinus rhythm, atrial tachycardia/flutter (regular), atrial paced, ventricular paced, ventricular rhythm or nodal rhythm. (A/B) Comparison of device AF-algorithm, 12-lead ECG interpretations and device ECG interpretations by independent reviewer. (C) Comparison of device interpretations and 12-lead ECG interpretations. Data are described as frequency. The independent reviewer was unable to assess rhythm on Eko DUO in one patient due to poor ECG quality. Abbreviations: AF: atrial fibrillation, ECG: electrocardiogram, SR: sinus rhythm

In ECGs that were classified by the device algorithm, the Withings algorithm correctly identified AF with 100% sensitivity and 100% specificity. The algorithm of Kardia 6L had 100% sensitivity and 97% specificity. The independent in-person evaluation identified all patients with AF correctly on all devices, with only few false-positive interpretations compared to the 12-lead ECG (sensitivity = 100% for all, specificity: Withings = 98% and Kardia 6L = 99%, Eko DUO = 99%).

QRS morphology

The results of the classification of QRS morphology of the devices and 12-lead ECG are listed in Table 3. The assessment of QRS morphology was classified as normal (68%), RBBB (24%), or other/wide (7%) on 12-lead ECG. Patients with tetralogy of Fallot (14 of 20, 70%) or pulmonary atresia with ventricular septal defect (7 of 8 patients, 88%) were likely to have complete RBBB. Patients with simple CHD (8 of 57, 14%) or patients without CHD repair (2 of 29, 7%) were unlikely to have complete RBBB. The physician evaluating Kardia 6L correctly classified QRS morphology in 159 patients (90% accuracy), significantly more compared when using Eko DUO (84%, p = 0.03), and similar to Withings (89%, p = 0.63).

Table 3. QRS morphology assessment compared to 12-Lead ECG

Overview of QRS morphology interpretations from devices compared to 12-lead ECG readings. One patient in which Kardia 6L ECG was not acquired was classified as no assessment. Abbreviations: ECG: electrocardiogram, RBBB: Right bundle branch block

ECG intervals

An overview of ECG intervals comparison is provided in Supplementary Table 1. The first set of analysis examined the percentage of assessed intervals, in which P-wave, QRS-complex, and QT-interval could be identified on the device ECG. The PR interval and QT interval could be identified and assessed more frequently in Kardia 6L (PR = 89%, QT = 97%) compared to Withings (PR = 70%, QT = 84%) and Eko DUO (PR = 68%, QT = 82%) (p < 0.001 for all). Overall, in all devices, comparable PR intervals were measured compared to the 12-lead ECG. QRS duration was assessed more frequently using Kardia 6L (99%) compared to Eko DUO (95%, p < 0.01). On average, QRS and QTc duration were underestimated on all devices (Supplementary Table 1B). In the Withings ECG, QTc-interval was more frequently (49%) over- or underestimated by more than 40 ms compared to both Eko DUO (30%) and Kardia 6L (26%) (p < 0.001 for both comparisons). The Bland-Altman plot for QTc-interval of Kardia 6L compared to the 12-lead ECG illustrates limits of agreements, which were independent of QTc-interval (Supplementary Figure 2). When analysing ECGs rated good or excellent in quality, devices had more robust correlation on QRS duration and QTc duration compared to the 12-lead ECG (Supplementary Table 1C).

Secondary outcomes

A total of 14 (8%) recordings were rated as very poor (grade 1 in 1–5 scale) by the independent reviewer (Supplementary Table 2). Overall, 51% of Withings recordings were of good or excellent quality as compared to 74% of Kardia and 70% of Eko DUO. There was no significant difference in quality on the 5-point scale between Kardia 6L and Eko DUO (p = 0.31). ECG quality of Withings was rated worse compared to Kardia 6L and Eko DUO (p < 0.01 compared to both).

A total of 140 patients (80%) responded that they were willing to use a device (Likert score = 4 or 5) in daily life. In addition, 13% of patients were in doubt (Likert score = 3) and 7% were unlikely to use any device (Likert score = 1 or 2). The majority of participants (54%) reported a preference for the Withings Scanwatch (p < 0.001, compared to Eko DUO and Kardia 6L). Fewer patients preferred Eko DUO (11%), Kardia 6L (23%), or had no preference (10%).

Discussion

This is the first study to compare accuracy of different ECG-based home monitoring devices compared to the 12-lead ECG in adults with CHD. We prospectively included a large and varied adult CHD population which reflects the broad spectrum of academic patients. Our result indicates that different methods of ECG acquisition translate into differences in accuracy that may inform the choice of home monitoring.

Although only a small subset of patients had atrial fibrillation in this cross-sectional study, our findings suggest that both algorithms and independent evaluation were accurately able to detect AF, with few false-positive results. The most notable difference between the Kardia 6L and Withings algorithm for AF detection was a higher rate of uninterpretable ECGs by Kardia 6L. In rhythms that were qualified, both algorithms achieve a high accuracy. However, results on sensitivity need to be interpreted with caution due to the low number of patients with AF. Reassuringly, independent evaluation of the ECGs revealed that all cases of atrial fibrillation were accurately diagnosed on all devices and only few ECGs were falsely classified as atrial fibrillation. Previous studies reported a lower percentage of uninterpretable ECGs of the Kardia algorithm. ^15,21–23 Although the exact specifics of the algorithms are unknown, during our study the Kardia algorithm used only information from lead 1 in the algorithm. It is conceivable that the abnormal QRS morphology in CHD patients contributes to a higher percentage of uninterpretable ECGs, as suggested by the longer QRS duration in uninterpretable ECGs. Previous studies reported sensitivity and specificity in the range of 82–94% using the AF algorithm by Kardia in a non-CHD population. ^15,21 Overall, devices and the algorithms may facilitate early atrial fibrillation detection in high-risk patients but are mostly hampered by uninterpretable ECGs. If in doubt, the patient should be advised to record another ECG, or to visit the (outpatient) clinic for a 12-lead ECG.

We evaluated the accuracy of various ECG-based devices on several other parameters relevant to the adult CHD population, although the devices were not designed or marketed for this purpose. The assessment of QRS morphology and P-wave is relevant in the adult CHD population as this is needed for classification of other arrhythmias such as supraventricular arrhythmias or ventricular tachycardia. Using Kardia 6L, we accurately classified QRS morphology into the three prespecified categories in 90%, more compared to Eko DUO. These results might be influenced by variation of QRS duration measurement on the devices which was required for QRS morphology classification. The small variations of Eko DUO precordial placement combined with anatomic variation could have influenced QRS morphology assessment. In addition, ECG intervals were assessed more frequently on Kardia 6L compared to Withings and Eko DUO. These results can be explained by both the six leads, which allow identification of the P-wave, QRS morphology, and QT-interval in more patients, and the good overall ECG quality. Overall, correlations with 12-lead ECG intervals were best for Kardia 6L although measurement of QTc-interval was still over- or underestimated by more than 40 ms in 26% of patients, a clearly unacceptable margin in clinical practice. Analysis restricted to high-quality ECGs found a better correlation but results need to further improve for accurate follow-up of QTc intervals. Likely, artificial intelligence will improve assessment of the QT interval. ²⁴ When comparing device measurements with the standard 12-lead ECG, all devices underestimated QTc and QRS duration. These results are in agreement with a study by Gropler et al. ²⁵ in 30 paediatric patients which showed that Kardia Mobile (lead I) underestimated QTc-interval and QRS duration in the majority of patients.

All device ECGs were classified into five different quality categories, based on the conceptual framework proposed by Liu et al. ¹⁹ The simplified system of classification helps to distinguish between the quantity of artefacts and noise. All devices are sensitive to poor skin contact, sensor placement, and motion artefact (muscle tremor and arm movement). ^25,26 Although Withings was strapped around the wrist, suggesting less movement, we frequently observed a noise in baseline, which was reflected by the overall lower ECG quality. Around 80% of patients were willing to use a smart device at home, similar to previous studies. ^7,27

Overall, considering patient preferences and the ability of the algorithm to rule out atrial fibrillation in a large proportion, Withings Scanwatch could be considered for AF screening in high-risk patients. In addition, Withings Scanwatch offers continuous photoplethysmography screening which could facilitate early AF detection in asymptomatic patients. ¹¹ Kardia 6L may be preferred to screen for arrhythmias in patients with intermittent symptomatic palpitations. Kardia 6L provides good overall quality and the six leads improve detection of P-waves or change in QRS morphology, this may better facilitate detection of non-sustained ventricular tachycardia or classify various supraventricular tachycardias. There is limited literature on at-home detection of non-AF arrhythmias but our results suggest the six lead ECG is likely more accurate than a single-lead ECG. ²⁸ Eko DUO acquired a precordial ECG with simultaneous auscultation and could be useful for AF detection when interpreted by a physician but its ECG had no clear advantages over lead I or six leads. Importantly, current devices are not advised to evaluate patients for unexplained syncope as ECGs are only recorded when activated by the patients. Availability of devices and AF algorithms studied may vary between countries.

Study limitations

This study was limited by the small proportion of patients with atrial fibrillation. Our study did not specifically target this group and further studies are needed to validate the AF algorithms in an adult CHD AF population. We could not directly assess reliability to detect ventricular arrhythmias or other types of supraventricular arrhythmias in this cross-sectional study but assessed accuracy on various parameters such as QRS morphology that are required for this purpose. We used all devices and the 12-lead ECG during the same patient visit (within minutes) although not simultaneously. Furthermore, we did not evaluate the devices during follow-up to determine yield of rhythm monitoring or to assess reliability of photoplethysmography to facilitate early AF detection. Finally, Eko DUO offers cardiac auscultation including murmur algorithms which we did not evaluate in this study, similar to the AF algorithm by Eko DUO which was not available.

Conclusions

This is the first study to compare the accuracy of ECG-based devices in adults with CHD. Overall, the Withings algorithm was accurate for AF screening with few uninterpretable results while Kardia 6L may be preferred to capture symptomatic palpitations. The insights gained from this study inform both caregivers and adult CHD patients on ECG-based home monitoring to ultimately facilitate early detection of arrhythmias and prevent late complications by timely intervention.