National survey of emergency physicians on the risk stratification and acceptable miss rate of acute aortic syndrome

Abstract

Objectives

One in four cases of acute aortic syndrome are missed. This national survey examined Canadian Emergency physicians’ opinion on risk stratification, the need for a clinical decision aid to risk stratify patients, and the required sensitivity of such a tool.

Methods

We surveyed 1,556 members of the Canadian Association of Emergency Physicians. We used a modified Dillman technique with a prenotification email and up to three survey attempts using electronic mail. Physicians were asked 21 questions about demographics, importance of certain high-risk features, investigation options, threshold for investigation, and if a clinical decision tool is required

Results

We had a response rate of 32%. Respondents were 66% male, and 49% practicing >10 years, with 59% in an academic teaching hospital. A total of 93% reported a need for a clinical decision aid to risk stratify for acute aortic syndrome. A total of 99.6% of physicians were pragmatic accepting a non-zero miss-rate, two-thirds accepting <1%, and the remaining accepting a higher miss-rate.

Conclusions

Our national survey determined that emergency physicians would use a highly sensitive clinical decision aid to determine which patients are at low, medium, or high-risk for acute aortic syndrome. The majority of clinicians have a low threshold (<1%) for investigating for acute aortic syndrome, but accept that a zero miss-rate is not feasible.

Résumé

Dans 1 cas sur 4, le syndrome aortique aigu passe inaperçu. L'enquête nationale menée parmi les médecins d'urgence au Canada visait à connaître leur opinion sur la stratification du risque, sur la nécessité d'un instrument d'aide à la décision clinique aux fins de stratification du risque chez les patients touchés et sur le degré souhaitable de sensibilité de l'instrument.

L'enquête a été menée parmi les 1556 membres de l'Association canadienne des médecins d'urgence, selon une version modifiée de la méthode de Dillman : un préavis a d'abord été envoyé par courriel, et des tentatives d'enquête, jusqu’à concurrence de 3, ont ensuite été transmises par courriel. Au total, 21 questions portaient sur différents sujets : données démographiques, importance de certains signes de risque élevé de syndrome aortique aigu, examens exploratoires, seuil d'exploration et nécessité ou non d'un instrument d'aide à la décision clinique.

Le taux de réponse a atteint 32%. Les répondants se répartissaient comme suit : 66% étaient des hommes, 49% pratiquaient depuis > 10 ans et 59% travaillaient dans un hôpital d'enseignement. Dans l'ensemble, 93% des participants ont confirmé la nécessité d'un instrument d'aide à la décision clinique leur permettant de stratifier le risque de syndrome aortique aigu. Enfin, 99,6% des médecins ont fait preuve de pragmatisme, jugeant inacceptable un taux nul (0) de cas passés inaperçus; toutefois, les deux tiers ont jugé acceptable un taux < 1% et les autres, un taux plus élevé.

D'après les résultats de l'enquête nationale, les médecins d'urgence seraient disposés à utiliser un instrument d'aide à la décision clinique très sensible leur permettant de classer les patients selon le degré de risque de syndrome aortique aigu, soit faible, moyen ou élevé. La majorité des cliniciens ont indiqué un faible seuil (< 1%) au regard des examens exploratoires du syndrome aortique aigu, mais ils reconnaissent qu'un taux nul de cas passés inaperçus est impossible à atteindre.

CLINICIAN'S CAPSULE

What is known about the topic?

There are no guidelines that are both safe and efficient in guiding who we should investigate for acute aortic syndrome.

What did this study ask?

Do we need a clinical decision aid for acute aortic syndrome? If so, what are the important components and required accuracy?

What did this study find?

A clinical decision aid requires a miss-rate of <1% and physicians rated clinical impression as important as classic high-risk features.

Why does this study matter to clinicians?

A clinical decision aid for acute aortic syndrome should be informed by physician required accuracy and clinical components.

INTRODUCTION

Acute aortic syndrome is a life-threatening clinical syndrome as a result of three distinct diagnoses; acute aortic dissection, penetrating atherosclerotic ulcer, and intramural hematoma.¹ It can present with a variety of symptoms (e.g., chest pain, back pain, or neurological findings) that overlap with numerous other common conditions (e.g., acute coronary syndrome or pulmonary embolism).^1,2 This contributes to one in four cases being missed on initial presentation.³ The acute aortic dissection detection risk score (ADD-RS) is the only decision aid available for acute aortic syndrome; however, its ability to rule in acute aortic syndrome is low, and it is not sufficiently accurate to rule out acute aortic syndrome.^4,5 A recent systematic review reported high-risk features that could potentially inform a new decision aid; however, it is unknown how these features are subjectively viewed by end users, such as emergency physicians.⁶

The characteristics of a decision aid are influenced by the diagnostic accuracy, miss-rate, and acceptable negative investigation rate required by the end user. To date, there has been little assessment of the acceptable level of missed diagnosis in emergency departments or acceptable rate of negative investigations. Two previous studies have looked at an acceptable miss rate for acute aortic syndrome based on a decision model incorporating probability of disease with risk of testing and treatment. The results range from 0.3% to 9%.^7,8 Although it is possible to calculate a miss-rate for acute aortic syndrome at which the patient will not benefit from further testing, this rate does not necessarily equate to the miss-rate that clinicians would be comfortable adopting or the miss-rate that patients would be comfortable in accepting. A consensus among clinicians and patients of an acceptable miss-rate or acceptable negative imaging rate for acute aortic syndrome would influence the sensitivity level for any new clinical decision tool deemed to be required by physicians.

The goal of this project is to obtain consensus on the need for development of a clinical decision tool, and if required, the acceptable diagnostic accuracy. In addition, we hope to assess for face validity of potential tool components.

METHODS

Study design and setting

We conducted a self-administered electronic mail survey of all emergency physicians listed in the Canadian Association of Emergency Physicians (CAEP) Directory. We used a modified Dillman tailored design method for survey design and administration (presurvey notification and up to three survey attempts). This study was coordinated by the Clinical Epidemiology Program of the Ottawa Hospital Research Institute between September 2018 and November 2018 and approved by Health Science North Research Ethics Board.

Survey content

Survey participants received a two-page questionnaire consisting of 21 questions. The variables (clinical and biochemical) included were derived from systematic review of the literature and expert consensus. The survey instrument was developed by the authors and revised following the feedback received after pretesting on 10 physicians.

Survey respondents were asked about the need and priority for a clinical decision tool. Physicians were asked about the importance of various clinical signs and symptoms associated with acute aortic syndrome on a four-point scale Likert scale, in addition to acceptable miss rate, rate of true positive computed tomographic scans, and use of D-dimer.

Survey administration

All participants received a prenotification correspondence that described the study and requested their participation. One week later, a second contact included (1) a cover letter describing the study, ensuring confidentiality, and providing instructions for completion of the survey; and (2) the link to the online survey using SurveyMonkey. Nonresponders were sent a minimum of three reminder correspondences.

Sample size calculation

All emergency physicians in Canada listed in the CAEP were surveyed. The sampling frame included 1,556 emergency physicians. We calculated that, with an anticipated response rate of 30% (based on similar surveys administered to emergency physicians), a 95% two-sided confidence interval around the most conservative estimate for a finite population proportion of 30% would have a margin of error of 5.6%.

RESULTS

We had a total response rate of 31.9% (478 of 1,499) the majority were >10 years in practice (48.7%), and working at an academic center (57.2%) (Table 1).

Table 1. Weighted importance of signs and symptoms for acute aortic syndrome

Only 6.9% believed we did not need a clinical decision tool for acute aortic syndrome, but if one were validated 98.2% would use it in clinical practice. The majority of proposed high-risk variables were deemed as important/very important (Table 1).

Two-thirds of physicians believed a miss rate of <1% was sufficient for any decision tool, and one-third a miss rate of 1–5% was acceptable. With regard to an acceptable number of positive computed tomographic scans, 43.1% deemed a rate of 5–10% acceptable (10%, 2–5%; 28%, 10–20%).

For further investigation, the majority of respondents deemed D-dimer potentially acceptable (82.9% yes or maybe) in risk stratification for acute aortic syndrome.

DISCUSSION

This survey is the first national study of emergency physicians risk assessment for acute aortic syndrome. We found that the majority of Canadian emergency physicians would consider using a well-validated, sensitive clinical decision aid to help risk stratify patients for acute aortic syndrome. The majority of physicians were pragmatic acknowledging that any tool may not be perfect and accepted a miss rate that was not zero.

The American Heart Association and European Society of Cardiology propose the use of the ADD-RS to aid in assessing pretest probability for acute aortic syndrome.^1,3 In a large prospective observational study, the miss-rate of the score was found to be 2.7%; this is above what survey respondents would accept.⁴ To achieve a <1% miss-rate, it has been proposed that D-dimer be used; however, the ADD-RS only offers who should not have a D-dimer (i.e., high risk), it does not offer guidance on who is low enough risk that they do not need a D-dimer to rule out acute aortic syndrome.⁴

Current practice based on unstructured clinical suspicion has a high specificity but low sensitivity leading to an unacceptably high miss rate.^2,4 The ADD-RS a pure clinical variable based score applied to those with a clinical suspicion for acute aortic syndrome has an extremely low specificity but high sensitivity.^4,6 Combining the clinical suspicion with specific clinical variables has the potential to improve risk stratification. Other clinical decision aids, such as the Wells score, incorporate clinical suspicion, but some clinicians are concerned about the subjective nature of this variable. Our results demonstrate that suspicion for acute aortic syndrome or an alternative diagnosis are potentially acceptable variables to include in a clinical decision aid.

LIMITATIONS

We acknowledge that our study has several potential limitations. Our low response rate and use of a single directory to recruit physicians may have led to a biased sample. Physicians who work in an academic center may be more likely to be registered in the organization registry. A small number of respondents did not have access to CT (4.3%). Rural and remote emergency physicians who practice in resource-constrained environments may be under-represented in this survey. We had lower numbers of physicians practicing in a community center than similar national surveys.⁹ The acceptable miss rate and imaging rate may be different when taken in context of the risk and benefits of transfer. Development of any proposed decision aid must take into account application in such an environment.

CONCLUSION

Our national survey determined that emergency physicians would use a highly sensitive clinical decision aid to determine which patients are at low, or high risk for acute aortic syndrome. The majority of clinicians have a low threshold (<1%) for investigating for acute aortic syndrome but accept that a zero miss-rate is not feasible.