EBM Review Series: International Validation of the Canadian Syncope Risk Score
VACEP’s Evidence-Based Medicine Review Series
for Emergency Physicians
Authors: Eastern Virginia Medical School Emergency Medicine Resident Jesse P. Cann, MD and EM Assistant Program Director and Associate Professor Martin D. Klinkhammer MD, MPH, FACEP
Reviewers: Georgetown University Hospital/Washington Hospital Center Emergency Medicine Residency Program: attending physician Rory Spiegel, MD and resident Connor Bracy, MD
Article: International Validation of the Canadian Syncope Risk Score: A Cohort Study | Annals of Internal Medicine, June 2022. Link to the full study.
CASE
Case: A 68-year-old man with a history of hypertension and high cholesterol presents to your Emergency Department (ED) after an unwitnessed syncopal episode. He has no known history of heart disease. The patient had been working outside all day in the heat, came inside to drink a glass of water, and after standing at the sink for a few minutes the patient’s wife states that she heard a loud “thud” from the living room and came in to find the patient on the ground. He was alert and oriented immediately afterwards, without confusion, and with no obvious signs of trauma. Vital signs on arrival are BP, 117/76; P, 91; R, 18; T, 37.0⁰ C (98.6⁰ F); and O2 saturation 97% on room air. Patient has no complaints currently, is overall well appearing, with a normal cardiovascular and abdominal exam and a non-focal neurological exam. EKG demonstrates normal sinus rhythm, no prolonged intervals or abnormal axes, and no acute ischemic changes. Labs are reassuring with normal blood counts, electrolytes, and cardiac enzymes.
Can you send this patient home?
About the EBM Review Series
VACEP’s Evidence-Based Medicine Review Series is a monthly literature review started by the University of Virginia’s Josh Easter, MD, MSc, a VACEP board member working to connect the academic community in Virginia. Physicians and residents submit their review of a recent journal article, and their summary is reviewed by emergency physician peers and published here. We invite each residency in Virginia (and D.C.) to create a faculty/resident team to submit and review articles. Email Executive Director Sarah Marshall to take the next step.
Goals
Provide a brief monthly synopsis of a high yield article germane to the practice of emergency medicine for distribution to all VACEP members
Provide an opportunity for a peer reviewed publication and invited presentation for faculty and trainees
Foster an academic community focused on evidenced based medicine for emergency medicine residency programs in the region
Syncope is among the more nebulous of chief complaints emergency physicians deal with on a daily basis. Its presentation can be as varied as its etiology. Accurately differentiating between the elderly man who had a syncopal episode related to his aortic dissection and the young college student who passed out at the sight of blood is essential to the flow and efficiency of any ED. Nevertheless, making this distinction becomes significantly more complicated when one considers the many shades of gray in between the very black-and-white admissions and discharges. Yet this is the task required of the emergency physician.
Furthermore, syncope admission rates in the U.S. are relatively high (about 35%) (1,2), and have moderate regional variability, suggesting that a decision instrument which could lend some scientific rationale towards admission practices would have significant utility.
Several risk assessment tools for syncope have previously been derived. These include the San Francisco Syncope Rule (SFSR), the risk stratification of syncope in the emergency department (ROSE), and the Short-Term Prognosis of Syncope (STePs), among others. However, these rules have either not been externally validated or have performed poorly when externally validated. Concurrently, international guidelines do not recommend their implementation (3).
More recently, the Canadian Syncope Risk Score (CSRS) was derived from a prospective cohort study of six large Canadian EDs enrolling 4,030 patients presenting with syncope (4). Its primary aim was to identify those patients with an increased 30-day risk for a serious adverse event (defined as death, arrhythmia, myocardial infarction, serious structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, severe hemorrhage, and any other condition causing syncope and procedural interventions for treatment of syncope). In other words: who is at risk for a bad outcome in the next 30 days after coming to the ED for syncope?
Figure 1.: CSRS scoring system.
Importantly, the derivation was carried out after excluding those with clear serious causes of syncope diagnosed in the ED. In the original derivation set, a score of ≤0 was found to confer a low rate of subsequent complications (the scoring system is displayed in Figure 1.) The scale uses scores ranging from -3 to 11 in which each additional point on the scale was associated with a higher estimated risk of an adverse event. Those with a score of zero or lower were estimated to be low risk with a risk less than 2% for any serious adverse event in the next 30 days.
Subsequently, the CSRS was validated in Canada in a paper published in 2020 (5). In this Canadian validation study, 3,819 patients ≥16 years old presenting to nine Canadian EDs with syncope were prospectively studied for the same negative outcomes as in the derivation set. Again, those patients adjudicated to have a serious underlying condition during the index ED visit were excluded. The patients were followed for 30 days to ascertain outcomes. As a diagnostic test, the CSRS performed well with an area under the receiver-operating characteristic curve (AUC) of 0.91 (95% CI, 0.88-0.93).
AUC was chosen as a way to demonstrate the diagnostic accuracy of the test through a range of sensitivities and specificities. The AUC is derived from a receiver operating characteristic (ROC) curve which plots the sensitivity and specificity of the test. AUC takes values from 0 to 1, where 0 indicates a perfectly inaccurate test and 1 is perfectly accurate (100% sensitivity and 100% specificity).
An AUC of 0.5 suggests no discriminatory ability (essentially a flip of the coin).
Generally in diagnostic testing, 0.7 to 0.8 is considered acceptable, 0.8 to 0.9 is considered excellent, and more than 0.9 is considered outstanding.(6)
Again, the AUC of the CSRS in the Canadian validation cohort was 0.91.
In the Canadian validation study, fewer than 1% of very-low-risk and low-risk patients (those with a score from -3 to 0) experienced a serious outcome within 30 days. Additionally, none of this cohort died or experienced a ventricular arrhythmia. Given that the low and very low risk patient groups included 75% of the total patients in the study ((1,631 + 1,254)/ 3,817), application of this rule to discharge the low and very-low risk patients has the potential to significantly reduce syncope admissions in the U.S.
This study was only conducted in Canada, so an international validation would have further utility to prove broad-reaching applicability.
Now, international investigators have conducted the International Validation of the Canadian Syncope Risk Score.(7) This was a prospective, international, multicenter study performed at fourteen hospitals across eight countries (U.S., Switzerland, Spain, Germany, Italy, Poland, New Zealand, and Australia). The study enrolled patients aged 40 years or older presenting to the ED with syncope that occurred within 12 hours prior to arrival. Patients with a nonsyncopal loss of consciousness (fall, seizure, stroke, or intoxication) were excluded. Additionally, as in the two previous studies, those with a significant serious diagnosis found during initial ED evaluation were excluded. Similar to the previous studies, the investigators followed 30-day serious outcomes (including death, life-threatening arrhythmia, myocardial infarction, serious structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, severe hemorrhage, any serious condition causing syncope, and procedural interventions for the treatment of syncope) as well as separately evaluating only purely clinical outcomes (excluding procedural interventions as a part of the composite outcome).
In contradistinction to the previous two studies, this study did not include asymptomatic arrhythmias nor symptomatic supraventicular arrhythmias as significant, with the rationale that these are more benign events and on their own may not justify hospitalization.
For comparison sake, the study compared the CSRS to the Osservatorio Epidemiologico sulla Sincope nel Lazio (OESIL) score. The OESIL is a previously validated risk stratification tool for syncope developed to identify those patients who had an increased risk of all-cause mortality within 12 months of presentation to the ED with syncope. The OESIL is a 4 point rule that includes age >65, history of cardiovascular disease, syncope without prodrome, and abnormal electrocardiogram as the only relevant markers, with 1 point given for each. A score of 0 or 1 is considered low risk. Importantly, the derivation included 270 patients and the validation only included 328 patients.(8) While the simplicity of the scoring system makes this tool easy to use, it was derived and validated only in Italy and the paucity of patient encounters makes its widespread applicability suspect.
The international validation of the CSRS study enrolled and analyzed 2,283 patients with a mean age of 68 years. Overall, 54% of patients were hospitalized. The primary composite outcome occurred in 7.2% (n=165), and the composite outcome excluding procedural interventions occurred in 3.1% (n=70).
Discrimination of the CSRS was good, with an AUC for the primary composite outcome at 30 days of 0.85 (95% CI, 0.83-0.88) compared to the OESIL score of 0.74 (95% CI 0.71-0.78).
For the composite outcome excluding procedural interventions, the CSRS had an AUC of 0.80 (95% CI 0.75-0.84) compared to the OESIL which scored 0.69 (95% CI 0.64-0.75)
Additionally, the CSRS triaged 60.8% of patients to the low-risk group, whereas the OESIL only triaged 48.4% to low risk.
Nonetheless, the “low risk” group captured by the CSRS (score ≤0) had a lower incidence of serious clinical events compared to the group identified by the OESIL (score 0 or 1), 0.6% vs 1.5% (7). In other words, when compared to the OESIL, the CSRS classified more true negatives as low risk while also misclassifying fewer false negatives.
Lastly, of the actual patients in the study, only 45.9% (n=1048) were discharged after ED evaluation. Utilizing either risk score and discharging those at low risk would have resulted in a greater number of discharged patients (CSRS 60.8% (1388/2282) vs. 48.4% (1104 of 2282) by OESIL). Lastly, patients at low risk by CSRS had lower rates of the primary composite outcome than those actually discharged from the ED (1.1% of low risk CSRS patients vs. 1.6% of those actually discharged). This was also true for the composite outcome excluding procedural interventions (0.6% vs. 1.0%)
Interestingly, one individual element of the score relating to the emergency clinician’s classification of syncope as cardiac, vasovagal, or other performed similarly in the identification of serious clinical outcomes (AUC 0.83 (95% CI 0.80 to 0.87)) when compared to the comprehensive CSRS (AUC 0.85 (95% CI 0.83-0.88)). This finding might suggest that the CSRS as a whole is excessive, since much of its power in this study ultimately came from the physician classification of syncope. This was not true for the original derivation or validation cohorts, and may suggest a difference in the way these international patients were worked up from the ED vs. those in Canada.
That said, this result seems unusual given that the results suggest that physician implementation of this rule in this very patient set would have resulted in an over 10% greater number of patients being discharged while resulting in potentially fewer unexpected deaths.
Limitations:
One limitation of this study relates to the setting in which it was utilized as these results were derived in academic emergency departments. As such, its applicability to other ED models, rural or county for instance, is questionable. Patients seen in more rural EDs who have less access to care may have undiagnosed medical conditions that would otherwise put them into a higher risk class (e.g. undiagnosed heart disease). In a similar sense, patients with less financial resources are more likely to have poorly controlled pre-existing medical comorbidities that may benefit from admission apart from their syncopal event. Therefore, an emergency department’s particular demographic of patients makes the rule more or less helpful depending on the overall health and access to care of the population being treated.
Another limitation is the implicit bias potentially generated in the treating provider by recording the CSRS. As with most clinical decision tools, the score is not intended to supplant experienced physician gestalt or to be used in a vacuum. Nevertheless, as one calculates a patient’s risk by using the score it may, either consciously or subconsciously, affect which tests are ordered as well as the decision to admit or discharge that patient. This has the potential of leading to excessive testing as well as both unnecessary hospital admissions and inappropriate ED discharges if not used in conjunction with clinical judgment.
Bottom Line:
The utility of the CSRS lies in its use as a consistent and evidence-based means of risk stratifying those patients who come to the ED for syncope. Importantly, the CSRS should only be used after a thorough ED evaluation as 3% or so of patients in all three studies were excluded due to important serious diagnoses being evident after initial ED evaluation. After that initial evaluation, this tool can be utilized as a disposition aid to identify those patients who are low risk and may be discharged home in the right clinical context. Nevertheless, it is worth reiterating that while this tool performs well in terms of diagnostic accuracy and reproducibility, its use is largely dependent on physician judgment (by judging vasovagal vs. cardiac syncope), and so certainly does not replace that judgment.
Conclusion:
With regards to the original case, we can use the Canadian Syncope Risk Score to help establish a safe disposition for this patient. Based on his history, exam, and workup he receives a score of 0, which classifies him in the low risk category. If this determination is consistent with the emergency provider’s overall assessment, then the patient can be discharged home with only a 1.9% risk of having a serious adverse event in the next 30 days.
REFERENCES
Probst MA, Kanzaria HK, Gbedemah M, Richardson LD, Sun BC. National trends in resource utilization associated with ED visits for syncope. Am J Emerg Med. 2015;33(8):998-1001. doi: 10.1016/j.ajem.2015.04.030
Sun BC, Emond JA, Camargo CA Jr. Characteristics and admission patterns of patients presenting with syncope to US emergency departments, 1992-2000. Acad Emerg Med. 2004;11(10):1029-1034. doi: 10.1016/j.annemergmed.2004.07.016
Shen WK, Sheldon RS, Benditt DG, Cohen MI, Forman DE, Goldberger ZD, Grubb BP, Hamdan MH, Krahn AD, Link MS, Olshansky B, Raj SR, Sandhu RK, Sorajja D, Sun BC, Yancy CW. 2017 ACC/AHA/HRS Guideline for the Evaluation and Management of Patients With Syncope: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society. Circulation. 2017 Aug 1;136(5):e60-e122. doi: 10.1161/CIR.0000000000000499. Epub 2017 Mar 9. Erratum in: Circulation. 2017 Oct 17;136(16):e271-e272. PMID: 28280231.
Thiruganasambandamoorthy V, Kwong K, Wells GA, Sivilotti MLA, Mukarram M, Rowe BH, Lang E, Perry JJ, Sheldon R, Stiell IG, Taljaard M. Development of the Canadian Syncope Risk Score to predict serious adverse events after emergency department assessment of syncope. CMAJ. 2016 Sep 6;188(12):E289-E298. doi: 10.1503/cmaj.151469. Epub 2016 Jul 4. PMID: 27378464; PMCID: PMC5008955.
Thiruganasambandamoorthy V, Sivilotti MLA, Le Sage N, Yan JW, Huang P, Hegdekar M, Mercier E, Mukarram M, Nemnom MJ, McRae AD, Rowe BH, Stiell IG, Wells GA, Krahn AD, Taljaard M. Multicenter Emergency Department Validation of the Canadian Syncope Risk Score. JAMA Intern Med. 2020 May 1;180(5):737-744. doi: 10.1001/jamainternmed.2020.0288. PMID: 32202605; PMCID: PMC7091474.
Mandrekar, Jayawant M. Receiver Operating Characteristic Curve in Diagnostic Test Assessment. Journal of Thoracic Oncology, Volume 5, Issue 9, 1315 - 1316
Zimmermann T, du Fay de Lavallaz J, Nestelberger T, et al; BASEL IX Investigators. International Validation of the Canadian Syncope Risk Score : A Cohort Study. Ann Intern Med. 2022 Jun;175(6):783-794. doi: 10.7326/M21-2313. Epub 2022 Apr 26. PMID: 35467933.
Colivicchi F, Ammirati F, Melina D, et al; OESIL (Osservatorio Epidemiologico sulla Sincope nel Lazio) Study Investigators. Development and prospective validation of a risk stratification system for patients with syncope in the emergency department: the OESIL risk score. Eur Heart J. 2003;24:811-9. [PMID: 12727148]
