Log In

Forgot Password?

OR

Not a member? Continue as a nonmember.

Become a Member

By becoming a member of the AAN, you can receive exclusive information to help you at every stage of your career. Benefits include:

Join Now See All Benefits

Loading... please wait

Abstract Details

Inter-Rater Reliability of EEG-Based Encephalopathy Grading
Epilepsy/Clinical Neurophysiology (EEG)
S30 - Epilepsy/Clinical Neurophysiology (EEG): Technology (4:54 PM-5:06 PM)
008

The VE-CAM-S (Visual EEG Confusion Assessment Method – Severity) scale quantifies encephalopathy severity based on EEG features. However, inter-rater reliability of experts using the scale has yet to be assessed.

To measure inter-rater reliability of experts in assessing encephalopathy severity using the VE-CAM-S grading system.

We created an online test with thirty-two 15-second EEG samples. Each question asked users to indicate the presence/absence of each of 29 EEG features, 11 of which were used in the VE-CAM-S. Gold-standard was based on consensus of 3 authors (IS, FN, MBW). Ten experts from 6 institutions participated. We quantified performance by average spearman correlation of VE-CAM-S scores with the gold standard, and average sensitivity/specificity. We performed a qualitative analysis to identify errors in recognizing EEG features that most affected VE-CAM-S scores.

The average [95%CI] correlation between VE-CAM-S scores with the gold standard was 0.73 [0.59-0.86]. Specificity was very high (>90%) for all but generalized delta (77%). Sensitivity was high (>70%) for all but brief generalized attenuations (69%), generalized periodic discharges (67%), generalized theta (63%), BIRDS (57%), generalized alpha (57%), extreme delta brushes/EDB (50%), and generalized beta (50%). Probable reasons for errors were subtlety of some findings; confusing some findings (e.g., generalized beta vs. myogenic artifact, burst suppression vs. brief generalized attenuations); failure to correctly recognize BIRDs (mislabeled as focal IEDs) and EDB (mislabeled as GRDA). The largest errors occurred when experts missed or falsely identified features that carry higher weight in the VE-CAM-S scoring rubric.

Expert agreement in VE-CAM-S scoring is high. Error analysis identified several ways to improve future versions, including breaking high-stakes features into smaller parts; creating a “cheat sheet” with scored examples to allow scorers to choose the closest match; and designing teaching materials to help scorers recognize subtle variations of high-stakes patterns.  

Authors/Disclosures
Fabio Nascimento, MD (Washington University Medical School)
PRESENTER
Dr. Nascimento has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Zogenix. The institution of Dr. Nascimento has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Biocodex. Dr. Nascimento has received personal compensation in the range of $500-$4,999 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for AAN. Dr. Nascimento has received publishing royalties from a publication relating to health care. Dr. Nascimento has a non-compensated relationship as a Editorial Team Member with Neurology RFS that is relevant to AAN interests or activities. Dr. Nascimento has a non-compensated relationship as a Production Team with Neurology Podcast that is relevant to AAN interests or activities.
Anika Zahoor, MD, MBBS Ms. Zahoor has nothing to disclose.
Jayme C. Banks, Other (MGH) Ms. Banks has nothing to disclose.
Ryan A. Tesh, Other (Beth Israel Deaconess Medical Center (BIDMC)) Mr. Tesh has nothing to disclose.
Christine Ann Eckhardt Dr. Eckhardt has stock in Moderna.
Haoqi Sun, PhD (Massachusetts General Hospital) Dr. Sun has nothing to disclose.
Adam Seth Greenblatt, MD (Washington University in St. Louis) Dr. Greenblatt has nothing to disclose.
Aline Herlopian, MD (Yale University) Dr. Herlopian has nothing to disclose.
Ioannis Karakis, MD, FAAN (Emory University) Dr. Karakis has received personal compensation for serving as an employee of Emory University School of Medicine. Dr. Karakis has received personal compensation in the range of $500-$4,999 for serving as a Consultant for UCB. Dr. Karakis has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for GSK. Dr. Karakis has received personal compensation in the range of $10,000-$49,999 for serving as a Consultant for Epitel. Dr. Karakis has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Ceribell. Dr. Karakis has received personal compensation in the range of $500-$4,999 for serving as an Expert Witness for Hill Ward Henderson.
Roohi Katyal, MD (Louisiana State University Health Sciences Center) Dr. Katyal has nothing to disclose.
Chetan Nayak, MD (University of Texas Health Science Center at Houston) Dr. Nayak has nothing to disclose.
Marcus Ng, MD (University of Manitoba) The institution of Dr. Ng has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Eisai Canada. The institution of Dr. Ng has received personal compensation in the range of $500-$4,999 for serving as a Consultant for Paladin Canada. The institution of Dr. Ng has received personal compensation in the range of $500-$4,999 for serving on a Speakers Bureau for Eisai Canada. The institution of Dr. Ng has received personal compensation in the range of $500-$4,999 for serving on a Speakers Bureau for UCB Canada. Dr. Ng has received publishing royalties from a publication relating to health care.
Jonathan Williams, MD (Washington University) Dr. Williams has nothing to disclose.
Irfan Sahail Sheikh, MD (UT Southwestern) Dr. Sheikh has nothing to disclose.
M. Brandon Westover, MD, PhD (MGH) Dr. Westover has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for Beacon Biosignals. Dr. Westover has stock in Beacon Biosignals. The institution of Dr. Westover has received research support from NIH. Dr. Westover has received publishing royalties from a publication relating to health care. Dr. Westover has a non-compensated relationship as a cofounder with Beacon Biosignals that is relevant to AAN interests or activities.