Jeremiah Schuur, MD, BWH Department of Emergency Medicine
With the rise in medical imaging over the last decade, it is logical to assume that some portion of that increase falls under the category of “inappropriate.” To be certain, this remains a serious issue and point of concern for the Centers for Medicare and Medicaid Services (CMS) as well as radiology groups themselves. Recently, in an effort to measure and possibly curb inappropriate head CT scans in the emergency department, CMS developed measure OP-15, “Use of Brain Computed Tomography in the Emergency Department for Atraumatic Headache,” which attempts to evaluate the use of brain CT in the ED for atraumatic headache in order to improve imaging efficiency.
Unfortunately, according to researchers at Brigham and Women’s Hospital (BWH), Boston, the measure fails to achieve meaningful results and may be inaccurately labeling procedures as inappropriate.
“Our study found that the measure OP-15 doesn’t reliably measure hospital emergency departments’ efficiency [regarding use] of head CT,” said Jeremiah Schuur, MD, BWH Department of Emergency Medicine.
The study, “Assessment of Medicare’s Imaging Efficiency Measure for Emergency Department Patients With Atraumatic Headache,” evaluated the CMS measure on three basic principles: reliability, validity, and accuracy. Researchers reviewed the visits of 748 patients that CMS labeled as having undergone inappropriate brain CTs based on billing data and found that 65% of the CT scans actually complied with Medicare’s measure and another 18% of patients had valid reasons for the CTs documented on their charts. While Schuur admits that not all inaccurate measures are ineffective, the degree to which OP-15 fails renders it largely useless.
“A measure can be useful—even if it’s not dead on reliable—if it gets the overall trend,” Schuur said. “If the number is not perfectly reliable but it does stratify low-performing hospitals from high-performing hospitals, it’s still a useful measure. Unfortunately, the score that Medicare came up with did not correlate well with the score from chart review. When you rank hospitals one through 20 and look at the top quarter versus the bottom quarter, looking at the chart review, a number of the hospitals in the top quarter fell to the bottom quarter and vice versa.”
The study and Schuur conclude that the CMS measure should be summarily thrown out rather than tweaked. He believes that there are far more useful clinical entities to track in order to determine when a head CT is appropriate. Moreover, the concern becomes that if and when OP-15 becomes publicly published, hospitals will be more concerned with performing “well” on an ineffective measure rather than improving patient care.
“There is good evidence [on the] dramatic increase in the use of CT scanning over the last 10 years,” Schuur said. “Although CT scanning is an amazing technology that allows us to diagnose serious conditions very early on, there’s good evidence that part of that rise in use is appropriate and part is inappropriate. The concern that we have is that this is not the place to start this measure. There are a number of other clinical entities and emergency benefits that have good scientific evidence about when to order a head CT or when to order another type of CT. I think it’s of concern to patients because they would be better served if Medicare was measuring other entities for which there was good clinical evidence.”
