Abstract
BACKGROUND AND PURPOSE: CTA provides high-resolution imaging of the head and neck vasculature but also of the soft tissues and bones. This results in a large volume of information to be interpreted. This study examines interpretation errors with head and neck CTAs and assesses whether double reading reduces miss rates.
MATERIALS AND METHODS: Consecutive CTAs of the neck and intracranial circulation were retrospectively identified and reviewed for vascular and nonvascular findings by a consensus of 2 neuroradiologists. The results were compared with the official report. Significant discrepancies were considered those that would have influenced follow-up or management.
RESULTS: We reviewed 503 studies; 144 were originally reported by a staff neuroradiologist alone, 209 by staff and diagnostic radiology resident, and 150 by staff and neuroradiology fellow. Twenty-six significant discrepancies were discovered in 20 studies, corresponding to 4.0% of studies with at least 1 miss, and an overall miss rate per study of 5.2%. There was at least 1 miss in 6.3% of studies interpreted by a staff neuroradiologist alone, 3.3% by staff and resident, and 2.7% by staff and fellow. The miss rate differences were not statistically significant. The most common misses were small aneurysms (50% of misses).
CONCLUSIONS: CTA neck and head datasets are now large, and there is a potential for missed findings. Significant discrepancies can occur with a low but not insignificant rate. Arterial pathology accounted for most discrepancies. This study emphasizes the need for careful systematic scrutiny for both vascular and nonvascular pathology regardless of indication. Double reading reduces error rates.
ABBREVIATIONS
- CTA
- CT angiography
- MPR
- multiplanar reformatted
CTA provides a reliable and noninvasive method for studying the neck and intracranial arteries. Its clinical applications include the identification and characterization of aneurysms, steno-occlusive diseases, arteriovenous malformations, vessel dissections, sino-venous thrombosis, idiopathic hemorrhage, and other vascular pathologies.1 The value of CTA in the diagnosis of vascular lesions has been established in relation to angiographic and surgical findings in several studies.2⇓⇓⇓⇓⇓⇓⇓–10
With the advent of multi-detector row CT scanners, CTA datasets increasingly provide high-resolution imaging of the vessels but also of the soft tissues and osseous structures. This results in a large volume of information that must be interpreted by the radiologist. Given the large volume of data contained within CTA studies, it is expected that the practical performance of the test may be limited by perceptual or cognitive errors. To date, such errors have not been addressed in the literature. The purpose of this study is to identify common diagnostic errors and their frequency of occurrence in the interpretation of CTA studies and to assess whether double reading reduces error rates.
Materials and Methods
Institutional research ethics review board approval was obtained. A retrospective study was performed: 507 sequential neck and intracranial CTA studies performed over a 6-month period were identified. Four were excluded because the vessels were incompletely visualized (inadequate vessel opacification in 3, inadequate coverage in 1), leaving 503 studies.
The CTAs were performed by using a LightSpeed Plus (GE Healthcare, Waukesha, Wisconsin) 4-section CT or VCT 64-section scanner. Images were obtained with 1.25-mm collimation (120 kVp; 350 mA) on the 4-section scanner and with 0.625-mm collimation on the 64-section scanner. Intravenous access was via an antecubital vein by using an 18- or 20-gauge angiocatheter. In total, 100–125 cc of Omnipaque 300 or Visipaque 320 (GE Healthcare) was injected at a rate of 4.0–4.5 mL/s, with either a 17-second delay or the use of Smart Prep (GE Medical Systems, Waukesha, Wisconsin). Coronal and sagittal MPR images were created at 7-mm thickness spaced by 3 mm. Bilateral 5° rotational MPRs were created at the carotid bifurcation and at the carotid terminus. 3D-rendered images were created on an Advantage workstation (GE Medical Systems). These reformations were performed as a routine for all CTAs. All images were viewed on an Impax 4.5 PACS (Agfa Healthcare, Mortsel, Belgium). Our institution has 3 head and neck CTA protocols: intracranial, including scanning from C4 to the vertex; carotid, including scanning from the arch to the vertex; and stroke, the same as carotid but also including an intracranial CT perfusion.
Demographic data including sex, age at the time of the scan, and indication for the CTA were recorded. The specific institutional CTA protocol (stroke, carotid, or intracranial) and imaging system (4- or 64-section) also were noted. The studies had been previously interpreted by a staff neuroradiologist alone (144 studies), a staff neuroradiologist and a diagnostic radiology resident (209 studies), or a staff neuroradiologist and a neuroradiology fellow (150 studies). The original reports were available. For this research, the CTAs were reviewed a second time by 2 neuroradiologists to detect misses. A consensus review of 2 neuroradiologists was chosen because the investigators believed that more potential misses would be detected by 2 rather than 1 reviewer and that any detected misses that were questionable could be settled as present or absent between the 2 reviewers. Each study was interpreted for vascular and nonvascular findings. The 2 reviewers were blinded to the original report and any other imaging studies, including conventional angiograms. The results were compared with the official report. All of the original studies were interpreted by neuroradiologists with sufficient qualifications to be senior members of the American Society of Neuroradiology. There were 5 original interpreting staff neuroradiologists with 2, 3, 16, 30, and 35 years of experience beyond fellowship. The 2 reviewing neuroradiologists had 1 and 3 years of experience. Significant discrepancies were identified as those that would have influenced follow-up or management. Minor discrepancies that were unlikely to affect patient care were omitted from this article. Significant findings were communicated to the interpreting radiologist and appropriate action was taken. If conventional angiographic studies were performed, these were only reviewed after the blinded review to confirm whether a perceived miss detected by the 2 reviewers was true. This only occurred in 1 case.
Results
Of the 503 studies were reviewed, 144 were originally reported by a staff neuroradiologist alone, 209 by staff and diagnostic radiology resident, and 150 by staff and a neuroradiology fellow. Twenty-six significant errors were uncovered in 20 studies. This corresponds to 4.0% (20/503) of studies with at least 1 error, and an overall error rate per study of 5.2% (26/503). All errors were false-negative interpretations or misses. Twenty-two of the errors were of vascular pathology (85%). Thirteen missed small intracranial aneurysms comprised the majority of the errors, corresponding to 50%. Six of the 13 aneurysms originated from the anterior communicating artery. The largest aneurysm missed was a 4-mm cavernous carotid artery aneurysm, difficult to see on CTA due to venous contamination in the cavernous sinus. The mean size of missed aneurysms was 2.5 mm. Other missed vascular pathologies included occlusions, a high-grade stenosis, sino-venous thrombosis, a dissection with a pseudoaneurysm, a spot sign,8 and a venous varix (Table and Figs 1⇓⇓–4). Missed nonvascular pathologies included nasopharyngeal, sellar, and parapharyngeal masses and a malpositioned nasogastric tube. There were no adverse clinical sequelae from the misses. None of the aneurysms required treatment, but appropriate serial follow-up was initiated. The occlusion and high-grade stenosis did not require intervention and were treated medically. The dissection was treated medically. The spot sign did not result in significant hematoma expansion. The venous varix did not seem to be related to an arteriovenous malformation or fistula; therefore, it was not treated. The nasopharyngeal mass was stable on follow-up and was likely a complicated Thornwaldt cyst with surrounding mucosal hyperplasia. The sellar mass was stable on follow-up and seemed to be a macroadenoma. The parapharyngeal mass with adjacent sino-venous thrombosis was probably a metastasis, but the patient died due to unrelated causes. The malpositioned nasogastric tube had been removed.
Types and frequencies of missed pathology from the primary interpretation of CTA studies
Axial CTA source image demonstrating a missed 3-mm left para-ophthalmic artery aneurysm (arrow).
Coronal CTA reformat demonstrating a missed small right middle cerebral artery aneurysm (long arrow) and a small missed right anterior communicating artery aneurysm (short arrow).
Axial CTA source image demonstrating a missed spot sign (arrow) at the edge of a right basal ganglia acute hemorrhage.
Axial CTA source image (A), sagittal reformat (B), and coronal reformat (C) demonstrating a missed left common carotid artery traumatic pseudoaneurysm.
There was at least 1 miss in 6.3% (9/144) of studies interpreted by a staff neuroradiologist alone, 3.3% (7/209) by staff and resident, and 2.7% (4/150) by staff and fellow. Of the 26 discrepancies, 10 were originally missed by staff alone (miss rate of 10 in 144 studies, or 6.9%), 9 by staff and resident (miss rate of 9 in 209 studies, or 4.3%), and 7 by staff and fellow (miss rate of 7 in 150 studies, or 4.7%). The miss rate differences were not statistically significant. Review of 216 studies performed on 4-section CT scanners demonstrated a higher error rate of 6.0%, compared with 287 studies performed on 64-section CT scanners with an error rate of 4.5%. The difference was not statistically significant.
Discussion
There is little available information about errors in the interpretation of neck and intracranial CTA studies in the literature. Therefore, no external data exist to allow direct comparisons with our current findings. A total error rate of 5.2% suggests that clinically significant missed findings on head and neck CTAs are fairly uncommon but not insignificant. Misses can be the result of either not recognizing the findings (perceptual error) or not understanding the ramifications of the findings (cognitive error). A study of on-call misses by radiology residents suggested that most misses on head CT studies are perceptual errors.11 Similarly, it is likely that the errors observed on CTAs in this study are perceptual as well. It is interesting that most missed pathologies were vascular in origin.
Although there are no prior studies on errors in the interpretation of CTA studies in the literature, numerous studies have examined interpretation errors in other imaging modalities as well as the value of having a second radiologist to provide secondary reads for quality assurance. These studies have a wide range of rates of significant diagnostic discrepancies for neuroradiologic imaging (CT and MR imaging), from 0.4% to 15.9%.12⇓⇓–15 Le et al14 found that performance among first-year fellows and general staff neuroradiologists in interpretation of head CTs was highly accurate (97.3%), without significant differences between the groups. It also was demonstrated that there was a low discrepancy rate between staff and trainees as well as between specialists and generalists, suggesting little utility in having a quality control program involving second reads.11 However, a study from the United Kingdom demonstrated a discrepancy rate of 15.9% between primary interpretations by general radiologists and second reads by neuroradiologists, suggesting the value of having a quality control program.14
Error rates in radiology interpretations will depend on how busy an individual radiologist is. At the time of this study, each neuroradiologist interpreted approximately 60 to 70 CT studies per day, including plain head CTs, neck CTs, spine CTs, and CTAs.
This study evaluated the benefits of double reading in 2 ways. First, the benefit of double reading with a resident or fellow was demonstrated. Errors rates were reduced from a staff alone rate of 6.3% to 3.3% when double read with a resident and to 2.7% when double read with a fellow. Second, the benefits of double reading were demonstrated with the second blinded review by 2 other staff neuroradiologists.
The benefits of double reading have been most studied with mammography. Double reading of screening mammograms has been shown to increase sensitivity of cancer detection while reducing specificity to an acceptable level. A review from 2001 indicates that double reading can increase cancer detection rates in the ranges of 3 to 12 (median, 4.4) cases per 10 000 women screened.16 The benefits were highly dependent on the method in which disagreements in diagnosis were resolved. Double reading by consensus (radiologists must agree on a diagnosis) and double reading by arbitration (a third radiologist makes the arbitrary decision in cases of disagreements) achieved relatively high cancer detection rates and low recall rates compared with double reading by unilateral recall (decision to recall patient is required from only 1 of the 2 radiologists).16 Independent readings also were shown to be superior to nonindependent readings. Double reading seemed to be cost-effective because 2 United Kingdom studies suggested that the incremental cost benefit per additional cancer detected by double reading ranges from £P1162 to £P2221.17,18 Although double reading has demonstrated clear benefits in mammography, its use in CTA has not been examined. Considering the notable error rate outlined by this study, and the improved error rates when double reading with a resident or fellow, it may be of value to explore the potential role of staff double reading in the interpretation of CTAs. Although the error rate (5.2%) is low, the consequences of missed pathologies such as aneurysms and occlusions can be serious. Future studies should examine the benefits and cost effectiveness of prospective staff double reading in the context of CTA.
The lower error rate observed in studies with 64-section in comparison with a 4-section CT scan can be explained by the higher resolution.
Potential weaknesses of this study are its retrospective design. Future studies on the benefits of double reading would best be designed prospectively. Another potential weakness is the sample size. Error rate differences were found between single staff read alone (6.3%) and double read by staff and resident (3.3%) or staff and fellow (2.7%). However, these errors rates were not statistically significant. With a larger sample size, statistical significance may have been achieved.
Conclusions
This study shows that significant errors in the interpretation of head and neck CTAs were rare but not insignificant. All observed errors were perceptual. Arterial pathology accounted for most missed findings. This study demonstrates the need for careful and systematic scrutiny for both vascular and nonvascular pathology on head and neck CTAs in all patients regardless of indication. Special care should be taken to identify small aneurysms as they are the most commonly missed pathology on CTA. Double reads may decrease the error rate. The greatest benefit was seen with a second staff double read, but double reading with a resident or fellow also reduced error rates.
Footnotes
-
Disclosures: Sean P. Symons, Research Support (including provision of equipment or materials): Sunnybrook Health Sciences Centre, Details: Sunnybrook Health Sciences Centre Brain Sciences Program Summer Scholarship $2000 (in the name of K.L.).
References
- Received August 8, 2010.
- Accepted after revision April 3, 2011.
- © 2011 by American Journal of Neuroradiology
In this issue
Jump to section
Related Articles
Cited By...
- Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center
- CT angiogram negative perimesencephalic subarachnoid hemorrhage: is a subsequent DSA necessary? A systematic review
- Further Refining the Analysis of Interpretation Errors in CT Angiography of the Head and Neck
- Reply: