Automated White Matter Total Lesion Volume Segmentation in Diabetes

LST Versus Subjective WM Lesion Scores

We demonstrate the LST to be comparable to subjective WM scores for determining severity of WM lesion load in a population with DM. A high degree of correlation was observed between TLV computed by use of the LST and GT manual segmentation. LST achieved a maximum correlation of 0.87, corresponding to a k-threshold of 0.25, and appeared robust in its segmentation performance, within a range of k-thresholds from 0.2–0.4, all demonstrating similar correlations. In comparison, the subjective WM scoring demonstrated slightly weaker correlation with GT, achieving a 0.84 correlation. This improved slightly to 0.85 by use of the log of the GT TLV. Additionally, the subjective WM scores demonstrated a nonlinear relationship to GT TLV. This is not surprising because the amount of visible WM disease increases substantially over the range of scores. That is, near total involvement of the brain WM for a grade of 9 is much greater in volume than 10 times the few lesions identified for a grade of 1. The use of a logarithmic transformation of the GT TLV provided a clear linear relationship to the subjective WM lesion scores. This finding has important implications for studies that use the subjective WM lesion rating scale. Studies that use the subjective WM lesion rating scale with standard statistical regression models violate assumptions of linearity and potentially affect the validity or significance of the results. The assumptions required for use of parametric testing (eg, Pearson correlation analysis) in this evaluation were fulfilled through log transform of the data. Alternatively, nonparametric testing can be used when these assumptions are not met. Repeating these analyses by use of the Spearman rank correlation test and Kendall τ again demonstrated higher values for the LST than subjective WM scores, but the differences did not achieve significance. The Pearson correlation, however, provides a more complete assessment of the associations between variables when the underlying assumptions are attained.

The k-threshold corresponding to the maximum correlation between subjective scoring and LST was lower than that for LST and GT (0.15 versus 0.25). The lower k value in the LST corresponds to a more relaxed threshold for detection of WM lesions. This suggests that subjective ratings overestimate the true degree of WM disease. The degree of reader bias toward overestimation may be greater at the lower disease burden range, which was typical of our sample.

LST Versus GT

LST achieved a maximum R² of 0.69 at a k-threshold of 0.25 in the sample with DM. In contrast, the LST achieved a maximum R² of 0.94 and optimum k-threshold of 0.3 in the recent evaluation of multiple sclerosis.⁹ The performance difference between DM and multiple sclerosis probably is an effect of disease severity. In the multiple sclerosis evaluation, there was greater disease burden, with TLV extending to >50 mL, compared with <35 mL for our DM population. More importantly, the performance in the multiple sclerosis population improved with increasing lesion volume, ranging from a mean Dice coefficient of 0.67 for lesion volumes <5 mL to 0.85 for lesions volume >15 mL. The multiple sclerosis evaluation did not appear to have a significant number of subjects with TLV <1 mL (if any), whereas for our population, most subjects had TLV <1 mL. Thus, the difference in performance for our group probably reflects disease severity, with the LST performing less optimally at low disease burdens, rather than a difference in lesion detectability between populations.

Automated Segmentation in Diabetes

Although automated WM lesion detection is an area of active investigation, there have been very few studies validating tools in a diabetic population. Jongen et al²⁰ described a k-nearest neighboring clustering algorithm used in a study of subjects with diabetes. de Bresser et al²¹ used the same algorithm in a separate study of subjects with diabetes and white matter lesion load. Tiehuis et al²² also used the same algorithm in a study of cognitive function, vascular disease and diabetes, and, in another study, showed that it performs favorably compared with subjective rating scales with reference to cognitive assessments.²³ For all of these studies, performance of the described algorithm was previously evaluated by using a leave-on-out cross-validation procedure, achieving a similarity index of 0.8.²⁴ This validation, however, was performed only on 10 elderly subjects with a history of vascular disease and not specifically on patients with diabetes. A recently described method with the use of support vector machines was evaluated on a subset of 45 subjects from a larger study on the treatment of diabetes.²⁵ A single rater was used as the reference standard, with 10 subjects used for training and the remaining 35 for testing. This method demonstrated a high sensitivity (>0.9) and a specificity of ∼0.85. Interestingly, this study had a second manual rater but only used the first rater as the reference standard, possibly because of their reported large inter-rater variability. In comparison to these studies, our study provides a more direct evaluation of lesion segmentation in diabetes, including a large sample size, and a more rigorous approach to ground-truth determination.

WM Disease and Diabetes

The LST performance was very similar between DM-affected and unaffected individuals, achieving similar maximum correlations and optimum k-thresholds. There was no significant difference in degree of WM lesion load between affected and unaffected groups, by use of any of the metrics (WM scores, LST, or GT). Whereas there is a convincing relationship between DM and lacunar infarcts and brain atrophy, the association with WM hyperintensities on conventional MR imaging is less clear.²⁶ There has been recent evidence suggesting that DM is an independent risk factor for deep WM lesions in the elderly.²⁷ Additionally, voxelwise analyses of diffusion tensor imaging and fractional anisotropy have demonstrated early changes in white matter microstructural properties that were associated with diabetes duration.²⁸ Our sample may not have been large enough to detect a between-group difference. Alternatively, the effects of aging may be a greater contributor to the presence of visible WM disease than DM. It should be noted that our DM population had very few subjects with WM scores >5, and none were >7. We drew a random sample from the Diabetes Heart Study, with WM disease burdens probably reflecting the distribution of WM disease in the general DM population. Conclusions about diabetes and WM disease should not be made on the basis of this study because we did not control for a variety of confounding effects, including disease duration, co-morbidities, and medications.

Limitations

No subject had WM scores >7 or TLV >35 mL. This limits evaluation at the extreme end of disease burden. Also, our population was skewed toward the lower end of disease burdens, with most having TLV <1 mL. Although this is a limitation in terms of disease distribution, it is also probably reflective of the true disease burden and incidence of WM disease in the general population. One critical limitation is generic to all studies of WM disease burden, which is that there is no accepted reference standard. Manual delineation by experienced readers comes closest to what most would accept as a reference standard. The laborious nature of manual WM segmentation makes it generally impractical to have multiple raters. Semi-automated methodologies to facilitate the procedure can introduce bias through predetermination of potential lesion borders. Even with multiple readers, some readers are typically more accurate (or meticulous). These limitations may be mitigated by machine learning approaches that can weight more accurate readers more heavily in the determination of GT. We attempted to address these potential biases in GT segmentation with the use of a multi-tiered manual segmentation approach with the final consensus manual segmentation performed by experienced neuroradiologists. This is a time-intensive and laborious approach but allows confidence in the quality of the GT determination.