Abstract
BACKGROUND AND PURPOSE: At present, no evidence-based lexicon exists for pediatric intracranial tumors. The Visually AcceSAble Rembrandt Images terminology describes reproducible MR imaging features of adult gliomas for prediction of tumor grade, molecular markers, and survival. Our aim was to assess the interrater reliability of the pre-resection features of Visually AcceSAble Rembrandt Images in pediatric brain tumors.
MATERIALS AND METHODS: Fifty consecutive pre-resection brain MR imaging examinations of pediatric intracranial neoplasms were independently reviewed by 3 neuroradiologists. The intraclass correlation coefficient for continuous variables and the Krippendorf alpha were used to evaluate the interrater agreement. Subgroup analysis was performed for 30 gliomas.
RESULTS: Parameters with almost perfect agreement (α > .8) included tumor location (F1) and proportion of enhancing tumor (F5). Parameters with substantial agreement (α = .61–.80) were side of tumor epicenter (F2), involvement of eloquent brain (F3), enhancement quality (F4), proportion of non-contrast-enhancing tumor (F6), and deep white matter invasion (F21). The other parameters showed either moderate (α = .41–.60; n = 11), fair (α = .21–.40; n = 5), or slight agreement (α = 0–.20; n = 1). Subgroup analysis of 30 gliomas showed almost perfect agreement for tumor location (F1), involvement of eloquent brain (F3), and proportion of enhancing tumor (F5); and substantial agreement for side of tumor epicenter (F2), enhancement quality (F4), proportion of noncontrast enhancing tumor (F6), cysts (F8), thickness of enhancing margin (F11), and deep white matter invasion (F21). The intraclass correlation coefficient for measurements in the axial plane was excellent in both the main group (0.984 [F29] and 0.982 [F30]) and the glioma subgroup (0.973 [F29] and 0.973 [F30]).
CONCLUSIONS: Nine features of Visually AcceSAble Rembrandt Images have an acceptable interrater agreement in pediatric brain tumors. For the subgroup of pediatric gliomas, 11 features of Visually AcceSAble Rembrandt Images have an acceptable interrater agreement. The low degree of reproducibility of the remainder of the features necessitates the use of features tailored to the pediatric age group and is likely related to the more heterogeneous imaging morphology of pediatric brain tumors.
ABBREVIATIONS:
- VASARI
- Visually AccesSAble Rembrandt Images
- REMBRANDT
- REpository for Molecular BRAin Neoplasia DaTA
The terminology for Visually AcceSAble Rembrandt Images (VASARI) was created for describing visual and subjective MR imaging features of adult gliomas.1 It was developed by The Cancer Genome Atlas radiology working group as part of The REpository for Molecular BRAin Neoplasia DaTA (REMBRANDT) project,2 which was, in turn, established to facilitate collaboration among researchers across the globe. The REMBRANDT data set comprises imaging, clinical, and genetic information from 630 patients with gliomas.2 VASARI features were developed following review of 88 MR imaging studies from this data set.1 The goal of the VASARI terminology was to facilitate accurate and reproducible MR imaging interpretations of gliomas across different sites with the help of a controlled terminology incorporating the major subjective features of these tumors. A total of 30 features (F1–F25 [with the exception of F15, which is no longer in use], and F26–F30) form the terminology. Of these, features 1–25 apply to pre-resection subjective MR imaging features, features 26–28 apply to post-resection MR imaging features, whereas features 29 and 30 refer to measurements in 2 planes.1 These features have been subsequently validated in adult gliomas for reproducibility, grading, prediction of molecular markers, and survival.3⇓⇓-6 Currently, no evidence-based standardized vocabulary exists for imaging features of pediatric brain tumors. We, therefore, aimed to assess the interrater reliability of VASARI features in a sample of consecutive pediatric brain tumors.
MATERIALS AND METHODS
This retrospective study was approved by the institutional research ethics board of the Hospital for Sick Children (REB 1000077073), and informed consent was waived due to the retrospective nature of the study.
Patient Population
A retrospective review of the data base of our institution was performed for patients who had undergone neuro-oncologic brain MR imaging between January 2018 and March 2020. Patients were included according to the following criteria: 1) age younger than 18 years, 2) presence of a de novo diagnosis of an intra-axial tumor on MR imaging, and 3) pretherapeutic status of the intracranial tumor. Exclusion criteria were image degradation by artifacts and incomplete image acquisition.
MR Imaging Technique
MR imaging of all except 4 patients was performed at our institution. MR imaging was performed on a 3T magnet (Magnetom Skyra, Siemens, or Achieva, Philips Healthcare) using a 32-channel head coil or on a 1.5T magnet (Achieva) with pediatric 8- or 16-channel head coils. The standardized sequence protocol at our institution included the following sequences: a sagittal 3D T1-weighted sequence with axial and coronal reformats; axial DWI, axial FLAIR, and coronal T2-weighted sequences; an axial SWI or a multiplanar gradient recalled acquisition; a gadolinium-based contrast agent–enhanced axial 3D T1-weighted sequence with coronal and sagittal reformats; and a gadolinium-based contrast agent–enhanced coronal T1-weighted spin-echo sequence. Other sequences performed variably for different patients on the basis of suspected tumor pathology or for preoperative purposes were axial TOF-MRA, contrast-enhanced axial FLAIR, and orbital or spinal imaging. For the 4 patients (glial series neoplasms = 2; embryonal tumor with multilayered rosettes = 1; atypical teratoid/rhabdoid tumor = 1) who were referred to our hospital following MR imaging performed at external sites, the images were uploaded onto our institutional PACS server, with all examinations including T1-weighted, T2-weighted, FLAIR, DWI, and contrast-enhanced T1-weighted sequences.
Analysis of the Study Cohort
Three pediatric neuroradiologists (A.B., M.W.W., B.B.E.-W.), blinded to clinical and neuropathologic data, were provided with the study accession IDs in random reading order (randomized in Excel [Microsoft]). These were read independently by each reader. Assessment of 26 pre-resection VASARI features (F1–F14, F16–F25, and F29–F30) was performed on the PACS workstations. The VASARI features for each study were entered into Google Forms (https://www.google.com/forms/about/) created and shared by The Cancer Imaging Archive Team.1 On completion of all 50 studies by each reader, the data were exported into Excel for further analysis.
Statistical Analysis
All statistical analyses were performed using R (Version 4.0.0; http://www.r-project.org/).7 The sample size (n = 50) was calculated using the kappaSize package in R with the confidence interval approach with binary outcome for a 1-sided 95% confidence interval, 0.2 precision, and an α level of .05. Intraclass correlation coefficients for continuous variables and the Krippendorf alpha8 were used to evaluate the interrater agreement. Subgroup analysis was performed for the 30 gliomas of the cohort.
RESULTS
Patient Demographics and Diagnoses
A total of 1097 consecutive neuro-oncologic brain MR imaging studies between January 1, 2018, and March 31, 2020, were reviewed for this study. A total of 1018 studies were acquired after initiation of therapy and were not considered for this study. We further excluded 18 studies with MRIs demonstrating image degradation by artifacts and 11 studies with incompletely acquired MRIs. The first 50 patients (mean age, 8.47 [SD, 5.33] years; range, 1–17 years; 28 males) who fulfilled the inclusion criteria were included for further analysis. The cohort consisted of biopsy-proved glial series neoplasms (n = 30), presumed glial series neoplasms (n = 7), biopsy-proved medulloblastomas (n = 6), a presumed medulloblastoma (n = 1), biopsy-proved embryonal tumors with multilayered rosettes (n = 2), biopsy-proved atypical teratoid/rhabdoid tumors (n = 2), a biopsy-proved ependymoma (n = 1), and a brain stem tumor that was presumed to be either a diffuse midline glioma or an embryonal tumor with multilayered rosettes (n = 1).
VASARI MR Imaging Analysis of 50 Consecutive Pediatric Brain Tumors
Parameters with almost perfect agreement (α > .8) included tumor location (F1) and proportion of enhancing tumor (F5). Those with substantial agreement (α = .61–.80) included side of tumor epicenter (F2), involvement of eloquent brain (F3), enhancement quality (F4), proportion of non-contrast-enhancing tumor (F6), and deep white matter invasion (F21). Those with moderate agreement (α = 0.41–.60) included proportion of necrosis (F7), cysts (F8), multifocal or multicentric tumor (F9), T1/FLAIR ratio (F10), thickness of enhancing margin (F11), definition of enhancing margin (F12), proportion of edema (F14), presence of hemorrhage (F16), diffusion characteristics (F17), pial invasion (F18), and satellites (F24). Those with fair agreement (α = 0.21–.40) included definition of nonenhancing margin (F13), ependymal invasion (F19), cortical involvement (F20), enhancing tumor crossing the midline (F23), and calvarial remodeling (F25), while those with slight agreement (α = .0–.20) included non-contrast-enhancing tumor crossing the midline. There was excellent agreement for measurements in the axial plane (F29 and F30), with intraclass correlation coefficients of 0.984 (range, 0.98–0.99) and 0.982 (range, 0.969–0.989), respectively. Results for each parameter are provided in the Online Supplemental Data.
VASARI MR Imaging Analysis of the Subgroup of 30 Consecutive Gliomas
Parameters with almost perfect agreement (α > .8) included tumor location (F1), involvement of eloquent brain (F3), and proportion of enhancing tumor (F5). Those with substantial agreement (α = .61–.80) included side of tumor epicenter (F2), enhancement quality (F4), proportion of non-contrast-enhancing tumor (F6), cysts (F8), thickness of enhancing margin (F11), and deep WM invasion (F21). Those with moderate agreement (α = .41–.60) included proportion necrosis (F7), definition of enhancing margin (F12), diffusion characteristics (F17), pial invasion (F18), and enhancing tumor crossing the midline (F23). Those with fair agreement (α = .21–.40) included T1/FLAIR ratio (F10), definition of nonenhancing margin (F13), proportion of edema (F14), hemorrhage (F16), ependymal invasion (F19), cortical involvement (F20), satellites (F24), and calvarial remodeling (F25), while those with slight agreement (α = .0–.20) included multifocal or multicentric tumor (F9) and non-contrast-enhancing tumor crossing the midline (F22). There was excellent agreement for measurements in the axial plane (F29 and F30), with an intraclass correlation coefficient of 0.973 (range, 0.951–0.986) and 0.973 (range, 0.948–0.987), respectively. Results for each parameter are provided in the Online Supplemental Data.
DISCUSSION
In this study, we analyzed the reliability of the VASARI terminology in pediatric brain tumors. In a group comprising 50 consecutive pediatric brain tumors, we found almost perfect interreader agreement for 2 features, substantial agreement for 5 features, and slight-to-moderate agreement for the other features. There was an excellent intraclass correlation coefficient for tumor measurement. In the subgroup comprising 30 consecutive pediatric gliomas, we found almost perfect interreader agreement for 3 features, substantial agreement for 6 features, and slight-to-moderate agreement for the other features. Similar to findings in the main group, an excellent intraclass correlation coefficient was found for tumor measurement.
In the adult population, imaging interpretation using VASARI features has been validated for reproducibility,3 prediction of molecular subtype,3,9 and prediction of survival10⇓⇓-13 in glioblastoma multiforme; and in predicting astrocytoma grade5 and molecular markers and survival in diffuse low-grade gliomas.4 The development of a controlled, reproducible terminology has allowed collaboration in adult glioma imaging across multiple sites.4,14,15
Our results for tumor location are similar to the results of the VASARI research project,1 in which there was high agreement for this parameter. However, there was relatively less agreement for side of tumor epicenter in both the main and subgroups of our study, possibly due to the higher incidence of midline/off-midline tumors such as medulloblastoma and diffuse midline gliomas in our cohort compared with the adult population.
In our study, in both the main group and the glioma subgroup, we showed higher agreement for ordinal parameters such as proportion of enhancing tumor and proportion of nonenhancing tumor but lower agreement for proportion of necrosis, proportion of edema, T1/FLAIR ratio, and thickness of enhancing margin. The lower interrater agreement for proportion of necrosis compared with the adult study was likely due to the difficulty in differentiating tiny cysts from areas of necrosis (Fig 1). There was also difficulty in differentiating peripheral cysts and necrosis from CSF clefts, especially in posterior fossa tumors. Re-definition of these parameters in the context of pediatric tumors, therefore, needs consideration, with perhaps separate terminologies for supratentorial and infratentorial tumors. The concept of the T1/FLAIR ratio was developed to differentiate expansive from infiltrative tumors.1 It is unclear, however, if this parameter has an important role in pediatric tumors. Moreover, there was uncertainty as to whether the FLAIR hyperintense signal resulted from tumor-related mass effect or infiltration, particularly in the posterior fossa. The role of this parameter may, therefore, be useful only in the setting of adult gliomas. Certain lesions had a poorly defined rim with scattered pockets of enhancement or focal nodular peripheral enhancement (Fig 2). This issue likely led to poor agreement for parameter F11 (thickness of enhancing margin). Modifying the definition to account for these enhancement patterns in pediatric brain tumors may help to better characterize this feature and may lead to better interreader agreement.
Depiction of cysts versus necrosis. Axial T2-weighted (A) and contrast-enhanced axial T1-weighted (B) images in a 57-week-old child with atypical teratoid/rhabdoid tumor demonstrate peripherally located “fluid-filled” structures (arrow in A) that do not parallel CSF intensity on T1-weighted imaging and do not show irregular rim enhancement (arrow in B), leading to difficulty in characterizing these as cysts or small necrotic pockets as per the VASARI definition.
Enhancement characteristics. Axial contrast-enhanced T1-weighted image in a 6-year-old girl with group 4 medulloblastoma (A) shows focal nodular enhancement along the periphery (arrow). Axial contrast-enhanced T1-weighted image in a 13-year-old girl with pilocytic astrocytoma (B) demonstrates spotty central enhancement (arrows) with faint rim enhancement (arrowhead).
For nominal parameters, there was near-perfect agreement for tumor location, with substantial agreement for side of tumor epicenter, eloquent brain, enhancement quality, and deep white matter invasion and lower agreement for other parameters such as cysts. In the subgroup of gliomas, however, there was substantial agreement for cysts. As we alluded to earlier, there was disparity in differentiating small cysts from necrosis and also peripherally located cysts from CSF clefts. A unifying definition incorporating fluid-filled structures as a single entity, with necrosis, cysts, and CSF clefts as subcategories, may help in better categorizing these components, especially if tumor location (supratentorial versus infratentorial) is also taken into account. Further evaluation of the cystic component (such as number of cysts, dominant cyst, peripheral cyst, cyst wall characteristics) may also prove useful in pediatric tumors, given the higher proportion of tumors with cystic components such as pilocytic astrocytoma, dysembryoplastic neuroepithelial tumors, and ganglioglioma in this population (Fig 3).16,17
Cysts in pediatric intracranial tumors. A and B, BRAF V600E–positive astrocytoma. Axial T2-weighted and contrast-enhanced axial T1-weighted images show a dominant cyst with an enhancing mural nodule. The cyst wall does not enhance (arrow). C and D, Pilocytic astrocytoma. Axial T2-weighted and contrast-enhanced axial T1-weighted images show a dominant cyst with an enhancing mural nodule. The cyst wall enhances (arrow).
Interpretation of diffusivity parameters in heterogeneous tumoral tissue and in the presence of hemorrhage was also challenging in both groups. Additional challenges encountered, particularly in posterior fossa tumors, were differentiation of cortical involvement from cortical distortion and obscuration from mass effect, and tumors crossing midline. “Crossing the midline,” by VASARI definition, included tumors that crossed over to the contralateral side via WM/commissural pathways. No clear definition existed, however, for lesions involving the brain stem. Another challenge we faced was ascertaining pial and/or ependymal invasion in instances in which the tumor reached the pial and or ependymal surface. Clearer definitions (such as pial enhancement contiguous with the tumor) may help improve interrater agreement in pediatric brain tumors. It would, however, still be challenging to assess nonenhancing components of the tumor that reach the pial and ependymal surfaces.
Slight agreement for calvarial remodeling in both groups may be explained by the absence of a VASARI definition for skull base remodeling and the higher variability of the individual skull base morphology in children, because posterior fossa tumors causing secondary osseous changes were rated inconsistently (Fig 4A). In addition, the ability of the calvaria of very young children to expand in response to raised intracranial pressure (due to open fontanelles and sutures)18 necessitates the need to update this definition for pediatric tumors (Fig 4B).
Osseous remodeling. Axial and sagittal T2-weighted images in a 4-month-old infant with a dysembryoplastic infantile ganglioglioma show asymmetric expansion of the calvaria (right > left, arrows in A), which occurs due to open sutures and fontanelles in this age group. Note the bulging anterior fontanelle (arrows in B).
There are several limitations to our study. First, the inclusion criteria were all consecutive tumors regardless of tumor pathology. These criteria also included a small number of cases that did not undergo biopsy. Because there is no semantic-based feature set in pediatric brain tumors, and given that in certain situations biopsy is not warranted, we decided to include all consecutive patients with brain tumors diagnosed on imaging. This choice led to a heterogeneous group of tumors with differing tumor biology but also highlighted the need for pediatric-specific and perhaps tumor- or location-specific terminology. We also performed a subgroup analysis only for pediatric gliomas. Second, the sample size was relatively limited but corresponded to our a priori calculations of sample size.
CONCLUSIONS
Although many VASARI features are reproducible in pediatric brain tumors with acceptable interrater agreement, the differing landscape and heterogeneity of pediatric tumors necessitates the use of tailored features, depending on tumor type and location.
Footnotes
M.W. Wagner and B.B. Ertl-Wagner are joint senior authors.
A. Biswas is partially funded by Ontasian Imaging Laboratory (OIL), Toronto.
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
References
- Received August 16, 2021.
- Accepted after revision October 20, 2021.
- © 2022 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- No citing articles found.