Reply:

In a previous article in the American Journal of Neuroradiology, we reviewed numerous methodologic difficulties encountered in observational studies on the natural history of cerebral AVMs. We concluded: “The estimates of risk of rupture per year are uncertain. Multiplying those uncertain numbers by the life expectancy of individuals can inflate error beyond control. Hence relying on these estimates to make clinical decisions may be dangerous.”¹

In their letter to the editor, Laakso et al asked a most important question regarding the management of AVMs, “What should we base our treatment decisions on, at present?” Before we turn our attention to this crucial issue, we must first clarify a few points that Laakso et al called “factual errors in our analysis”:

• 1) As much as we would like to reconcile estimates from different studies, no matter how diverging the results, we must be careful not to create, out of diversity, selection, and statistical models, a “natural history of AVMs” that will make us believe we know more than we actually do. If similarities between the annual rupture rates calculated in Toronto (4.6% per year) and the ones in Helsinki (2.4%) are so desperately needed, why publish the 2.4% hemorrhage rate in the conclusion of the abstract in the first place or why use patient-years or report the entire follow-up period?^2,3

• 2) We questioned when the clock should start counting the length of the follow-up period. The question regards the biases introduced by the fact that some lesions can be revealed by symptoms such as seizures, while others can only present with hemorrhages. Presumably, all lesions are present from birth. Hence the length of the follow-up period used in the denominators of these calculations is, we suspect, more a reflection of detection bias than a witness of various specific risk factors for ruptures.

• 3) It is true that the authors of the Helsinki study provided confidence intervals for the cumulative rupture rate for the first 5 years. Unfortunately confidence intervals were not given for the annual rupture rates (ie, the 4.7% or 1.6% or 2.4% we are supposed to compare with the 4.6% from Toronto). Confidence intervals only reflect sample variation. They take no account of other sources of uncertainty, such as loss to follow-up or censuring because of treatments.

• 4) If the rupture rates decrease so markedly with time, it does not matter whether clinicians use p or 1- (1-p)^t. The warning we are trying to give is that these numbers are not only too uncertain to be multiplied, but they probably should not be used at all to justify risky preventive interventions.

We must then apologize if the authors thought they were unfairly quoted. They did their best to provide the community with estimates of the risks of hemorrhage of brain AVMs, but as they now concede, these types of observational studies are fraught with insurmountable problems. When what is needed is to acknowledge the uncertainty, studies that add pseudoprecision to the analysis of unreliable data do not help. It is just another way of evading the problem raised in the title of their letter, “Would Our Treatment Decisions Be Better Justified in the Absence of Observational Data?”

How, we ask, are we supposed to use observational data to make treatment decisions? More important, how could these data justify our risky interventions? This, then, is the crux of the matter: In the absence of reliable evidence, should we make clinical decisions at all, pretending we know, and then attempt to do the biased research without the consent of participants, to justify, after the fact, what we have already done? The authors mentioned the ARUBA (A Randomized Trial of Unruptured Brain AVMs) study, dedicated to unruptured AVMs but could not wait for the results of the trial. For unruptured AVMs, the Helsinki study reported 17 events that occurred in 99 patients during more than 60 years between 1940 and 2005. Rare events are further split into size, location, venous drainage, and so forth. How much knowledge can one extract from so little data, and how many clinical decisions are to be guided by this meager experience? Clinicians who cannot wait for the results of the trial should simply recruit patients into the trial. Trials are designed for that very purpose.⁴

Observational studies of patients selected for conservative management were never meant to replace trials to guide clinical interventions. They cannot test nor show the benefit or harm of therapy.⁵ We look up to the leaders of this field, such as the Helsinki group, to guide us into the future, a future where treatment decisions are made on reliable evidence that they do more good than harm. Rather than attempt to justify our decisions after the fact, we must squarely confront the uncertainty, reveal it in a transparent manner to our patients, and design trials that will help us act in a scientifically sound and ethically right manner.

• © 2011 by American Journal of Neuroradiology