When the results of the ORBITA trial were published last November, demonstrating that stents for stable angina were no more effective at relieving pain than a placebo surgery, many cardiologists were vehemently critical of the study on Twitter.
Now, several ORBITA skeptics have published letters in The Lancet with their feedback – and the authors have responded. Study authors Dr. Darrel P. Francis and Dr. Rasha Al-Lamee wrote a rebuttal to these letters that can only be described as a “mic drop.”
Here are some of the common criticisms of the study that they knocked down:
“The study is too small!”
A common criticism of ORBITA is that it was “underpowered” – there weren’t enough patients in the sample to detect a difference from the intervention. However, the study had enough people to detect a 30-second difference in exercise time, which is a conservative effect estimate given that previous trials detected a 90-second effect. Also, the sample size of ORBITA (200) is similar to that of previous stent trials which were positive (the ACME trial had 212 people).
“The study is too short!”
Some commentators thought that six weeks was not a long enough time to measure patient symptoms, and that measuring exercise time over a longer period would yield a better result. The authors wrote that they don’t know why someone would assume better long-term results, because “anatomical and haemodynamic relief is immediate, and the 6-week stress echo shows almost total eradication of ischaemia.”
“Those patients weren’t sick enough to benefit from a stent!”
Several writers contended that some patients in the study did not have severe enough angina to benefit from a stent, so they shouldn’t have expected a large change in exercise time. However, all the patients in ORBITA had already had the clinical decision before they were randomized – so their doctor believed that they needed a stent. Additionally, the authors note that “the proportion of patients had not had recent angina at randomisation (11%) is similar to other unblinded trials showing impressive angina relief: 9% in ACME, 11% in FAME 2, and 13% in COURAGE.”
“You did the stats wrong!”
Some commenters suggested using different statistical tests instead of the ones the authors used. The authors remind readers that other trials used the same statistical tests as ORBITA, and no one suggested they change the test for previous trials which were positive. More importantly, it would be unethical to change statistical methods once they have the data, just to get the result they wanted – that’s why prespecified analysis exists.
Francis and Al-Lamee write that they were disappointed that stents didn’t do better than placebo, but they refuse to “apply a positive spin” on the study because, “as authors we have a duty to preserve scientific integrity.”
Read all of the letters and the authors’ rebuttal on The Lancet website.