Slide 1: How do we evaluate non-inferiority trials?

Slide 2: What are non-inferiority trials, again? In comparison to superiority trials, which ask, is a treatment better than a standard of care…

A non-inferiority trial asks, is a treatment not much worse than a standard of care?

Graphic showing non-inferiority margin. Data beyond this threshold would signify that the new therapy is unacceptably worse than the control.

Slide 3: How is the non-inferiority margin chosen? There is no universally accepted method for choosing a non-inferiority margin. Some use the approach adopted by federal agencies.

One. Determine the smallest plausible benefit of the existing standard of care versus placebo, often the bound of the confidence interval nearest to no effect.

Two. The new treatment must preserve at least some fraction of the absolute benefit of the standard of care.

Slide 4: Questions to ask in non-inferiority trials. One. Was the trial actually planning as non-inferiority? Trialists may try to salvage a failed superiority trial by re-analyzing it as non-inferiority trial. Changing the analysis after the fact introduces bias.

Slide 5: Questions to ask in non-inferiority trials. Two. Was the control treatment administered to the full standard of care? Poor adherence to the control treatment or use of a substandard control may make the treatment effect seem more compelling.

Slide 6: Questions to ask in non-inferiority trials. Three. Did the investigators perform both per protocol and intention to treat analysis? Per protocol analysis exaggerates between groups difference when the intervention is truly inferior. Therefore it is less likely to result in a false positive conclusion of non-inferiority. Intention to treat analysis makes groups prognostically equivalent and preserves the benefit of randomization. When intention to treat and per protocol analysis gives the same results, confidence in the conclusion is strengthened.

Slide 7: Real life non-inferiority trial: EXCEL. Question: Is PCI non-inferior to CABG in the treatment of left main disease for a primary composite endpoint of death, stroke, or MI after 3 years of follow up? The non-inferiority margin was agreed on at 4.2% by the study leadership. They believe this number was “consistent with an interpretation of equipoise between the two treatments.”

Slide 8: Summarizing results of EXCEL trial: at 3 year follow up 15.4% of the PCI patients vs 14.7% of the CABG patients experienced a primary end-point event. This was a between-group difference of 0.7%.

Slide 9: Interpreting results of the EXCEL trial: non-inferiority was demonstrated (p=0.02), but in the worst case, 4% (the upper bound of the CI) more patients with PCI may suffer an adverse event than those with CABG – a big effect! On average 0.7% more patients have an event with PCI. This signal favoring CABG suggests that it is reasonable to advocate for CABG over PCI and patients with low surgical risk.

Slide 10: Take home points. Some sources of bias in non-inferiority trials are: substandard care in the control group, re-analysis of a planned superiority trial as inferiority, failing to perform both per protocol and intention to treat analyses. Think critically about whether the non-inferiority margin represents an acceptable level of risk for your patient!