Slide 1: Bias in Diagnostic Studies: Part Two

Slide 2: Question: You are evaluating a rapid antigen test for COVID-19. The manufacturer reports that the test has 98% sensitivity and 99% specificity. The test was studied in ICU patients with PCR-confirmed COVID-19 (cases) and PCR negative community dwelling adults (controls). All patients received both the rapid antigen test and the PCR. Which of the following types of bias is present in this study design? A. Spectrum, B. Verification, C. Selection, D. Lead time, E. Ascertainment.

Slide 3: Answer: Spectrum bias.

Spectrum bias describes the challenges to performance of a diagnostic test (sensitivity and specificity) that occur when it is used in different settings across spectrum of disease. It is analogous to selection bias, which describes the loss of applicability (external validity) in interventional studies when patients studied different from those seen in practice.

Specificity. The percent of health patients a test can exclude. Sensitivity. The percent of diseased patients a test can capture.

Slide 4: A good diagnostic test should minimize false negatives and false positives. Three features to think about when minimizing false negatives: pathologic, clinical, and comorbid.

Slide 5: Disease spectrum. Pathologic. Pathologic determinants of spectrum: investigators must define the disease (pathology) carefully. For instance, a nuclear scan for myocardial infarction might have strong sensitivity for large MI but perform worse when small infarctions are included in the definition. The best test will minimize false negatives and appropriately define the disease being studied.

Slide 6: Disease spectrum. Clinical. Clinical determinants of spectrum: a good test will be positive in patients with severe, mild, and asymptomatic disease. In our example of a COVID test, it was studies in ICU patients but not less sick patients.

Slide 7: Disease spectrum. Comorbid. Comorbid determinants of spectrum: coexisting conditions that may influence the test. Obesity decreases the sensitivity of BNP for heart failure. A breath test could have false negative results in a patient with lung disease.

Slide 8: In non-diseased patients, a good test will minimize false positives. The spectrum of comparative patients included in studies should have disease mimickers – for instance, a COVID test may not distinguish active infection from long COVID. A test of a marker for cholecystitis should include patients with chronic liver disease. A blood test for staph endocarditis should include patients with staph at other sides such as bone.

Slide 9: Spectrum bias or spectrum effect? Differences in spectrum are unavoidable, since researchers must choose which patients to include.

Furthermore, small differences in spectrum can have BIG effects on test characteristics. For instance, in one study, an exercise stress test had a positive likelihood ratio of 3.8 in patients with high blood pressure but 17 in normotensive patients!

For this reason, some advocate referring instead to spectrum effect, acknowledging that different groups may have different test characteristics.