How do you decide whether a clinical trial “worked”? In Part 2 of our Galleri series, we examine the landmark randomized trial of a blood test designed to detect more than 50 cancers. We explore why different outcome measures led to dramatically different headlines, discuss primary versus secondary outcomes, pre-registration, hierarchical testing, and post hoc analyses, and explain why mortality remains the outcome everyone is waiting for. Along the way, we uncover a statistical mystery involving dozens of missing cancers and discover how a little arithmetic can sometimes reveal more than a press release.
Statistical topics
- cancer screening
- exploratory analyses
- hierarchical testing
- missing data
- multiple testing
- outcome measures
- post hoc analyses
- pre-registration
- primary and secondary outcomes
- randomized clinical trials
- screening tests
Methodologic Morals
- “When the simple numbers don't add up, pay attention. The arithmetic may be trying to tell you something.”
- “The first question should not be, did it work? It should be, what counts as success?”
References
Common biases in cancer screening studies
Cancer screening studies are subject to several well-known biases that can make a screening test appear more effective than it actually is. Three of the most important are:
Lead-time bias: Screening advances the time of diagnosis, making survival from diagnosis appear longer even if the patient's lifespan is unchanged. For example, if a screening test detects a Stage II cancer at age 60 that otherwise would have been diagnosed because of symptoms at age 62, but the patient dies at age 68 regardless, survival from diagnosis appears to increase from 6 years to 8 years even though the patient did not live any longer.
Length bias: Screening preferentially detects slower-growing, less aggressive cancers because they remain detectable for longer than fast-growing cancers. For example, a slow-growing cancer that remains in Stage I for 5 years is much more likely to be found by screening than an aggressive cancer that progresses to symptoms within months. This can make screened patients appear to have better survival simply because screening preferentially found the less aggressive cancers.
Overdiagnosis: Screening detects cancers that would never have caused symptoms or death during a person's lifetime, leading to unnecessary diagnosis and treatment. For example, a screening test may detect a very slow-growing prostate or thyroid cancer in an older adult that would never have become clinically important if it had remained undiscovered.
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Writing in the Sciences
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn &ReginaNuzzo.com
- (00:00) - Intro
- (03:39) - The Claim: Not Ready for Primetime
- (03:58) - Trial Design: 142,000 Participants
- (07:50) - The Primary Outcome Problem
- (20:29) - The Primary Endpoint: Complete Miss
- (22:14) - Three Arguments for the Defense
- (28:29) - - Statistical Sleuthing: Missing Cancers
- (41:14) - - The Stage Shift Argument
- (50:30) - - Rating the Claim
