Alexander Gibson is a PhD student at the Queensland University of Technology and the Australian Centre for Health Services Innovation studying the intersection of metascience and clinical machine learning. One of his focus areas is data provenance, the Who, What, Where, When, Why, and How of datasets, and how neglecting this can lead to bad outcomes in medical machine learning not only in research, but also for clinical practice and medical device approval.
CONTACT RANDY:
Feedback: metasciencematters@gmail.com
EPISODE LINKS:
Alex's preprint on unreliable diabetes and stroke datasets:
https://www.medrxiv.org/content/10.64898/2026.02.24.26347028v2
OUTLINE:
0:00 - Introduction
3:14 - The beginning of Alex's interest in clinical predictive modeling
5:05 - Alex's interest in metascience
6:42 - Choosing a dissertation topic/metrics hacking in machine learning
9:49 - Preprint on data provenance in medical datasets
12:33 - The diabetes and stroke datasets Alex investigated
16:46 - Major irregularities in the data
23:29 - TRIPOD+AI guidelines for auditing machine learning studies
25:26 - How unreliable studies can impact clinical practice and medical device patents
26:42 - Citation networks
27:37 - AI-generated formulaic medical machine learning studies
31:50 - Strategies for high-quality data provenance
33:53 - Patents citing unreliable studies, and how to integrate data provenance into peer review
35:23 - The biggest problems for clinical predictive modeling studies
37:02 - Resources and tools for improving rigor in machine learning
38:45 - Metrics reporting
40:45 - Choosing decision thresholds in predictive models
42:59 - The importance of clinical context in metrics reporting
45:21 - The unreasonable effectiveness of age and sex as predictors
47:53 - The roles of academia and industry in improving clinical machine learning studies
50:07 - Explanation versus prediction
52:51 - Advice and resources for students
54:27 - Outro
