Signal and Noise
Avsnitt

Are AI Agents Actually Infiltrating Your Surveys? We Have the Data. | Signal & Noise Ep 41

Signal and Noise
Dela

Brian and Andrew sit down with Karine Pepin of The Research Heads and Andrew Gordon of Prolific to break down one of the most important data quality studies the industry has produced. They analyzed 5,200 respondents across 10 platforms and the findings will surprise you.

AI agents are not in your data. Outside of MTurk, detection rates were below 0.5% across every platform type. And the things getting flagged on MTurk? Old-school bots, not LLMs. When they ran actual AI agents through the same survey, those agents outperformed real human respondents on nearly every quality measure.

The actual crisis is your human respondents and the platform you are buying them from.

Key Takeaways:

  • Agent infiltration is not the crisis the industry thinks it is

  • What is actually getting flagged as bots on MTurk, and what it really means

  • Why the human quality gap between direct panels & marketplace platforms is the story nobody is telling

  • Why the cheapest sample per click is often the most expensive data you will ever buy

Link to the study "AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers"

If you loved the episode, have comments, or want to appear on the show, connect with us down below!

Connect with us:

Connect with Karine Pepin:

Connect with Andrew Gordon:

Rss Apple Podcaster
Podden och tillhörande omslagsbild på den här sidan tillhör ROI Rocket, Brian Lamar and Andrew DeCilles. Innehållet i podden är skapat av ROI Rocket, Brian Lamar and Andrew DeCilles och inte av, eller tillsammans med, Poddtoppen.