What happens when the chatbot answering your applicants gets it wrong, and nobody notices?
In this episode, I speak with Martha Horler, founder of Sparkline Advisory and known to many in the sector as the data goddess. Martha has spent two decades in UK higher education working in student data, statutory returns, digital systems and organisational change. She started out answering the phone to students in programme administration. She now supports institutions with data governance, digital strategy and AI literacy.
Martha developed the ICE/BERG framework after noticing that the questions coming out of her AI training sessions had little to do with prompting. People wanted to know if they could put student data in, which tools were safe, whether they would be replaced, and whether they could trust what came out. The tip of the iceberg is the everyday cycle of intention, communication and evaluation. Underneath sit boundaries, ethics, risk and governance. Sustainability, added after a conversation with a colleague in further education, is the sea it floats in.
Much of our conversation focuses on Martha's case study of an applicant enquiry chatbot. Chatbots feel familiar, so they get treated as low risk. But they talk to many people at once. When Martha got something wrong on the phone, a colleague could correct her. A chatbot might give the same wrong answer to hundreds of applicants first. We talk about what happens when policy changes quickly, as it did during the pandemic, who in a university should own keeping these tools up to date, and why the responsibility stays with the institution rather than the vendor. Martha also makes the point that if a chatbot is the only way to get an answer, the applicant never chose to use AI at all.
She is open about where the framework falls short, especially for procured systems where the vendor has already decided how the tool behaves. And she ends with a case for engaging with AI rather than ignoring it, knowing what it should and shouldn't be used for, and keeping up as the technology shifts.
Essential listening for anyone working in student services, data, digital strategy or AI policy in higher education.
You can read Martha's chatbot case study here.
AI Ethics Now
Exploring the ethical dilemmas of AI in Higher Education and beyond.
A University of Warwick IATL Podcast
This podcast series was developed by Dr Tom Ritchie, lead of The AI Revolution: Ethics, Technology, and Society at the University of Warwick and Elon University. The module explores the history, current state, and potential futures of artificial intelligence, examining its profound impact on society, individuals, and the very definition of 'humanness.'
This podcast was initially designed to provide a deeper dive into the key themes explored each week in class. We want to share the discussions we have had to help offer a broader, interdisciplinary perspective on the ethical and societal implications of artificial intelligence to a wider audience.
Join each fortnight for new critical conversations on AI Ethics with local, national, and international experts.
We will discuss:
- Ethical Dimensions of AI: Fairness, bias, transparency, and accountability
- Societal Implications: How AI is transforming industries, economies, and our understanding of humanity
- The Future of AI: Potential benefits, risks, and shaping a future where AI serves humanity
If you want to join the podcast as a guest, contact Tom.Ritchie@warwick.ac.uk
