Episode 3: "LLMs, RAG, Ragas and Testing"

In this episode Vitaly, Maryia and Anupam engage with guest Rahul Verma to discuss the evolving landscape of testing in the context of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). The conversation explores the challenges of testing LLMs, the importance of understanding the principles behind these technologies, and the need for testers to contribute to the development of frameworks and benchmarks. The discussion emphasizes the significance of approaching LLMs critically, rather than blindly trusting them or, conversely, rejecting them altogether.

Links:

1. Anupam's research on RAG evaluation using RAGAS: https://beyondquality.org/research/rag-evaluation

2. Rahul's book "The Last Book on Testing": https://www.amazon.com/Last-Book-Testing-Rahul-Verma-ebook/dp/B0DZRJFX65

Sponsors:

1. Qase https://qase.io/

2. Regent AB https://regent.se

3. AnuKrit https://anukrit.de