# Why Your RAG System Is Broken, and How to Fix It with Jason Liu - #709

Page: https://stenobird.com/podcast/twiml-ai-podcast/why-your-rag-system-is-broken-and-how-to-fix-it-with-jason-liu-709
Text version: https://stenobird.com/podcast/twiml-ai-podcast/why-your-rag-system-is-broken-and-how-to-fix-it-with-jason-liu-709.md
Podcast: [The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)](https://stenobird.com/podcast/twiml-ai-podcast)
Published: 2024-11-11T15:55:00+00:00
Episode link: https://twimlai.com/podcast/twimlai/why-your-rag-pipeline-is-broken-and-how-to-fix-it/
Audio file: https://pscrb.fm/rss/p/traffic.megaphone.fm/MLN3653850871.mp3?updated=1731384027
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/why-your-rag-system-is-broken-and-how-to-fix-it-with-jason-liu-709
Duration seconds: 3483

## Resource

Today, we're joined by Jason Liu, freelance AI consultant, advisor, and creator of the Instructor library to discuss all things retrieval-augmented generation (RAG). We dig into the tactical and strategic challenges companies face with their RAG system, the different signs Jason looks for to identify looming problems, the issues he most commonly encounters, and the steps he takes to diagnose these issues. We also cover the significance of building out robust test datasets, data-driven experimentation, evaluation tools, and metrics for different use cases. We also touched on fine-tuning strategies for RAG systems, the effectiveness of different chunking strategies, the use of collaboration tools like Braintrust, and how future models will change the game. Lastly, we cover Jason’s interest in teaching others how to capitalize on their own AI experience via his AI consulting course. The complete show notes for this episode can be found at https://twimlai.com/go/709.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/why-your-rag-system-is-broken-and-how-to-fix-it-with-jason-liu-709/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/twiml-ai-podcast/why-your-rag-system-is-broken-and-how-to-fix-it-with-jason-liu-709.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.