# Metrics Driven Development

Page: https://stenobird.com/podcast/practical-ai/metrics-driven-development
Text version: https://stenobird.com/podcast/practical-ai/metrics-driven-development.md
Podcast: [Practical AI](https://stenobird.com/podcast/practical-ai)
Published: 2024-08-29T20:45:00+00:00
Episode link: https://share.transistor.fm/s/4873f35c
Audio file: https://pscrb.fm/rss/p/dts.podtrac.com/redirect.mp3/media.transistor.fm/4873f35c/d4cac168.mp3
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/practical-ai/episodes/metrics-driven-development
Duration seconds: 2532

## Resource

How do you systematically measure, optimize, and improve the performance of LLM applications (like those powered by RAG or tool use)? Ragas is an open source effort that has been trying to answer this question comprehensively, and they are promoting a “Metrics Driven Development” approach. Shahul from Ragas joins us to discuss Ragas in this episode, and we dig into specific metrics, the difference between benchmarking models and evaluating LLM apps, generating synthetic test data and more. Sponsors: Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. Featuring: Shahul Es – GitHub , LinkedIn , X Daniel Whitenack – Website , GitHub , X Show Notes: Ragas Upcoming Events: Register for upcoming webinars here !

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/practical-ai/episodes/metrics-driven-development/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/practical-ai/metrics-driven-development.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.