# Evaluating and Building AI Systems - ML 166

Page: https://stenobird.com/podcast/adventures-in-machine-learning/evaluating-and-building-ai-systems-ml-166
Text version: https://stenobird.com/podcast/adventures-in-machine-learning/evaluating-and-building-ai-systems-ml-166.md
Podcast: [Adventures in Machine Learning](https://stenobird.com/podcast/adventures-in-machine-learning)
Published: 2024-09-19T10:00:00+00:00
Episode link: https://www.spreaker.com/episode/evaluating-and-building-ai-systems-ml-166--62029593
Audio file: https://dts.podtrac.com/redirect.mp3/api.spreaker.com/download/episode/62029593/ml_166.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/adventures-in-machine-learning/episodes/evaluating-and-building-ai-systems-ml-166
Duration seconds: 3837

## Resource

Building effective RAG pipelines requires mastering the tension between data chunking strategies and embedding context windows. This episode explores how to navigate the complexities of retrieval-augmented generation and the evolving role of AI engineers.

## Highlights
- Failure mode: Large text chunks risk truncation by embedding models, while overly small chunks lack sufficient semantic context for retrieval
- Practical takeaway: Use frameworks like LangChain or LlamaIndex for rapid prototyping, but be prepared to build custom solutions for edge cases
- Main idea: The frontier of AI development is moving from simple RAG-enabled chatbots toward more complex agentic systems
- Main idea: Synthetic data generation via LLMs is becoming a primary solution for overcoming data collection and evaluation bottlenecks
- Practical takeaway: Career transitions in AI are best achieved through horizontal movement and demonstrating hands-on skill sets within an organization

## Topics

Retrieval-Augmented Generation, RAG Pipelines, Vector Databases, Data Chunking, Embedding Models, Agentic Systems, Synthetic Data, Machine Learning Engineering

## Chapters
- 1:00 — The Developer's Journey to AI: Richmond Alake discusses his transition from full-stack JavaScript development to AI and the role of technical content in learning.
- 6:05 — Content Strategy and Technical Writing: A discussion on the effectiveness of listicles and technical tutorials for engaging developers on platforms like Medium.
- 11:20 — The RAG Pipeline Challenge: Deep dive into the unsolved problems of RAG, specifically focusing on the impact of chunking strategies on retrieval quality.
- 38:20 — Leveraging Database Abstractions: Evaluating the use of MongoDB's aggregation pipeline and frameworks like LangChain to accelerate AI application development.
- 49:10 — Synthetic Data and the Future of Evaluation: Exploring how synthetic data generation is addressing the scarcity of high-quality training and evaluation datasets.
- 54:50 — From Chatbots to Agentic Systems: Analyzing the increasing complexity of AI as the industry moves beyond basic retrieval toward autonomous agentic workflows.
- 1:00:10 — Navigating the AI Career Landscape: Advice on filtering signal from noise in the AI field and transitioning roles through continuous learning and internal networking.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/adventures-in-machine-learning/episodes/evaluating-and-building-ai-systems-ml-166/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/adventures-in-machine-learning/evaluating-and-building-ai-systems-ml-166.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.