Stenobird

Episode

#528: Python apps with LLM building blocks

Podcast
Talk Python To Me
Published
Nov 30, 2025
Duration seconds
4606
Processing state
processed
Canonical source
https://talkpython.fm/episodes/show/528/python-apps-with-llm-building-blocks
Audio
https://talkpython.fm/episodes/download/528/python-apps-with-llm-building-blocks.mp3
JSON
/v1/public/podcasts/talk-python-to-me/episodes/528-python-apps-with-llm-building-blocks
Markdown
/podcast/talk-python-to-me/528-python-apps-with-llm-building-blocks.md

Actions

  • POST https://stenobird.com/v1/public/podcasts/talk-python-to-me/episodes/528-python-apps-with-llm-building-blocks/transcription-requests
    Idempotently request low-priority transcript generation for this episode.
  • GET https://stenobird.com/podcast/talk-python-to-me/528-python-apps-with-llm-building-blocks.md
    Read the agent-friendly Markdown representation of this episode resource.

Summary

Treat Large Language Models as standard, decoupled APIs within your Python architecture rather than magic black boxes. Learn how to implement clear boundaries, focused endpoints, and robust monitoring to build reliable AI-integrated applications.

Topics

  • Python
  • LLM
  • Software Architecture
  • API Design
  • Machine Learning
  • Caching
  • Data Engineering
  • AI Integration

Highlights

  • Main idea: Treat LLM calls as standard API dependencies with defined boundaries and small, focused endpoints
  • Practical takeaway: Use caching and inspection patterns to manage costs and verify the reliability of non-deterministic responses
  • Failure mode: Avoid treating LLMs as a replacement for structured logic; instead, use them as building blocks within a larger, typed system
  • Practical takeaway: Implement monitoring and evaluations to ensure LLM performance meets application requirements
  • Main idea: Leverage abstraction layers to switch between different providers like OpenAI and Anthropic without rewriting core logic

Chapters

  1. 6:45 The AI Hype Landscape: A discussion on the current state of AI hype and the impact of LLM advancements on the developer ecosystem.
  2. 12:25 Software Engineering for Data Science: Exploring how software engineering best practices can bring reliability to data science and machine learning workflows.
  3. 24:05 LLMs as Modular Building Blocks: Architectural strategies for treating LLM APIs as interchangeable components regardless of the provider.
  4. 29:50 Caching and Storage Strategies: Using local storage and SQLite for caching LLM responses to improve performance and reduce latency.
  5. 41:50 Abstraction Layers and Tooling: Using libraries to abstract across different LLM providers like Anthropic and OpenAI.
  6. 53:10 Structured Output and Type Safety: Leveraging modern LLM capabilities for structured tasks and the importance of using types in Python.
  7. 1:10:50 Recommended Tools and Final Advice: A roundup of useful libraries and tools for building production-ready Python applications with LLMs.