# How can you test your code when you don’t know what’s in it?

Page: https://stenobird.com/podcast/the-stack-overflow-podcast/how-can-you-test-your-code-when-you-don-t-know-what-s-in-it
Text version: https://stenobird.com/podcast/the-stack-overflow-podcast/how-can-you-test-your-code-when-you-don-t-know-what-s-in-it.md
Podcast: [The Stack Overflow Podcast](https://stenobird.com/podcast/the-stack-overflow-podcast)
Published: 2026-03-31T04:30:00+00:00
Episode link: https://rss.art19.com/episodes/4322d3fc-87b7-4276-abce-c4b3c3b7da8a.mp3?rss_browser=BAhJIg90cmFuc2NyaWJyBjoGRVQ%3D--952c5701c84ad333c69d5faa668f8177091704f0
Audio file: https://rss.art19.com/episodes/4322d3fc-87b7-4276-abce-c4b3c3b7da8a.mp3?rss_browser=BAhJIg90cmFuc2NyaWJyBjoGRVQ%3D--952c5701c84ad333c69d5faa668f8177091704f0
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/the-stack-overflow-podcast/episodes/how-can-you-test-your-code-when-you-don-t-know-what-s-in-it
Duration seconds: 1818

## Resource

The shift toward Model Context Protocol (MCP) and LLM-driven agents introduces non-determinism that breaks traditional software testing. This discussion explores how developers can validate agentic workflows when the sequence of tool calls is decided on the fly by an AI.

## Highlights
- Main idea: MCP is becoming the new foundational layer for AI agents, shifting the abstraction level from LLM prompts to tool invocation
- Failure mode: Relying on rigid, hard-coded workflows for AI agents defeats the purpose of the LLM's ability to decide the best path dynamically
- Practical takeaway: Testing must move toward validating outcomes and bounds rather than checking specific, hand-authored code paths
- Economic trade-off: AI-generated code may prioritize development velocity over performance, requiring a new pricing strategy to cover higher compute costs
- Future outlook: As source code becomes a commodity, the value in engineering will shift toward data construction and managing the complexity of agentic interactions

## Topics

Model Context Protocol, AI Agents, Software Testing, LLM-driven Development, MCP Servers, Agentic Workflows, Non-determinism, Software Architecture

## Chapters
- 1:00 — Guest Background: Fitz Nowlan discusses his journey from PhD research in distributed systems to leading AI architecture at SmartBear.
- 3:20 — The Challenge of Non-deterministic Testing: The difficulty of testing MCP servers where the LLM decides the tool sequence on the fly, making rigid workflows impossible.
- 7:50 — Prompt Engineering vs. Long-term Value: Why developers should avoid falling in love with specific prompts that may be rendered obsolete by newer, better models.
- 12:35 — The Future of Unit Testing: A debate on whether traditional unit testing loses relevance when AI can effectively handle assertion-based testing.
- 21:20 — Commoditization of CRUD Apps: How AI-driven development lowers margins for basic applications and shifts the focus toward more complex engineering problems.
- 27:50 — MCP as the New Foundation: The transition from LLMs being the core focus to MCP providing the essential infrastructure for agentic tool use.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/the-stack-overflow-podcast/episodes/how-can-you-test-your-code-when-you-don-t-know-what-s-in-it/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/the-stack-overflow-podcast/how-can-you-test-your-code-when-you-don-t-know-what-s-in-it.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.