Episode
EP 45: Why AI Agents Break in Production: The Missing Harness in Your Data Stack
- Podcast
- Data Science With Sam
- Published
- Jul 15, 2026
- Duration seconds
- 1952
- Processing state
not_requested- Canonical source
- https://rss.com/podcasts/data-science-with-sam/3006403
Actions
POST https://stenobird.com/v1/public/podcasts/data-science-with-sam-4345646/episodes/ep-45-why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/data-science-with-sam-4345646/ep-45-why-ai-agents-break-in-production-the-missing-harness-in-your-data-stack.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
An independent evaluation of Snowflake's Cortex Analyst found 6 in 10 AI-generated queries were wrong — but they all compiled and ran without a single error. That's not a model problem. That's a missing harness problem. Pradnesh Patil is the Co-Founder and CEO of Altimate AI — a platform bringing agentic AI to data engineering with tools trusted by Fortune 500s and downloaded more than 1 million times across 200+ countries. Before Altimate, he spent a decade in product leadership at Palo Alto Networks, Cisco, and VMware. IN THIS EPISODE: ▪ The five components of an agentic data engineering harness: context, governance, MCP tools, shared skills, and agent infrastructure — and why missing any one of them causes silent failures ▪ Why a system prompt cannot substitute for a harness: a prompt tells the model what to do, a harness tells it what is actually true ▪ Where the 27–33% phantom table references and 78% silent wrong joins come from — and why it's not the LLM's fault ▪ How Altimate Code topped ADE-Bench using Sonnet while competitors used Opus — proof that the harness matters more than the model ▪ The deterministic vs LLM boundary: why validation, cost checks, and query correctness are deterministic jobs and should never go to an LLM ▪ Context compaction innovation: why standard LLM compaction destroys long-running data engineering tasks — and how Altimate fixed it ▪ The $5,000 Cortex AI query bill — and how permission-based governance controls prevent agents from going rogue on your cloud bill ▪ The future of data engineering: days of writing SQL by hand are ending — what the data engineer's role becomes in an agentic world ▪ Pradnesh's advice: build open source, build cross-platform — avoid siloed AI feat…