Episode
Episode 70: 1,400 Production AI Deployments
- Podcast
- Vanishing Gradients
- Published
- Feb 12, 2026
- Duration seconds
- 4192
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/vanishing-gradients-4989163/episodes/episode-70-1-400-production-ai-deployments/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/vanishing-gradients-4989163/episode-70-1-400-production-ai-deployments.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
There’s a company who spent almost $50,000 because an agent went into an infinite loop and they forgot about it for a month. It had no failures and I guess no one was monitoring these costs. It’s nice that people do write about that in the database as well. After it happened, they said: watch out for infinite loops. Watch out for cascading tool failures. Watch out for silent failures where the agent reports it has succeeded when it didn’t! We Discuss: * Why the most successful teams are ripping out and rebuilding their agent systems every few weeks as models improve, and why over-engineering now creates technical debt you can’t afford later; * The $50,000 infinite loop disaster and why “silent failures” are the biggest risk in production: agents confidently report success while spiraling into expensive mistakes; * How ELIOS built emergency voice agents with sub-400ms response times by aggressively throwing away context every few seconds, and why these extreme patterns are becoming standard practice; * Why DoorDash uses a three-tier agent architecture (manager, progress tracker, and specialists) with a persistent workspace that lets agents collaborate across hours or days; * Why simple text files and markdown are emerging as the best “continual learning” layer: human-readable memory that persists across sessions without fine-tuning models; * The 100-to-1 problem : for every useful output, tool-calling agents generate 100 tokens of noise, and the three tactics (reduce, offload, isolate) teams use to manage it; * Why companies are choosing Gemini Flash for document processing and Opus for long reasoning chains , and how to match models to your actual usage patterns; * The debate over vector databases versus simple grep and cat , and why giving agents standard command-line…