Stenobird

Episode

Baseten CEO and co-founder Tuhin Srivastava on inference and feedback loops

Podcast
Scaling DevTools
Published
Nov 14, 2025
Duration seconds
1451
Processing state
processed
Canonical source
https://podcast.scalingdevtools.com/episodes/scaling-ai-infrastructure-with-basetens-tuhin-srivastava
Audio
https://media.transistor.fm/5e289b9a/31f1913a.mp3
JSON
/v1/public/podcasts/scaling-devtools/episodes/baseten-ceo-and-co-founder-tuhin-srivastava-on-inference-and-feedback-loops
Markdown
/podcast/scaling-devtools/baseten-ceo-and-co-founder-tuhin-srivastava-on-inference-and-feedback-loops.md

Actions

  • POST https://stenobird.com/v1/public/podcasts/scaling-devtools/episodes/baseten-ceo-and-co-founder-tuhin-srivastava-on-inference-and-feedback-loops/transcription-requests
    Idempotently request low-priority transcript generation for this episode.
  • GET https://stenobird.com/podcast/scaling-devtools/baseten-ceo-and-co-founder-tuhin-srivastava-on-inference-and-feedback-loops.md
    Read the agent-friendly Markdown representation of this episode resource.

Summary

Baseten CEO Tuhin Srivastava discusses the strategic shift from building ML primitives to scaling a high-growth GenAI inference platform. He explores how to navigate rapid market shifts by prioritizing developer experience and tight customer feedback loops.

Topics

  • GenAI
  • Inference Infrastructure
  • Developer Experience
  • Machine Learning
  • DevTools
  • Scaling Startups
  • Cloud Computing
  • Software Engineering

Highlights

  • Main idea: As model capabilities commoditize, the real value shifts to the underlying inference infrastructure and developer experience
  • Practical takeaway: Minimize the distance between you and your customers to create ultra-fast feedback loops
  • Failure mode: Avoid making irreversible early architectural abstractions before you have fully honed the product's developer experience
  • Strategic insight: The future of enterprise software lies in being either AI-first or AI-enabled, driving massive demand for inference-centric tools
  • Product philosophy: Focus on making the easy things easy and the hard things possible without blocking the developer's workflow

Chapters

  1. 1:00 The Pre-GenAI Build Phase: Reflecting on the years spent building machine learning primitives before the massive market demand arrived in 2023.
  2. 2:45 Navigating Market Shifts: How Baseten repositioned itself to meet the sudden explosion of interest in generative AI.
  3. 8:05 The Power of Feedback Loops: Why small companies must prioritize direct, rapid communication with customers to iterate effectively.
  4. 11:40 The Future of Inference: Discussing why every company will eventually need inference-based infrastructure as models become commoditized.
  5. 15:20 Prioritizing Developer Experience: The importance of building tools that provide observability and ease of use without creating friction.
  6. 20:45 Advice for DevTool Founders: Lessons on managing early technical decisions and the importance of staying close to the user.