Episode
AI Inference Costs Are Crushing SaaS Gross Margins — Here's What to Do About It
- Podcast
- SaaS Metrics School
- Published
- Apr 21, 2026
- Duration seconds
- 359
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/saas-metrics-school-6467825/episodes/ai-inference-costs-are-crushing-saas-gross-margins-here-s-what-to-do-about-it/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/saas-metrics-school-6467825/ai-inference-costs-are-crushing-saas-gross-margins-here-s-what-to-do-about-it.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
Is your AI SaaS company skating on thin ice because of exploding compute costs you're not tracking? In episode #365, Ben Murray tackles one of the most pressing financial challenges facing AI-first SaaS companies: the structural margin compression caused by LLM inference costs. Traditional SaaS was built on near-zero marginal cost per customer — that era is over. If you're building on top of AI, every prompt, query, and agentic workflow is a hard COGS line that scales with revenue, and if you're not managing it, it will quietly destroy your unit economics. Why AI-first SaaS companies are running 50–60% gross margins (vs. 70–80% for legacy SaaS) — and what Bessemer data shows about AI supernovas with margins as low as 25%. How inference and compute costs differ fundamentally from traditional SaaS COGS — and why they won't scale down the way hosting costs did Why token costs vary wildly (from $1–2 per million to $30–180+ for frontier models) and how that variability makes feature-level economics a CFO priority 5 tactical ways to reduce LLM spend: model routing, prompt caching, context compaction, semantic caching, and batch processing How to set up your GL accounts and COGS tracking to allocate inference costs by feature — so you actually understand the economics of what you've built Tune in before your next board meeting — because if you're not tracking AI inference costs at the feature level, you're flying blind on your most important unit economics. Resources Mentioned The SaaS CFO: https://www.thesaascfo.com/ Ray Rike — AI to ROI Newsletter: https://ai2roi.substack.com/ Tomas Tunguz: https://tomtunguz.com/ Fungies.io — 5 Ways to Save on LLM Costs: https://fungies.io