# EP 38: The Local AI Stack Nobody Talks About (But Should)

Page: https://stenobird.com/podcast/data-science-with-sam-4345646/ep-38-the-local-ai-stack-nobody-talks-about-but-should
Text version: https://stenobird.com/podcast/data-science-with-sam-4345646/ep-38-the-local-ai-stack-nobody-talks-about-but-should.md
Podcast: [Data Science With Sam](https://stenobird.com/podcast/data-science-with-sam-4345646)
Published: 2026-04-22T10:00:00+00:00
Episode link: https://rss.com/podcasts/data-science-with-sam/3006411
Audio file: https://content.rss.com/episodes/395137/3006411/data-science-with-sam/2026_07_20_13_47_15_776f2908-b763-43f2-860e-aac2d8e2529c.mp3
Processing state: not_requested
JSON: https://stenobird.com/v1/public/podcasts/data-science-with-sam-4345646/episodes/ep-38-the-local-ai-stack-nobody-talks-about-but-should
Duration seconds: 2473

## Resource

You want to run AI locally. You have questions: What hardware do I actually need? Which framework should I use? How much will this cost? What's the realistic performance? In this episode, Sam brings back Trent Rossiter, founder of Logical Data Solutions, for a practical walkthrough of building a production-grade local AI lab. Trent has built real systems for enterprise clients, tested frameworks on multiple hardware stacks, and made the hardware choices that matter. This is not theory. This is what actually works. WHAT WE COVER: ▪  Hardware & Framework Choices: VRAM is the critical metric (not all VRAM is equal — memory throughput matters as much as capacity).  ▪  Model Architecture & Capability: Mixture of Experts (MoE) lets you fit more power into less VRAM by using fewer active parameters.  ▪  Real Enterprise Applications: Computer vision for quality assurance on assembly lines. Proprietary data handling without cloud exposure.  ▪  Your Starter Stack (All Free): Langflow (agentic workflow builder), Goose (MCP-enabled chat), AnythingLLM (with vector stores for RAG), MCP servers (Model Context Protocol — standardised tool integration).  ▪  Agentic AI & Security: OpenClaw is powerful but controversial — manages email, Telegram, calendars, creates sub-agents. Trent runs it in Docker on an isolated machine for safety. NVIDIA's NemoClaw is the enterprise version (security-first, nothing-allowed-by-default, explicit permissions). HARDWARE TRENT MENTIONS: NVIDIA DGX Spark — 128GB unified memory, CUDA stack Apple MacBook Pro/Mac mini — up to 512GB unified memory, market leader for personal AI AMD integrated AI PCs — emerging competitor NVIDIA RTX gaming cards (30/40/50/60 series) — high VRAM, high power consumption, co…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/data-science-with-sam-4345646/episodes/ep-38-the-local-ai-stack-nobody-talks-about-but-should/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/data-science-with-sam-4345646/ep-38-the-local-ai-stack-nobody-talks-about-but-should.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.