# Sergey Levine - Building LLMs for the Physical World - [Invest Like the Best, EP.465]

Page: https://stenobird.com/podcast/invest-like-the-best/sergey-levine-building-llms-for-the-physical-world-invest-like-the-best-ep-465
Text version: https://stenobird.com/podcast/invest-like-the-best/sergey-levine-building-llms-for-the-physical-world-invest-like-the-best-ep-465.md
Podcast: [Invest Like the Best with Patrick O'Shaughnessy](https://stenobird.com/podcast/invest-like-the-best)
Published: 2026-03-31T08:00:00+00:00
Episode link: https://colossus.com/episode/building-general-physical-intelligence
Audio file: https://traffic.megaphone.fm/CLS7427739184.mp3?updated=1774904050
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/invest-like-the-best/episodes/sergey-levine-building-llms-for-the-physical-world-invest-like-the-best-ep-465
Duration seconds: 3995

## Resource

Sergey Levine argues that the path to general-purpose robotics lies in building foundation models that learn across diverse environments rather than training narrow specialists. He explores the tension between simulation-heavy humanoid training and real-world data-driven manipulation.

## Highlights
- Main idea: Scalability in robotics comes from generality and the ability for models to improve through diverse, unlabelled data
- Failure mode: Over-optimizing for 'cool' robotic feats like backflips instead of focusing on practical, useful utility in everyday environments
- Technical tension: The divide between simulation-heavy approaches for humanoids and real-world data-heavy approaches for robotic manipulation
- Practical takeaway: End-to-end learning and the 'bitter lesson' of not programming machines manually are central to achieving robotic generality
- Future outlook: The potential for models to move beyond language-based instructions to more complex, multi-modal sensory inputs

## Topics

Robotics, Foundation Models, Embodied AI, Machine Learning, Reinforcement Learning, Physical Intelligence, Automation, Computer Vision

## Chapters
- 6:00 — The Challenge of Generality: Discussing the difficulties of scaling robotic learning when moving from specific tasks to general-purpose capabilities.
- 11:00 — History of End-to-End Control: A look back at the origins of end-to-end learning in autonomous driving systems from the 1980s.
- 21:10 — Utility vs. Novelty: Evaluating the strategy of building useful robotic systems that can be applied to various real-world tasks.
- 26:10 — Learning from High-Level Instructions: How models are beginning to improve through supervision using high-level human instructions in new environments.
- 41:40 — The Bitter Lesson in Robotics: Exploring the importance of general-purpose learning and the debate over end-to-end learning architectures.
- 52:20 — The Economics of Technological Change: Reflecting on how advancements in AI and coding tools alter the landscape for software engineering and business.
- 57:30 — Evaluating AI Progress: Distinguishing between impressive social media demonstrations and the actual underlying capability of AI models.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/invest-like-the-best/episodes/sergey-levine-building-llms-for-the-physical-world-invest-like-the-best-ep-465/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/invest-like-the-best/sergey-levine-building-llms-for-the-physical-world-invest-like-the-best-ep-465.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.