Episode
The Pre-Training Wall and the Treadmill After It
- Podcast
- CoRecursive: Coding Stories
- Published
- May 9, 2026
- Duration seconds
- 3370
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/corecursive-coding-stories/episodes/the-pre-training-wall-and-the-treadmill-after-it/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/corecursive-coding-stories/the-pre-training-wall-and-the-treadmill-after-it.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
I've been confusing Don with frontier-lab links late at night for a bit. Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall. A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion. Don was confused. So he came over and we made an episode about it. Episode Page Support The Show Subscribe To The Podcast Join The Newsletter