# The Pre-Training Wall and the Treadmill After It

Page: https://stenobird.com/podcast/corecursive-coding-stories/the-pre-training-wall-and-the-treadmill-after-it
Text version: https://stenobird.com/podcast/corecursive-coding-stories/the-pre-training-wall-and-the-treadmill-after-it.md
Podcast: [CoRecursive: Coding Stories](https://stenobird.com/podcast/corecursive-coding-stories)
Published: 2026-05-09T10:00:00+00:00
Episode link: https://corecursive.com/the-pre-training-wall-and-the-treadmill-after-it/
Audio file: https://traffic.libsyn.com/secure/corecursive/121.mp3?dest-id=628353
Processing state: not_requested
JSON: https://stenobird.com/v1/public/podcasts/corecursive-coding-stories/episodes/the-pre-training-wall-and-the-treadmill-after-it
Duration seconds: 3370

## Resource

I've been confusing Don with frontier-lab links late at night for a bit. Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall. A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion. Don was confused. So he came over and we made an episode about it. Episode Page Support The Show Subscribe To The Podcast Join The Newsletter

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/corecursive-coding-stories/episodes/the-pre-training-wall-and-the-treadmill-after-it/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/corecursive-coding-stories/the-pre-training-wall-and-the-treadmill-after-it.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.