Episode

Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil

Podcast: Training Data
Published: Jul 29, 2026
Duration seconds: 2951
Processing state: processed
Canonical source: https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI4789823358.mp3
Audio: https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI4789823358.mp3
JSON: /v1/public/podcasts/training-data/episodes/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil
Markdown: /podcast/training-data/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil.md

Actions

POST https://stenobird.com/v1/public/podcasts/training-data/episodes/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil/transcription-requests
Idempotently request low-priority transcript generation for this episode.
GET https://stenobird.com/podcast/training-data/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil.md
Read the agent-friendly Markdown representation of this episode resource.

Summary

The transformer architecture has reached a scaling plateau where the next leap in intelligence requires architectural innovation rather than just more data. Core Automation founders Jerry Tworek and Rohan Anil discuss moving beyond pre-training and RL toward systems capable of true continual learning and test-time adaptation.

Topics

Transformer Architecture
Artificial General Intelligence
Reinforcement Learning
Continual Learning
Machine Learning Research
Core Automation
Model Scaling
Test-time Adaptation

Highlights

Main idea: The current era of AI has mastered large-scale pre-training and RL, but the transformer architecture lacks the fundamental capability for continual learning
Failure mode: Transformers suffer from a lack of test-time adaptability and are increasingly reliant on distilling older models via internet-scale data
Practical takeaway: To find the next breakthrough, researchers must move away from making transformers more efficient and instead focus on making architectures more expressive
Main idea: The bottleneck for frontier labs is the release cycle; they are too locked into the current coding-agent race to experiment with radical architectural shifts
Practical takeaway: The path to superior architecture lies in building an automated lab that can execute high-frequency experiments, starting with automating kernel generation

Chapters

1:00 The Vision for Core Automation: Introduction to the founders and their transition from OpenAI and Google to building a new type of AI research lab.
5:00 The Limits of the Transformer: An analysis of why the transformer architecture is reaching its functional limits and why scaling alone is insufficient.
12:00 The Timing of Architectural Research: Discussing why the market is currently optimized for scaling and why the window for new architectural breakthroughs is opening.
19:00 Beyond Scaling: The Need for Expressivity: The argument that current technology only scales to a subset of human intelligence and needs more powerful, adaptive structures.
23:00 The Age of Experience and RL: Exploring the nuances of reinforcement learning versus behavioral cloning and the potential for more efficient learning algorithms.
34:00 The Automated Lab Strategy: How Core Automation intends to use automation to iterate on kernels and architectures faster than traditional research cycles.
41:00 Finding the Breakthrough: The search for the 'perfect plot' and the goal of creating systems that improve through their own daily operations.