{"podcast":{"title":"Training Data","slug":"training-data","podcast_index_feed_id":null,"rss_url":"https://feeds.megaphone.fm/trainingdata","website_url":"https://www.sequoiacap.com/","image_url":"https://megaphone.imgix.net/podcasts/8ca8a61c-08b4-11ef-97ab-9fe273d59030/image/4a16fd33b06708003827556209cd42b7.png?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress","author":"Sequoia Capital","episode_count":104,"summary":"Join us as we train our neural nets on the theme of the century: AI. Sonya Huang, Pat Grady and more Sequoia Capital partners host conversations with leading AI builders and researchers to ask critical questions and develop a deeper understanding of the evolving technologies—and their implications for technology, business and society. The content of this podcast does not constitute investment advice, an offer to provide investment advisory services, or an offer to sell or solicitation of an offer to buy an interest in any investment fund.","last_synced_at":"2026-08-02T15:06:49.231983+00:00","page_url":"https://stenobird.com/podcast/training-data"},"episode":{"title":"Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil","slug":"building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil","published_at":"2026-07-29T09:00:00+00:00","page_url":"https://stenobird.com/podcast/training-data/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil","show_page_url":"https://stenobird.com/podcast/training-data","url":"https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI4789823358.mp3","audio_url":"https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI4789823358.mp3","summary":"The transformer architecture has reached a scaling plateau where the next leap in intelligence requires architectural innovation rather than just more data. Core Automation founders Jerry Tworek and Rohan Anil discuss moving beyond pre-training and RL toward systems capable of true continual learning and test-time adaptation.","meta_description":"Core Automation founders discuss why the transformer architecture is hitting a wall and how an automated lab can discover the next era of AI architecture.","key_points":["Main idea: The current era of AI has mastered large-scale pre-training and RL, but the transformer architecture lacks the fundamental capability for continual learning","Failure mode: Transformers suffer from a lack of test-time adaptability and are increasingly reliant on distilling older models via internet-scale data","Practical takeaway: To find the next breakthrough, researchers must move away from making transformers more efficient and instead focus on making architectures more expressive","Main idea: The bottleneck for frontier labs is the release cycle; they are too locked into the current coding-agent race to experiment with radical architectural shifts","Practical takeaway: The path to superior architecture lies in building an automated lab that can execute high-frequency experiments, starting with automating kernel generation"],"chapters":[{"start_ms":60000,"title":"The Vision for Core Automation","summary":"Introduction to the founders and their transition from OpenAI and Google to building a new type of AI research lab."},{"start_ms":300000,"title":"The Limits of the Transformer","summary":"An analysis of why the transformer architecture is reaching its functional limits and why scaling alone is insufficient."},{"start_ms":720000,"title":"The Timing of Architectural Research","summary":"Discussing why the market is currently optimized for scaling and why the window for new architectural breakthroughs is opening."},{"start_ms":1140000,"title":"Beyond Scaling: The Need for Expressivity","summary":"The argument that current technology only scales to a subset of human intelligence and needs more powerful, adaptive structures."},{"start_ms":1380000,"title":"The Age of Experience and RL","summary":"Exploring the nuances of reinforcement learning versus behavioral cloning and the potential for more efficient learning algorithms."},{"start_ms":2040000,"title":"The Automated Lab Strategy","summary":"How Core Automation intends to use automation to iterate on kernels and architectures faster than traditional research cycles."},{"start_ms":2460000,"title":"Finding the Breakthrough","summary":"The search for the 'perfect plot' and the goal of creating systems that improve through their own daily operations."}],"topics":["Transformer Architecture","Artificial General Intelligence","Reinforcement Learning","Continual Learning","Machine Learning Research","Core Automation","Model Scaling","Test-time Adaptation"],"duration_seconds":2951,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/training-data/episodes/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/training-data/building-the-automated-agi-lab-core-automation-s-jerry-tworek-and-rohan-anil.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}