{"podcast":{"title":"The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)","slug":"twiml-ai-podcast","podcast_index_feed_id":1045879,"rss_url":"https://feeds.megaphone.fm/MLN2155636147","website_url":"https://twimlai.com","image_url":"https://megaphone.imgix.net/podcasts/35230150-ee98-11eb-ad1a-b38cbabcd053/image/TWIML_AI_Podcast_Official_Cover_Art_1400px.png?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress","author":"TWIML","episode_count":785,"summary":"Machine learning and artificial intelligence are dramatically changing the way businesses operate and people live. The TWIML AI Podcast brings the top minds and ideas from the world of ML and AI to a broad and influential community of ML/AI researchers, data scientists, engineers and tech-savvy business and IT leaders. Hosted by Sam Charrington, a sought after industry analyst, speaker, commentator and thought leader. Technologies covered include machine learning, artificial intelligence, deep learning, natural language processing, neural networks, analytics, computer science, data science and more.","last_synced_at":null,"page_url":"https://stenobird.com/podcast/twiml-ai-podcast"},"episode":{"title":"Rethinking Pre-Training for Agentic AI with Aakanksha Chowdhery - #759","slug":"rethinking-pre-training-for-agentic-ai-with-aakanksha-chowdhery-759","published_at":"2025-12-17T19:24:00+00:00","page_url":"https://stenobird.com/podcast/twiml-ai-podcast/rethinking-pre-training-for-agentic-ai-with-aakanksha-chowdhery-759","show_page_url":"https://stenobird.com/podcast/twiml-ai-podcast","url":"https://twimlai.com/podcast/twimlai/rethinking-pretraining-for-agentic-ai/","audio_url":"https://pscrb.fm/rss/p/traffic.megaphone.fm/MLN3462034138.mp3?updated=1766003076","summary":"To move beyond static benchmarks, AI pre-training must shift from simple next-token prediction to supporting multi-step reasoning and environmental interaction. Aakanksha Chowdhery argues that true agentic capabilities like error recovery and tool use require fundamental changes to training objectives and data trajectories.","meta_description":"Aakanksha Chowdhery discusses why pre-training must evolve beyond next-token prediction to enable agentic reasoning, planning, and tool use.","key_points":["Main idea: Agentic AI requires a fundamental rethink of pre-training objectives rather than relying solely on post-training refinements","Failure mode: Relying on static benchmarks like GSM8K fails to measure a model's ability to interact with dynamic environments","Practical takeaway: Training on 'trajectory' data is essential for teaching models to plan multiple steps ahead and recover from failed actions","Main idea: Scaling remains the primary driver for discovering emergent capabilities like cross-modal reasoning and dynamic tool learning","Practical takeaway: High-quality, representative data curation is just as critical as scale for achieving efficiency in modern models"],"chapters":[{"start_ms":60000,"title":"Foundations of Large Scale Pre-training","summary":"Aakanksha discusses her experience building the distributed systems for PaLM and Gemini, highlighting the complexities of scaling models to hundreds of billions of parameters."},{"start_ms":275000,"title":"The Need for Fundamental Pre-training Shifts","summary":"The limitations of current benchmarks and the argument that agentic capabilities cannot be solved through post-training alone."},{"start_ms":495000,"title":"Attention Mechanisms and Reasoning","summary":"How the attention mechanism serves as the fundamental engine for long-form reasoning and multi-step planning."},{"start_ms":975000,"title":"Beyond Next-Token Prediction","summary":"Examining why the standard next-token prediction objective is insufficient for the complex requirements of autonomous agents."},{"start_ms":1210000,"title":"Data Curation and Efficiency","summary":"Insights into how data quality and curation drive competition and efficiency in the current LLM landscape."},{"start_ms":1660000,"title":"Designing Better Benchmarks","summary":"The importance of breaking down real-world workflows into measurable sub-problems to create meaningful evaluation metrics."},{"start_ms":2165000,"title":"Predictive Planning and Error Recovery","summary":"The necessity of training models to 'think ahead' and develop the ability to self-correct during inference."}],"topics":["Agentic AI","Pre-training","Large Language Models","Machine Learning","Reasoning","Artificial Intelligence","Neural Networks","Model Evaluation"],"duration_seconds":3174,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/rethinking-pre-training-for-agentic-ai-with-aakanksha-chowdhery-759/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/twiml-ai-podcast/rethinking-pre-training-for-agentic-ai-with-aakanksha-chowdhery-759.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}