{"podcast":{"title":"Gradient Dissent: Conversations on AI","slug":"gradient-dissent","podcast_index_feed_id":1020509,"rss_url":"https://feeds.captivate.fm/gradient-dissent/","website_url":"https://wandb.ai/site/resources/podcast","image_url":"https://artwork.captivate.fm/25fd1181-b46e-459b-85a5-d397eec4cdcf/JDLDW81K-wlJoAWL7ZnxLdTp.jpg","author":"Lukas Biewald","episode_count":136,"summary":"Join Lukas Biewald on Gradient Dissent, an AI-focused podcast brought to you by Weights & Biases. Dive into fascinating conversations with industry giants from NVIDIA, Meta, Google, Lyft, OpenAI, and more. Explore the cutting-edge of AI and learn the intricacies of bringing models into production.","last_synced_at":null,"page_url":"https://stenobird.com/podcast/gradient-dissent"},"episode":{"title":"The Startup Powering The Data Behind AGI","slug":"the-startup-powering-the-data-behind-agi","published_at":"2025-09-16T10:00:00+00:00","page_url":"https://stenobird.com/podcast/gradient-dissent/the-startup-powering-the-data-behind-agi","show_page_url":"https://stenobird.com/podcast/gradient-dissent","url":"https://wandb.ai/site/resources/podcast","audio_url":"https://episodes.captivate.fm/episode/becd4fd5-189b-4644-b956-4efd1c5756c1.mp3","summary":"Surge AI CEO Edwin Chen explains why high-quality, expert-led human data is the critical bottleneck for frontier LLMs. He argues that traditional labeling is broken and that the future of AGI depends on moving beyond simple benchmarks toward complex, multi-day reasoning tasks.","meta_description":"Learn how Surge AI is powering AGI by replacing low-quality labeling with high-skill human expertise in math, coding, and scientific reasoning.","key_points":["Main idea: The industry is moving from simple classification to high-complexity tasks requiring days of human expertise","Failure mode: Relying on inter-annotator agreement or simple checkboxes fails to capture subjective quality in creative or complex domains","Practical takeaway: Effective model training requires understanding the researcher's underlying goal rather than just following rigid instructions","Critical insight: Benchmark hacking on academic datasets is creating a disconnect between leaderboard performance and real-world utility","Future trend: The ratio of spend on data versus compute should increase as models require more nuanced, specialized human feedback"],"chapters":[{"start_ms":60000,"title":"The Data Collection Landscape","summary":"An overview of the massive, constant spend required for data in foundation model training."},{"start_ms":315000,"title":"Scaling Human Networks","summary":"How Surge initially sourced workers and built its early network."},{"start_ms":810000,"title":"The Myth of the PhD Solution","summary":"Why simply hiring experts like PhDs doesn't solve the fundamental problems of data quality."},{"start_ms":1570000,"title":"The Shift to High-Cognitive Tasks","summary":"Moving from five-second labeling tasks to complex problems that take days to solve."},{"start_ms":1825000,"title":"The Danger of Benchmark Hacking","summary":"How optimizing for leaderboards like LMSYS can degrade real-world model performance."},{"start_ms":2360000,"title":"Data for Scientific Discovery","summary":"The role of specialized data in training models for chemistry and advanced reasoning."},{"start_ms":2860000,"title":"Synthetic vs. Human Data","summary":"The limitations of synthetic data and the necessity of messy, real-world human inputs."}],"topics":["LLM Training","Data Labeling","AGI","Reinforcement Learning","Machine Learning Benchmarks","Human-in-the-loop","Synthetic Data","Surge AI"],"duration_seconds":3375,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/the-startup-powering-the-data-behind-agi/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/gradient-dissent/the-startup-powering-the-data-behind-agi.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}