{"podcast":{"title":"Latent Space: The AI Engineer Podcast","slug":"latent-space-ai-engineer","podcast_index_feed_id":6058902,"rss_url":"https://api.substack.com/feed/podcast/1084089.rss","website_url":"https://www.latent.space/podcast","image_url":"https://substackcdn.com/feed/podcast/1084089/ca7468da5614a246d2906ee8926f6de7.jpg","author":"Latent.Space","episode_count":204,"summary":"The AI Engineer newsletter + Top technical AI podcast. How leading labs build Agents, Models, Infra, & AI for Science. See https://latent.space/about for highlights from Greg Brockman, Andrej Karpathy, George Hotz, Simon Willison, Soumith Chintala et al!","last_synced_at":null,"page_url":"https://stenobird.com/podcast/latent-space-ai-engineer"},"episode":{"title":"World Models & General Intuition: Khosla's largest bet since LLMs & OpenAI","slug":"world-models-general-intuition-khosla-s-largest-bet-since-llms-openai","published_at":"2025-12-06T16:00:00+00:00","page_url":"https://stenobird.com/podcast/latent-space-ai-engineer/world-models-general-intuition-khosla-s-largest-bet-since-llms-openai","show_page_url":"https://stenobird.com/podcast/latent-space-ai-engineer","url":"https://www.latent.space/p/world-models-and-general-intuition","audio_url":"https://api.substack.com/feed/podcast/186610516/549025ec6e575cd11565a6db9aeeb42f.mp3","summary":"From building Medal into a 12M-user game clipping platform with 3.8B highlight moments to turning down a reported $500M offer from OpenAI ( https://www.theinformation.com/articles/openai-offered-pay-500-million-startup-videogame-data ) and raising a $134M seed from Khosla ( https://techcrunch.com/2025/10/16/general-intuition-lands-134m-seed-to-teach-agents-spatial-reasoning-using-video-game-clips/ ) to spin out General Intuition , Pim is betting that world models trained on peak human gameplay are the next frontier after LLMs. We sat down with Pim to dig into why game highlights are “episodic memory for simulation” (and how Medal’s privacy-first action labels became a world-model goldmine https://medal.tv/blog/posts/enabling-state-of-the-art-security-and-protections-on-medals-new-apm-and-controller-overlay-features ), what it takes to build fully vision-based agents that just see frames and output actions in real time, how General Intuition transfers from games to real-world video and then into robotics, why world models and LLMs are complementary rather than rivals, what founders with proprietary datasets should know before selling or licensing to labs, and his bet that spatial-temporal foundation models will power 80% of future atoms-to-atoms interactions in both simulation and the real world. We discuss: * How Medal’s 3.8B action-labeled highlight clips became a privacy-preserving goldmine for world models * Building fully vision-based agents that only see frames and output actions yet play like (and sometimes better than) humans * Transferring from arcade-style games to realistic games to real-world video using the same perception–action recipe * Why world models need actions, memory, and partial observability (smoke, occlusion, camera shake) vs. “just” pretty vide…","meta_description":"From building Medal into a 12M-user game clipping platform with 3.8B highlight moments to turning down a reported $500M offer from OpenAI ( https://www.th…","key_points":[],"chapters":[],"topics":[],"duration_seconds":3857,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/latent-space-ai-engineer/episodes/world-models-general-intuition-khosla-s-largest-bet-since-llms-openai/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/latent-space-ai-engineer/world-models-general-intuition-khosla-s-largest-bet-since-llms-openai.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}