{"podcast":{"title":"Machine Learning Street Talk (MLST)","slug":"machine-learning-street-talk","podcast_index_feed_id":781643,"rss_url":"https://anchor.fm/s/1e4a0eac/podcast/rss","website_url":"https://podcasters.spotify.com/pod/show/machinelearningstreettalk","image_url":"https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/4981699/4981699-1757416025703-f026fa81b6d04.jpg","author":"Machine Learning Street Talk (MLST)","episode_count":250,"summary":"Welcome! We engage in fascinating discussions with pre-eminent figures in the AI field. Our flagship show covers current affairs in AI, cognitive science, neuroscience and philosophy of mind with in-depth analysis. Our approach is unrivalled in terms of scope and rigour – we believe in intellectual diversity in AI, and we touch on all of the main ideas in the field with the hype surgically removed. MLST is run by Tim Scarfe, Ph.D (https://www.linkedin.com/in/ecsquizor/) and features regular appearances from MIT Doctor of Philosophy Keith Duggar (https://www.linkedin.com/in/dr-keith-duggar/).","last_synced_at":null,"page_url":"https://stenobird.com/podcast/machine-learning-street-talk"},"episode":{"title":"Prof. Subbarao Kambhampati - LLMs don't reason, they memorize (ICML2024 2/13)","slug":"prof-subbarao-kambhampati-llms-don-t-reason-they-memorize-icml2024-2-13","published_at":"2024-07-29T18:32:42+00:00","page_url":"https://stenobird.com/podcast/machine-learning-street-talk/prof-subbarao-kambhampati-llms-don-t-reason-they-memorize-icml2024-2-13","show_page_url":"https://stenobird.com/podcast/machine-learning-street-talk","url":"https://podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/Prof--Subbarao-Kambhampati---LLMs-dont-reason--they-memorize-ICML2024-213-e2mjcse","audio_url":"https://anchor.fm/s/1e4a0eac/podcast/play/89813326/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2024-6-29%2F24b87b76-4192-b8fd-013e-4d817810331f.mp3","summary":"Prof. Subbarao Kambhampati argues that while LLMs are impressive and useful tools, especially for creative tasks, they have fundamental limitations in logical reasoning and cannot provide guarantees about the correctness of their outputs. He advocates for hybrid approaches that combine LLMs with external verification systems. MLST is sponsored by Brave: The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api. TOC (sorry the ones baked into the MP3 were wrong apropos due to LLM hallucination!) [00:00:00] Intro [00:02:06] Bio [00:03:02] LLMs are n-gram models on steroids [00:07:26] Is natural language a formal language? [00:08:34] Natural language is formal? [00:11:01] Do LLMs reason? [00:19:13] Definition of reasoning [00:31:40] Creativity in reasoning [00:50:27] Chollet's ARC challenge [01:01:31] Can we reason without verification? [01:10:00] LLMs cant solve some tasks [01:19:07] LLM Modulo framework [01:29:26] Future trends of architecture [01:34:48] Future research directions Youtube version: https://www.youtube.com/watch?v=y1WnHpedi2A Refs: (we didn't have space for URLs here, check YT video description instead) Can LLMs Really Reason and Plan? On the Planning Abilities of Large Language Models : A Critical Investigation Chain of Thoughtlessness? An Analysis of CoT in Planning On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to…","meta_description":"Prof. Subbarao Kambhampati argues that while LLMs are impressive and useful tools, especially for creative tasks, they have fundamental limitations in log…","key_points":[],"chapters":[],"topics":[],"duration_seconds":6147,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/machine-learning-street-talk/episodes/prof-subbarao-kambhampati-llms-don-t-reason-they-memorize-icml2024-2-13/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/machine-learning-street-talk/prof-subbarao-kambhampati-llms-don-t-reason-they-memorize-icml2024-2-13.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}