Episode
The Mysterious Math Behind LLMs - Anil Ananthaswamy - #537
- Published
- Jan 23, 2026
- Duration seconds
- 4256
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/into-the-impossible-with-brian-keating-324783/episodes/the-mysterious-math-behind-llms-anil-ananthaswamy-537/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/into-the-impossible-with-brian-keating-324783/the-mysterious-math-behind-llms-anil-ananthaswamy-537.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
WANTED: Developers and STEM experts! Get paid to create benchmarks and improve AI models. Sign up for Alignerr using our link: https://alignerr.com/?referral-source=briankeating One of the most powerful AI systems we’ve ever built is succeeding for reasons we still don’t understand. And worse, they may succeed for reasons that might lock us into the wrong future for humanity. Today’s guest is Anil Ananthaswamy, an award-winning science writer and one of the clearest thinkers on the mathematical foundations of machine learning. In this conversation, we’re not just talking about new demos, incremental improvements, or updates on new models being released. We’re asking even harder questions: Why does the mathematics of machine learning work at all? How do these models succeed when they suffer from problems like overparameterization and lack of training data? And are large language models revealing deep structure, or are they just producing very convincing illusions and causing us to face an increasingly AI-slop-driven future? KEY TAKEAWAYS 00:00 — Book explores why ML works through math 02:47 — Perceptron proof shows simple math guarantees learning 05:11 — Early AI failed due to single-layer limits 07:12 — Nonlinear limits caused the first AI winter 09:04 — Backpropagation revived neural networks 10:59 — GPUs + big data enabled deep learning 15:25 — AI success risks technological lock-in 17:30 — LLMs lack human-like learning and embodiment 22:57 — High-dimensional spaces power ML behavior 27:36 — Data saturation may slow future gains 31:11 — Continual learning is still missing in AI 33:46 — Neuromorphic chips promise energy efficiency 41:49 — Overparameterized models still generalize well 45:05 — SGD succeeds via randomness in complex landscapes 48:27 — Perceptrons remain…