{"podcast":{"title":"Best AI papers explained","slug":"best-ai-papers-explained-7258006","podcast_index_feed_id":7258006,"rss_url":"https://anchor.fm/s/1026675f8/podcast/rss","website_url":"https://podcasters.spotify.com/pod/show/ehwkang","image_url":"https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/43252366/43252366-1744500070152-e62b760188d8.jpg","author":"Enoch H. Kang","episode_count":789,"summary":"Cut through the noise. We curate and break down the most important AI papers so you don’t have to.","last_synced_at":"2026-07-19T16:17:08.576018+00:00","page_url":"https://stenobird.com/podcast/best-ai-papers-explained-7258006"},"episode":{"title":"How Much Do Language Models Memorize?","slug":"how-much-do-language-models-memorize","published_at":"2026-07-09T05:52:54+00:00","page_url":"https://stenobird.com/podcast/best-ai-papers-explained-7258006/how-much-do-language-models-memorize","show_page_url":"https://stenobird.com/podcast/best-ai-papers-explained-7258006","url":"https://podcasters.spotify.com/pod/show/ehwkang/episodes/How-Much-Do-Language-Models-Memorize-e3lrl61","audio_url":"https://anchor.fm/s/1026675f8/podcast/play/122589825/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2026-6-9%2Ff2743d01-0f1a-bd3e-aaf0-800c7362b867.m4a","summary":"This research paper investigates language model capacity by introducing a new method to measure how much a model truly memorizes versus what it generalizes. The authors distinguish between unintended memorization, which is specific data storage, and generalization, which is the understanding of broader patterns. By testing the GPT family, they determine these models possess a storage capacity of approximately 3.6 bits-per-parameter. The study reveals that the double descent phenomenon occurs specifically when a dataset's size surpasses the model's total bit capacity. Furthermore, the researchers established scaling laws to predict the success of membership inference attacks, which identify if a specific datapoint was used in training. Their findings suggest that modern models are trained on so much data that standard membership inference is increasingly difficult for average samples.","meta_description":"This research paper investigates language model capacity by introducing a new method to measure how much a model truly memorizes versus what it generalize…","key_points":[],"chapters":[],"topics":[],"duration_seconds":1432,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/best-ai-papers-explained-7258006/episodes/how-much-do-language-models-memorize/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/best-ai-papers-explained-7258006/how-much-do-language-models-memorize.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}