{"podcast":{"title":"Super Data Science: ML & AI Podcast with Jon Krohn","slug":"super-data-science","podcast_index_feed_id":220402,"rss_url":"https://feeds.megaphone.fm/SUPERDATASCIENCEPTYLTD9836501887","website_url":"https://www.superdatascience.com/podcast","image_url":"https://megaphone.imgix.net/podcasts/efa92454-1c31-11ef-9e30-03596b470c27/image/c3e0edc239c962f8bcd144000fafa5aa.jpeg?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress","author":"Jon Krohn","episode_count":991,"summary":"The latest machine learning, A.I., and data career topics from across both academia and industry are brought to you by host Dr. Jon Krohn on the Super Data Science Podcast. As the quantity of data on our planet doubles every couple of years and with this trend set to continue for decades to come, there's an unprecedented opportunity for you to make a meaningful impact in your lifetime. In conversation with the biggest names in the data science industry, Jon cuts through hype to fuel that professional impact. Whether you're curious about getting started in a data career or you're a deep technical expert, whether you'd like to understand what A.I. is or you'd like to integrate more data-driven processes into your business, we have inspiring guests and lighthearted conversation for you to enjoy. We cover tools, techniques, and implementation tricks across data collection, databases, analytics, predictive modeling, visualization, software engineering, real-world applications, commercialization, and entrepreneurship − everything you need to crush it with data science.","last_synced_at":null,"page_url":"https://stenobird.com/podcast/super-data-science"},"episode":{"title":"978: A Post-Transformer Architecture Crushes Sudoku (Transformers Solve ~0%)","slug":"978-a-post-transformer-architecture-crushes-sudoku-transformers-solve-0","published_at":"2026-03-27T11:00:00+00:00","page_url":"https://stenobird.com/podcast/super-data-science/978-a-post-transformer-architecture-crushes-sudoku-transformers-solve-0","show_page_url":"https://stenobird.com/podcast/super-data-science","url":"https://www.podtrac.com/pts/redirect.mp3/chrt.fm/track/E581B9/arttrk.com/p/VI4CS/pscrb.fm/rss/p/traffic.megaphone.fm/SUPERDATASCIENCEPTYLTD5403184044.mp3?updated=1774606789","audio_url":"https://www.podtrac.com/pts/redirect.mp3/chrt.fm/track/E581B9/arttrk.com/p/VI4CS/pscrb.fm/rss/p/traffic.megaphone.fm/SUPERDATASCIENCEPTYLTD5403184044.mp3?updated=1774606789","summary":"Leading LLMs like o3-mini and Claude 3.7 Sonnet fail completely at solving extreme Sudoku puzzles, scoring effectively 0% accuracy. Pathway's new BDH architecture achieves 97.4% accuracy by using a post-transformer design focused on internalized reasoning and constraint satisfaction.","meta_description":"Discover why Transformers fail at complex reasoning tasks like Sudoku and how Pathway's BDH architecture uses sparse activations to achieve 97.4% accuracy.","key_points":["Failure mode: Transformers struggle with constraint satisfaction because they process information token-by-token, locking in decisions without the ability to backtrack","Main idea: The BDH architecture uses sparse positive activations, activating only about 5% of neurons to mimic biological efficiency","Technical breakthrough: Unlike attention mechanisms, BDH is a state-based model that maintains and updates an internal state, similar to biological synaptic updates","Practical takeaway: Moving beyond text-based chain-of-thought toward 'generative strategy' could enable AI to solve complex problems in medicine, law, and operations","Current limitation: BDH has been demonstrated at a billion-parameter scale, and while promising, it has not yet reached the massive scale of frontier models like GPT-4"],"chapters":[{"start_ms":60000,"title":"The 0% Accuracy Problem","summary":"Leading LLMs fail at extreme Sudoku puzzles that humans can solve easily, exposing a fundamental weakness in transformer-based reasoning."},{"start_ms":105000,"title":"Sudoku as a Reasoning Benchmark","summary":"Why Sudoku serves as a perfect test for constraint satisfaction, search, and backtracking capabilities in AI."},{"start_ms":145000,"title":"The Transformer Bottleneck","summary":"An analysis of how token-by-token processing and limited latent space prevent Transformers from holding multiple candidate strategies in parallel."},{"start_ms":235000,"title":"Internalized Reasoning with BDH","summary":"Comparing the BDH architecture to a chess grandmaster who navigates search spaces through internalized patterns rather than verbalized steps."},{"start_ms":320000,"title":"Sparse Activations and Biological Plausibility","summary":"How BDH uses sparse positive activations to achieve efficiency and mimic the energy-saving mechanisms of the human brain."},{"start_ms":370000,"title":"State-Based Modeling","summary":"Exploring how BDH maintains an internal state through mechanisms related to Hebbian learning, moving beyond standard attention."},{"start_ms":455000,"title":"The Future of Generative Strategy","summary":"The potential for post-transformer architectures to move from summarizing text to generating complex, constraint-aware strategies."}],"topics":["Transformer Architecture","BDH Architecture","Machine Learning","Constraint Satisfaction","Artificial Intelligence","Neural Networks","Pathway","LLM Reasoning"],"duration_seconds":639,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/super-data-science/episodes/978-a-post-transformer-architecture-crushes-sudoku-transformers-solve-0/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/super-data-science/978-a-post-transformer-architecture-crushes-sudoku-transformers-solve-0.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}