{"podcast":{"title":"Smart Enterprises: AI Frontiers","slug":"smart-enterprises-ai-frontiers-7077810","podcast_index_feed_id":7077810,"rss_url":"https://anchor.fm/s/fcb30f54/podcast/rss","website_url":"https://podcasters.spotify.com/pod/show/ali-mehedi0","image_url":"https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/42295933/42295933-1729711732301-3d9f09647417d.jpg","author":"Ali Mehedi","episode_count":93,"summary":"Welcome to Smart Enterprises: AI Frontiers, where we explore the cutting-edge of AI technology and its impact on enterprise and business transformation. Join us as we dive into the latest innovations, strategies, and success stories, helping businesses harness the power of AI to stay competitive in an ever-evolving market. Whether you're an industry leader or just getting started with AI, this podcast is your go-to resource for actionable insights and expert analysis.","last_synced_at":null,"page_url":"https://stenobird.com/podcast/smart-enterprises-ai-frontiers-7077810"},"episode":{"title":"LLM Unpacked: A Deep Dive into Modern AI Architectures","slug":"llm-unpacked-a-deep-dive-into-modern-ai-architectures","published_at":"2025-07-29T22:17:15+00:00","page_url":"https://stenobird.com/podcast/smart-enterprises-ai-frontiers-7077810/llm-unpacked-a-deep-dive-into-modern-ai-architectures","show_page_url":"https://stenobird.com/podcast/smart-enterprises-ai-frontiers-7077810","url":"https://podcasters.spotify.com/pod/show/ali-mehedi0/episodes/LLM-Unpacked-A-Deep-Dive-into-Modern-AI-Architectures-e3670qd","audio_url":"https://anchor.fm/s/fcb30f54/podcast/play/106184973/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2025-6-29%2F404788191-44100-2-6d2666ff4b505.m4a","summary":"Join us for an insightful exploration into the cutting-edge design of today's Large Language Models . Seven years on from the original GPT architecture, have we truly seen groundbreaking changes, or are we simply refining existing foundations? This podcast focuses on the architectural developments that define flagship open models in 2025, moving beyond benchmark performance or training algorithms. In this episode, we'll unpack the key ingredients contributing to LLM performance, examining how developers are pushing the boundaries of efficiency, memory management, and training stability. Discover the evolution and intricacies of: Attention Mechanisms: From Multi-Head Attention (MHA) to the more efficient Grouped-Query Attention (GQA), and innovative approaches like Multi-Head Latent Attention (MLA) used in DeepSeek-V3, which compresses key and value tensors for memory savings. We also delve into Sliding Window Attention from Gemma 3, which restricts context size for local efficiency. Normalization Layers: Explore the shift from LayerNorm to RMSNorm and the crucial placement of these layers (Pre-Norm, Post-Norm) as seen in OLMo 2 and Gemma 3, including the addition of QK-Norm for enhanced training stability. Mixture-of-Experts (MoE): Understand why this approach has seen a significant resurgence in 2025. Learn how MoE, as implemented in models like DeepSeek-V3, Llama 4, and Qwen3's sparse variants, allows for massive total parameter counts (e.g., DeepSeek-V3's 671 billion parameters) while activating only a small subset (e.g., 37 billion) per inference step for remarkable efficiency. Positional Embeddings: Discover how positional information is handled, from rotational positional embeddings (RoPE) to the radical concept of No Positional Embeddings (NoPE) in SmolLM3, whic…","meta_description":"Join us for an insightful exploration into the cutting-edge design of today's Large Language Models . Seven years on from the original GPT architecture, h…","key_points":[],"chapters":[],"topics":[],"duration_seconds":2501,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/smart-enterprises-ai-frontiers-7077810/episodes/llm-unpacked-a-deep-dive-into-modern-ai-architectures/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/smart-enterprises-ai-frontiers-7077810/llm-unpacked-a-deep-dive-into-modern-ai-architectures.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}