{"podcast":{"title":"The Generative AI Meetup Podcast","slug":"generative-ai-meetup","podcast_index_feed_id":6763937,"rss_url":"https://podcast.genaimeetup.com/feed.xml","website_url":"https://podcast.genaimeetup.com","image_url":"https://pbcdn1.podbean.com/imglogo/image-logo/17891792/Gemini_Generated_Image_sqmm5isqmm5isqmm_minified.png","author":"Mark and Shashank","episode_count":70,"summary":"Hosted by Mark and Shashank, software engineers and organizers in Silicon Valley. Get their grounded perspective each week as they explore the generative AI landscape through news analysis, tech discussions, hands-on experiments, and clear explanations. Dive into the latest language models, AI agent capabilities, and RAG techniques. Understand the hardware race, key research, startup trends, benchmarks, and the real-world impact of AI across industries like healthcare, robotics, and creative work. We also test AI limits, explain core concepts, discuss ethics, and interview builders shaping the field. For engineers, developers, researchers, and anyone seeking a practical understanding of AI’s rapid evolution and its applications.","last_synced_at":null,"page_url":"https://stenobird.com/podcast/generative-ai-meetup"},"episode":{"title":"Can you trust LLM Leaderboards?","slug":"can-you-trust-llm-leaderboards","published_at":"2025-03-17T13:34:10+00:00","page_url":"https://stenobird.com/podcast/generative-ai-meetup/can-you-trust-llm-leaderboards","show_page_url":"https://stenobird.com/podcast/generative-ai-meetup","url":"https://podcast.genaimeetup.com/e/can-you-trust-llm-leaderboards/","audio_url":"https://mcdn.podbean.com/mf/web/m72hinnunkyxa85b/audio_18dfg9.mp3","summary":"This conversation delves into the latest developments in AI, particularly focusing on Google's Gemma models and their capabilities. The discussion covers the differences between various types of language models, the significance of multimodal inputs, and the training techniques employed in AI models. The hosts also explore the implications of open-source versus proprietary models, the hardware requirements for running these models, and the limitations of benchmarks in evaluating AI performance. Additionally, they touch on the future of robotics and the cultural differences in AI adoption, particularly between Japan and the United States.takeaways Open source models are pushing the boundaries of AI.Gemma models are capable of multimodal inputs.Different types of LLMs serve different purposes.Benchmarks can be misleading and should be approached with caution.Training techniques like RLHF are crucial for model performance.The hardware requirements for AI models vary significantly.Cultural differences affect the adoption of robotics and AI.Robots are increasingly filling labor gaps in societies with declining populations.AI benchmarks should be tailored to specific use cases.The future of robotics and AI feels imminent and exciting. Chapters 00:00 Introduction to the Week's AI Developments00:50 Exploring Google's Gemma Models03:21 Understanding Different Types of LLMs05:32 Gemma's Multimodal and Multilingual Capabilities08:45 Training Techniques Behind Gemma15:48 Open Source Models and Their Impact20:34 Benchmarking AI Models28:30 Gaming Benchmarks in AI34:10 The Ethics of Benchmarking in AI44:56 Language Learning and AI Models49:12 The Importance of Benchmarks52:35 Vibe Checks and User Preferences01:01:09 Top AI Models and Their Performance01:13:35 Robotics and the Future…","meta_description":"This conversation delves into the latest developments in AI, particularly focusing on Google's Gemma models and their capabilities. The discussion covers…","key_points":[],"chapters":[],"topics":[],"duration_seconds":5388,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/generative-ai-meetup/episodes/can-you-trust-llm-leaderboards/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/generative-ai-meetup/can-you-trust-llm-leaderboards.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}