{"podcast":{"title":"muckrAIkers","slug":"muckraikers-7026051","podcast_index_feed_id":7026051,"rss_url":"https://feeds.transistor.fm/muckraikers","website_url":"https://kairos.fm/muckraikers/","image_url":"https://img.transistorcdn.com/2Xoj9Q8V0g3EZ4QkfrR-DMBxWLBu5eO1bikePOFS7Ng/rs:fill:0:0:1/w:1400/h:1400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS83MzQ3/NzVkMTk3MWFiM2Nl/ZDRmNzFhMGQxZDQ3/MzM3Yi5wbmc.jpg","author":"Jacob Haimes","episode_count":25,"summary":"Join us as we dig a tiny bit deeper into the hype surrounding \"AI\" press releases, research papers, and more. Each episode, we'll highlight ongoing research and investigations, providing some much needed contextualization, constructive critique, and even a smidge of occasional good will teasing to the conversation, trying to find the meaning under all of this muck.","last_synced_at":"2026-07-15T04:21:26.211565+00:00","page_url":"https://stenobird.com/podcast/muckraikers-7026051"},"episode":{"title":"OpenAI's o1 System Card, Literally Migraine Inducing","slug":"openai-s-o1-system-card-literally-migraine-inducing","published_at":"2024-12-23T02:00:00+00:00","page_url":"https://stenobird.com/podcast/muckraikers-7026051/openai-s-o1-system-card-literally-migraine-inducing","show_page_url":"https://stenobird.com/podcast/muckraikers-7026051","url":"https://kairos.fm/muckraikers/e009/","audio_url":"https://op3.dev/e/media.transistor.fm/c2ec3a0e/32301ce9.mp3","summary":"The idea of model cards , which was introduced as a measure to increase transparency and understanding of LLMs, has been perverted into the marketing gimmick characterized by OpenAI's o1 system card. To demonstrate the adversarial stance we believe is necessary to draw meaning from these press-releases-in-disguise, we conduct a close read of the system card. Be warned, there's a lot of muck in this one. Note: All figures/tables discussed in the podcast can be found on the podcast website at https://kairos.fm/muckraikers/e009/ (00:00) - Recorded 2024.12.08 (00:54) - Actual intro (03:00) - System cards vs. academic papers (05:36) - Starting off sus (08:28) - o1.continued (12:23) - Rant #1: figure 1 (18:27) - A diamond in the rough (19:41) - Hiding copyright violations (21:29) - Rant #2: Jacob on \"hallucinations\" (25:55) - More ranting and \"hallucination\" rate comparison (31:54) - Fairness, bias, and bad science comms (35:41) - System, dev, and user prompt jailbreaking (39:28) - Chain-of-thought and Rao-Blackwellization (44:43) - \"Red-teaming\" (49:00) - Apollo's bit (51:28) - METR's bit (59:51) - Pass@??? (01:04:45) - SWE Verified (01:05:44) - Appendix bias metrics (01:10:17) - The muck and the meaning Links o1 system card OpenAI press release collection - 12 Days of OpenAI Additional o1 Coverage NIST + AISI [report] - US AISI and UK AISI Joint Pre-Deployment Test Apollo Research's paper - Frontier Models are Capable of In-context Scheming VentureBeat article - OpenAI launches full o1 model with image uploads and analysis, debuts ChatGPT Pro The Atlantic article - The GPT Era Is Already Ending On Data Labelers 60 Minutes article + video - Labelers training AI say they're overworked, underpaid and exploited by big American tech companies Reflections article - The hidden he…","meta_description":"The idea of model cards , which was introduced as a measure to increase transparency and understanding of LLMs, has been perverted into the marketing gimm…","key_points":[],"chapters":[],"topics":[],"duration_seconds":4597,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/muckraikers-7026051/episodes/openai-s-o1-system-card-literally-migraine-inducing/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/muckraikers-7026051/openai-s-o1-system-card-literally-migraine-inducing.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}