Episode
"How concerned should we be about OpenAI’s recurrent architecture rumors?" by Rauno Arike
- Published
- Sep 3, 2026
- Duration seconds
- 1131
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/lesswrong-curated-popular-5643401/episodes/how-concerned-should-we-be-about-openai-s-recurrent-architecture-rumors-by-rauno-arike/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/lesswrong-curated-popular-5643401/how-concerned-should-we-be-about-openai-s-recurrent-architecture-rumors-by-rauno-arike.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
Yesterday, The Information reported that OpenAI's upcoming model, Astra, is built with a looped transformer architecture. Given that Zvi sounds (understandably) tired and this topic is somewhat in my wheelhouse, I'll try to spare him this one and provide a Zvi-style overview of what we know about the situation. I'll cover Astra's likely architecture and the case for and against concern. I'll also discuss how neuralese concerns should change with increases in hidden serial depth. What archi...