Episode
What comes after attention? This startup says it already knows.
- Podcast
- The New Stack Podcast
- Published
- Jul 7, 2026
- Duration seconds
- 1219
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/the-new-stack-podcast-1092634/episodes/what-comes-after-attention-this-startup-says-it-already-knows/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/the-new-stack-podcast-1092634/what-comes-after-attention-this-startup-says-it-already-knows.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
Subquadratic is beginning to back up its ambitious claims with benchmarks and third-party validation for its SubQ 1.1 Small model, which uses its proprietary Sparse Attention (SSA) architecture to dramatically improve long-context performance. Rather than comparing every token to every other token, SSA selectively processes relationships, enabling near-linear scaling while maintaining high accuracy across context windows of up to 12 million tokens. The company reports near-perfect retrieval performance, competitive coding and reasoning benchmarks, and compute savings of up to 1,000x at maximum context lengths.