Episode
Red-Teaming after Mythos — Zico Kolter & Matt Fredrikson, Gray Swan
- Published
- Jun 22, 2026
- Duration seconds
- 3983
- Processing state
processed- Canonical source
- https://www.latent.space/p/gray-swan
Actions
POST https://stenobird.com/v1/public/podcasts/latent-space-ai-engineer/episodes/red-teaming-after-mythos-zico-kolter-matt-fredrikson-gray-swan/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/latent-space-ai-engineer/red-teaming-after-mythos-zico-kolter-matt-fredrikson-gray-swan.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
AI agents introduce a new class of vulnerabilities, specifically indirect prompt injections, that traditional cybersecurity cannot solve. Gray Swan founders Zico Kolter and Matt Fredrikson explain why we must move beyond simple guardrails toward automated, scalable red-teaming and robust safety systems.
Topics
- AI Security
- Prompt Injection
- Red-Teaming
- AI Agents
- Machine Learning Robustness
- Cybersecurity
- LLM Vulnerabilities
- Gray Swan Events
Highlights
- Main idea: AI security is fundamentally different from traditional software security because models are non-deterministic and susceptible to indirect prompt injections
- Failure mode: Agents that ingest external, untrusted data create a massive attack surface for attackers to hijack tool calls and leak credentials
- Practical takeaway: Robustness requires specialized safety layers like 'Cygnal' that can enforce enterprise policies and generalize to new attack vectors
- Main idea: The rise of 'gray swan' events—unlikely but predictable AI security breaches—is driving the need for AI insurance and compliance frameworks
- Practical takeaway: Automated red-teaming and specialized models are necessary to outpace human-led adversarial attacks in complex coding environments
Chapters
1:00The New AI Attack Surface: Defining the risks of AI agents, specifically the danger of malicious tool calls and credential theft.6:00Indirect Prompt Injection: How parsing untrusted external data allows attackers to manipulate AI systems via the network.11:00Automated Red-Teaming: The shift from manual human testing to using specialized models to find vulnerabilities at scale.16:00Mechanistic Interpretability: Discussing the potential for coding agents to turn model robustness into a measurable science.21:00Human vs. Model Performance: Analyzing the effectiveness of different red-teaming approaches and the ranking of human testers.26:00The Trade-off of Robustness: Balancing model capability and intelligence with the necessity of defensive guardrails.31:00Enforcing Enterprise Policies: The difficulty of implementing specific, non-bypassable security policies within LLM prompts.