# Yoshua Bengio - Designing out Agency for Safe AI

Page: https://stenobird.com/podcast/machine-learning-street-talk/yoshua-bengio-designing-out-agency-for-safe-ai
Text version: https://stenobird.com/podcast/machine-learning-street-talk/yoshua-bengio-designing-out-agency-for-safe-ai.md
Podcast: [Machine Learning Street Talk (MLST)](https://stenobird.com/podcast/machine-learning-street-talk)
Published: 2025-01-15T19:21:54+00:00
Episode link: https://podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/Yoshua-Bengio---Designing-out-Agency-for-Safe-AI-e2thu9p
Audio file: https://anchor.fm/s/1e4a0eac/podcast/play/97105657/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2025-0-15%2Fec5f2ba3-9244-22fc-7c18-43ca121cc9e3.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/machine-learning-street-talk/episodes/yoshua-bengio-designing-out-agency-for-safe-ai
Duration seconds: 6113

## Resource

Professor Yoshua Bengio is a pioneer in deep learning and Turing Award winner. Bengio talks about AI safety, why goal-seeking “agentic” AIs might be dangerous, and his vision for building powerful AI tools without giving them agency. Topics include reward tampering risks, instrumental convergence, global AI governance, and how non-agent AIs could revolutionize science and medicine while reducing existential threats. Perfect for anyone curious about advanced AI risks and how to manage them responsibly. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events? They are hosting an event in Zurich on January 9th with the ARChitects, join if you can. Goto https://tufalabs.ai/ *** Interviewer: Tim Scarfe Yoshua Bengio: https://x.com/Yoshua_Bengio https://scholar.google.com/citations?user=kukA0LcAAAAJ&hl=en https://yoshuabengio.org/ https://en.wikipedia.org/wiki/Yoshua_Bengio TOC: 1. AI Safety Fundamentals [00:00:00] 1.1 AI Safety Risks and International Cooperation [00:03:20] 1.2 Fundamental Principles vs Scaling in AI Development [00:11:25] 1.3 System 1/2 Thinking and AI Reasoning Capabilities [00:15:15] 1.4 Reward Tampering and AI Agency Risks [00:25:17] 1.5 Alignment Challenges and Instrumental Convergence 2. AI Architecture and Safety Design [00:33:10] 2.1 Instrumental Goals and AI Safety Fundamentals [00:35:02] 2.2 Separating Intelligence from Goals in AI Systems [00:40:40] 2.3 Non-Agent AI as Scientific Tools [00:44:25…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/machine-learning-street-talk/episodes/yoshua-bengio-designing-out-agency-for-safe-ai/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/machine-learning-street-talk/yoshua-bengio-designing-out-agency-for-safe-ai.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.