Episode
130. Edouard Harris - New Research: Advanced AI may tend to seek power *by default*
- Podcast
- Towards Data Science
- Published
- Oct 12, 2022
- Duration seconds
- 3502
- Processing state
failed
Actions
POST https://stenobird.com/v1/public/podcasts/towards-data-science/episodes/130-edouard-harris-new-research-advanced-ai-may-tend-to-seek-power-by-default/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/towards-data-science/130-edouard-harris-new-research-advanced-ai-may-tend-to-seek-power-by-default.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
Progress in AI has been accelerating dramatically in recent years, and even months. It seems like every other day, there’s a new, previously-believed-to-be-impossible feat of AI that’s achieved by a world-leading lab. And increasingly, these breakthroughs have been driven by the same, simple idea: AI scaling. For those who haven’t been following the AI scaling sage, scaling means training AI systems with larger models, using increasingly absurd quantities of data and processing power. So far, empirical studies by the world’s top AI labs seem to suggest that scaling is an open-ended process that can lead to more and more capable and intelligent systems, with no clear limit. And that’s led many people to speculate that scaling might usher in a new era of broadly human-level or even superhuman AI — the holy grail AI researchers have been after for decades. And while that might sound cool, an AI that can solve general reasoning problems as well as or better than a human might actually be an intrinsically dangerous thing to build. At least, that’s the conclusion that many AI safety researchers have come to following the publication of a new line of research that explores how modern AI systems tend to solve problems, and whether we should expect more advanced versions of them to perform dangerous behaviours like seeking power. This line of research in AI safety is called “power-seeking”, and although it’s currently not well understood outside the frontier of AI safety and AI alignment research, it’s starting to draw a lot of attention. The first major theoretical study of power seeking was led by Alex Turner, who’s appeared on the podcast before, and was published in NeurIPS (the world’s top AI conference), for example. And today, we’ll be hearing from Edouard Harris, an AI…