# Neel Nanda - Mechanistic Interpretability (Sparse Autoencoders)

Page: https://stenobird.com/podcast/machine-learning-street-talk/neel-nanda-mechanistic-interpretability-sparse-autoencoders
Text version: https://stenobird.com/podcast/machine-learning-street-talk/neel-nanda-mechanistic-interpretability-sparse-autoencoders.md
Podcast: [Machine Learning Street Talk (MLST)](https://stenobird.com/podcast/machine-learning-street-talk)
Published: 2024-12-07T21:14:42+00:00
Episode link: https://podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/Neel-Nanda---Mechanistic-Interpretability-Sparse-Autoencoders-e2s186i
Audio file: https://anchor.fm/s/1e4a0eac/podcast/play/95510162/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2024-11-7%2Fc6f11920-f06a-6c65-f767-1b957d252a38.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/machine-learning-street-talk/episodes/neel-nanda-mechanistic-interpretability-sparse-autoencoders
Duration seconds: 13356

## Resource

Neel Nanda, a senior research scientist at Google DeepMind, leads their mechanistic interpretability team. In this extensive interview, he discusses his work trying to understand how neural networks function internally. At just 25 years old, Nanda has quickly become a prominent voice in AI research after completing his pure mathematics degree at Cambridge in 2020. Nanda reckons that machine learning is unique because we create neural networks that can perform impressive tasks (like complex reasoning and software engineering) without understanding how they work internally. He compares this to having computer programs that can do things no human programmer knows how to write. His work focuses on "mechanistic interpretability" - attempting to uncover and understand the internal structures and algorithms that emerge within these networks. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/ *** SHOWNOTES, TRANSCRIPT, ALL REFERENCES (DONT MISS!): https://www.dropbox.com/scl/fi/36dvtfl3v3p56hbi30im7/NeelShow.pdf?rlkey=pq8t7lyv2z60knlifyy17jdtx&st=kiutudhc&dl=0 We riff on: * How neural networks develop meaningful internal representations beyond simple pattern matching * The effectiveness of chain-of-thought prompting and why it improves model performance * The importance of hands-on coding over extensive paper reading for new researchers * His jour…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/machine-learning-street-talk/episodes/neel-nanda-mechanistic-interpretability-sparse-autoencoders/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/machine-learning-street-talk/neel-nanda-mechanistic-interpretability-sparse-autoencoders.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.