# Stealing Part of a Production Language Model with Nicholas Carlini - #702

Page: https://stenobird.com/podcast/twiml-ai-podcast/stealing-part-of-a-production-language-model-with-nicholas-carlini-702
Text version: https://stenobird.com/podcast/twiml-ai-podcast/stealing-part-of-a-production-language-model-with-nicholas-carlini-702.md
Podcast: [The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)](https://stenobird.com/podcast/twiml-ai-podcast)
Published: 2024-09-23T19:21:00+00:00
Episode link: https://twimlai.com/podcast/twimlai/stealing-part-of-a-production-language-model/
Audio file: https://pscrb.fm/rss/p/traffic.megaphone.fm/MLN7516431304.mp3?updated=1727119766
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/stealing-part-of-a-production-language-model-with-nicholas-carlini-702
Duration seconds: 3810

## Resource

Today, we're joined by Nicholas Carlini, research scientist at Google DeepMind to discuss adversarial machine learning and model security, focusing on his 2024 ICML best paper winner, “Stealing part of a production language model.” We dig into this work, which demonstrated the ability to successfully steal the last layer of production language models including ChatGPT and PaLM-2. Nicholas shares the current landscape of AI security research in the age of LLMs, the implications of model stealing, ethical concerns surrounding model privacy, how the attack works, and the significance of the embedding layer in language models. We also discuss the remediation strategies implemented by OpenAI and Google, and the future directions in the field of AI security. Plus, we also cover his other ICML 2024 best paper, “Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining,” which questions the use and promotion of differential privacy in conjunction with pre-trained models. The complete show notes for this episode can be found at https://twimlai.com/go/702.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/stealing-part-of-a-production-language-model-with-nicholas-carlini-702/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/twiml-ai-podcast/stealing-part-of-a-production-language-model-with-nicholas-carlini-702.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.