# Shaping AI Benchmarks with Together AI Co-Founder Percy Liang

Page: https://stenobird.com/podcast/gradient-dissent/shaping-ai-benchmarks-with-together-ai-co-founder-percy-liang
Text version: https://stenobird.com/podcast/gradient-dissent/shaping-ai-benchmarks-with-together-ai-co-founder-percy-liang.md
Podcast: [Gradient Dissent: Conversations on AI](https://stenobird.com/podcast/gradient-dissent)
Published: 2024-05-09T16:00:00+00:00
Episode link: https://wandb.ai/site/resources/podcast/episodes/shaping-ai-benchmarks-with-together-ai-co-founder-percy-lang
Audio file: https://podcasts.captivate.fm/media/f95149f1-2e28-4ad4-a3f4-9ae566b6f3ba/GD011-Podcast-Final.mp3
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/shaping-ai-benchmarks-with-together-ai-co-founder-percy-liang
Duration seconds: 3200

## Resource

In this episode of Gradient Dissent, Together AI co-founder and Stanford Associate Professor Percy Liang joins host, Lukas Biewald, to discuss advancements in AI benchmarking and the pivotal role that open-source plays in AI development. He shares his development of HELM—a robust framework for evaluating language models. The discussion highlights how this framework improves transparency and effectiveness in AI benchmarks. Additionally, Percy shares insights on the pivotal role of open-source models in democratizing AI development and addresses the challenges of English language bias in global AI applications. This episode offers in-depth insights into how benchmarks are shaping the future of AI, highlighting both technological advancements and the push for more equitable and inclusive technologies. ✅ Subscribe to Weights & Biases → http://wandb.me/yt_subscribe Connect with Percy Liang: https://www.linkedin.com/in/percy-liang-717b8a/ https://twitter.com/percyliang Anticipatory Music Composer: https://stanford.io/3y5VycN Blog Post: https://crfm.stanford.edu/2024/02/18/helm-instruct.html Follow Weights & Biases: https://twitter.com/weights_biases https://www.linkedin.com/company/wandb

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/shaping-ai-benchmarks-with-together-ai-co-founder-percy-liang/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/gradient-dissent/shaping-ai-benchmarks-with-together-ai-co-founder-percy-liang.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.