# Ion Stoica — Spark, Ray, and Enterprise Open Source

Page: https://stenobird.com/podcast/gradient-dissent/ion-stoica-spark-ray-and-enterprise-open-source
Text version: https://stenobird.com/podcast/gradient-dissent/ion-stoica-spark-ray-and-enterprise-open-source.md
Podcast: [Gradient Dissent: Conversations on AI](https://stenobird.com/podcast/gradient-dissent)
Published: 2022-01-20T12:00:00+00:00
Episode link: https://wandb.ai/site/resources/podcast
Audio file: https://podcasts.captivate.fm/media/3072eb79-f1fd-4f1b-9ed9-28a0c94e149c/gd-ion-stoica-v2.mp3
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/ion-stoica-spark-ray-and-enterprise-open-source
Duration seconds: 3222

## Resource

Ion Stoica is co-creator of the distributed computing frameworks Spark and Ray, and co-founder and Executive Chairman of Databricks and Anyscale. He is also a Professor of computer science at UC Berkeley and Principal Investigator of RISELab, a five-year research lab that develops technology for low-latency, intelligent decisions. Ion and Lukas chat about the challenges of making a simple (but good!) distributed framework, the similarities and differences between developing Spark and Ray, and how Spark and Ray led to the formation of Databricks and Anyscale. Ion also reflects on the early startup days, from deciding to commercialize to picking co-founders, and shares advice on building a successful company. The complete show notes (transcript and links) can be found here: http://wandb.me/gd-ion-stoica --- Timestamps: 0:00 Intro 0:56 Ray, Anyscale, and making a distributed framework 11:39 How Spark informed the development of Ray 18:53 The story behind Spark and Databricks 33:00 Why TensorFlow and PyTorch haven't monetized 35:35 Picking co-founders and other startup advice 46:04 The early signs of sky computing 49:24 Breaking problems down and prioritizing 53:17 Outro --- Subscribe and listen to our podcast today! 👉 Apple Podcasts: http://wandb.me/apple-podcasts​​ 👉 Google Podcasts: http://wandb.me/google-podcasts​ 👉 Spotify: http://wandb.me/spotify​

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/ion-stoica-spark-ray-and-enterprise-open-source/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/gradient-dissent/ion-stoica-spark-ray-and-enterprise-open-source.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.