Episode
Ion Stoica — Spark, Ray, and Enterprise Open Source
- Published
- Jan 20, 2022
- Duration seconds
- 3222
- Processing state
failed- Canonical source
- https://wandb.ai/site/resources/podcast
Actions
POST https://stenobird.com/v1/public/podcasts/gradient-dissent/episodes/ion-stoica-spark-ray-and-enterprise-open-source/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/gradient-dissent/ion-stoica-spark-ray-and-enterprise-open-source.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
Ion Stoica is co-creator of the distributed computing frameworks Spark and Ray, and co-founder and Executive Chairman of Databricks and Anyscale. He is also a Professor of computer science at UC Berkeley and Principal Investigator of RISELab, a five-year research lab that develops technology for low-latency, intelligent decisions. Ion and Lukas chat about the challenges of making a simple (but good!) distributed framework, the similarities and differences between developing Spark and Ray, and how Spark and Ray led to the formation of Databricks and Anyscale. Ion also reflects on the early startup days, from deciding to commercialize to picking co-founders, and shares advice on building a successful company. The complete show notes (transcript and links) can be found here: http://wandb.me/gd-ion-stoica --- Timestamps: 0:00 Intro 0:56 Ray, Anyscale, and making a distributed framework 11:39 How Spark informed the development of Ray 18:53 The story behind Spark and Databricks 33:00 Why TensorFlow and PyTorch haven't monetized 35:35 Picking co-founders and other startup advice 46:04 The early signs of sky computing 49:24 Breaking problems down and prioritizing 53:17 Outro --- Subscribe and listen to our podcast today! 👉 Apple Podcasts: http://wandb.me/apple-podcasts 👉 Google Podcasts: http://wandb.me/google-podcasts 👉 Spotify: http://wandb.me/spotify