Stenobird

Episode

PubNub with Stephen Blum

Podcast
Rustacean Station
Published
Oct 13, 2024
Duration seconds
1968
Processing state
processed
Canonical source
https://rustacean-station.org/episode/stephen-blum/
Audio
https://dts.podtrac.com/redirect.mp3/audio.rustacean-station.org/file/rustacean-station/2024-10-13-stephen-blum.mp3
JSON
/v1/public/podcasts/rustacean-station/episodes/pubnub-with-stephen-blum
Markdown
/podcast/rustacean-station/pubnub-with-stephen-blum.md

Actions

  • POST https://stenobird.com/v1/public/podcasts/rustacean-station/episodes/pubnub-with-stephen-blum/transcription-requests
    Idempotently request low-priority transcript generation for this episode.
  • GET https://stenobird.com/podcast/rustacean-station/pubnub-with-stephen-blum.md
    Read the agent-friendly Markdown representation of this episode resource.

Summary

PubNub CTO Stephen Blum discusses the strategic migration from C to Rust to handle massive-scale real-time messaging. The conversation explores the trade-offs between performance, memory safety, and the engineering challenges of managing billions of connected devices.

Topics

  • Rust programming language
  • Pub/Sub architecture
  • Cloud infrastructure
  • Microservices
  • Kubernetes
  • Real-time communication
  • Memory safety
  • Software migration

Highlights

  • Main idea: PubNub is migrating its core infrastructure from C to Rust to improve developer velocity and memory safety without sacrificing performance
  • Practical takeaway: Using Rust allows for a safer concurrency model, which is critical when managing a pub/sub network handling trillions of monthly messages
  • Failure mode: Relying on Go for high-scale infrastructure can lead to higher costs due to lower price-per-dollar performance compared to Rust or C
  • Engineering strategy: The team is opting for full microservice rewrites in Rust rather than incremental C-to-Rust library ports to simplify the codebase
  • Security focus: Implementing tools like cargo-vet and cargo-audit into Docker builds is essential for maintaining trust in a large-scale data pipeline

Chapters

  1. 1:00 The Scale of PubNub: An overview of PubNub's infrastructure, handling three trillion JSON messages per month across billions of devices like cars and gaming consoles.
  2. 6:05 The Case for Rust: Why memory safety and concurrency are non-negotiable for modern large-scale connected device networks.
  3. 8:35 Rust vs. Go at Scale: A comparison of language efficiency, focusing on the need for high performance-per-dollar in massive cloud deployments.
  4. 15:50 Kubernetes and Global Deployment: How PubNub uses Kubernetes and Amazon's data centers to achieve automatic scaling and regional fault tolerance.
  5. 23:20 The Migration Strategy: Discussing the decision to perform full microservice rewrites rather than incremental piece-by-piece migrations.
  6. 30:25 Securing the Supply Chain: Integrating cargo-vet and cargo-audit into the build pipeline to ensure crate security and prevent data leakage.