Episode
Kubernetes at LinkedIn, with Ahmet Alp Balkan and Ronak Nathani
- Published
- Mar 25, 2025
- Duration seconds
- 2522
- Processing state
processed- Canonical source
- https://e780d51f-f115-44a6-8252-aed9216bb521.libsyn.com/kubernetes-at-linkedin-with-ahmet-alp-balkan-and-ronak-nathani
Actions
POST https://stenobird.com/v1/public/podcasts/kubernetes-podcast-from-google/episodes/kubernetes-at-linkedin-with-ahmet-alp-balkan-and-ronak-nathani/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/kubernetes-podcast-from-google/kubernetes-at-linkedin-with-ahmet-alp-balkan-and-ronak-nathani.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
LinkedIn engineers reveal how they manage massive Kubernetes clusters on bare metal, focusing on handling latency-sensitive workloads and hardware fragmentation. The discussion explores the complexities of building an opinionated platform that abstracts infrastructure for developers while maintaining high performance.
Topics
- Kubernetes
- Bare Metal
- Infrastructure at Scale
- Cloud Native
- Custom Controllers
- LinkedIn Engineering
- Site Reliability Engineering
- Container Orchestration
Highlights
- Main idea: LinkedIn uses a custom, opinionated platform on bare metal to manage hardware generations and prevent fragmentation
- Practical takeaway: Use node profiles and specific pools to isolate latency-sensitive applications from older hardware generations
- Failure mode: Large-scale clusters can cause controller informers to fail to sync, potentially leading to catastrophic state loss like label clearing
- Main idea: Kubernetes can be adapted for stateful workloads if you control the full stack from bare metal to disk attachment
- Practical takeaway: Abstracting Kubernetes complexity via custom APIs helps developers focus on resources like CPU and memory rather than low-level networking
Chapters
1:00Kubernetes Ecosystem News: Updates on CubeFS graduation to CNCF and Canonical's announcement regarding 12-year Kubernetes Long Term Support.4:10Running Kubernetes on Bare Metal: An exploration of LinkedIn's transition from legacy containerization to a large-scale Kubernetes implementation on bare metal.10:25Networking and Flat Data Center Architectures: Discussion on bypassing standard CNI plugins like Flannel in favor of a flat, routable data center network.16:40Managing Hardware Skews and Fragmentation: How to handle different CPU generations and the challenges of preventing resource fragmentation in multi-tenant pools.22:50The Power of Custom Controllers: The necessity and reality of developing custom Kubernetes controllers and CRDs to extend platform functionality.32:15Abstracting Complexity for Developers: Providing an opinionated interface that allows engineers to specify compute resources without managing low-level Kubernetes primitives.38:45Lessons from Large-Scale Failures: A deep dive into how controller informer timeouts in massive clusters can lead to unintended side effects like mass label deletion.