# Multi-Cluster Orchestrator, with Nick Eberts and Jon Li

Page: https://stenobird.com/podcast/kubernetes-podcast-from-google/multi-cluster-orchestrator-with-nick-eberts-and-jon-li
Text version: https://stenobird.com/podcast/kubernetes-podcast-from-google/multi-cluster-orchestrator-with-nick-eberts-and-jon-li.md
Podcast: [Kubernetes Podcast from Google](https://stenobird.com/podcast/kubernetes-podcast-from-google)
Published: 2025-05-28T15:21:00+00:00
Episode link: https://e780d51f-f115-44a6-8252-aed9216bb521.libsyn.com/multi-cluster-orchestrator-with-nick-eberts-and-jon-li
Audio file: https://traffic.libsyn.com/secure/e780d51f-f115-44a6-8252-aed9216bb521/KPOD253.mp3?dest-id=3486674
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/kubernetes-podcast-from-google/episodes/multi-cluster-orchestrator-with-nick-eberts-and-jon-li
Duration seconds: 1291

## Resource

The Multi-Cluster Orchestrator (MCO) solves the challenge of managing workloads across fragmented, non-uniform cloud resources. It provides a standardized way to scale AI inference engines from zero to one across multiple clusters based on real-time capacity and hardware availability.

## Highlights
- Main idea: MCO addresses the shift from 'infinite' cloud capacity to a reality of hardware stockouts and non-uniform GPU availability
- Practical takeaway: Use MCO to automate workload placement recommendations, allowing tools like Argo CD to scale workloads from zero to one in available regions
- Technical detail: The system utilizes a normalized 'Cluster Profile' spec to unify cluster inventories across different providers and tools
- Failure mode: Outdated cluster profiles can lead to MCO making recommendations based on stale capacity data
- Practical takeaway: For AI inference, leverage custom metrics like KV cache utilization or queue depth to drive precise scaling decisions

## Topics

Kubernetes, Multi-Cluster Orchestrator, AI Inference, GKE, Cloud Infrastructure, GPU Availability, Cluster Management, LLM Deployment

## Chapters
- 1:00 — Kubernetes & GKE Updates: Brief overview of Kubernetes 1.33 availability in GKE Rapid channel and Kyverno 1.14.0 release.
- 2:35 — Introduction to MCO: An introduction to the Multi-Cluster Orchestrator and the shift in cloud computing assumptions.
- 4:05 — The Problem: Hardware Scarcity: Discussing why the assumption of infinite, uniform cloud capacity no longer holds true for modern workloads.
- 5:40 — Defining Multi-Cluster Orchestrator: Explaining how MCO manages workloads across multiple clusters to optimize for proximity and availability.
- 7:15 — Scaling AI Inference: How MCO handles the 'zero to one' scaling problem for compute-intensive inference engines.
- 8:45 — Capacity and Metrics: Using open, accessible metrics to determine regional capacity and drive workload placement.
- 10:15 — Latency in LLM Inference: The challenges of managing high-latency, multi-modal LLM requests compared to traditional microservices.
- 11:55 — Standardizing Cluster Inventories: Using Cluster Profiles to create a normalized, open-source specification for multi-cluster management.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/kubernetes-podcast-from-google/episodes/multi-cluster-orchestrator-with-nick-eberts-and-jon-li/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/kubernetes-podcast-from-google/multi-cluster-orchestrator-with-nick-eberts-and-jon-li.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.