{"podcast":{"title":"DevOps and Docker Talk: Cloud Native Interviews and Tooling","slug":"devops-and-docker-talk-cloud-native-interviews-and-tooling","podcast_index_feed_id":79609,"rss_url":"https://feeds.transistor.fm/devops-and-docker-talk","website_url":"https://podcast.bretfisher.com","image_url":"https://img.transistorcdn.com/cAiLhBy2mqgPbwU4-TJ749hfmjqYMhUBIDgZxM_G5aI/rs:fill:0:0:1/w:1400/h:1400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9iZGUz/NzE4NjE5OWI1NDhm/ZmQ3YTNiNjVhMzA0/NmVhYi5qcGc.jpg","author":"Bret Fisher","episode_count":193,"summary":"Interviews from Bret Fisher's live show with co-host Nirmal Mehta. Topics cover container and cloud topics like Docker, Kubernetes, Swarm, Cloud Native development, DevOps, SRE, GitOps, DevSecOps, platform engineering, and the full software lifecycle. Full show notes and more info available at https://podcast.bretfisher.com","last_synced_at":null,"page_url":"https://stenobird.com/podcast/devops-and-docker-talk-cloud-native-interviews-and-tooling"},"episode":{"title":"Docker Model Runner","slug":"docker-model-runner","published_at":"2025-04-21T18:44:48+00:00","page_url":"https://stenobird.com/podcast/devops-and-docker-talk-cloud-native-interviews-and-tooling/docker-model-runner","show_page_url":"https://stenobird.com/podcast/devops-and-docker-talk-cloud-native-interviews-and-tooling","url":"https://podcast.bretfisher.com/episodes/docker-model-runner","audio_url":"https://media.transistor.fm/b8689db1/ab36fce8.mp3","summary":"Docker Model Runner simplifies running LLMs locally by using a single command to manage models via llama.cpp. This episode explores the architecture, OCI artifact integration, and practical use cases for local AI inference.","meta_description":"Learn how to run LLMs easily with Docker Model Runner. Explore the internals of OCI artifacts, llama.cpp integration, and the future of Docker AI.","key_points":["Main idea: Docker Model Runner provides a streamlined interface for running LLMs using the 'docker model' command","Technical detail: Models are distributed as OCI artifacts containing the model blob and license files, rather than full container images","Practical takeaway: Use Open WebUI with Docker Model Runner to create a private, local ChatGPT-like experience","Failure mode: Large models can cause timeouts or system freezes, occasionally requiring a Docker Desktop restart","Future roadmap: Upcoming support for Windows, Docker CE, and MLX for significant performance boosts on Apple Silicon"],"chapters":[{"start_ms":60000,"title":"The Agentic DevOps Guild","summary":"An introduction to the new community for accelerating AI adoption in DevOps, CI/CD, and Platform Engineering."},{"start_ms":185000,"title":"Docker Model Runner Elevator Pitch","summary":"A high-level overview of how Docker Model Runner lowers the barrier to entry for running local LLMs."},{"start_ms":245000,"title":"Enabling Docker Model Runner","summary":"How to enable the feature in Docker Desktop and the distinction between Model Runner and Docker AI."},{"start_ms":450000,"title":"Downloading Models via Docker Hub","summary":"Exploring the new packaging format for models and how to pull them from the Docker Hub AI account."},{"start_ms":590000,"title":"Architecture and llama.cpp","summary":"A deep dive into the underlying use of llama.cpp and how models are dynamically loaded into memory."},{"start_ms":785000,"title":"OCI Artifacts and ORAS","summary":"Understanding the technical implementation of models as OCI artifacts and the role of tools like ORAS."},{"start_ms":850000,"title":"Troubleshooting and Future Roadmap","summary":"Addressing current limitations like model size issues and discussing upcoming Windows and Linux support."}],"topics":["Docker Model Runner","LLM","llama.cpp","OCI Artifacts","Docker Hub","Open WebUI","AI Infrastructure","DevOps Automation"],"duration_seconds":924,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/devops-and-docker-talk-cloud-native-interviews-and-tooling/episodes/docker-model-runner/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/devops-and-docker-talk-cloud-native-interviews-and-tooling/docker-model-runner.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}