# On-Device AI Agents in Production: Privacy, Performance, and Scale // Varun Khare & Neeraj Poddar // #340

Page: https://stenobird.com/podcast/mlops-community/on-device-ai-agents-in-production-privacy-performance-and-scale-varun-khare-neeraj-poddar-340
Text version: https://stenobird.com/podcast/mlops-community/on-device-ai-agents-in-production-privacy-performance-and-scale-varun-khare-neeraj-poddar-340.md
Podcast: [MLOps.community](https://stenobird.com/podcast/mlops-community)
Published: 2025-09-30T16:00:04+00:00
Episode link: https://podcasters.spotify.com/pod/show/mlops/episodes/On-Device-AI-Agents-in-Production-Privacy--Performance--and-Scale--Varun-Khare--Neeraj-Poddar--340-e38o3na
Audio file: https://anchor.fm/s/174cb1b8/podcast/play/108842154/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2025-8-26%2F408182272-44100-2-53ae65c4bbbb9.mp3
Processing state: failed
JSON: https://stenobird.com/v1/public/podcasts/mlops-community/episodes/on-device-ai-agents-in-production-privacy-performance-and-scale-varun-khare-neeraj-poddar-340
Duration seconds: 2770

## Resource

On-Device AI Agents in Production: Privacy, Performance, and Scale // MLOps Podcast #340 with NimbleEdge's Varun Khare, Founder/CEO and Neeraj Poddar, Co-founder & CTO. Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter // Abstract AI agents are transitioning from experimental stages to performing real work in production; however, they have largely been limited to backend task automation. A critical frontier in this evolution is the on-device AI agent, enabling sophisticated, AI-native experiences directly on mobile and embedded devices. While cloud-based AI faces challenges like constant connectivity demands, increased latency, privacy risks, and high operational costs, on-device breaks through these trade-offs. We'll delve into the practical side of building and deploying AI agents with “DeliteAI”, an open-source on-device AI agentic framework. We'll explore how lightweight Python runtimes facilitate the seamless orchestration of end-to-end workflows directly on devices, allowing AI/ML teams to define data preprocessing, feature computation, model execution, and post-processing logic independently of frontend code. This architecture empowers agents to adapt to varying tasks and user contexts through an ecosystem of tools natively supported on Android/iOS platforms, handling all the permissions, model lifecycles, and many more. // Bio Varun Khare Varun is the Founder and CEO of NimbleEdge, an AI startup pioneering privacy-first, on-device intelligence. With an academic foundation in AI and neuroscience from UC Berkeley, MPI Frankfurt, and IIT Kanpur, Varun brings deep expertise at the intersection of technology and science. Before founding NimbleEdge, Varun led open-source projects at OpenMined, focu…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/mlops-community/episodes/on-device-ai-agents-in-production-privacy-performance-and-scale-varun-khare-neeraj-poddar-340/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/mlops-community/on-device-ai-agents-in-production-privacy-performance-and-scale-varun-khare-neeraj-poddar-340.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.