LlamaIndex podcast mentions

Quote

We're facing a uh a you know, uh a choice on whether or not to build versus buy. Um Um whether or not they want to maintain how they're gonna maintain this, etc. that's the case. Um other folks that we bump into, it's really uneven because it depends on the persona and the maturity level. So for less mature like folks that are more to the prototyping phase. Um maybe they're doing more with like doc like intelligent document processing tooling, like what LAN is doing or Llama Index. doing, etc. And they're just needing lots of levers in order to uh calibrate that data to help to try and get the the the it advanced production. On the other end of the spectrum around like the data engineering persona. you have some um kind of starter kits that are available to the CSPs. So like AWS Glue, um Azure AI. document intelligence, does it have a couple connectors? Snowflake has been building a little bit. around this. Um but they're not really comp they're not really positioning it and they're not really competing for production workloads. It's more like, hey you'll continue to grow with us and prototype w w with the CSPs. And so in that segment in particular, it's really a buy versus build choice. I think that w

Quote

Build the future of multi-agent software with agency, AGNT. The agency is an open source collective building the Internet of Windows. agents. It's a collaboration layer where AI agents can discover, connect, and work across frameworks. For developers, this means standardized agent discovery tools. Seamless protocols for interagent communication, and modular components to compose and scale. multi-agent workflows. Join Crew AI, Langchain, Llama Index, Browser base, Cisco, and dozens more. The agency is dropping code, specs, and and services all with no strings attached. Build with other engineers who care About high-quality multi-agent software. Visit agency.org and add your software. Support. That's AGNTCY. It is an interesting time for business. Tariff and Trade policies are dynamic, supply chains squeezed, and cash flow tighter than. ever. If your business can't adapt in real time, you are in a world of hurt. You need total visibility, from global shipments to tariff impacts to real-time cash flow and the That's NetSuite by Oracle, your AI powered business management suite, trusted by over. 42,000 businesses. NetSuite is the number one cloud ERP for many reasons.

Quote

key things that is is pretty much the the the bread and butter of vector search. having the functionality itself retrieved the appropriate document based on the query. and chunk chung in and different um ho holding different embedding sizes. But one thing we also deal with about MongoDB is we recognize that MongoDB is not a model provider. We're not line chained. We're not llama index. We're not open AI, we're not, we're not on through pay, but we can bring the expertise of these folks. together. Um and we have a new program called the MAP program that I call it the Avengers of the AI because what you get if you come onto the map program is expertise from all these AI companies. I'm talking the Lamaindex. the land chain a lot of companies that focus on evaluation some companies are focused on um providing this models and we bring that expertise to our customers. So we're really meeting our customers and developers at varying level. Um and that's that's where I see MongoDB investing a lot in as well. Got it. So what I heard was vector databases are the bread and butter Models are not the bread and butter and then from there you guys look to be scrapped.

Quote

That I was fortunate enough to moderate at the PyTorch conference in late October here in San Francisco. This was a fantastic panel comprised of So the panelists are Samuel, and the other one is the first time. and chief product officer at Rise AI. Adam Jones, member of the technical staff at Anthropic, and also a member of the model content. Text protocol team at Anthropic and Jerry Liu, CEO and co-founder of Llama Index. I hope you enjoy this episode. Alright, so we're here with many of the people who built some of our most favorite tools and they're all in the front lines of building uh agents. So I think I I'm gonna start out on a positive note and have them uh maybe comment on So maybe uh starting with Samuel at the far end here. So what are some architectural patterns? or agents that you've seen in the wild that really impress you and so and what are some of the key lessons from the I get to go first and I get to use a really easy one which is obviously coding agents or work much better than I think anyone would have predicted at the beginning of this year. And I said I would get this in as a hobby horse and I'm getting in right at the beginning. I think one of the things they absolutely love is type safety. No one

Quote

Yeah, and I I think part of that is like kind of what you're describing is if we take a look at the AI journey that companies go through, right? Chat interfaces. That's the prime. primary thing that came out that is still I think the dominant way that people interface with L LMs and AI, right? And so you have, let's say, interactive AI. That's where people tend to start building apps, building AI products, right? They'll go and grab a Langchain or they'll go and grab a Llama index and uh and really expect to either have like a human in the loop or you know interactive on the other end like an agent waiting for like an AI response. that comes from these LLMs, right? And I think that's where companies tend to be a good thing. to start primarily because that's where most of the tooling exists now. Then they tried they already start seeing value, right? Okay, we're building this product. some value, but now I really want to go and scale this. Like how do I now do AI? at scale, how do I actually build a product in production? And the challenge there is the nature of all these models are non-deterministic, right? And so a lot of the concepts that were once used for data teams with structured and tabular Data don't necessarily apply for unstructured data because of the nature of non determined And so what what what our theory is and what our thesis is is that

Quote

Regardless of how they're built, who built them, or where they run. just announced several key updates including interoperability with anthropics model context pro protocol, MCP, a new observability data schema enriched with concepts. specific to multi-agent systems, as well as new extensions to the open agenda. So are you ready to build the internet of agent? Agents, get started with agency and join Crew AI, Langchain, Llama Index, browser base, Cisco, and dozens more. Visit talkpython.fm slash agency To get started today, that's talkpython.fm/slash agency. The link is in your podcast. Player's show notes and on the episode page. Thank you to the agency for supporting TalkPython. There's certainly areas where the cloud can go like sideways. Um In the book, I mentioned a story about Kara, I believe, and um that Was this project that this woman I think in Hong Kong or South Korea I'm a I can't remember which I think it's South Korea. Anyway, created this as she's a photographer and really. hates AI generated art. So created this service that would say, hey, give me a piece of of art and I'll tell you if it's AI generated or not or something vaguely like that. And it her thing took off in the app store and was like number six. And her cloud bill at version.

Quote

building these systems. Like where do you see this going here? What the implications are at least. Yeah. So I think the the big problems I see today are actually m um at the like the practical problems with production that I see today is like um more well like I I guess like it's less sexy. It's about like okay I have a bunch of PDFs. How do I parse the text? How do I parse the information out? Right. And so uh this is why, you know, like I'm I'm a I'm a big fan. of Llama index and like Llama Parts, for example, does a great job of the colour. Oh heck yeah exactly, right? Yeah. Uh well we gotta invite Jerry by the way, but that that's a side note. Yeah. Yeah. Yeah. And and so this is where I think models like late interaction models like Cole Poly, for example, um it has great premise. I I think we're like that that that model uh it still takes a lot of effort to get it working well. Uh but we're I think we're like in in terms of late interaction or approaches to to visualize visually and directly extract information from images, we're very close. So uh I I think that's the big problem folks building multimodal applications today are are trying to solve. Um and then number two is then the next thing is like compound systems versus you know, compound systems being connected by an external DAG, if you will, versus like a model or or system that basically

Quote

And OpenAI are facing. They have to do with what data do you use to train the model? That's a data governance and lineage issue. Like if you could trace it down and say this model was trained on these ten data sources, it's an easy question. So the challenge is the data goes through so many steps of pre-processing and post-processing that by the time the model gets to it, it's It's hard to figure out where it came from. For AI governance, we need to solve those kinds of fundamental issues around data governance. What should you build with it? I asked Jerry Yu, the CEO at Llama Index. Literally everybody is using coding assistance these days. You know, if you ask any engineer at a tech company, they're all using like cursor, Windsurf, ChatGPT, etcetera. And then of course, like, you know, you're seeing AI agents deliver real ROI in certain enterprise verticals. So this includes like customer service. This includes like IT help desk stuff. This also includes areas that we're actually very specifically excited about, which is uh document workflow automation, you know, whether whether you're a finance team or procurement legal being able to sort through massive volumes of unstructured data and getting insights from it. That's a great high level answer but Within each industry, there also have specific use cases. Let's consider healthcare. Aldo Faisal is a professor of AI and neuro…

Quote

engagement via chat, mobile, et cetera. You had the systems of record, and you had a bunch of enterprise data sitting out there from databases, applications like Notion, Coda, TOE. PowerPoint, et cetera, Google Sheets, the big SaaS apps we've mentioned like ServiceNow, Workday, Salesforce. What companies are now looking at building is how do I connect connect and bridge all that information sitting in a Dropbox folder to a foundation model and then have my end users or customers interact with it. And so we're pretty excited that we think there's a new AI infrastructure stack being built. Uh there's companies like Llama Index that we invested in, that's kind of this data framework that helps bridge, you know, your. private data, your personal data, enterprise data with foundation models to let foundation models kind of interact with all your data actually be super useful. Llama is index and other frameworks like Langchain help build agents, these autonomous agents that let you do like amazing things behind the scenes, like interact with data, actually take actions on on you know, business process flows or personal process flows. The whole use of vector embeddings, quote unquote, the memory for these large language models we're watching. So all the vector data databases out there. We're trying to understand, you know, how that evolves into an independent category. Or do current…

Quote

Ben Wilson. I work on quarterly plans at Databricks. Today we are not joined by a guest. Uh we are joined by myself and Ben and what we're gonna talk about is a really Informative experience for me and uh we're gonna walk through basically how Ben approaches approached teaching me and sort of mentoring me uh through a very challenging project and this project was building the Llama Index flavor for ML flow. So if you're not familiar, MLflow manages machine learning model lifecycles. So model And uh in ML flow there are things called flavors, which are essentially wrappers around common projects. So Sidekit Learn, TensorFlow, Keras, PyTorch. You name it. And I just built the flavor for Llama Index with some help from another engine. engineer on the MO flow team. And uh this project was a big uh learning experience because I wanted to learn basically how software is done, uh specifically a data. And I'm not gonna lie, it was hell. There were like some really challenging times. There were some very long Saturday nights. Uh well maybe not Saturday. So much Saturday nights, but there's some long Sunday nights, long Friday nights. And uh

Quote

with someone for a very small amount of time. Um but I typically play the TL role in a lot of my projects now where I do that initial design, that mock-up, um and provide a design Design doc to someone to actually implement it. And if I'm doing it end to end End I can do it so much faster. Like this crew AI integration Um, I was actually thinking about what are the tips that I wish I knew when I was doing the llama index flavor. And Lama index flavor took me like legitimately six months of like actual Working on it a fair amount. And um I learned so much along the way but there's just like basic things that are really valuable like don't Don't close Visual Studio until the PR is merged. Like just leave it open. There's gonna be a comment, you're gonna need to refactor something, you're gonna need to rerun an example. Just leave it open, minimize the window. And that has saved me I don't know like Ondeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, ondex loans. Up to $400,000 help make it happen fast. Rated A plus by the Better Business. Bureau and earning thousands of five-star trust pilot reviews, OnDeck delivers funding you can

Quote

And And then right now, the execution of that tool. It's really left up to the developer. And that's actually one of the problems that stuff like MCP um and even arcade are trying to solve. Um and there's a lot of attempts at this. But beforehand they were just in Lang chain and llama index and you just ran it wherever the L L M climbed like this definition. I think with the tools, the agent can now access Information that it doesn't have memorized, right? And so it's coming in um it's able to access stuff on the fly. So which is still very different than reasoning. is think step by step, but tools is how do you retrieve it and then be able And that suddenly expands the score. So if you've used the latest chat GPT, that's the same thing. has tool use built in'cause you can see like it is searching the web, it is doing stuff. Right. And imagine the possibilities if it can now access Let's say your private database in the enterprise you have files located somewhere, it can read those.