Together AI podcast mentions

Quote

They actually started as a research project focused on programmatic data. Data labeling. Seven years in though, the company has evolved considerably, achieving evaluation of I think one point. three billion in twenty twenty five and Alex's team is now doing some of the most interesting work. in both AI evalves, benchmark design, uh model weights, data. data set labeling, including a a major open benchmarks grant program with partners like Hugging. Face, Together AI, and PyTorch. And Alex also did his PhD at Stanford. where he built the Snorkel open source project and help establish data centric AI. as a field. He, I believe, is an affiliate professor of computer science at the University of Washington, go dogs. That's my alma mater. Alex, welcome to Chain of Thought. Well, thanks so much for having. And uh uh affiliate assistant professor. So I I uh you know in startup world uh title are all made up and academic world uh the titles have a little bit more uh more weight behind them. Right. So I I uh but but was fortunate enough to to get some time there. Uh you that was amazing. And then then, you know, snorkel snorkel sucked me into the vortex a couple of years back. Um But excited to chat today. Thanks for having me on. Yeah, super excited for our conversation. farther in though I do want to do a quick thank you to our presenting sponsors Galileo uh they They were recently ac…

Quote

Version of access to that model. This would be like AWS bedrock, right? runs a deep seek model in their servers and you connect through an AWS endpoint in your AWS V P C to access this model, which you are not running. Running manually on servers e in your AWS or anywhere. But that AWS is managing that on your behalf as a managed service or you know, this would be like together AI or whoever's running the model, right? So just in summary there's kind of those possibilities. Now when we talk about an open or a closed model, the open model at some point goes through that process of how kind of opens up this wider range of possibilities of how you can use the model. model would never kind of leave those weights and that model code that inference code would never leave the vendor's infrastructure on purpose. Um because they consider that their IP. So you could connect to their SAS. product and interact with the model maybe in a no-code way or you could interact with their API. The REST API to use the model, but it's that model interaction is always

Quote

That's an interesting trend. I I didn't foresee, but seems like they are talking about it, right? So that's uh that's an interesting aspect from an economic economic standpoint. So yeah, I think uh um and then um from an AI economic standpoint Um as inference prices drop, new hardware optimized inference tax are coming up as well as you know uh providers like uh together AI, fireworks, they're all building optimized stacks that are dramatically reducing operational costs. So Now uh with uh uh with with this whole notion of uh uh as they call like uh you know mixture of experts architecture where they want to uh they want to kind of pull in the right uh model and agent for the right tasks so that You optimize um you know uh even the task. Exactly, exactly. So that that becomes the that becomes a kind of a very optimal engine that is built in into the orchestration layer that is being talked about. So it's an interesting um it all kind of points to the beginning of 22 2026 trend where AI orchestration is going to be the story this year and then these are all like feeders into it that is that we are seeing.

Quote

like that protein folding was solved, I always thought that was inevitable. But the fact that it was solved and on like your desktop, This is the first episode of the New AI for science podcast on the Lase and Space Network. I'm Brandon. I work On RNA therapeutics using machine learning at Atomic AI. My name is R.J. Haneke. I'm the co-founder of Mira Omics, where we build spatial transcriptomics AI models. The point of this podcast is to bring together AI engineers and scientists or bring together the communities. These are two communities which have been developed independently for quite some time. Time, but there's been some attempt to combine them and only now after you Many years are we starting to see some of the big developments start to play out in the real world and start to solve you know key scientific problems. There's no like one size fits all solution, you need domain expertise, you need ki people on both sides of the aisle. who can really talk to each other and really work together and understand both the modeling and all of the real subtleties of the system you're actually trying to work on. We hope that we can connect these communities and that we can provide a starting point for this new era of AI and science to move forward. So without further ado, let's get started on the first podcast. We're really happy to have in the studio today and

Quote

So the gist here is four thirteen billion activated parameters during inference, not at Next we've got another open source released this time It's more of an R L trained coding agents coming from Together AI, they call it Deep SWE, deep software engineer and this is based on on top So the focus is on particularly. the the training of the model to be a software engineering agent. And uh among uh open rate models this is a leading one. I think we don't yet have benchmarks that really capture the But uh regardless, yeah clearly making a push in the first time.

Quote

So we'll see. Um and then I I did a lot of reflecting on the other side. On why i this work situation ended up being so hard for me. career as a whole and like what are the patterns and when things were were hard for me. I also just jumped So the curve is a very good thing. conference, uh which is a a conference that kind of brings together AI Skeptics um and AI safety people and like people kind of on all sides of the issue of like that was like having its first iteration while I was on sabbatical so I was able to like uh Uh kind of get involved with that more and like try to be helpful more than I like uh could have been if I had a full-time job which was really cool. Um did some writing. Um Most of that writing hasn't been published, um, but it was still good for me to do. Um but yeah, it it kind of went by really fast, honestly. There was a lot of stuff to think about. Yeah. What what sorts of reflections did you have on I guess your your your career So far and your motivation and I guess like what had been difficult in in in twenty twenty three and twenty twenty four. Yeah. Um so I think in terms of twenty twenty three and twenty twenty four specifically, um

Quote

the texture renderers and things that are actually you know, still used as has different types of caches and and stuff that that have wonky names that don't really translate to the LLM world. So, but there's so many patterns that have been built up that are and if you take something like flash attention. you know, um that was built by the the chief research scientist over at Together AI, right? He's a famous guy. Tree yeah, tree jumping. Dow, yeah. Yeah, yeah, he's pretty badass. Um uh Yeah, he he really brought light to this like mechanical sympathy. type of thing. And the the big revelation was is let's use the fastest memory for the things that that need to move, you know uh the the like memory that's closest to the chip. Let's put things that are constantly moving in And out let's keep it there so that doesn't have to travel across all the other slower um, you know, bandwidth uh like interconnects and and just just m that One small change made huge performance gains. And that's why with every generation uh flash attention has to get like has to get rebuilt and reoptimized because it's not fully taking advantage. And you know, there's a flash of tension

Quote

You can do both. It's where the most trusted creators and powerful AI can. That's why YouTube drives high. Higher long-term return on ad spend versus TV, paid social, or streaming. No more choosing between brand or results. With one platform, you get both. Learn more at G dot co slash business slash YouTube the right. Technology can strengthen human judgment. That's why Deloitte brings together AI and data. Data analytics with multidisciplinary teams who can help you connect the dots across your enterprise. From risk to operations to customer needs, so opportunities don't slip by. And surprises don't spread, because the smarter your systems, the sharper your inspiration. Instincts. That's how technology makes people better at what they do best. Deloitte. Together makes progress. Learn more at Deloitte.com slash together makes progress. Well, Casey, we're coming down to the end of twenty twenty five. And as is our custom on this show, it is time to stare into our crystal balls. But before we talk about our predictions, We should make our disclosures. The New York Times company is suing Open AI and Microsoft over copyright violation. And my boyfriend works at Anthropic. So Casey, remind me what your

Quote

It became powerful players in the AI computing ecosystem. Mosaic has continued to scale after Databricks acquisition, establishing the data platform as a leader in training custom language models for customers and producing the data. Producing state-of-the-art models like DBRX. The Snowflake team now provides a full-stack AI platform. These developments have enabled both Databricks and Snowflake to offer attractive distribution. Additionally, NVIDIA's partnerships and sizable investments have enabled a growing class of small, VC backed AI-specific cloud companies to thrive. Large rounds of financing and or GPU allocation from NVIDIA has enabled companies like Coreweave, Together AI, Crusoe and Lambda Labs to acquire customers with flexible compute options and availability compared to the often constrained big three clouds. All of this shows we've entered the big four. four era of cloud. The Big Four collectively power the entire AI ecosystem and often lead the largest financing rounds. Each are racing to establish crucial partnerships with startups to build up their own. AI Edge. The record-breaking financing rounds are heavily tilted towards large language model providers. Microsoft alone put $13 billion into OpenAI in 2023. AWS recently invested two point seven billion in Anthropic, and Perplexity AI's rapid ascension to higher and higher valuations was achieved in part by b…

Quote

T fours all all of these new chips started coming out. Uh and then it was It was great to see. It's like these new types of workloads, great new types of workloads, let's make them run on North Like. Uh Um so helping a a a lot of startups get their um yeah training. And for us it just becomes a good thing. More compute, more workloads, more clouds, right? Now people are looking for neo clouds, they wanna be on Core Weave, they wanna be on Together AI, they want to be on uh Lambda Labs. All of those folks uh so they started out selling you know the the the the you know the research social machines or the gaming machines or they started out in crypto. They've all come in to build these massive highly capable, highly performant developer clouds for AI. Started out bare metal. Now pretty much all of them are now trying to default to offering all of their compute through Kubernetes. So we've gone through the same cycle as Northlank's primary objective. Fix Kubernetes. Make Kubernetes work for your developers. So North Flank's actually a really great place for you to build and deploy your AI workloads, even though that's not our core mission. or original idea.

Quote

assume that the product is gonna be so good that everyone's gonna immediately get it because With developers, you know, we need to know who we're we're selling to and developers have very if Efficient ways of working that they've built out over the last twenty years. And we want to make sure that we constantly And to do that we need to accept the same thing. an episode with Together AI maybe a year ago or so. And we were talking about what inference speed actually we needed and they always argued we need to get to like five thousand think about factory how much do you think you're like bound by the speed speed of these models. Do you know like if the models were like a lot faster, would you just complete things quicker? Would you like maybe sp fan out more in parallel? Like what are like the limits I want to let Eno answer this, but immediately every time this thing comes up, I always just think about the memory that Chrome tabs take, which is like it's never enough and you always Always want more, but then it's also it lets you be lazier. Yeah. But anyway, I wanna No, for sure. And and I think that the this is kind of a a a funny question. It it kind of

Quote

Basically only for Chat GPT. Right. And maybe for this year No Book LM. Like sticking anything in there, it'll mostly be correct. And then for basically everyone else, it's like Thanks. Yeah, yeah, yeah. All right, code interpreting we talked about a bunch of times We soft announced the E2B fundraising uh on this podcast. Code sandbox got acquired by Together AI last week. Um which they're now gonna offer as an API so uh more and more activity which is great. Yeah, and then uh w in the last uh two episodes ago with bolt, we talked about the web container stuff. I think like there's maybe the spectrum occurring to the first time. interpreting, which is like, you know, dedicated SDK. There's like yeah, the modals of the world which is like hey we got a sandbox now you just kinda run the commands and orchestrate all of that. I think this is one of the I mean Easter B scrooth has just been crazy just because I mean Everybody needs to run code, right? And I think now I all the products and the Everybody's graduating to like okay, it's not enough to just do chat. So perplexity Which is uh easy to be customers, they do all these nice charts for like finance and all these different things. It's like the products are maturing and I think this is becoming more and more of kinda like a hair on