{"podcast":{"title":"The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)","slug":"twiml-ai-podcast","podcast_index_feed_id":1045879,"rss_url":"https://feeds.megaphone.fm/MLN2155636147","website_url":"https://twimlai.com","image_url":"https://megaphone.imgix.net/podcasts/35230150-ee98-11eb-ad1a-b38cbabcd053/image/TWIML_AI_Podcast_Official_Cover_Art_1400px.png?ixlib=rails-4.3.1&max-w=3000&max-h=3000&fit=crop&auto=format,compress","author":"TWIML","episode_count":790,"summary":"Machine learning and artificial intelligence are dramatically changing the way businesses operate and people live. The TWIML AI Podcast brings the top minds and ideas from the world of ML and AI to a broad and influential community of ML/AI researchers, data scientists, engineers and tech-savvy business and IT leaders. Hosted by Sam Charrington, a sought after industry analyst, speaker, commentator and thought leader. Technologies covered include machine learning, artificial intelligence, deep learning, natural language processing, neural networks, analytics, computer science, data science and more.","last_synced_at":"2026-08-02T15:06:58.791455+00:00","page_url":"https://stenobird.com/podcast/twiml-ai-podcast"},"episode":{"title":"Why Models Are AI’s Next Training Dataset with Damian Borth - #772","slug":"why-models-are-ai-s-next-training-dataset-with-damian-borth-772","published_at":"2026-07-27T21:40:00+00:00","page_url":"https://stenobird.com/podcast/twiml-ai-podcast/why-models-are-ai-s-next-training-dataset-with-damian-borth-772","show_page_url":"https://stenobird.com/podcast/twiml-ai-podcast","url":"https://twimlai.com/podcast/twimlai/why-models-are-ais-next-training-dataset","audio_url":"https://pscrb.fm/rss/p/traffic.megaphone.fm/MLN5471902114.mp3","summary":"As high-quality training data becomes scarce, researchers are pivoting to 'weight-space learning,' treating the parameters of existing models as a new primary dataset. This approach aims to leverage the distilled knowledge within millions of GPU hours of previous training to accelerate model development.","meta_description":"Explore the frontier of weight-space learning: using trained neural network weights as a new training modality to build more efficient foundation models.","key_points":["Main idea: Weight-space learning treats the parameters of pre-trained models as an input modality, rather than just the end product of training","Practical takeaway: This method can significantly reduce the cost and time required to create specialized models for specific tasks","Failure mode: Scaling alone is insufficient; the diversity of the model zoo used for training is critical to preventing stagnation","Technical insight: Effective weight-based learning requires specialized machinery to handle varying architectures, sequence lengths, and tokenizers","Future direction: The field is moving toward building foundation models of neural networks that can generate or analyze arbitrary architectures"],"chapters":[{"start_ms":60000,"title":"The Shift to Weight-Space Learning","summary":"An introduction to the concept of using trained weights as a new source of data to accelerate model creation."},{"start_ms":240000,"title":"Weights as Neural DNA","summary":"Understanding the configuration of parameters as the fundamental information extracted from massive compute cycles."},{"start_ms":480000,"title":"Predicting Model Performance","summary":"How extracting features from weights can allow researchers to predict downstream task accuracy."},{"start_ms":660000,"title":"Scaling via Auto-encoders","summary":"Using encoder architectures to learn discriminative features from a large collection of neural networks."},{"start_ms":1320000,"title":"Training on the Model Zoo","summary":"The transition from training on a fixed set of models to leveraging the vast, diverse library of Hugging Face."},{"start_ms":1500000,"title":"Architecture-Agnostic Generation","summary":"Techniques for training backbones that can sample and generate different architectures, such as ResNets or EfficientNets."},{"start_ms":2580000,"title":"Frequency Domains and Neural Artifacts","summary":"Exploring the potential of applying FFT to weights and studying gradients and activations as learnable representations."}],"topics":["Weight-space learning","Neural network weights","Foundation models","Machine learning optimization","Model Zoo","Parameter efficiency","Representation learning","Neural architecture search"],"duration_seconds":2820,"processing_state":"processed","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/twiml-ai-podcast/episodes/why-models-are-ai-s-next-training-dataset-with-damian-borth-772/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/twiml-ai-podcast/why-models-are-ai-s-next-training-dataset-with-damian-borth-772.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}