{"podcast":{"title":"Software Engineering Institute (SEI) Podcast Series","slug":"software-engineering-institute-sei-podcast-series-389154","podcast_index_feed_id":389154,"rss_url":"https://cmu-sei-podcasts.libsyn.com/rss","website_url":"https://www.sei.cmu.edu/publications/podcasts/index.cfm","image_url":"https://static.libsyn.com/p/assets/4/5/0/6/4506812e834bc5af16c3140a3186d450/5966_Libsyn.png","author":"Carnegie Mellon University Software Engineering Institute","episode_count":435,"summary":"The SEI Podcast Series presents conversations in software engineering, cybersecurity, and future technologies.","last_synced_at":"2026-08-01T02:18:38.616051+00:00","page_url":"https://stenobird.com/podcast/software-engineering-institute-sei-podcast-series-389154"},"episode":{"title":"From Data to Performance: Understanding and Improving Your AI Model","slug":"from-data-to-performance-understanding-and-improving-your-ai-model","published_at":"2025-11-10T21:21:00+00:00","page_url":"https://stenobird.com/podcast/software-engineering-institute-sei-podcast-series-389154/from-data-to-performance-understanding-and-improving-your-ai-model","show_page_url":"https://stenobird.com/podcast/software-engineering-institute-sei-podcast-series-389154","url":"https://cmu-sei-podcasts.libsyn.com/from-data-to-performance-understanding-and-improving-your-ai-model","audio_url":"https://traffic.libsyn.com/clean/secure/cmu-sei-podcasts/SEIP_09_17_Nolan_Testa.mp3?dest-id=762491","summary":"Modern data analytic methods and tools—including artificial intelligence (AI) and machine learning (ML) classifiers—are revolutionizing prediction capabilities and automation through their capacity to analyze and classify data. To produce such results, these methods depend on correlations. However, an overreliance on correlations can lead to prediction bias and reduced confidence in AI outputs. Drift in data and concept, evolving edge cases, and emerging phenomena can undermine the correlations that AI classifiers rely on. As the U.S. government increases its use of AI classifiers and predictors, these issues multiply (or use increase again) . Subsequently, users may grow to distrust results. To address inaccurate erroneous cor r elation s and predictions , we need new methods for ongoing testing and evaluation of AI and ML accuracy. In this podcast from the Carnegie Mellon University Software Engineering Institute (SEI) , Nicholas Testa, a senior data scientist in the SEI's Software Solutions Division (SSD) , and Crisanne Nolan, and Agile transformation engineer, also in SSD, sit down with Linda Parker Gates, Principal Investigator for this research and initiative lead for Software Acquisition Pathways at the SEI, to discuss the AI Robustness (AIR) tool , which allows users to gauge AI and ML classifier performance with data-based confidence.","meta_description":"Modern data analytic methods and tools—including artificial intelligence (AI) and machine learning (ML) classifiers—are revolutionizing prediction capabil…","key_points":[],"chapters":[],"topics":[],"duration_seconds":1602,"processing_state":"not_requested","actions":[{"name":"request_transcript","method":"POST","url":"https://stenobird.com/v1/public/podcasts/software-engineering-institute-sei-podcast-series-389154/episodes/from-data-to-performance-understanding-and-improving-your-ai-model/transcription-requests","description":"Idempotently request low-priority transcript generation for this episode."},{"name":"read_markdown","method":"GET","url":"https://stenobird.com/podcast/software-engineering-institute-sei-podcast-series-389154/from-data-to-performance-understanding-and-improving-your-ai-model.md","description":"Read the agent-friendly Markdown representation of this episode resource."}]}}