# #236: Building ML Products at Compare the Market, with Conor O'Neill, the Head of Data Science at Compare The Market

Page: https://stenobird.com/podcast/data-futurology-leadership-and-strategy/236-building-ml-products-at-compare-the-market-with-conor-o-neill-the-head-of-data-science-at-compare-the-market
Text version: https://stenobird.com/podcast/data-futurology-leadership-and-strategy/236-building-ml-products-at-compare-the-market-with-conor-o-neill-the-head-of-data-science-at-compare-the-market.md
Podcast: [Data Futurology - Leadership And Strategy in Artificial Intelligence, Machine Learning, Data Science](https://stenobird.com/podcast/data-futurology-leadership-and-strategy)
Published: 2023-06-13T23:22:32+00:00
Episode link: https://podcasters.spotify.com/pod/show/datafuturology/episodes/236-Building-ML-Products-at-Compare-the-Market--with-Conor-ONeill--the-Head-of-Data-Science-at-Compare-The-Market-e25mibr
Audio file: https://anchor.fm/s/3fab060/podcast/play/72091451/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2023-5-15%2Ff4effd8c-3e93-9db0-cdf7-29ffb06dbc50.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/data-futurology-leadership-and-strategy/episodes/236-building-ml-products-at-compare-the-market-with-conor-o-neill-the-head-of-data-science-at-compare-the-market
Duration seconds: 3154

## Resource

Conor O'Neill explains how to transition from reactive data science to a product-centric machine learning approach. He details the structural changes required to align data engineering, architecture, and science within a large organization.

## Highlights
- Main idea: Treating machine learning models as internal products via APIs ensures better consumption and scalability across the organization
- Practical takeaway: Involve data scientists in the early stages of solution design and architecture to prevent downstream delivery bottlenecks
- Failure mode: Relying on siloed data science work without foundational data engineering and unified documentation leads to unscalable results
- Strategic insight: Transitioning from a practitioner to a leader requires shifting focus from model accuracy to business value and reusable infrastructure
- Future trend: Generative AI may shift the data scientist's role toward 'data science as a service,' focusing on stewardship and integration rather than just model building

## Topics

Machine Learning Product Management, Data Transformation, Data Science Leadership, MLOps, Predictive Analytics, Data Engineering, Generative AI, Solution Architecture

## Chapters
- 1:00 — Coordinating Data Science and Engineering: The challenges of synchronizing data scientists with API development and data ingestion pipelines.
- 5:00 — From Astrophysics to Data Science: Conor's career pivot and his experience building the early data science function at Flight Centre.
- 8:50 — Building Data Capability: The journey of establishing a dedicated data science function within an established organization.
- 12:50 — Predictive Use Cases: Using machine learning to predict customer purchase behavior and shorten the sales cycle.
- 16:50 — Dimensionality Reduction in Practice: How complex customer inputs are distilled into actionable scores for business decision-making.
- 20:40 — Modernizing the Data Stack: Leveraging Databricks to improve modeling, documentation, and cohesive data modeling.
- 24:40 — Managing Transformation Dependencies: The complexities of coordinating new API development with downstream machine learning needs.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/data-futurology-leadership-and-strategy/episodes/236-building-ml-products-at-compare-the-market-with-conor-o-neill-the-head-of-data-science-at-compare-the-market/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/data-futurology-leadership-and-strategy/236-building-ml-products-at-compare-the-market-with-conor-o-neill-the-head-of-data-science-at-compare-the-market.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.