# Data! Data! Data! With Francesco Tisiot

Page: https://stenobird.com/podcast/arrested-devops/data-data-data-with-francesco-tisiot
Text version: https://stenobird.com/podcast/arrested-devops/data-data-data-with-francesco-tisiot.md
Podcast: [Arrested DevOps](https://stenobird.com/podcast/arrested-devops)
Published: 2023-06-01T16:07:50+00:00
Episode link: https://www.arresteddevops.com/data-data-data/
Audio file: https://media.blubrry.com/arresteddevops/content.blubrry.com/arresteddevops/arrested-devops-podcast-episode186.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/arrested-devops/episodes/data-data-data-with-francesco-tisiot
Duration seconds: 2793

## Resource

Uncontrolled data growth creates significant security risks and operational debt. This episode explores how to manage data lineage, prevent manual 'Excel-based' workarounds, and build resilient pipelines using technologies like Kafka.

## Highlights
- Failure mode: Allowing unmanaged data pipelines leads to 'shadow' processes and security vulnerabilities like exposed S3 buckets
- Practical takeaway: Empowering users with proper tools prevents them from reverting to manual, insecure Excel exports
- Main idea: Data lineage and visibility are more critical for scaling than simply implementing the latest shiny technology
- Practical takeaway: Use technologies like Kafka to enable data replayability and schema versioning for more resilient systems
- Main idea: The real challenge in modern data engineering isn't innovation, but the unglamorous work of data cleansing and management

## Topics

Data Engineering, Data Lineage, Kafka, Data Governance, Data Pipelines, DevOps, Data Security, Scalability

## Chapters
- 4:40 — The Intersection of People and Data: Francesco discusses his transition from computer science to a role focused on the intersection of technology and human interaction through data.
- 8:10 — The Data Journey: An exploration of how data points become valuable assets as they move through various stages of a company's lifecycle.
- 17:55 — The Danger of Undocumented Pipelines: The risks of 'orphaned' data pipelines where the origin, ownership, and cost of data movement become unknown.
- 25:25 — Preventing the 'Excel Way': How a lack of proper data infrastructure forces engineers into manual, insecure data handling processes.
- 29:25 — Leveraging Kafka for Connectivity: Using Kafka Connect and JSON payloads to parse and link disparate data sources effectively.
- 36:20 — Building Trustworthy Systems: Moving beyond speed to focus on error recovery, reliability, and building trust in data outputs.
- 40:00 — The Power of Data Replay: How Kafka's log retention allows for replaying historical data to support new consumers and schema evolutions.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/arrested-devops/episodes/data-data-data-with-francesco-tisiot/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/arrested-devops/data-data-data-with-francesco-tisiot.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.