Episode
SIEM vs. Data Lake: Why We Ditched Traditional Logging?
- Podcast
- Cloud Security Podcast
- Published
- Dec 2, 2025
- Duration seconds
- 2813
- Processing state
not_requested
Actions
POST https://stenobird.com/v1/public/podcasts/cloud-security-podcast-612345/episodes/siem-vs-data-lake-why-we-ditched-traditional-logging/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/cloud-security-podcast-612345/siem-vs-data-lake-why-we-ditched-traditional-logging.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
In this episode, Cliff Crosland , CEO & co-founder of Scanner.dev , shares his candid journey of trying (and initially failing) to build an in-house security data lake to replace an expensive traditional SIEM. Cliff explains the economic breaking point where scaling a SIEM became "more expensive than the entire budget for the engineering team". He details the technical challenges of moving terabytes of logs to S3 and the painful realization that querying them with Amazon Athena was slow and costly for security use cases . This episode is a deep dive into the evolution of logging architecture, from SQL-based legacy tools to the modern "messy" data lake that embraces full-text search on unstructured data. We discuss the "data engineering lift" required to build your own, the promise (and limitations) of Amazon Security Lake, and how AI agents are starting to automate detection engineering and schema management. Guest Socials - Cliff's Linkedin Podcast Twitter - @CloudSecPod If you want to watch videos of this LIVE STREAMED episode and past episodes - Check out our other Cloud Security Social Channels: - Cloud Security Podcast- Youtube - …