# 481: Dev Horror Stories

Page: https://stenobird.com/podcast/the-bike-shed/481-dev-horror-stories
Text version: https://stenobird.com/podcast/the-bike-shed/481-dev-horror-stories.md
Podcast: [The Bike Shed](https://stenobird.com/podcast/the-bike-shed)
Published: 2025-11-04T08:00:00+00:00
Episode link: https://bikeshed.thoughtbot.com/481
Audio file: https://aphid.fireside.fm/d/1437767933/167c01a1-0eb9-4640-b488-c2f6d6866650/410c5a32-441a-4a31-a402-7ef85e2ea2d6.mp3
Processing state: processed
JSON: https://stenobird.com/v1/public/podcasts/the-bike-shed/episodes/481-dev-horror-stories
Duration seconds: 2602

## Resource

Software engineers share cautionary tales of production disasters, from corrupted data and broken database indexes to the dangers of unvalidated JSON inputs. The discussion explores how technical debt and architectural over-engineering create increasingly complex 'horrors' for senior developers to unwind.

## Highlights
- Failure mode: Using the database console to manually reset passwords can lead to unpredictable production states
- Practical takeaway: Always use EXPLAIN ANALYZE to verify that your query plans are actually utilizing your indexes
- Main idea: Data corruption is significantly more dangerous than code bugs because code can be redeployed, but corrupted state is often permanent
- Failure mode: Relying on massive, unvalidated JSON blobs in admin interfaces creates high-risk environments for non-technical staff
- Main idea: Seniority increases the stakes of mistakes, as architectural decisions like premature microservices are much harder to reverse than simple logic bugs

## Topics

Software Engineering, Database Management, Technical Debt, Production Outages, PostgreSQL, System Architecture, Data Integrity, DevOps

## Chapters
- 1:00 — ThoughtBot Summit in Amsterdam: The hosts discuss their recent experience at the ThoughtBot summit and the history of Amsterdam's canal expansion.
- 7:35 — The Perils of Query Tweaks: A discussion on how small changes to queries can inadvertently break database indexes and degrade performance.
- 13:50 — The Terror of Data Corruption: Why data-related tasks carry much higher stakes and anxiety than deploying buggy code.
- 20:15 — Troubleshooting During Holidays: The difficulty of debugging production issues when team communication is limited by holidays and manual interventions.
- 23:25 — The JSON Blob Admin Panel: A horror story about an admin interface that required customer support staff to manually edit raw JSON strings.
- 40:00 — AI-Generated Pull Request Gremlins: Reflections on the risks of AI-generated tests and the emergence of 'vibe-coded' software.
- 43:15 — The Weight of Seniority: How architectural decisions made by senior developers create much harder-to-fix long-term technical debt.

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/the-bike-shed/episodes/481-dev-horror-stories/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/the-bike-shed/481-dev-horror-stories.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.