Episode
Experimental Results from a Self-Improving Retrieval System for Conversational Memory
- Published
- May 8, 2026
- Duration seconds
- 2671
- Processing state
not_requested- Canonical source
- https://share.transistor.fm/s/3ad0c965
Actions
POST https://stenobird.com/v1/public/podcasts/tech-stories-tech-brief-by-hackernoon-6365648/episodes/experimental-results-from-a-self-improving-retrieval-system-for-conversational-memory/transcription-requests
Idempotently request low-priority transcript generation for this episode.GET https://stenobird.com/podcast/tech-stories-tech-brief-by-hackernoon-6365648/experimental-results-from-a-self-improving-retrieval-system-for-conversational-memory.md
Read the agent-friendly Markdown representation of this episode resource.
Summary
This story was originally published on HackerNoon at: https://hackernoon.com/experimental-results-from-a-self-improving-retrieval-system-for-conversational-memory . Eighteen retrieval experiments on agent memory: why BM25 dominates, what clustered retrieval-induced forgetting actually does, and the Rust port that shipped. Check more stories related to tech-stories at: https://hackernoon.com/c/tech-stories . You can also check exclusive content about #agent-memory , #rag , #bm25 , #retrieval-systems , #cross-encoder-reranking , #longmemeval , #faiss , #hackernoon-top-story , and more. This story was written by: @teimurjan . Learn more about this writer by checking @teimurjan's about page, and for more stories, please visit hackernoon.com . The biology-inspired mutation layer didn't work. A learned MLP adapter and segmentation mutation both produced ~zero NDCG lift on LongMemEval. The control loop was sound; the perturbations weren't load-bearing. A recall diagnostic reframed the project: 78% of relevant entries never reached the cross-encoder. Bi-encoder recall was the ceiling, not the mutation layer. Standard IR wins compounded: 0.95-cosine dedup plus BM25 alongside vector plus cross-encoder rerank took NDCG@10 from 0.22 to 0.34. BM25 alone beat pretrained embeddings by 76% on this corpus. Clustered retrieval-induced forgetting (Anderson 1994, ported as far as I can tell for the first time) added +1.9pp NDCG with p=0.0001 on LongMemEval. Regresses on NFCorpus: the mechanism is scoped to single-user long-term conversation memory, not general IR. Write-time LLM enrichment (gist plus anticipated queries via Haiku) was the biggest single lever: +8.3pp NDCG on covered queries. A regex-tokenizer fix that BM25 had been missing was worth +1.4pp NDCG on the headline benchmark.…