# Many-Shot CoT-ICL: Making In-Context Learning Truly Learn

Page: https://stenobird.com/podcast/daily-paper-cast-7079649/many-shot-cot-icl-making-in-context-learning-truly-learn
Text version: https://stenobird.com/podcast/daily-paper-cast-7079649/many-shot-cot-icl-making-in-context-learning-truly-learn.md
Podcast: [Daily Paper Cast](https://stenobird.com/podcast/daily-paper-cast-7079649)
Published: 2026-05-15T04:59:08+00:00
Episode link: https://share.transistor.fm/s/8b97cdae
Audio file: https://media.transistor.fm/8b97cdae/910993ea.mp3
Processing state: not_requested
JSON: https://stenobird.com/v1/public/podcasts/daily-paper-cast-7079649/episodes/many-shot-cot-icl-making-in-context-learning-truly-learn
Duration seconds: 1440

## Resource

🤗 Upvotes: 28 | cs.CL, cs.AI Authors: Tsz Ting Chung, Lemao Liu, Mo Yu, Dit-Yan Yeung Title: Many-Shot CoT-ICL: Making In-Context Learning Truly Learn Arxiv: http://arxiv.org/abs/2605.13511v1 Abstract: In-context learning (ICL) adapts large language models (LLMs) to new tasks by conditioning on demonstrations in the prompt without parameter updates. With long-context models, many-shot ICL can use dozens to hundreds of examples and achieve performance comparable to fine-tuning, yet current understanding of its scaling behavior is largely derived from non-reasoning tasks. We study many-shot chain-of-thought in-context learning (CoT-ICL) for reasoning and show that standard many-shot rules do not transfer. Across non-reasoning and reasoning-oriented LLMs and across non-reasoning and reasoning tasks, we find: (i) a setting-dependent scaling effect, where increasing the number of CoT demonstrations is unstable for non-reasoning LLMs and benefits mainly reasoning-oriented LLMs; (ii) similarity-based retrieval helps on non-reasoning tasks but fails on reasoning, since semantic similarity poorly predicts procedural (i.e., CoT) compatibility; and (iii) an order-scaling effect, where performance variance grows with more CoT demonstrations. We interpret these behaviors by viewing many-shot CoT-ICL as in-context test-time learning rather than scaled pattern matching, and suggests two principles: (i) demonstrations should be easy for the target model to understand, and (ii) they should be ordered to support a smooth conceptual progression. Guided by the principle, we propose Curvilinear Demonstration Selection (CDS), a simple ordering method that yields up to a 5.42 percentage-point gain on geometry with 64 demonstrations. Overall, our results reframe the long context window from a…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/daily-paper-cast-7079649/episodes/many-shot-cot-icl-making-in-context-learning-truly-learn/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/daily-paper-cast-7079649/many-shot-cot-icl-making-in-context-learning-truly-learn.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.