# LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning

Page: https://stenobird.com/podcast/daily-paper-cast-7079649/latentomni-rethinking-omni-modal-understanding-via-unified-audio-visual-latent-reasoning
Text version: https://stenobird.com/podcast/daily-paper-cast-7079649/latentomni-rethinking-omni-modal-understanding-via-unified-audio-visual-latent-reasoning.md
Podcast: [Daily Paper Cast](https://stenobird.com/podcast/daily-paper-cast-7079649)
Published: 2026-05-23T04:27:07+00:00
Episode link: https://share.transistor.fm/s/60bce2d7
Audio file: https://media.transistor.fm/60bce2d7/61f2d843.mp3
Processing state: not_requested
JSON: https://stenobird.com/v1/public/podcasts/daily-paper-cast-7079649/episodes/latentomni-rethinking-omni-modal-understanding-via-unified-audio-visual-latent-reasoning
Duration seconds: 1323

## Resource

🤗 Upvotes: 35 | cs.CL, cs.CV Authors: Yifan Dai, Zhenhua Wu, Bohan Zeng, Daili Hua, Jialing Liu, Bozhou Li, Yuran Wang, Chengzhuo Tong, Hao Liang, Xiaochen Ma, Junbo Niu, Tianyu Guo, Yang Shi, Yue Ding, Yiyan Ji, Bingyin Mei, Yushuo Guan, Yuanxing Zhang, Pengfei Wan, Fangcheng Fu, Wentao Zhang Title: LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning Arxiv: http://arxiv.org/abs/2605.22012v1 Abstract: Joint audio-visual reasoning is essential for omnimodal understanding, yet current multimodal large language models (MLLMs) still struggle when reasoning requires fine-grained evidence from both modalities. A central limitation is that explicit text-based chain-of-thought (CoT) compresses continuous audio-visual signals into discrete tokens, weakening temporal grounding and shifting intermediate reasoning toward language priors. We argue that a unified latent space is a better medium for such reasoning because it preserves dense sensory information while remaining compatible with autoregressive generation. Based on this insight, we propose \textbf{LatentOmni}, a cross-modal reasoning framework that interleaves textual reasoning with audio-visual latent states. LatentOmni introduces feature-level supervision to align latent reasoning states with task-relevant sensory features and uses Omni-Sync Position Embedding (OSPE) to maintain temporal consistency between latent audio and visual states. We further construct \textbf{LatentOmni-Instruct-35K}, a dataset of audio-visual interleaved reasoning trajectories for supervising latent-space reasoning. Comprehensive evaluation across multiple audio-visual reasoning benchmarks demonstrates that LatentOmni achieves the best performance among the evaluated open-source models and consistently outp…

## Actions

- request_transcript: `POST https://stenobird.com/v1/public/podcasts/daily-paper-cast-7079649/episodes/latentomni-rethinking-omni-modal-understanding-via-unified-audio-visual-latent-reasoning/transcription-requests` — Idempotently request low-priority transcript generation for this episode.
- read_markdown: `GET https://stenobird.com/podcast/daily-paper-cast-7079649/latentomni-rethinking-omni-modal-understanding-via-unified-audio-visual-latent-reasoning.md` — Read the agent-friendly Markdown representation of this episode resource.

A page view does not enqueue transcription. Agents should invoke `request_transcript` explicitly when they need this episode processed.

## Transcript

Full transcripts are not published on public pages unless there is a clear rights basis.