Light-Omni

Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory

Chang Nie Jiaju Wei Junlan Feng^* Chaoyou Fu Caifeng Shan^*

Nanjing University

* Corresponding authors

Demo

Light-Omni web demo with reflective responses and sleep-time memory consolidation. This demo is built on Qwen3-Omni-30B-A3B-Instruct.

Introduction

Agentic video understanding equips multimodal models with long-term memory to process continuous, long-horizon streams. Existing video agents often rely on detective-style iterative reasoning for search control and evidence aggregation, which introduces high latency and computational cost.

Light-Omni reframes this process as reflexive video understanding. Instead of repeatedly planning and searching, it maintains dual contextual states that provide global context and generate semantically aligned retrieval embeddings in a single forward pass.

Comparison between iterative video agents and Light-Omni — Light-Omni replaces costly iterative reasoning with global context and semantically aligned retrieval for low-latency video understanding.

The Light-Omni Framework

Light-Omni builds a multimodal long-term memory system with identity profiles, semantic memory, and episodic memory. Sleep-time memory consolidation constructs a compact global state from episodic memory, preserving recent details while summarizing long-range observations.

Conditioned on this global state, Light-Omni derives a latent state that directly controls autonomous actions and produces retrieval embeddings. This coupled design enables Light-Omni to perform reflexive evidence aggregation and response generation, achieving real-time interaction.

Light-Omni framework — Dual contextual states enable global video context, reflexive action control, and aligned evidence retrieval.

Results

+2.4% average accuracy over M3-Agent

12.1x speedup over M3-Agent

2.6x GPU memory efficiency improvement

Light-Omni performance results — Light-Omni achieves strong long-video performance while maintaining near-constant latency as video duration increases.

Citation

Please cite Light-Omni if this project is useful for your research.

@misc{nie2026lightomni,
  title={Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory},
  author={Nie, Chang and Wei, Jiaju and Feng, Junlan and Fu, Chaoyou and Shan,
  Caifeng},
  year={2026},
  eprint={2607.05511},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={http://arxiv.org/abs/2607.05511}
}