Streaming-Video-Lmm

Series guide. This article is the bounded-memory thematic survey. Start with the development-history hub for the six-path map, read the ReKV note for the exact-archive baseline, and use the pipeline survey for the stage-by-stage architecture. A streaming Video-LMM has to make decisions before it knows which detail a future user will care about. ReKV avoids write-time eviction or compression by storing every generated visual KV block, then retrieves a small subset after the question is known. This does not recover unsampled pixels or information absent from those model states. Fixed-state methods make the opposite trade: they evict, merge, or summarize history during the stream so that memory no longer grows with video duration. ...

Series guide. This is the development-history hub for my streaming Video-LMM notes. For a stage-by-stage systems view, see the pipeline survey. For dataset and protocol details, see the benchmark index. The complete reading map is at the end of this article. Calling a model streaming no longer identifies one technical problem. One paper may process a causal video prefix but retain every historical KV block. Another may keep a fixed cache but permanently forget old evidence. A proactive assistant may know the user’s request before the stream begins, while a delayed question-answering system must decide what to preserve before any question is known. A benchmark may measure the age of remembered evidence without measuring whether the implementation can keep up with a live camera. ...

Paper: WeaveTime: Streaming from Earlier Frames into Emergent Memory in VideoLLMs Preprint (earlier title: Stream from Earlier Frames): arXiv:2602.22142 PDF: CVPR 2026 Open Access Supplement: CVPR 2026 Supplementary Material Project: WeaveTime Code: zhangyl4/weavetime Series guide. PCDF bounds when and how much history is read, not how much ReKV-style history is stored. See the bounded-memory survey for the resource comparison and the development-history hub for the complete series. Core Idea WeaveTime can be summarized as: ReKV-style historical visual KV memory + temporal-order SFT + uncertainty-gated coarse-to-fine retrieval. ...

Paper: MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering Proceedings: CVPR 2026 Open Access Code: IMBALDY/MuKV Series guide. A fixed per-question read can still sit on a growing multi-grained archive. See the bounded-memory survey for that accounting and the development-history hub for the complete series. Background Long streaming VideoQA has a simple but painful constraint: the video keeps arriving, while the future user questions are unknown. KV-cache methods such as ReKV make this setting more practical. Instead of recomputing historical video tokens when a question arrives, the model can prefill the video stream in advance, store the visual KV cache, retrieve the relevant cache blocks later, and answer with much lower online cost. ...

Series guide. This article is the evaluation-protocol index. Start with the development-history hub for the six research paths, and use the pipeline survey for the stage-by-stage architecture view. This post is a small index for the benchmarks that appear repeatedly in recent streaming video / long-video VLM papers. The main split is simple: online streaming benchmarks test whether the model can answer while the video is still coming in; offline long-video benchmarks test long-context video understanding, but usually assume the whole video is already available; standard video QA benchmarks are useful for comparability, but they are not the real target of streaming-memory papers. The tables below are copied or compacted from the corresponding method papers. They should not be read as one unified leaderboard: backbones, frame rates, memory budgets, judge versions, subtitle settings, and dataset splits often differ. ...

Series guide. This article is the stage-by-stage architecture map. Start with the development-history hub for the six research paths, and use the benchmark index for evaluation settings and protocols. Related papers: ReKV: Paper / Code StreamKV: Paper / Code InfiniPot-V: Paper / Code StreamMem: Paper LiveVLM: Paper / Code StreamingTOM: Paper / Code rLiVS: Paper / Code Core Question All of these papers are trying to solve the same systems problem: When a video stream keeps arriving, the user question is not known yet, and GPU memory is limited, how should a Video-LLM process the stream, compress memory, retrieve evidence, and generate an answer with low latency? ...

Paper: LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval Code: sjtu-zhao-lab/LiveVLM Acceptance: DAC 2026 program Version note. This article follows the substantially revised arXiv v2 (2026-04-23) / DAC 2026 method: Vision Sink Bucketing and Position-agnostic KV Retrieval. The 2025 v1 used a different chunk-compression and FIFO mechanism; its results should not be mixed with the v2 design. Series guide. For the distinction between bounded VSB state and query-time PaR reads, see the bounded-memory survey. The development-history hub places LiveVLM in the complete series. ...

Paper: StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding Project: StreamMem Series guide. For how proxy-guided retention fits between exact archives and summary-state memory, see the bounded-memory survey. The development-history hub places this method in the complete series. Background Streaming video understanding is hard because the model has to process frames as they arrive, without knowing: how long the video will be; what future user questions will ask; which past details will become important later. For long videos, the visual tokens and their KV cache keep growing over time. Even if a long-context MLLM can technically accept many tokens, storing and attending to all historical KV entries is still expensive. ...

Paper: InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding Preprint: arXiv:2506.15745 Research reimplementation: aiha-lab/InfiniPot-V Series guide. For why a fixed retained KV cache differs from a bounded read over a growing archive, see the bounded-memory survey. The development-history hub places this shift in the complete series. Background Streaming video understanding is more constrained than offline long-video understanding. In offline settings, the model can see the whole video first, maybe even the user query first, and then decide how to compress tokens or KV cache. But in streaming settings: ...

Series guide. This note audits one method in detail. Start with the development-history hub for the field-level map, or use the pipeline survey to place ReKV’s write, index, and read stages in a complete system. The bounded-memory survey compares its growing exact-KV archive with later fixed-state methods. Paper: Streaming Video Question-Answering with In-context Video KV-Cache Retrieval Proceedings: ICLR 2025 Preprint: arXiv:2503.00540 Code: Becomebright/ReKV ReKV was first public on arXiv on 2025-03-01 and was published at ICLR 2025. The paper, appendix, and official code were rechecked on 2026-09-03. Unless a sentence is explicitly marked as my interpretation, mechanism and measurement claims below describe the authors’ paper or released code. ...