Streaming

Series guide. This article is the bounded-memory thematic survey. Start with the development-history hub for the six-path map, read the ReKV note for the exact-archive baseline, and use the pipeline survey for the stage-by-stage architecture. A streaming Video-LMM has to make decisions before it knows which detail a future user will care about. ReKV avoids write-time eviction or compression by storing every generated visual KV block, then retrieves a small subset after the question is known. This does not recover unsampled pixels or information absent from those model states. Fixed-state methods make the opposite trade: they evict, merge, or summarize history during the stream so that memory no longer grows with video duration. ...

Series guide. This is the development-history hub for my streaming Video-LMM notes. For a stage-by-stage systems view, see the pipeline survey. For dataset and protocol details, see the benchmark index. The complete reading map is at the end of this article. Calling a model streaming no longer identifies one technical problem. One paper may process a causal video prefix but retain every historical KV block. Another may keep a fixed cache but permanently forget old evidence. A proactive assistant may know the user’s request before the stream begins, while a delayed question-answering system must decide what to preserve before any question is known. A benchmark may measure the age of remembered evidence without measuring whether the implementation can keep up with a live camera. ...

Paper: OpenReview PDF: Memento: Toward an All-Day Proactive Assistant for Ultra-Long Streaming Video Supplement: OpenReview Supplementary Material Base implementation: showlab/videollm-online Core Idea Memento turns a streaming Video-LLM into a persistent visual monitor. A user registers a request such as tell me every time I hold a phone, and the model keeps observing the stream, updating its history, and deciding at every frame whether to remain silent or respond. The method can be summarized as: ...

Paper: OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning PDF: CVPR 2026 Open Access Code: Solus-sano/OASIS Core Idea OASIS is a training-free external memory and agentic RAG system for streaming video. It organizes the past into an online event hierarchy, reasons from a small working context by default, and retrieves historical keyframes only when the MLLM decides that more evidence is needed. The method can be summarized as: short/medium visual working memory + an online tree of event summaries and keyframes + prompt-based history routing + planned-query retrieval. ...

Paper: Accelerating Streaming Video Large Language Models via Hierarchical Token Compression PDF: CVPR 2026 Open Access Code: lern-to-write/STC Release History STC first appeared as an arXiv preprint on November 30, 2025. The current arXiv version was uploaded on February 11, 2026, and the paper was subsequently accepted by CVPR 2026. It is therefore an older preprint that later received a 2026 conference label, rather than a method first released with the conference proceedings. ...

Paper: WeaveTime: Streaming from Earlier Frames into Emergent Memory in VideoLLMs Preprint (earlier title: Stream from Earlier Frames): arXiv:2602.22142 PDF: CVPR 2026 Open Access Supplement: CVPR 2026 Supplementary Material Project: WeaveTime Code: zhangyl4/weavetime Series guide. PCDF bounds when and how much history is read, not how much ReKV-style history is stored. See the bounded-memory survey for the resource comparison and the development-history hub for the complete series. Core Idea WeaveTime can be summarized as: ReKV-style historical visual KV memory + temporal-order SFT + uncertainty-gated coarse-to-fine retrieval. ...

Paper: StreamingVLM: Real-Time Understanding for Infinite Video Streams Project: StreamingVLM Code: mit-han-lab/streaming-vlm Datasets: Inf-Stream-Train / Inf-Stream-Eval Core Idea StreamingVLM is designed for a different target from most streaming VideoQA papers: it keeps watching a video and producing synchronized commentary, instead of waiting for a future question and then retrieving historical evidence. Its main idea is to align a simple streaming inference cache with the context pattern used during supervised fine-tuning. At inference time, the model keeps only: ...

Paper: A Simple Baseline for Streaming Video Understanding Project: SimpleStream Code: EvolvingLMMs-Lab/SimpleStream Core Idea SimpleStream is a deliberately simple streaming baseline: when a question arrives, keep only the most recent N observed frames and feed them with the question to an off-the-shelf VLM. It uses: 1 fps sampling under the observed-only streaming protocol; a short recent-frame window, usually 2 / 4 / 8 frames; Qwen2.5-VL-7B or Qwen3-VL-8B as the base VLM; no new memory bank; no retrieval module; no KV cache compression; no extra training. So this is not a new architecture. It is a strong recency baseline for asking whether complex streaming memory is actually helping. ...

Paper: MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering Proceedings: CVPR 2026 Open Access Code: IMBALDY/MuKV Series guide. A fixed per-question read can still sit on a growing multi-grained archive. See the bounded-memory survey for that accounting and the development-history hub for the complete series. Background Long streaming VideoQA has a simple but painful constraint: the video keeps arriving, while the future user questions are unknown. KV-cache methods such as ReKV make this setting more practical. Instead of recomputing historical video tokens when a question arrives, the model can prefill the video stream in advance, store the visual KV cache, retrieve the relevant cache blocks later, and answer with much lower online cost. ...

Series guide. This article is the evaluation-protocol index. Start with the development-history hub for the six research paths, and use the pipeline survey for the stage-by-stage architecture view. This post is a small index for the benchmarks that appear repeatedly in recent streaming video / long-video VLM papers. The main split is simple: online streaming benchmarks test whether the model can answer while the video is still coming in; offline long-video benchmarks test long-context video understanding, but usually assume the whole video is already available; standard video QA benchmarks are useful for comparability, but they are not the real target of streaming-memory papers. The tables below are copied or compacted from the corresponding method papers. They should not be read as one unified leaderboard: backbones, frame rates, memory budgets, judge versions, subtitle settings, and dataset splits often differ. ...