Streaming

Paper: OpenReview PDF: Memento: Toward an All-Day Proactive Assistant for Ultra-Long Streaming Video Supplement: OpenReview Supplementary Material Base implementation: showlab/videollm-online Core Idea Memento turns a streaming Video-LLM into a persistent visual monitor. A user registers a request such as tell me every time I hold a phone, and the model keeps observing the stream, updating its history, and deciding at every frame whether to remain silent or respond. The method can be summarized as: ...

Paper: OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning PDF: CVPR 2026 Open Access Code: Solus-sano/OASIS Core Idea OASIS is a training-free external memory and agentic RAG system for streaming video. It organizes the past into an online event hierarchy, reasons from a small working context by default, and retrieves historical keyframes only when the MLLM decides that more evidence is needed. The method can be summarized as: short/medium visual working memory + an online tree of event summaries and keyframes + prompt-based history routing + planned-query retrieval. ...

Paper: Accelerating Streaming Video Large Language Models via Hierarchical Token Compression PDF: CVPR 2026 Open Access Code: lern-to-write/STC Release History STC first appeared as an arXiv preprint on November 30, 2025. The current arXiv version was uploaded on February 11, 2026, and the paper was subsequently accepted by CVPR 2026. It is therefore an older preprint that later received a 2026 conference label, rather than a method first released with the conference proceedings. ...

Paper: WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMs PDF: CVPR 2026 Open Access Supplement: CVPR 2026 Supplementary Material Project: WeaveTime Code: zhangyl4/weavetime Core Idea WeaveTime can be summarized as: ReKV-style historical visual KV memory + temporal-order SFT + uncertainty-gated coarse-to-fine retrieval. It does not introduce a new compressed KV format or a bounded long-term memory. Instead, it improves two parts of a ReKV-style streaming system: temporal interpretation: teach the Video-LLM to distinguish event order rather than treat video as an unordered bag of visual evidence; memory access: answer from the current/recent context when possible and retrieve historical KV only when the model appears uncertain. The individual ingredients are familiar. The paper’s main contribution is their integration and evaluation under a causal streaming VideoQA protocol. ...

Paper: StreamingVLM: Real-Time Understanding for Infinite Video Streams Project: StreamingVLM Code: mit-han-lab/streaming-vlm Datasets: Inf-Stream-Train / Inf-Stream-Eval Core Idea StreamingVLM is designed for a different target from most streaming VideoQA papers: it keeps watching a video and producing synchronized commentary, instead of waiting for a future question and then retrieving historical evidence. Its main idea is to align a simple streaming inference cache with the context pattern used during supervised fine-tuning. At inference time, the model keeps only: ...

Paper: A Simple Baseline for Streaming Video Understanding Project: SimpleStream Code: EvolvingLMMs-Lab/SimpleStream Core Idea SimpleStream is a deliberately simple streaming baseline: when a question arrives, keep only the most recent N observed frames and feed them with the question to an off-the-shelf VLM. It uses: 1 fps sampling under the observed-only streaming protocol; a short recent-frame window, usually 2 / 4 / 8 frames; Qwen2.5-VL-7B or Qwen3-VL-8B as the base VLM; no new memory bank; no retrieval module; no KV cache compression; no extra training. So this is not a new architecture. It is a strong recency baseline for asking whether complex streaming memory is actually helping. ...

Paper: MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering Code: IMBALDY/MuKV Background Long streaming VideoQA has a simple but painful constraint: the video keeps arriving, while the future user questions are unknown. KV-cache methods such as ReKV make this setting more practical. Instead of recomputing historical video tokens when a question arrives, the model can prefill the video stream in advance, store the visual KV cache, retrieve the relevant cache blocks later, and answer with much lower online cost. ...

This post is a small index for the benchmarks that appear repeatedly in recent streaming video / long-video VLM papers. The main split is simple: online streaming benchmarks test whether the model can answer while the video is still coming in; offline long-video benchmarks test long-context video understanding, but usually assume the whole video is already available; standard video QA benchmarks are useful for comparability, but they are not the real target of streaming-memory papers. The tables below are copied or compacted from the corresponding method papers. They should not be read as one unified leaderboard: backbones, frame rates, memory budgets, judge versions, subtitle settings, and dataset splits often differ. ...

Paper: Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge Code: hmxiong/StreamChat Background Most Video-LLMs are still awkward in a real streaming setting. Offline video QA usually assumes: the whole video is already available; the question is known before inference; the interaction is single-turn. But a streaming assistant has a different problem: video frames keep arriving; the user may ask questions at arbitrary timestamps; the system should remember previous conversation turns; the answer should come back with low latency. This is close to the motivation of ReKV, Flash-VStream, LiveVLM, and rLiVS, but StreamChat chooses a different abstraction. ...

Related papers: ReKV: Paper / Code StreamKV: Paper / Code InfiniPot-V: Paper / Code StreamMem: Paper LiveVLM: Paper / Code StreamingTOM: Paper / Code rLiVS: Paper / Code Core Question All of these papers are trying to solve the same systems problem: When a video stream keeps arriving, the user question is not known yet, and GPU memory is limited, how should a Video-LLM process the stream, compress memory, retrieve evidence, and generate an answer with low latency? The methods look different if we read them one by one: KV cache retrieval, semantic chunking, TaR / VaN, chat-template proxy queries, VSB, CTR, OQM, caption RAG, and so on. ...