KV Cache

Paper: WeaveTime: Stream from Earlier Frames into Emergent Memory in VideoLLMs PDF: CVPR 2026 Open Access Supplement: CVPR 2026 Supplementary Material Project: WeaveTime Code: zhangyl4/weavetime Core Idea WeaveTime can be summarized as: ReKV-style historical visual KV memory + temporal-order SFT + uncertainty-gated coarse-to-fine retrieval. It does not introduce a new compressed KV format or a bounded long-term memory. Instead, it improves two parts of a ReKV-style streaming system: temporal interpretation: teach the Video-LLM to distinguish event order rather than treat video as an unordered bag of visual evidence; memory access: answer from the current/recent context when possible and retrieve historical KV only when the model appears uncertain. The individual ingredients are familiar. The paper’s main contribution is their integration and evaluation under a causal streaming VideoQA protocol. ...

Paper: StreamingVLM: Real-Time Understanding for Infinite Video Streams Project: StreamingVLM Code: mit-han-lab/streaming-vlm Datasets: Inf-Stream-Train / Inf-Stream-Eval Core Idea StreamingVLM is designed for a different target from most streaming VideoQA papers: it keeps watching a video and producing synchronized commentary, instead of waiting for a future question and then retrieving historical evidence. Its main idea is to align a simple streaming inference cache with the context pattern used during supervised fine-tuning. At inference time, the model keeps only: ...

Paper: MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering Code: IMBALDY/MuKV Background Long streaming VideoQA has a simple but painful constraint: the video keeps arriving, while the future user questions are unknown. KV-cache methods such as ReKV make this setting more practical. Instead of recomputing historical video tokens when a question arrives, the model can prefill the video stream in advance, store the visual KV cache, retrieve the relevant cache blocks later, and answer with much lower online cost. ...

Related papers: ReKV: Paper / Code StreamKV: Paper / Code InfiniPot-V: Paper / Code StreamMem: Paper LiveVLM: Paper / Code StreamingTOM: Paper / Code rLiVS: Paper / Code Core Question All of these papers are trying to solve the same systems problem: When a video stream keeps arriving, the user question is not known yet, and GPU memory is limited, how should a Video-LLM process the stream, compress memory, retrieve evidence, and generate an answer with low latency? The methods look different if we read them one by one: KV cache retrieval, semantic chunking, TaR / VaN, chat-template proxy queries, VSB, CTR, OQM, caption RAG, and so on. ...

Paper: StreamingTOM: Streaming Token Compression for Efficient Video Understanding Code: YIGE24/StreamingTOM Background Streaming video understanding has two constraints that offline video understanding does not really need to respect: causality: the model cannot use future frames to decide how to compress current frames; accumulation: tokens and KV cache keep growing as the video stream becomes longer. Most recent training-free streaming methods mainly work on the post-LLM KV cache: ReKV stores historical KV blocks and retrieves relevant ones at question time; StreamKV improves the segmentation / compression / retrieval pipeline; InfiniPot-V and StreamMem keep a bounded KV memory with query-agnostic compression; LiveVLM combines query-agnostic KV compression with query-time retrieval. These methods are useful, but they still have one important blind spot: ...

Paper: LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval Code: sjtu-zhao-lab/LiveVLM Background Online video understanding is harder than offline long-video QA. In the offline setting, the model usually receives a video and a question together. It can then sample, compress, or retrieve content with the query already known. In the online setting, the model has two separate phases: encoding phase: video frames arrive continuously before any question appears; response phase: when a user asks a question, the model should answer quickly from the already processed stream. This creates three constraints at the same time: ...

Paper: StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding Background Streaming video understanding is hard because the model has to process frames as they arrive, without knowing: how long the video will be; what future user questions will ask; which past details will become important later. For long videos, the visual tokens and their KV cache keep growing over time. Even if a long-context MLLM can technically accept many tokens, storing and attending to all historical KV entries is still expensive. ...

Paper: InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding Code: aiha-lab/InfiniPot-V Background Streaming video understanding is more constrained than offline long-video understanding. In offline settings, the model can see the whole video first, maybe even the user query first, and then decide how to compress tokens or KV cache. But in streaming settings: frames arrive continuously; future queries are unknown; memory is fixed; KV cache still grows roughly linearly with time. This is exactly the part that makes many existing KV compression methods awkward for real streaming scenarios. ...

Paper: StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression Code: sou1p0wer/StreamKV Background Streaming video question-answering (StreamingVQA) requires a model to continuously process incoming video, preserve useful historical context, and answer questions online with low latency. ReKV showed that video QA can be reformulated as retrieve relevant KV caches first, then answer with the retrieved KV. But it still has several weaknesses: It uses uniform segmentation, which may cut through semantic boundaries. It keeps essentially the whole historical visual context, so memory usage is still large. Its retrieval strategy is not flexible enough, especially when the useful information is distributed differently across layers. Core Idea StreamKV extends the ReKV line in two directions at the same time: ...

Paper: Streaming Video Question-Answering with In-context Video KV-Cache Retrieval Code: Becomebright/ReKV Background Consider the problem of streaming video question-answering (StreamingVQA), it presents three challenges: Efficient Video Encoding: we need to efficiently process incoming frames without access to future frames or frequent revisiting of distant past frames. Video Context Preservation: models must preserve relevant information from earlier frames. Real-Time Response: models must provide accurate answers with minimum delay. Core Idea The attention calculation makes it possible to decouple video encoding from question answering. So we can pre-produce KV and reuse KV in QA. ...