Benchmark

Paper: A Simple Baseline for Streaming Video Understanding Project: SimpleStream Code: EvolvingLMMs-Lab/SimpleStream Core Idea SimpleStream is a deliberately simple streaming baseline: when a question arrives, keep only the most recent N observed frames and feed them with the question to an off-the-shelf VLM. It uses: 1 fps sampling under the observed-only streaming protocol; a short recent-frame window, usually 2 / 4 / 8 frames; Qwen2.5-VL-7B or Qwen3-VL-8B as the base VLM; no new memory bank; no retrieval module; no KV cache compression; no extra training. So this is not a new architecture. It is a strong recency baseline for asking whether complex streaming memory is actually helping. ...

This post is a small index for the benchmarks that appear repeatedly in recent streaming video / long-video VLM papers. The main split is simple: online streaming benchmarks test whether the model can answer while the video is still coming in; offline long-video benchmarks test long-context video understanding, but usually assume the whole video is already available; standard video QA benchmarks are useful for comparability, but they are not the real target of streaming-memory papers. The tables below are copied or compacted from the corresponding method papers. They should not be read as one unified leaderboard: backbones, frame rates, memory budgets, judge versions, subtitle settings, and dataset splits often differ. ...