This post is a small index for the benchmarks that appear repeatedly in recent streaming video / long-video VLM papers.

The main split is simple:

  • online streaming benchmarks test whether the model can answer while the video is still coming in;
  • offline long-video benchmarks test long-context video understanding, but usually assume the whole video is already available;
  • standard video QA benchmarks are useful for comparability, but they are not the real target of streaming-memory papers.

The tables below are copied or compacted from the corresponding method papers. They should not be read as one unified leaderboard: backbones, frame rates, memory budgets, judge versions, subtitle settings, and dataset splits often differ.

Online / Streaming Benchmarks

BenchmarkWhat It Mainly TestsPapers Using It
RVS-EgoStreaming QA on egocentric videos with timestamped questions. Good for testing whether old visual evidence remains accessible.ReKV, StreamMem, LiveVLM, StreamingTOM, rLiVS, MuKV
RVS-MovieStreaming QA on movie-style videos. More narrative and event-heavy than RVS-Ego.ReKV, StreamMem, LiveVLM, StreamingTOM, rLiVS, MuKV
StreamingBenchBroader streaming benchmark. Recent papers often use the real-time visual understanding subset for causal evaluation, while the full benchmark also covers omni-source and contextual understanding.StreamKV, LiveVLM, InfiniPot-V, MuKV, SimpleStream, WeaveTime, STC, OASIS
OVO-BenchTimestamp-conditioned online evaluation with real-time perception, backward tracing, and forward active responding tracks. Useful for separating present grounding from historical recall and future-event waiting.SimpleStream, StreamingVLM, WeaveTime, STC, OASIS
Inf-Streams-EvalDense, per-second commentary over complete sports broadcasts averaging more than two hours. It evaluates synchronized continuous generation rather than question answering.StreamingVLM
StreamBenchOnline multi-turn video QA with memory-heavy question types such as object search, long-term memory search, short-term memory search, conversational interaction, knowledge QA, and simple factual QA.StreamChat; baselines include Video-online and Flash-VStream

RVS-Ego

Metric: open-ended streaming VideoQA, usually reported as LLM-judge Acc. and a 1-5 Score / Sco.. Judge versions differ across papers, so rows are grouped by paper setting rather than normalized.

Method / PaperBackbone or SettingAcc.Score / Sco.Source
ReKV Internal RetrievalLLaVA-OV-7B; 0.5 FPS; KV offload63.74.0ReKV Table 5
StreamMemLLaVA-OneVision-7B; GPU memory < 28 GB57.63.8StreamMem Table 2
LiveVLMLLaVA-OneVision-Qwen2-7B-OV; 24 GB; no CPU offload57.83.9LiveVLM Table 3
StreamingTOMLLaVA-OV-7B; GPT-3.5-turbo-0125; 28 GB58.33.9StreamingTOM Table 2
rLiVSLLaVA-OV-7B; 0.5 FPS; 10K context65.34.0rLiVS Table 3
rLiVSQwen2.5-VL-7B; RVS streaming setting68.14.0rLiVS Table 3
MuKVLLaVA-OV-7B; 8.3K inference tokens; 59K memory tokens59.5-MuKV Table 1

RVS-Movie

Metric: open-ended streaming VideoQA, again reported with LLM-judge accuracy and sometimes a 1-5 score. MuKV uses GPT-3.5-turbo after GPT-3.5-turbo-0613 was deprecated, so its accuracy is not directly comparable with older judge-version rows.

Method / PaperBackbone or SettingAcc.Score / Sco.Source
ReKV Internal RetrievalLLaVA-OV-7B; 0.5 FPS; KV offload54.43.6ReKV Table 5
StreamMemLLaVA-OneVision-7B; GPT-3.5-turbo-0125; GPU memory < 28 GB52.73.4StreamMem Table 2
LiveVLMLLaVA-OneVision-Qwen2-7B-OV; 24 GB; no CPU offload53.43.6LiveVLM Table 3
StreamingTOMLLaVA-OV-7B; GPT-3.5-turbo-0125; 28 GB53.23.5StreamingTOM Table 2
rLiVSLLaVA-OV-7B; 0.5 FPS; 10K context57.73.6rLiVS Table 3
rLiVSQwen2.5-VL-7B; 0.5 FPS; 10K context56.13.6rLiVS Table 3
MuKVLLaVA-OV-7B; GPT-3.5-turbo; 8.3K inference tokens; 59K memory tokens48.5-MuKV Table 1

StreamingBench

Metric scope varies by paper: some use the full benchmark, some use real-time visual understanding, and some aggregate real-time plus omni-source tasks.

Method / PaperBackbone or SettingStreamingBench ScopeMain MetricScoreSource
StreamKVLLaVA-OneVision-Qwen2-7B-OV; 0.5 FPS; 60% KV compressionFull: real-time, omni-source, contextualOverall58.9StreamKV Table 1
LiveVLMLLaVA-OneVision-Qwen2-7B-OV; 12K cache; 40% retrievalReal-time + omni-sourceOverall63.10LiveVLM Table 4
InfiniPot-VQwen-2.5-VL-7B; 4K memory budgetReal-time visual understandingStreamingBench76.4InfiniPot-V Table 4
MuKVLLaVA-OV-7B; 0.5 FPS; 59K memory tokens per 300 framesReal-time visual understandingAll64.4MuKV Table 1
SimpleStreamQwen3-VL-8B + 4 recent frames; 1 FPS causal prefixRTVU accuracyStreamingBench RTVU80.59SimpleStream Table 1
WeaveTimeLLaVA-OV-7B; 1 FPS; multi-turn evaluationReal-time visual understandingReal-Time AVG72.13WeaveTime Table 2
WeaveTimeQwen2-VL-7B; 1 FPS; multi-turn evaluationReal-time visual understandingReal-Time AVG75.39WeaveTime Table 2
STC-Cacher & PrunerLLaVA-OneVision-7B + ReKV; 0.5 FPSReal-time visual understandingAVG65.2STC Table 2
OASISQwen3-VL-8B; 0.5 FPS; official All Context settingReal-time visual understandingReal-Time All78.22OASIS Table 2

OVO-Bench

Metric handling varies by paper. The first table keeps the observed-only aggregate used by SimpleStream, with OVO RT Avg., OVO Bwd Avg., and OVO Avg..

Method / PaperBackbone or SettingOVO RT Avg.OVO Bwd Avg.OVO Avg.Source
Qwen2.5-VL-7B1 FPS causal prefix59.944.752.28SimpleStream Table 1
StreamForest-7B1 FPS61.252.056.60SimpleStream Table 1
HERMES-7B1 FPS; Qwen2.5-VL-7B + HERMES 4K tokens69.049.459.20SimpleStream Table 1
SimpleStreamQwen2.5-VL-7B + 4 recent frames78.451.965.13SimpleStream Table 1
SimpleStreamQwen3-VL-8B + 4 recent frames81.454.067.70SimpleStream Table 1

Other recent papers report only selected OVO-Bench tracks or use a paper-specific causal protocol. Their results are kept in a separate table rather than filling a synthetic overall average.

Method / PaperBackbone or SettingReported OVO-Bench MetricsSource
StreamingVLMQwen2.5-VL-7B; streaming SFT; VQA evaluationRealtime: 61.96StreamingVLM Table 3
WeaveTimeLLaVA-OV-7B; 1 FPS; multi-turn evaluationReal-Time AVG: 68.82WeaveTime Table 2
WeaveTimeQwen2-VL-7B; 1 FPS; multi-turn evaluationReal-Time AVG: 66.28WeaveTime Table 2
STC-Cacher & PrunerLLaVA-OneVision-7B + ReKV; 0.5 FPSReal-Time: 62.5; Backward: 63.3; Forward: 52.0STC Table 1
OASISQwen2.5-VL-7B; 0.5 FPSPerception: 67.26; Backward: 52.61OASIS Table 1
OASISQwen3-VL-8B; 0.5 FPSPerception: 78.14; Backward: 57.21OASIS Table 1

Inf-Streams-Eval

Metric: GPT-5 pairwise preference against a named baseline, with access to reference commentary. This is a dense continuous-commentary evaluation, not an accuracy score and not directly comparable with streaming VideoQA tables.

Method / PaperSettingComparisonWin RateSource
StreamingVLMInfinite streaming mode; complete sports games average 2.12 hoursvs. GPT-4o mini in 100-second chunks66.18%StreamingVLM Table 1

StreamBench

Metrics: Sco., Acc., Coh., and RPD; the full paper also reports six-task breakdowns for object search, long-term memory, short-term memory, conversational interaction, knowledge QA, and simple factual QA.

Method / PaperSettingFPSSco.Acc.Coh.RPDSource
Video-onlineStreaming baseline53.1156.41.941.07StreamChat Table 4
Flash-VStreamStreaming baseline12.8952.12.214.15StreamChat Table 4
StreamChat SlowLongVA + CLIP-L-P14153.4864.71.760.90StreamChat Table 4
StreamChat BaseLongVA + CLIP-L-P14203.4263.81.790.89StreamChat Table 4
StreamChat FastLongVA + CLIP-L-P14323.2861.71.810.85StreamChat Table 4

Offline Long-video Benchmarks

BenchmarkWhat It Mainly TestsPapers Using It
MLVULong-video multiple-choice understanding. Often used as a compact proxy for long-context video reasoning.ReKV, StreamMem, LiveVLM, StreamingTOM, InfiniPot-V, AdaVideoRAG, MuKV, WeaveTime, STC
Video-MMEGeneral long-video multimodal understanding across short, medium, and long videos. Papers often report the no-subtitle setting.StreamMem, LiveVLM, StreamingTOM, InfiniPot-V, AdaVideoRAG, ViG-RAG, MuKV, StreamingVLM, STC
EgoSchemaLong-range egocentric video reasoning. Useful for memory and temporal reasoning evaluation.ReKV, StreamMem, StreamingTOM, InfiniPot-V, MuKV, STC
LongVideoBenchLong-video QA with stronger pressure on long-context multimodal reasoning.LiveVLM, StreamingTOM, InfiniPot-V, ViG-RAG, StreamingVLM
HourVideoHour-scale egocentric video QA over videos lasting 20–120 minutes.OASIS
HiVUHierarchical long-video benchmark for knowledge-rich videos. It separates questions into different reasoning levels, making it useful for adaptive VideoRAG evaluation.AdaVideoRAG
LongerVideosLong-form and multi-video benchmark used to test retrieval and reasoning over extended videos, especially for graph-RAG style methods.ViG-RAG
ActivityNet-QAOpen-ended video QA with longer activity videos.ReKV, StreamChat
QAEGO4DEgocentric long-video QA.ReKV, WeaveTime
CG-Bench / CGBenchClue-grounded long-video QA, useful for retrieval-heavy methods.ReKV, rLiVS
MovieChatLong movie/video understanding.rLiVS
VS-Ego / VS-MovieOffline long-video evaluation around egocentric and movie scenarios.rLiVS

MLVU

Metric: accuracy. Most streaming-memory papers report a single MLVU score, while AdaVideoRAG reports MLVU_test category scores plus AVG.

Method / PaperBackbone or SettingSplit / MetricScoreSource
ReKVLLaVA-OV-7B; 0.5 FPS -> 64 framesMLVU dev Acc.68.5 (+3.8)ReKV Table 6
StreamMemLLaVA-OneVision-7B; 0.5/0.2 FPS; KV size 6KMLVU66.9StreamMem Table 1
LiveVLMLLaVA-OneVision-7B; 0.5/0.2 FPSMLVU68.1LiveVLM Table 2
StreamingTOMLLaVA-OV-7B; 0.5/0.2 FPSMLVU67.9StreamingTOM Table 1
InfiniPot-VQwen-2-VL-7B; 768 frames; 6K budgetMLVU65.8InfiniPot-V Table 1
AdaVideoRAGVideoLLaMA3-7B; 1 FPS; 180 framesMLVU_test AVG53.2AdaVideoRAG Table 1
MuKVLLaVA-OV-7B; 0.5 FPS; 5.9K memory tokensMLVU67.8MuKV Table 9
MuKVQwen3-VL-4B; 0.5 FPS; 5.9K memory tokensMLVU66.0MuKV Table 9
WeaveTime C2FLLaVA-OV-7B + ReKV; retrieval ablationMLVU Acc.68.9WeaveTime Table 4
STC-Cacher & PrunerLLaVA-OneVision-7B + ReKV; offline evaluationMLVU-dev67.0STC Table 3

Video-MME

Metric: accuracy-style Video-MME scores. Subtitle settings and split labels differ; All, Overall, Ovl., and VideoMME are kept close to the paper wording.

Method / PaperBackbone or SettingScopeShortMediumLongOverall / AllSource
StreamMemLLaVA-OneVision-7B; 0.5/0.2 FPS; KV size 6Kw/o subtitles-56.650.159.4StreamMem Table 1
LiveVLMLLaVA-OneVision-Qwen2-7B-OV; 12K cachew/o subtitles-57.051.359.6LiveVLM Table 2
StreamingTOMLLaVA-OV-7B; 0.5/0.2 FPSw/o subtitles71.357.850.659.9StreamingTOM Table 1
InfiniPot-VQwen-2-VL-7B; 768 frames; 6K budgetw/o subtitles---62.8InfiniPot-V Table 1
AdaVideoRAGVideoLLaMA3-7B; 1 FPS; 180 framessubtitle setting not stated80.365.459.868.5AdaVideoRAG Table 2
ViG-RAGLLaVA-Video-72B; 32 frames; text 2.1Ksubtitle setting not stated79.371.172.874.4ViG-RAG Table 2
MuKVQwen3-VL-4B; 0.5 FPS; 5.9K memory tokenssubtitle setting not restated-61.851.063.6MuKV Table 9
StreamingVLMQwen2.5-VL-7B; streaming SFTw/o subtitles---65.10StreamingVLM Table 3
STC-Cacher & PrunerLLaVA-OneVision-7B + ReKV; offline evaluationsubtitle setting not restated67.353.948.356.5STC Table 3

EgoSchema

Metric: accuracy. StreamMem, StreamingTOM, and InfiniPot-V explicitly use the official dev/development split; MuKV treats the offline setting as asking at the video end.

Method / PaperBackbone or SettingAcc.Source
ReKVLLaVA-OV-7B; 0.5 FPS -> 64 frames60.7 (+0.9)ReKV Table 4
StreamMemQwen2-VL-7B; 4.0/0.5 FPS; KV size 6K67.2StreamMem Table 1
StreamingTOMLLaVA-OV-7B; 0.5/0.2 FPS63.7StreamingTOM Table 1
InfiniPot-VLLaVA-Next-7B; 128 frames; 6K budget65.8InfiniPot-V Table 1
MuKVQwen3-VL-4B; 0.5 FPS; 5.9K memory tokens67.0MuKV Table 9
STC-Cacher & PrunerLLaVA-OneVision-7B + ReKV; offline evaluation59.0STC Table 3

LongVideoBench

Metric: accuracy-style LongVideoBench / LVB score. ViG-RAG discusses LongVideoBench in the dataset section but does not report a standalone LongVideoBench result table.

Method / PaperBackbone or SettingSplit / ColumnScoreSource
LiveVLMLLaVA-OneVision-7B; 0.5/0.2 FPSLVB56.1LiveVLM Table 2
StreamingTOMLLaVA-OV-7B; 0.5 FPSAccuracy56.3StreamingTOM Table 5
InfiniPot-VQwen-2-VL-7B; 768 frames; 6K budgetLVB58.4InfiniPot-V Table 1
InfiniPot-VLLaVA-Next-7B; 128 frames; 6K budgetLVB60.9InfiniPot-V Table 1
StreamingVLMQwen2.5-VL-7B; streaming SFTLongVideoBench59.00StreamingVLM Table 3

HourVideo

Metric: overall accuracy on the hour-scale egocentric benchmark. OASIS reports this result in its supplementary long-horizon evaluation rather than its main benchmark table.

Method / PaperBackbone or SettingAccuracySource
Qwen3-VL-8BOASIS reproduction baseline35.11OASIS Supplementary Table 7
OASISQwen3-VL-8B + hierarchical event memory37.35OASIS Supplementary Table 7

HiVU

Metric: LLM-judged pairwise win rate. AdaVideoRAG uses DeepSeek-32B as the arbiter in the HiVU evaluation.

ComparisonMethodSplitComprehensivenessEmpowermentTrustworthinessDepthDensityOverall WinnerSource
Base vs AdaVideoRAGVideoLLaMA3 baselineOverall35.98%30.88%30.58%26.23%31.03%30.58%AdaVideoRAG Table 4
Base vs AdaVideoRAGAdaVideoRAGOverall64.02%69.12%69.42%73.77%68.97%69.42%AdaVideoRAG Table 4
VideoRAG vs AdaVideoRAGVideoRAGOverall45.33%43.81%46.40%40.88%44.10%44.10%AdaVideoRAG Table 4
VideoRAG vs AdaVideoRAGAdaVideoRAGOverall54.67%56.19%53.60%59.12%55.90%55.90%AdaVideoRAG Table 4

LongerVideos

Metric: win-rate comparison over long-form videos. ViG-RAG repeats the comparison against several graph/text RAG baselines; this compact table shows the GraphRAG-l group.

Method / PaperSettingComprehensivenessClarityDepthRelevancePractical ValueOverallSource
GraphRAG-lLongerVideos overall group22.59%23.33%21.99%22.96%21.24%22.42%ViG-RAG Table 1
VideoRAGLongerVideos overall group33.41%34.43%35.31%35.62%36.26%35.81%ViG-RAG Table 1
ViG-RAGLongerVideos overall group44.00%42.24%42.70%41.42%42.50%41.77%ViG-RAG Table 1

ActivityNet-QA

Metric: open-ended QA Acc. and 1-5 Score / Sco..

Method / PaperBackbone or SettingAcc.Score / Sco.Source
ReKVLLaVA-OV-0.5B + ReKV; 0.5 FPS -> 64 frames52.1 (+1.6)3.15 (+0.13)ReKV Table 4
ReKVLLaVA-OV-7B + ReKV; 0.5 FPS -> 64 frames60.4 (+3.8)3.52 (+0.23)ReKV Table 4
StreamChatSTREAMCHAT; offline setting50.12.78StreamChat Table 6

QAEGO4D

Metric: QaEgo4D_test-mc accuracy. ReKV also reports retrieval recall in a separate ablation table; this table keeps the offline benchmark comparison rows.

Method / PaperBackbone or Settingtest Acc.Source
Flash-VStream-7BOffline VideoQA comparison38.2ReKV Table 4
LLaVA-OV-0.5BBase model; 64 frames42.6ReKV Table 4
ReKVLLaVA-OV-0.5B; 0.5 FPS -> 64 frames50.0 (+7.4)ReKV Table 4
LLaVA-OV-7BBase model; 64 frames52.8ReKV Table 4
ReKVLLaVA-OV-7B; 0.5 FPS -> 64 frames56.0 (+3.2)ReKV Table 4
WeaveTime C2FLLaVA-OV-7B + ReKV; retrieval ablation55.2WeaveTime Table 4

CG-Bench / CGBench

Metric: clue-grounded benchmark accuracy. In rLiVS, the 0.5 FPS / 10K context setup is described in the experimental setup; Table 1 itself reports the final rLiVS (Ours) row.

Method / PaperBackbone or SettingAcc.Source
Chat-UniVirLiVS comparison table baseline25.9rLiVS Table 1
rLiVSLLaVA-OV-7B; 0.5 FPS; 10K context33.1rLiVS Table 1
ReKVLLaVA-OV-7B; 0.5 FPS -> 64 frames33.9 (+2.8)ReKV Table 6
ReKVLLaVA-OV-72B; 0.1 FPS -> 32 frames40.5 (+3.3)ReKV Table 6

MovieChat

Metric: long-video QA Acc. and 1-5 Sco..

Method / PaperBackbone or SettingAcc.Sco.Source
GoldfishrLiVS comparison table baseline67.64.2rLiVS Table 1
rLiVSLLaVA-OV-7B; 1 FPS; 10K context78.04.0rLiVS Table 1
rLiVS w/o recurrencyAblation74.13.9rLiVS Table 4

VS-Ego / VS-Movie

Metric: offline VS-Stream accuracy and 1-5 score.

Method / PaperBackbone or SettingVS-Ego Acc.VS-Ego Sco.VS-Movie Acc.VS-Movie Sco.Source
Flash-VStream-7BrLiVS comparison table baseline59.03.956.13.4rLiVS Table 1
rLiVSLLaVA-OV-7B; 0.5 FPS; 10K context61.03.959.33.6rLiVS Table 1

Standard Video QA Benchmarks

BenchmarkWhat It Mainly TestsPapers Using It
NExT-QA / NextQA-valsetShorter video QA and temporal reasoning. Often useful for token-selection ablations.StreamChat, rLiVS
MSVD-QAOpen-ended QA on short web videos.StreamChat
MSRVTT-QAOpen-ended QA on short web videos, broader than MSVD.StreamChat

NExT-QA / NextQA-valset

Metric: StreamChat reports open-ended Sco. and Acc. on NExT-QA; rLiVS reports NextQA-valset accuracy only in token-selection ablations.

Method / PaperSettingSco.Acc.NextQA-valset Acc.Source
STREAMCHATOffline benchmark setting2.8450.5-StreamChat Table 6
rLiVS Full ModelLLaVA-OneVision-7B; token-selection ablation--78.6rLiVS Table 7
rLiVS Attention6% selected visual tokens--77.0rLiVS Table 7
rLiVS Attention12% selected visual tokens--78.4rLiVS Table 7

MSVD-QA

Metric: open-ended QA Sco. and Acc..

Method / PaperSettingSco.Acc.Source
STREAMCHATOffline benchmark setting3.0858.7StreamChat Table 6

MSRVTT-QA

Metric: open-ended QA Sco. and Acc..

Method / PaperSettingSco.Acc.Source
STREAMCHATOffline benchmark setting2.3843.4StreamChat Table 6