StreamingVLM

Paper: StreamingVLM: Real-Time Understanding for Infinite Video Streams Project: StreamingVLM Code: mit-han-lab/streaming-vlm Datasets: Inf-Stream-Train / Inf-Stream-Eval Core Idea StreamingVLM is designed for a different target from most streaming VideoQA papers: it keeps watching a video and producing synchronized commentary, instead of waiting for a future question and then retrieving historical evidence. Its main idea is to align a simple streaming inference cache with the context pattern used during supervised fine-tuning. At inference time, the model keeps only: ...