FEATUREDAI HOT (Curated Pool)· aihot-apiZH00:00 · 08·15
→MOSS-VL: An open VLM family that treats real-time interaction as a first-class capability
Fudan's MOSS-VL makes real-time interaction—perceiving while speaking—a first-class capability. Gated cross-attention keeps visual tokens outside the decoded sequence, giving it a 2.8× to 5.1× time-to-first-token advantage over same-backbone Qwen3-VL-8B. MOSS-VL-Realtime tops three of four streaming benchmarks, hitting 66.0 vs. 37.5 on OmniMMI Proactive Alerting. The offline variant leads temporal-reasoning video sets at comparable scale. All five checkpoints, the training curriculum, and inference code are open.
#Vision#Reasoning#MOSS-VL#OpenMOSS
why featured
Featured · importance 74 · hook + knowledge
editor take
MOSS-VL makes real-time perception a native capability, cutting time-to-first-token by 2.8–5.1× vs. same-backbone Qwen3-VL-8B, with all five checkpoints open.
sharp
The reason to click: this isn't just another VLM—it's architected so the model can keep watching while it talks. Visual tokens sit outside the decoded sequence and get pulled in through gated cross-attention, so growing visual context doesn't bloat the generation cache. The real-time variant hits 66.0 on OmniMMI Proactive Alerting against 37.5 for the best open baseline. That gap is large enough to take seriously.
Two discounts. First, 11.3B parameters isn't light, and the paper doesn't spell out real-time memory footprint or throughput—deployment won't be trivial. Second, the proactive behaviors (when to speak, stay silent, revise) are supervised by a synthetic interaction corpus; real-world robustness is unproven. The upside: the offline variant leads temporal-reasoning video benchmarks at comparable scale, so the architecture didn't sacrifice static understanding for streaming. All five checkpoints and inference code are open—if you're building streaming multimodal apps, you can run this today.
HKR breakdown
hook ✓knowledge ✓resonance —