curated · 2026-08-10

This piece is worth opening because a16z plots a steep climb on OSWorld-Verified: ~30% a year ago, now Claude Fable 5 at 85%, above the 72% human baseline. Their call is blunt—the model is no longer the bottleneck, and the fight has moved from "can the agent use a computer?" to "can it reliably do this job inside a real company?" I'd discount the score a bit. OSWorld tests standardized desktop tasks; real back-offices have messy permissions, missing process docs, and edge cases that break agents fast. The post admits agents get brittle when work drifts off the runbook, and costs don't pencil out everywhere when caching is intractable. The useful bit is where they see production deployments happening: standardized back-office work—updating legacy systems, moving data through portals, processing tickets—where no clean API exists and manual clicking is the baseline. If you're tracking computer-use agent timelines, the framework matters more than the score. The question isn't whether the model can click the right button; it's whether engineering teams can stitch together permissions, process knowledge, error handling, and caching into something that doesn't fall over.

The post is worth opening for the concrete numbers: a 30B multimodal model with a 128k+ context window, running on a single RTX 5090 with NVFP4 quantization and DFlash speculative decoding, hitting 236 tok/s per user and 1,452 tok/s total throughput. The hybrid sliding-window plus full-attention design is clearly aimed at keeping local inference costs down. I'd discount this a bit — it's an engineering integration announcement from LMSYS and Meta, not a third-party benchmark. The 236 tok/s figure is with NVFP4; the native BF16 checkpoint needs an H100. Apple Silicon gets MLX support but no speculative decoding yet. The thing to watch: Meta is positioning a 30B model as a base for local agentic workflows, not general-purpose chat. If they later publish agent-task success rates and end-to-end latency, that'll matter more than raw token speed.

The thing here is the routing logic: instead of fixed rules, it looks at what models the community actually used for similar tasks over the past 7 days and builds a Pareto-optimal curve from that. At the default tier, MMLU Pro hits 85.2% while cost drops from $393 to $141. At max tier, SWE-Atlas QnA jumps from 2.4% to 60.7%, but cost climbs to $1,325. I'd discount this a bit. The post doesn't spell out how task classification works or how often the routing curve refreshes. It does mention sticky behavior to avoid switching models mid-conversation and rebuilding caches — that matters for real-world use but won't show up in benchmarks. Don't read this as "routing beats single models." A fairer take: OpenRouter turned its usage data into a dynamic model recommendation engine, saving you the work of picking models and comparing prices yourself. If the task classification is accurate, this is more flexible than fixed routing. But we only have their own benchmark numbers right now — no external validation yet.